Deband
- JPulowski
-
Topic Author

Please Log in or Create an account to join the conversation.
- lowenz
-
C:\ReShade 4\Shaders\Deband2.fx(136,2-13): error X3511: unable to unroll loop, loop does not appear to terminate in a timely manner (212 iterations) or unrolled loop is too large, use the [unroll(n)] attribute to force an exact higher number
Please Log in or Create an account to join the conversation.
- lowenz
-

Please Log in or Create an account to join the conversation.
- JPulowski
-
Topic Author
Seems due to changing tex2Dlod to tex2D. Thanks, will fix it in the next beta.lowenz wrote: C:\ReShade 4\Shaders\Deband2.fx(86,15-31): warning X3553: can't use gradient instructions in loops with break, forcing loop to unroll
C:\ReShade 4\Shaders\Deband2.fx(136,2-13): error X3511: unable to unroll loop, loop does not appear to terminate in a timely manner (212 iterations) or unrolled loop is too large, use the [unroll(n)] attribute to force an exact higher number
Please Log in or Create an account to join the conversation.
- Enterprise24
-

Please Log in or Create an account to join the conversation.
- JPulowski
-
Topic Author
Changelog:
- Added preset based basic setting and moved thresholds to advanced settings
- Fixed the problem where for loops were unable to be unroll in d3d9
- Added gradient angle checking (disabled by default)
I think the new thing is the gradient angle checking. From my limited testing, I did not see much benefit from it. It also causes a decrease in performance. Depending on your feedback I might remove it all together. It might be more efficient in video content and current parameters might not be optimized for video game content. Basically, when it detects a significant a gradient angle it boosts the thresholds for further debanding. Feel free to play with in addition to standard thresholds. (it is controlled by maximum angle and angle boost parameters)
I have also added madVR-like preset system. The current preset parameters are based on madshi's notes and do not reflect the final parameters. That actually depends on your feedback. So feel free to test them and tell me what do you think. You can also use "Custom" preset to input your own values in the advanced section. The default values in the advanced section are based on the medium preset by the way.
Again, if you encounter any bugs, errors or warnings feel free to let me know.
/**
* Deband shader by haasn
* https://github.com/haasn/gentoo-conf/blob/xor/home/nand/.mpv/shaders/deband-pre.glsl
*
* Copyright (c) 2015 Niklas Haas
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*
* Modified and optimized for ReShade by JPulowski
* https://reshade.me/forum/shader-presentation/768-deband
*
* Do not distribute without giving credit to the original author(s).
*
* 1.0 - Initial release
* 1.1 - Replaced the algorithm with the one from MPV
* 1.1a - Minor optimizations
* Removed unnecessary lines and replaced them with ReShadeFX intrinsic counterparts
* 2.0 - Replaced "grain" with CeeJay.dk's ordered dithering algorithm and enabled it by default
* The configuration is now more simpler and straightforward
* Some minor code changes and optimizations
* Improved the algorithm and made it more robust by adding some of the madshi's
* improvements to flash3kyuu_deband which should cause an increase in quality. Higher
* iterations/ranges should now yield higher quality debanding without too much decrease
* in quality.
* Changed licensing text and original source code URL
*/
#include "ReShadeUI.fxh"
uniform int threshold_preset < __UNIFORM_COMBO_INT1
ui_label = "Debanding strength";
ui_items = "Low\0Medium\0High\0Custom\0";
> = 0;
uniform float Range < __UNIFORM_SLIDER_FLOAT1
ui_min = 1.0;
ui_max = 32.0;
ui_step = 1.0;
ui_label = "Initial radius";
ui_tooltip = "The radius increases linearly for each iteration. A higher radius will find more gradients, but a lower radius will smooth more aggressively.";
> = 24.0;
uniform bool gradient_angle_check_enabled < __UNIFORM_RADIO_BOOL2
ui_label = "Gradient angle check";
ui_tooltip = "Additionally checks for gradient angles of the reference pixels and boosts the thresholds if necessary. In somes cases it might preserve some banding instead of details, it also demands more performance. Can be disabled for an extra performance boost.";
> = false;
uniform float custom_avgdiff < __UNIFORM_SLIDER_FLOAT1
ui_min = 0.0;
ui_max = 255.0;
ui_step = 0.1;
ui_label = "Average threshold";
ui_tooltip = "Threshold for the difference between the average of reference pixel values and the original pixel value. Higher numbers increase the debanding strength but progressively diminish image details. In pixel shaders a 8-bit color step equals to 1.0/255.0";
ui_category = "Advanced";
> = 1.8;
uniform float custom_maxdiff < __UNIFORM_SLIDER_FLOAT1
ui_min = 0.0;
ui_max = 255.0;
ui_step = 0.1;
ui_label = "Maximum threshold";
ui_tooltip = "Threshold for the difference between the maximum difference of one of the reference pixel values and the original pixel value. Higher numbers increase the debanding strength but progressively diminish image details. In pixel shaders a 8-bit color step equals to 1.0/255.0";
ui_category = "Advanced";
> = 4.0;
uniform float custom_middiff < __UNIFORM_SLIDER_FLOAT1
ui_min = 0.0;
ui_max = 255.0;
ui_step = 0.1;
ui_label = "Middle threshold";
ui_tooltip = "Threshold for the difference between the average of diagonal reference pixel values and the original pixel value. Higher numbers increase the debanding strength but progressively diminish image details. In pixel shaders a 8-bit color step equals to 1.0/255.0";
ui_category = "Advanced";
> = 2.0;
uniform float custom_maxangle < __UNIFORM_SLIDER_INT1
ui_min = 0.0;
ui_max = 255.0;
ui_step = 1.0;
ui_label = "Maximum angle";
ui_tooltip = "Threshold for the difference between the maximum gradient angle difference of one of the reference pixel values and the original pixel gradient angle value. If the maximum difference is smaller than this value other threshold values will be multiplied by angle boost value. In pixel shaders a 8-bit color step equals to 1.0/255.0";
ui_category = "Advanced";
> = 22.0;
uniform float custom_angleboost < __UNIFORM_SLIDER_FLOAT1
ui_min = 1.0;
ui_max = 255.0;
ui_step = 0.1;
ui_label = "Angle boost";
ui_tooltip = "The boost multiplier to be used with threshold values when there is a significant a gradient angle detected.";
ui_category = "Advanced";
> = 1.6;
uniform int iterations < __UNIFORM_SLIDER_INT1
ui_min = 1;
ui_max = 4;
ui_label = "Iterations";
ui_tooltip = "The number of debanding steps to perform per sample. Each step reduces a bit more banding, but takes time to compute.";
ui_category = "Advanced";
> = 1;
#include "ReShade.fxh"
// Reshade uses C rand for random, max cannot be larger than 2^15-1
uniform int drandom < source = "random"; min = 0; max = 32767; >;
uniform float2 pingpong < source = "pingpong"; min = 0; max = 1; step = 1; >;
float rand(float x)
{
return frac(x / 41.0);
}
float permute(float x)
{
return ((34.0 * x + 1.0) * x) % 289.0;
}
float3 gradient_angle_check(sampler2D tex, float2 texcoord) {
float3 p00 = tex2Dlod(tex, float4(texcoord.x - ReShade::PixelSize.x * 20.0, texcoord.y - ReShade::PixelSize.y * 20.0, 0.0, 0.0)).rgb;
float3 p10 = tex2Dlod(tex, float4(texcoord.x, texcoord.y - ReShade::PixelSize.y * 20.0, 0.0, 0.0)).rgb;
float3 p20 = tex2Dlod(tex, float4(texcoord.x + ReShade::PixelSize.x * 20.0, texcoord.y - ReShade::PixelSize.y * 20.0, 0.0, 0.0)).rgb;
float3 p01 = tex2Dlod(tex, float4(texcoord.x - ReShade::PixelSize.x * 20.0, texcoord.y, 0.0, 0.0)).rgb;
float3 p21 = tex2Dlod(tex, float4(texcoord.x + ReShade::PixelSize.x * 20.0, texcoord.y, 0.0, 0.0)).rgb;
float3 p02 = tex2Dlod(tex, float4(texcoord.x - ReShade::PixelSize.x * 20.0, texcoord.y + ReShade::PixelSize.y * 20.0, 0.0, 0.0)).rgb;
float3 p12 = tex2Dlod(tex, float4(texcoord.x, texcoord.y + ReShade::PixelSize.y * 20.0, 0.0, 0.0)).rgb;
float3 p22 = tex2Dlod(tex, float4(texcoord.x + ReShade::PixelSize.x * 20.0, texcoord.y + ReShade::PixelSize.y * 20.0, 0.0, 0.0)).rgb;
float3 gx = (p20 + p21 * 2.0 + p22) - (p00 + p01 * 2.0 + p02);
float3 gy = (p00 + p10 * 2.0 + p20) - (p02 + p12 * 2.0 + p22);
float3 angle = (abs(gx) < 0.01 / 255.0) ? 1.0 : (atan(gy / gx) / 3.1415927 + 0.5);
return angle;
}
float3 analyze_gradients(sampler2D tex, float2 texcoord, float2 range, float2 dir, float3 ori_g) {
// South-east
float3 ref_g = gradient_angle_check(tex, float2(texcoord + range * float2(dir.x, dir.y)));
float3 gra_max_diff = abs(ori_g - ref_g);
// North-west
ref_g = gradient_angle_check(tex, float2(texcoord + range * float2(-dir.x, -dir.y)));
gra_max_diff = max(gra_max_diff, abs(ori_g - ref_g));
// North-east
ref_g = gradient_angle_check(tex, float2(texcoord + range * float2(-dir.y, dir.x)));
gra_max_diff = max(gra_max_diff, abs(ori_g - ref_g));
// South-west
ref_g = gradient_angle_check(tex, float2(texcoord + range * float2( dir.y, -dir.x)));
gra_max_diff = max(gra_max_diff, abs(ori_g - ref_g));
return gra_max_diff;
}
void analyze_pixels(float3 ori, sampler2D tex, float2 texcoord, float2 range, float2 dir,
out float3 ref_avg,
out float3 ref_avg_diff,
out float3 ref_max_diff,
out float3 ref_mid_diff1,
out float3 ref_mid_diff2)
{
// Sample at quarter-turn intervals around the source pixel
// South-east
float3 ref = tex2Dlod(tex, float4(texcoord + range * dir, 0.0, 0.0)).rgb;
float3 diff = abs(ori - ref);
ref_max_diff = diff;
ref_avg = ref;
ref_mid_diff1 = ref;
// North-west
ref = tex2Dlod(tex, float4(texcoord + range * -dir, 0.0, 0.0)).rgb;
diff = abs(ori - ref);
ref_max_diff = max(ref_max_diff, diff);
ref_avg += ref;
ref_mid_diff1 = abs(((ref_mid_diff1 + ref) * 0.5) - ori);
// North-east
ref = tex2Dlod(tex, float4(texcoord + range * float2(-dir.y, dir.x), 0.0, 0.0)).rgb;
diff = abs(ori - ref);
ref_max_diff = max(ref_max_diff, diff);
ref_avg += ref;
ref_mid_diff2 = ref;
// South-west
ref = tex2Dlod(tex, float4(texcoord + range * float2( dir.y, -dir.x), 0.0, 0.0)).rgb;
diff = abs(ori - ref);
ref_max_diff = max(ref_max_diff, diff);
ref_avg += ref;
ref_mid_diff2 = abs(((ref_mid_diff2 + ref) * 0.5) - ori);
ref_avg *= 0.25; // Normalize avg
ref_avg_diff = abs(ori - ref_avg);
}
float3 PS_Deband(float4 vpos : SV_Position, float2 texcoord : TexCoord) : SV_Target
{
// Settings
float _avgdiff;
float _maxdiff;
float _middiff;
float angle_boost;
float _maxangle;
if (threshold_preset == 0) {
_avgdiff = 0.6;
_maxdiff = 1.9;
_middiff = 1.2;
angle_boost = 1.9;
_maxangle = 10.0;
}
else if (threshold_preset == 1) {
_avgdiff = 1.8;
_maxdiff = 4.0;
_middiff = 2.0;
angle_boost = 1.6;
_maxangle = 22.0;
}
else if (threshold_preset == 2) {
_avgdiff = 3.4;
_maxdiff = 6.8;
_middiff = 3.3;
}
else if (threshold_preset == 3) {
_avgdiff = custom_avgdiff;
_maxdiff = custom_maxdiff;
_middiff = custom_middiff;
angle_boost = custom_angleboost;
_maxangle = custom_maxangle;
}
if (!gradient_angle_check_enabled || threshold_preset == 2)
angle_boost = 1.0;
// Normalize
_avgdiff /= 255.0;
_maxdiff /= 255.0;
_middiff /= 255.0;
_maxangle /= 255.0;
// Initialize the PRNG by hashing the position + a random uniform
float h = permute(permute(permute(texcoord.x) + texcoord.y) + drandom / 32767.0);
float3 ref_avg; // Average of 4 reference pixels
float3 ref_avg_diff; // The difference between the average of 4 reference pixels and the original pixel
float3 ref_max_diff; // The maximum difference between one of the 4 reference pixels and the original pixel
float3 ref_mid_diff1; // The difference between the average of SE and NW reference pixels and the original pixel
float3 ref_mid_diff2; // The difference between the average of NE and SW reference pixels and the original pixel
float3 gra_max_diff; // The maximum difference between gradient angle of one of the 4 reference pixels and the original pixel's
float3 ori = tex2Dlod(ReShade::BackBuffer, float4(texcoord, 0.0, 0.0)).rgb; // Original pixel
float3 ori_g = gradient_angle_check(ReShade::BackBuffer, texcoord); // Gradient angle of the original pixel
float3 res; // Final pixel
// Compute a random angle
float dir = rand(permute(h)) * 6.2831853;
float2 o = float2(cos(dir), sin(dir));
#if (__RESHADE_PERFORMANCE_MODE__)
[unroll]
#endif
for (int i = 1; i <= iterations; ++i)
{
// Compute a random distance
float dist = rand(h) * Range * i;
float2 pt = dist * ReShade::PixelSize;
analyze_pixels(ori, ReShade::BackBuffer, texcoord, pt, o,
ref_avg,
ref_avg_diff,
ref_max_diff,
ref_mid_diff1,
ref_mid_diff2);
[branch] if (angle_boost > 1.0) {
gra_max_diff = analyze_gradients(ReShade::BackBuffer, texcoord, pt, o, ori_g);
angle_boost = gra_max_diff < (_maxangle * i) ? angle_boost : 1.0;
}
// Thresholds roughly based on madshi's "medium" settings
float3 ref_avg_diff_threshold = _avgdiff * i * angle_boost;
float3 ref_max_diff_threshold = _maxdiff * i * angle_boost;
float3 ref_mid_diff_threshold = _middiff * i * angle_boost;
// Fuzzy logic based pixel selection
float3 factor = pow(saturate(3.0 * (1.0 - ref_avg_diff / ref_avg_diff_threshold)) *
saturate(3.0 * (1.0 - ref_max_diff / ref_max_diff_threshold)) *
saturate(3.0 * (1.0 - ref_mid_diff1 / ref_mid_diff_threshold)) *
saturate(3.0 * (1.0 - ref_mid_diff2 / ref_mid_diff_threshold)), 0.1);
res = lerp(ori, ref_avg, factor);
h = permute(h);
}
const float dither_bit = 8.0; //Number of bits per channel. Should be 8 for most monitors.
/*------------------------.
| :: Ordered Dithering :: |
'------------------------*/
//Calculate grid position
float grid_position = frac(dot(texcoord, (ReShade::ScreenSize * float2(1.0 / 16.0, 10.0 / 36.0)) + 0.25));
//Calculate how big the shift should be
float dither_shift = 0.25 * (1.0 / (pow(2, dither_bit) - 1.0));
//Shift the individual colors differently, thus making it even harder to see the dithering pattern
float3 dither_shift_RGB = float3(dither_shift, -dither_shift, dither_shift); //subpixel dithering
//modify shift acording to grid position.
dither_shift_RGB = lerp(2.0 * dither_shift_RGB, -2.0 * dither_shift_RGB, grid_position); //shift acording to grid position.
//shift the color by dither_shift
res += dither_shift_RGB;
return res;
}
technique Deband <
ui_tooltip = "Alleviates color banding by trying to approximate original color values.";
>
{
pass
{
VertexShader = PostProcessVS;
PixelShader = PS_Deband;
}
}
Please Log in or Create an account to join the conversation.
- lowenz
-

Please Log in or Create an account to join the conversation.
- Enterprise24
-
Old deband have more strength but there is noticeably flicker from noise.
New deband cause much less flicker (on low setting - don't wan't to touch medium or higher since flicker is more noticeable).
Please Log in or Create an account to join the conversation.
- Scorpio82CO
-
Please Log in or Create an account to join the conversation.
- JBeckman
-
Scorpio82CO wrote: Doesn´t work for me in reshade 4.2.1, could you update the version??
Do you have the ReShadeUI.fxh file?
From this new version of the shader.#include "ReShadeUI.fxh"
github.com/crosire/reshade-shaders/blob/...haders/ReShadeUI.fxh
raw.githubusercontent.com/crosire/reshad...haders/ReShadeUI.fxh
That's accompanying ReShade.fxh now since a few changes back to the shader files but since Deband hasn't updated in some time it might still have been from before that change got added to the shader repository.
Unless ReShade.exe has a way to check for changes or you periodically re-download everything or manually check the shader Github repository it's very easy to miss changes to shader files even for core functionality such as this adjustment.
Just a suggestion, logger in the ReShade UI itself might have some additional info from breaking errors (red color.) to warnings (yellow color.) and perhaps more.

Please Log in or Create an account to join the conversation.
- Duran.te
-
Since pictures are worth a thousand words, here's some screenshots from Sekiro - Colours Band Twice:
(Yep, once again with over saturation)
Values are similar to default ones, with just few changes:
 - The Old Debanding Initial Radius has been changed from 16.000 to 24.000 like beta versions.
 - Threshold from Beta Debandings modified to 1.02
    (in the screenshot, I used the default "Low" strength preset with the updated version to show the new preset system behaviour)
 - Iterations of every version edited to 8 (2x max value) to better highlight the differences.
[hr]Huge fan of the new menu setting: The preset system is very intuitive and the Advanced line it's a definitely cleaner solution!
I've also tested the new gradient angle check trying to find out if there were significant difference with it on (except for performance decreasing

Please Log in or Create an account to join the conversation.
- JPulowski
-
Topic Author
Here is the next beta, which is also the release candidate.
Changelog:
- Removed gradient angle checking since it doesn't help with debanding, in the future i will probably replace it with a better suited local spatial frequency activity analysis function (e.g. contrast, uniformity, entropy etc.).
- Cleaned up the code.
- Added a new "Debug view" parameter which should help with determining range and iterations to make sure that the blurred image captures all the banding in the image.
/**
* Deband shader by haasn
* https://github.com/haasn/gentoo-conf/blob/xor/home/nand/.mpv/shaders/deband-pre.glsl
*
* Copyright (c) 2015 Niklas Haas
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
* of this software and associated documentation files (the "Software"), to deal
* in the Software without restriction, including without limitation the rights
* to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
* copies of the Software, and to permit persons to whom the Software is
* furnished to do so, subject to the following conditions:
*
* The above copyright notice and this permission notice shall be included in
* all copies or substantial portions of the Software.
*
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
* IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
* FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
* AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
* LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
* OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*
* Modified and optimized for ReShade by JPulowski
* https://reshade.me/forum/shader-presentation/768-deband
*
* Do not distribute without giving credit to the original author(s).
*
* 1.0 - Initial release
* 1.1 - Replaced the algorithm with the one from MPV
* 1.1a - Minor optimizations
* Removed unnecessary lines and replaced them with ReShadeFX intrinsic counterparts
* 2.0 - Replaced "grain" with CeeJay.dk's ordered dithering algorithm and enabled it by default
* The configuration is now more simpler and straightforward
* Some minor code changes and optimizations
* Improved the algorithm and made it more robust by adding some of the madshi's
* improvements to flash3kyuu_deband which should cause an increase in quality. Higher
* iterations/ranges should now yield higher quality debanding without too much decrease
* in quality.
* Changed licensing text and original source code URL
*/
#include "ReShadeUI.fxh"
uniform int threshold_preset < __UNIFORM_COMBO_INT1
ui_label = "Debanding strength";
ui_items = "Low\0Medium\0High\0Custom\0";
ui_tooltip = "Debanding presets. Use Custom to be able to use custom thresholds in the advanced section.";
> = 0;
uniform float range < __UNIFORM_SLIDER_FLOAT1
ui_min = 1.0;
ui_max = 32.0;
ui_step = 1.0;
ui_label = "Initial radius";
ui_tooltip = "The radius increases linearly for each iteration. A higher radius will find more gradients, but a lower radius will smooth more aggressively.";
> = 24.0;
uniform int iterations < __UNIFORM_SLIDER_INT1
ui_min = 1;
ui_max = 4;
ui_label = "Iterations";
ui_tooltip = "The number of debanding steps to perform per sample. Each step reduces a bit more banding, but takes time to compute.";
> = 1;
uniform float custom_avgdiff < __UNIFORM_SLIDER_FLOAT1
ui_min = 0.0;
ui_max = 255.0;
ui_step = 0.1;
ui_label = "Average threshold";
ui_tooltip = "Threshold for the difference between the average of reference pixel values and the original pixel value. Higher numbers increase the debanding strength but progressively diminish image details. In pixel shaders a 8-bit color step equals to 1.0/255.0";
ui_category = "Advanced";
> = 1.8;
uniform float custom_maxdiff < __UNIFORM_SLIDER_FLOAT1
ui_min = 0.0;
ui_max = 255.0;
ui_step = 0.1;
ui_label = "Maximum threshold";
ui_tooltip = "Threshold for the difference between the maximum difference of one of the reference pixel values and the original pixel value. Higher numbers increase the debanding strength but progressively diminish image details. In pixel shaders a 8-bit color step equals to 1.0/255.0";
ui_category = "Advanced";
> = 4.0;
uniform float custom_middiff < __UNIFORM_SLIDER_FLOAT1
ui_min = 0.0;
ui_max = 255.0;
ui_step = 0.1;
ui_label = "Middle threshold";
ui_tooltip = "Threshold for the difference between the average of diagonal reference pixel values and the original pixel value. Higher numbers increase the debanding strength but progressively diminish image details. In pixel shaders a 8-bit color step equals to 1.0/255.0";
ui_category = "Advanced";
> = 2.0;
uniform bool debug_output < __UNIFORM_RADIO_BOOL1
ui_label = "Debug view";
ui_tooltip = "Shows the low-pass filtered (blurred) output. Could be useful when making sure that range and iterations capture all of the banding in the picture.";
ui_category = "Advanced";
> = false;
#include "ReShade.fxh"
// Reshade uses C rand for random, max cannot be larger than 2^15-1
uniform int drandom < source = "random"; min = 0; max = 32767; >;
float rand(float x)
{
return frac(x / 41.0);
}
float permute(float x)
{
return ((34.0 * x + 1.0) * x) % 289.0;
}
void analyze_pixels(float3 ori, sampler2D tex, float2 texcoord, float2 _range, float2 dir,
out float3 ref_avg,
out float3 ref_avg_diff,
out float3 ref_max_diff,
out float3 ref_mid_diff1,
out float3 ref_mid_diff2)
{
// Sample at quarter-turn intervals around the source pixel
// South-east
float3 ref = tex2Dlod(tex, float4(texcoord + _range * dir, 0.0, 0.0)).rgb;
float3 diff = abs(ori - ref);
ref_max_diff = diff;
ref_avg = ref;
ref_mid_diff1 = ref;
// North-west
ref = tex2Dlod(tex, float4(texcoord + _range * -dir, 0.0, 0.0)).rgb;
diff = abs(ori - ref);
ref_max_diff = max(ref_max_diff, diff);
ref_avg += ref;
ref_mid_diff1 = abs(((ref_mid_diff1 + ref) * 0.5) - ori);
// North-east
ref = tex2Dlod(tex, float4(texcoord + _range * float2(-dir.y, dir.x), 0.0, 0.0)).rgb;
diff = abs(ori - ref);
ref_max_diff = max(ref_max_diff, diff);
ref_avg += ref;
ref_mid_diff2 = ref;
// South-west
ref = tex2Dlod(tex, float4(texcoord + _range * float2( dir.y, -dir.x), 0.0, 0.0)).rgb;
diff = abs(ori - ref);
ref_max_diff = max(ref_max_diff, diff);
ref_avg += ref;
ref_mid_diff2 = abs(((ref_mid_diff2 + ref) * 0.5) - ori);
ref_avg *= 0.25; // Normalize avg
ref_avg_diff = abs(ori - ref_avg);
}
float3 PS_Deband(float4 vpos : SV_Position, float2 texcoord : TexCoord) : SV_Target
{
// Settings
float _avgdiff;
float _maxdiff;
float _middiff;
if (threshold_preset == 0) {
_avgdiff = 0.6;
_maxdiff = 1.9;
_middiff = 1.2;
}
else if (threshold_preset == 1) {
_avgdiff = 1.8;
_maxdiff = 4.0;
_middiff = 2.0;
}
else if (threshold_preset == 2) {
_avgdiff = 3.4;
_maxdiff = 6.8;
_middiff = 3.3;
}
else if (threshold_preset == 3) {
_avgdiff = custom_avgdiff;
_maxdiff = custom_maxdiff;
_middiff = custom_middiff;
}
// Normalize
_avgdiff /= 255.0;
_maxdiff /= 255.0;
_middiff /= 255.0;
// Initialize the PRNG by hashing the position + a random uniform
float h = permute(permute(permute(texcoord.x) + texcoord.y) + drandom / 32767.0);
float3 ref_avg; // Average of 4 reference pixels
float3 ref_avg_diff; // The difference between the average of 4 reference pixels and the original pixel
float3 ref_max_diff; // The maximum difference between one of the 4 reference pixels and the original pixel
float3 ref_mid_diff1; // The difference between the average of SE and NW reference pixels and the original pixel
float3 ref_mid_diff2; // The difference between the average of NE and SW reference pixels and the original pixel
float3 ori = tex2Dlod(ReShade::BackBuffer, float4(texcoord, 0.0, 0.0)).rgb; // Original pixel
float3 res; // Final pixel
// Compute a random angle
float dir = rand(permute(h)) * 6.2831853;
float2 o = float2(cos(dir), sin(dir));
#if (__RESHADE_PERFORMANCE_MODE__)
[unroll]
#endif
for (int i = 1; i <= iterations; ++i)
{
// Compute a random distance
float dist = rand(h) * range * i;
float2 pt = dist * ReShade::PixelSize;
analyze_pixels(ori, ReShade::BackBuffer, texcoord, pt, o,
ref_avg,
ref_avg_diff,
ref_max_diff,
ref_mid_diff1,
ref_mid_diff2);
float3 ref_avg_diff_threshold = _avgdiff * i;
float3 ref_max_diff_threshold = _maxdiff * i;
float3 ref_mid_diff_threshold = _middiff * i;
// Fuzzy logic based pixel selection
float3 factor = pow(saturate(3.0 * (1.0 - ref_avg_diff / ref_avg_diff_threshold)) *
saturate(3.0 * (1.0 - ref_max_diff / ref_max_diff_threshold)) *
saturate(3.0 * (1.0 - ref_mid_diff1 / ref_mid_diff_threshold)) *
saturate(3.0 * (1.0 - ref_mid_diff2 / ref_mid_diff_threshold)), 0.1);
if (debug_output)
res = ref_avg;
else
res = lerp(ori, ref_avg, factor);
h = permute(h);
}
const float dither_bit = 8.0; //Number of bits per channel. Should be 8 for most monitors.
/*------------------------.
| :: Ordered Dithering :: |
'------------------------*/
//Calculate grid position
float grid_position = frac(dot(texcoord, (ReShade::ScreenSize * float2(1.0 / 16.0, 10.0 / 36.0)) + 0.25));
//Calculate how big the shift should be
float dither_shift = 0.25 * (1.0 / (pow(2, dither_bit) - 1.0));
//Shift the individual colors differently, thus making it even harder to see the dithering pattern
float3 dither_shift_RGB = float3(dither_shift, -dither_shift, dither_shift); //subpixel dithering
//modify shift acording to grid position.
dither_shift_RGB = lerp(2.0 * dither_shift_RGB, -2.0 * dither_shift_RGB, grid_position); //shift acording to grid position.
//shift the color by dither_shift
res += dither_shift_RGB;
return res;
}
technique Deband <
ui_tooltip = "Alleviates color banding by trying to approximate original color values.";
>
{
pass
{
VertexShader = PostProcessVS;
PixelShader = PS_Deband;
}
}
Please Log in or Create an account to join the conversation.
- Duran.te
-
An advice I could give, according to my experience, would be to boost up a little bit the dithering effect as, in many cases, just raising up each threshold is not enough to get rid of color banding without loosing too many details.
Also, it could be added an extra very high profile for all of those gradient-based games with excessive banding issue.
These are some few examples:
From top to bottom: Inside (2016) - Ibb & Obb (2014) - Necropolis (2016)
Please Log in or Create an account to join the conversation.
- empleat
-
I tried latest bundle, i found: reshade 1.1 + sweetfx 2.0 and it didn't worked, while default shaders did.
I highly increased treshold to 0.12 and nothing.
I added in - i don't know exactly how it called, there were values like this: USE_CARTOON 0.
So i simply wrote USE_DEBAND 1 - that's how i named my shader and i pressed scroll lock and it said it is on so.
No idea why it didn't worked.
Please Log in or Create an account to join the conversation.
- lowenz
-
Please Log in or Create an account to join the conversation.
- empleat
-
- lowenz
-
CTRL+click and you'll get the DLLs (ReShade32.dll + ReShade64.dll)empleat wrote: ? There is no dll file...
Please Log in or Create an account to join the conversation.
- JPulowski
-
Topic Author

Thanks for your feedback. It doesn't have anything to do with dithering. Dithering is already optimized for 8 bpc output. The problem is current implementation is static which means for every region the shader applies the same thresholds. Which is why you either have too much debanding or it's good but still some banding remains. In order to overcome that I need to apply a spatial frequency analysis function to moderate thresholds. Current version only has additional sanity checking compared to the previous ones, but the thresholds themselves are still static, they are not adapting to the image content (gradient vs. texture). I am already reviewing some known methods and will push an update once I have the time to develop a prototype, again I will share it here first, then will make it public. Also feel free to suggest values for the very high preset.Duran.te wrote: These values are pretty okayish.
An advice I could give, according to my experience, would be to boost up a little bit the dithering effect as, in many cases, just raising up each threshold is not enough to get rid of color banding without loosing too many details.
Also, it could be added an extra very high profile for all of those gradient-based games with excessive banding issue.
Please Log in or Create an account to join the conversation.
- empleat
-
Because it is more simple and i put into mpc folder ReShade32.dll and Deband.h file and which i renamed to Reshade.fx. Still not working.
I used reshade 2.0 there is a link at first post on site 1.
Please Log in or Create an account to join the conversation.
- JPulowski
-
Topic Author
Please Log in or Create an account to join the conversation.