Possible compiler error on D3D10/11
- luluco250
-
Topic Author
More
7 years 6 months ago - 7 years 6 months ago #1
by luluco250
Possible compiler error on D3D10/11 was created by luluco250
Hi, I found out this weird glitch in D3D10/11 where colors can go negative after RGB-to-HSV and HSV-to-RGB conversions.
The following functions are used for the conversion itself (ported from HERE ):
And used as such:
In D3D9 no problems occur, however in both D3D10 and 11 (tested in Crysis (DX10) and Postal 2 (DgVoodoo2 D3D8-to-D3D11)), the following happens after blurring (for bloom):
From some testing, I've come to the conclusion that due to some sort of compiler error, dark pixels sometimes go negative, which in turn glitches out the bloom effect done afterwards.
Simply using max(0, col) (after hsv2rgb(col)) fixes this completely, as shown below:
As far as I know that simple fix seems to do the trick and no other problems seem to occur from this specific case.
Here are two more screenshots showing only the bloom effect itself for the sake of completely demonstrating the problem:
So far I have only seen this problem in 3.0, including the beta and final versions, I haven't tested it in 2.0.
The following functions are used for the conversion itself (ported from HERE ):
//convert RedGreenBlue to HueSaturationValue
float3 rgb2hsv(float3 c) {
float4 K = float4(0.0, -1.0 / 3.0, 2.0 / 3.0, -1.0);
float4 p = c.g < c.b ? float4(c.bg, K.wz) : float4(c.gb, K.xy);
float4 q = c.r < p.x ? float4(p.xyw, c.r) : float4(c.r, p.yzx);
float d = q.x - min(q.w, q.y);
float e = 1.0e-10;
return float3(abs(q.z + (q.w - q.y) / (6.0 * d + e)), d / (q.x + e), q.x);
}
//convert HueSaturationValue to RedGreenBlue
float3 hsv2rgb(float3 c) {
float4 K = float4(1.0, 2.0 / 3.0, 1.0 / 3.0, 3.0);
float3 p = abs(frac(c.xxx + K.xyz) * 6.0 - K.www);
return c.z * lerp(K.xxx, saturate(p - K.xxx), c.y);
}And used as such:
float3 Threshold(float4 pos : SV_Position, float2 uv : TEXCOORD0) : SV_Target {
float3 col = rgb2hsv(tex2D(ReShade::BackBuffer, uv).rgb);
float weight = lerp(1.0, fMagicBloom_DepthOverbright, ReShade::GetLinearizedDepth(uv)) * fMagicBloom_Overbright;
col.z = smoothstep(fMagicBloom_Threshold, 1.0, col.z) * weight;
col.z *= fMagicBloom_Intensity;
return hsv2rgb(col);
}In D3D9 no problems occur, however in both D3D10 and 11 (tested in Crysis (DX10) and Postal 2 (DgVoodoo2 D3D8-to-D3D11)), the following happens after blurring (for bloom):
From some testing, I've come to the conclusion that due to some sort of compiler error, dark pixels sometimes go negative, which in turn glitches out the bloom effect done afterwards.
Simply using max(0, col) (after hsv2rgb(col)) fixes this completely, as shown below:
As far as I know that simple fix seems to do the trick and no other problems seem to occur from this specific case.
Here are two more screenshots showing only the bloom effect itself for the sake of completely demonstrating the problem:
So far I have only seen this problem in 3.0, including the beta and final versions, I haven't tested it in 2.0.
Last edit: 7 years 6 months ago by luluco250.
Please Log in or Create an account to join the conversation.