[SOLVED] Strange problem regarding discard with ReShade 4.5

  • Posts: 273
5 months 2 weeks ago - 5 months 2 weeks ago #1 by brussell
With Reshade 4.5 I noticed a strange issue with my bloom shader, which results in a look like this: i.imgur.com/1IKR48k.jpg
I had a hard time finding the cause, but I could pinpoint it to some weird texture lookup and the discard function.
So here's an example ReShade.fx:
//Textures And Samplers
texture tex1 { Width = BUFFER_WIDTH; Height = BUFFER_HEIGHT; Format = RGBA8; };
sampler Sampler1 { Texture = tex1; };

texture tex2 { Width = BUFFER_WIDTH; Height = BUFFER_HEIGHT; Format = RGBA8; };
sampler Sampler2 { Texture = tex2; };

//Vertex Shader 
void VS(in uint id : SV_VertexID, out float4 position : SV_Position, out float2 texcoord : TEXCOORD)
{
    texcoord.x = (id == 2) ? 2.0 : 0.0;
    texcoord.y = (id == 1) ? 2.0 : 0.0;
    position = float4(texcoord * float2(2.0, -2.0) + float2(-1.0, 1.0), 0.0, 1.0);
}

//Pixel Shader
float4 PS1(float4 pos : SV_Position, float2 texcoord : TEXCOORD) : SV_Target
{
    //if (texcoord.x > 0.5) { return 0.0.xxxx; }
    if (texcoord.x > 0.5) { discard; }
    return 0.5;
}

float4 PS2(float4 pos : SV_Position, float2 texcoord : TEXCOORD) : SV_Target
{
    return tex2Dlod(Sampler1, float4(texcoord, 0, 0)) ;
}

float4 PS3(float4 pos : SV_Position, float2 texcoord : TEXCOORD) : SV_Target
{
    return tex2Dlod(Sampler2, float4(texcoord, 0, 0)) * 1.1;
}

float4 PS4(float4 pos : SV_Position, float2 texcoord : TEXCOORD) : SV_Target
{
    return tex2Dlod(Sampler2, float4(texcoord, 0, 0));
}

//Technique 
technique Effects < enabled = 1;  >
{
    pass 
    {
        VertexShader = VS;
        PixelShader = PS1;
        RenderTarget = tex1;
    }
    pass 
    {
        VertexShader = VS;
        PixelShader = PS2;
        RenderTarget = tex2;
    }
    //This pass shouldn't affect the final image at all, but it does
    pass 
    {
        VertexShader = VS;
        PixelShader = PS3;
        RenderTarget = tex1;
    }
    pass
    {
        VertexShader = VS;
        PixelShader = PS4;
    }
}
The expeced result should look like this (this is what I get with Reshade 4.4):


Instead we get this (ignore the wrong grey color caused by compression):


The issue only shows when pass PS3 is enabled, even if it shouldn't have any effect on the image. And it only happens with "* 1.1" (or any other multiplication).
It also works if using "return 0.0.xxxx;" instead of "discard;".

Please Log in or Create an account to join the conversation.

  • Posts: 3723
5 months 2 weeks ago #2 by crosire
Weird, I can't reproduce that on any API. Always get the first image. Does this happen everywhere or a specific game only? Are your drivers up-to-date?

Please Log in or Create an account to join the conversation.

  • Posts: 273
5 months 2 weeks ago - 5 months 2 weeks ago #3 by brussell
Well, I thought I already ruled it out yesterday, but it seems I was wrong: The issue only happen when using SGSSAA. I tried it with two dx9 games and both show the weird behavior with ReShade.
Here are the settings I use (as you can see I have the newest driver):


Even if it seems odd to use the Diablo3 AA flag, it works well for every game I know without problems.
As I said, I never had issues with SGSSAA and ReShade until 4.5

I always use the COD4 MW Demo for testing purposes. You can download it here (if you don't have a dx9 game on hand now):
www.4players.de/4players.php/download_in...rn_Warfare/Demo.html

Please Log in or Create an account to join the conversation.

  • Posts: 3723
5 months 1 week ago #4 by crosire
Still happening in 4.5.1?

Please Log in or Create an account to join the conversation.

  • Posts: 273
5 months 1 week ago #5 by brussell
Yes, still happening.

Please Log in or Create an account to join the conversation.

  • Posts: 273
5 months 1 week ago - 5 months 1 week ago #6 by brussell
So I found out it happens even without SGSSAA.
With the following example ReShade.fx you should definitely see the bug (tested with a dx9 and dx11 game):
//Textures And Samplers
texture tex1 { Width = BUFFER_WIDTH; Height = BUFFER_HEIGHT; Format = RGBA8; };
sampler Sampler1 { Texture = tex1; };

texture tex2 { Width = BUFFER_WIDTH; Height = BUFFER_HEIGHT; Format = RGBA8; };
sampler Sampler2 { Texture = tex2; };

//Vertex Shader
void VS(in uint id : SV_VertexID, out float4 position : SV_Position, out float2 texcoord : TEXCOORD)
{
    texcoord.x = (id == 2) ? 2.0 : 0.0;
    texcoord.y = (id == 1) ? 2.0 : 0.0;
    position = float4(texcoord * float2(2.0, -2.0) + float2(-1.0, 1.0), 0.0, 1.0);
}

//Pixel Shader
float4 PS1(float4 pos : SV_Position, float2 texcoord : TEXCOORD) : SV_Target
{
    //if (texcoord.x > 0.5) { return 0.0.xxxx; }
    if (texcoord.x > 0.5) { discard; }
    return 0.5;
}

float4 PS2(float4 pos : SV_Position, float2 texcoord : TEXCOORD) : SV_Target
{
    return tex2Dlod(Sampler1, float4(texcoord, 0, 0));
}

float4 PS3(float4 pos : SV_Position, float2 texcoord : TEXCOORD) : SV_Target
{
    return 1.0;
}

float4 PS4(float4 pos : SV_Position, float2 texcoord : TEXCOORD) : SV_Target
{
    return tex2Dlod(Sampler2, float4(texcoord, 0, 0));
}

//Technique
technique Effects < enabled = 1;  >
{
    pass 
    {
        VertexShader = VS;
        PixelShader = PS1;
        RenderTarget = tex1;
    }
    pass 
    {
        VertexShader = VS;
        PixelShader = PS2;
        RenderTarget = tex2;
    }
    //This pass shouldn't affect the final image at all, but it does
    pass 
    {
        VertexShader = VS;
        PixelShader = PS3;
        RenderTarget = tex1;
    }
    pass
    {
        VertexShader = VS;
        PixelShader = PS4;
    }
}

Expected Result:Result:

Pass 3 samples tex1 which gets white.
Pass 4 samples tex2 which should produce the expected image (left grey, right black), but somehow the right side becomes white from pass 3.

Please Log in or Create an account to join the conversation.

  • Posts: 3723
5 months 1 week ago - 5 months 1 week ago #7 by crosire
Aha! Nope, this does exactly what it is supposed to do =). You just have to think in multiple frames:

Frame 1:
> tex1 is completly black.
> PS1 only writes to the left part of the texture and does not touch the right part (because it is discarded).
> tex1 is now half grey, half black.
> PS3 clears the entire texture to 1.
> tex1 is now completly white.

Frame 2:
> tex1 is completly white (see end of last frame)
> PS1 only writes to the left part of the texture ...
> tex1 is now half grey, half white (since the right side was not touched)
...

If you want it to stay black on the right you need to tell ReShade to clear the texture before you start a new pass. This has been the default before, but as of 4.5 the default has changed to "false" for better performance (since such a clear operation is not free):
technique Effects < enabled = 1;  >
{
    pass 
    {
        VertexShader = VS;
        PixelShader = PS1;
        RenderTarget = tex1;
        ClearRenderTargets = true; 
    }
    ...
}
The following user(s) said Thank You: brussell

Please Log in or Create an account to join the conversation.

  • Posts: 273
5 months 1 week ago - 5 months 1 week ago #8 by brussell
Ahh, makes perfect sense now that I think about it! The change of default behavior just made it look like a bug.
Thanks for the explanation.

Please Log in or Create an account to join the conversation.