[SOLVED] Strange problem regarding discard with ReShade 4.5

  • brussell
  • Topic Author
More
4 years 11 months ago - 4 years 11 months ago #1 by brussell Strange problem regarding discard with ReShade 4.5 was created by brussell
With Reshade 4.5 I noticed a strange issue with my bloom shader, which results in a look like this: i.imgur.com/1IKR48k.jpg
I had a hard time finding the cause, but I could pinpoint it to some weird texture lookup and the discard function.
So here's an example ReShade.fx:
//Textures And Samplers
texture tex1 { Width = BUFFER_WIDTH; Height = BUFFER_HEIGHT; Format = RGBA8; };
sampler Sampler1 { Texture = tex1; };

texture tex2 { Width = BUFFER_WIDTH; Height = BUFFER_HEIGHT; Format = RGBA8; };
sampler Sampler2 { Texture = tex2; };

//Vertex Shader 
void VS(in uint id : SV_VertexID, out float4 position : SV_Position, out float2 texcoord : TEXCOORD)
{
    texcoord.x = (id == 2) ? 2.0 : 0.0;
    texcoord.y = (id == 1) ? 2.0 : 0.0;
    position = float4(texcoord * float2(2.0, -2.0) + float2(-1.0, 1.0), 0.0, 1.0);
}

//Pixel Shader
float4 PS1(float4 pos : SV_Position, float2 texcoord : TEXCOORD) : SV_Target
{
    //if (texcoord.x > 0.5) { return 0.0.xxxx; }
    if (texcoord.x > 0.5) { discard; }
    return 0.5;
}

float4 PS2(float4 pos : SV_Position, float2 texcoord : TEXCOORD) : SV_Target
{
    return tex2Dlod(Sampler1, float4(texcoord, 0, 0)) ;
}

float4 PS3(float4 pos : SV_Position, float2 texcoord : TEXCOORD) : SV_Target
{
    return tex2Dlod(Sampler2, float4(texcoord, 0, 0)) * 1.1;
}

float4 PS4(float4 pos : SV_Position, float2 texcoord : TEXCOORD) : SV_Target
{
    return tex2Dlod(Sampler2, float4(texcoord, 0, 0));
}

//Technique 
technique Effects < enabled = 1;  >
{
    pass 
    {
        VertexShader = VS;
        PixelShader = PS1;
        RenderTarget = tex1;
    }
    pass 
    {
        VertexShader = VS;
        PixelShader = PS2;
        RenderTarget = tex2;
    }
    //This pass shouldn't affect the final image at all, but it does
    pass 
    {
        VertexShader = VS;
        PixelShader = PS3;
        RenderTarget = tex1;
    }
    pass
    {
        VertexShader = VS;
        PixelShader = PS4;
    }
}
The expeced result should look like this (this is what I get with Reshade 4.4):


Instead we get this (ignore the wrong grey color caused by compression):


The issue only shows when pass PS3 is enabled, even if it shouldn't have any effect on the image. And it only happens with "* 1.1" (or any other multiplication).
It also works if using "return 0.0.xxxx;" instead of "discard;".
Last edit: 4 years 11 months ago by brussell.

Please Log in or Create an account to join the conversation.

  • crosire
More
4 years 11 months ago #2 by crosire Replied by crosire on topic Strange problem regarding discard with ReShade 4.5
Weird, I can't reproduce that on any API. Always get the first image. Does this happen everywhere or a specific game only? Are your drivers up-to-date?

Please Log in or Create an account to join the conversation.

  • brussell
  • Topic Author
More
4 years 11 months ago - 4 years 11 months ago #3 by brussell Replied by brussell on topic Strange problem regarding discard with ReShade 4.5
Well, I thought I already ruled it out yesterday, but it seems I was wrong: The issue only happen when using SGSSAA. I tried it with two dx9 games and both show the weird behavior with ReShade.
Here are the settings I use (as you can see I have the newest driver):


Even if it seems odd to use the Diablo3 AA flag, it works well for every game I know without problems.
As I said, I never had issues with SGSSAA and ReShade until 4.5

I always use the COD4 MW Demo for testing purposes. You can download it here (if you don't have a dx9 game on hand now):
www.4players.de/4players.php/download_in...rn_Warfare/Demo.html
Last edit: 4 years 11 months ago by brussell.

Please Log in or Create an account to join the conversation.

  • crosire
More
4 years 11 months ago #4 by crosire Replied by crosire on topic Strange problem regarding discard with ReShade 4.5
Still happening in 4.5.1?

Please Log in or Create an account to join the conversation.

  • brussell
  • Topic Author
More
4 years 11 months ago #5 by brussell Replied by brussell on topic Strange problem regarding discard with ReShade 4.5
Yes, still happening.

Please Log in or Create an account to join the conversation.

  • brussell
  • Topic Author
More
4 years 11 months ago - 4 years 11 months ago #6 by brussell Replied by brussell on topic Strange problem regarding discard with ReShade 4.5
So I found out it happens even without SGSSAA.
With the following example ReShade.fx you should definitely see the bug (tested with a dx9 and dx11 game):
//Textures And Samplers
texture tex1 { Width = BUFFER_WIDTH; Height = BUFFER_HEIGHT; Format = RGBA8; };
sampler Sampler1 { Texture = tex1; };

texture tex2 { Width = BUFFER_WIDTH; Height = BUFFER_HEIGHT; Format = RGBA8; };
sampler Sampler2 { Texture = tex2; };

//Vertex Shader
void VS(in uint id : SV_VertexID, out float4 position : SV_Position, out float2 texcoord : TEXCOORD)
{
    texcoord.x = (id == 2) ? 2.0 : 0.0;
    texcoord.y = (id == 1) ? 2.0 : 0.0;
    position = float4(texcoord * float2(2.0, -2.0) + float2(-1.0, 1.0), 0.0, 1.0);
}

//Pixel Shader
float4 PS1(float4 pos : SV_Position, float2 texcoord : TEXCOORD) : SV_Target
{
    //if (texcoord.x > 0.5) { return 0.0.xxxx; }
    if (texcoord.x > 0.5) { discard; }
    return 0.5;
}

float4 PS2(float4 pos : SV_Position, float2 texcoord : TEXCOORD) : SV_Target
{
    return tex2Dlod(Sampler1, float4(texcoord, 0, 0));
}

float4 PS3(float4 pos : SV_Position, float2 texcoord : TEXCOORD) : SV_Target
{
    return 1.0;
}

float4 PS4(float4 pos : SV_Position, float2 texcoord : TEXCOORD) : SV_Target
{
    return tex2Dlod(Sampler2, float4(texcoord, 0, 0));
}

//Technique
technique Effects < enabled = 1;  >
{
    pass 
    {
        VertexShader = VS;
        PixelShader = PS1;
        RenderTarget = tex1;
    }
    pass 
    {
        VertexShader = VS;
        PixelShader = PS2;
        RenderTarget = tex2;
    }
    //This pass shouldn't affect the final image at all, but it does
    pass 
    {
        VertexShader = VS;
        PixelShader = PS3;
        RenderTarget = tex1;
    }
    pass
    {
        VertexShader = VS;
        PixelShader = PS4;
    }
}

Expected Result: Result:

Pass 3 samples tex1 which gets white.
Pass 4 samples tex2 which should produce the expected image (left grey, right black), but somehow the right side becomes white from pass 3.
Last edit: 4 years 11 months ago by brussell.

Please Log in or Create an account to join the conversation.

  • crosire
More
4 years 11 months ago - 4 years 11 months ago #7 by crosire Replied by crosire on topic Strange problem regarding discard with ReShade 4.5
Aha! Nope, this does exactly what it is supposed to do =). You just have to think in multiple frames:

Frame 1:
> tex1 is completly black.
> PS1 only writes to the left part of the texture and does not touch the right part (because it is discarded).
> tex1 is now half grey, half black.
> PS3 clears the entire texture to 1.
> tex1 is now completly white.

Frame 2:
> tex1 is completly white (see end of last frame)
> PS1 only writes to the left part of the texture ...
> tex1 is now half grey, half white (since the right side was not touched)
...

If you want it to stay black on the right you need to tell ReShade to clear the texture before you start a new pass. This has been the default before, but as of 4.5 the default has changed to "false" for better performance (since such a clear operation is not free):
technique Effects < enabled = 1;  >
{
    pass 
    {
        VertexShader = VS;
        PixelShader = PS1;
        RenderTarget = tex1;
        ClearRenderTargets = true; 
    }
    ...
}
Last edit: 4 years 11 months ago by crosire.
The following user(s) said Thank You: brussell

Please Log in or Create an account to join the conversation.

  • brussell
  • Topic Author
More
4 years 11 months ago - 4 years 11 months ago #8 by brussell Replied by brussell on topic Strange problem regarding discard with ReShade 4.5
Ahh, makes perfect sense now that I think about it! The change of default behavior just made it look like a bug.
Thanks for the explanation.
Last edit: 4 years 11 months ago by brussell.

Please Log in or Create an account to join the conversation.

We use cookies
We use cookies on our website. Some of them are essential for the operation of the forum. You can decide for yourself whether you want to allow cookies or not. Please note that if you reject them, you may not be able to use all the functionalities of the site.