- Posts: 351
[SOLVED] Bugs in generated HLSL code (2.0.3f1)
- OtisInf
-
Topic Author
- Offline
Less More
6 years 1 month ago - 6 years 1 month ago #1 by OtisInf
[SOLVED] Bugs in generated HLSL code (2.0.3f1) was created by OtisInf
Hi,
I think I ran into several issues.
First:
gives on the tex2Dfetch line:
error X3017: '__tex2Dfetch': cannot implicitly convert from 'const int2' to 'int4'
Using tex2D() works (with float coords)
Then:
What I want is this:So I want to NOT write to the render target if the key isn't ON, so discard the pixel. However discard here doesn't discard the pixel, it will write black (0,0,0,0) to the texture as if it's re-initialized each time.
So I thought, maybe it's me being stupid, so I'll just use clip(-1); but when I use clip(-1) like:
I get:not sure, but it looks like 'clip' isn't a known function? I'm using Dx11, on a 780, which IMHO is way inside the required shader model.
(edit) I see 'clip' isn't in the list of known intrinsic functions for reshade, is this because there's no equivalent under OpenGL? I can live without using clip, I just want to discard the complete rendering of the pixel so it will keep the previous value written to the texture. I recon this isn't possible?
The whole idea is that in a pixel shader pass ran after the pass specified above I read from the fog color sampler, which contains the last color on the mouse coordinates when the toggle key was pressed. This way I can pick a color from the scene and keep it after moving the mouse. However when I toggle the key off again, I go back to black, so it overwrites the pixel stored with a value, even though discard is used.
I think I ran into several issues.
First:
texture Otis_FogColorFromMouseTarget { Width=1; Height=1; Format= Otis_RENDERMODE;};
sampler2D Otis_FogColorFromMouseSampler { Texture = Otis_FogColorFromMouseTarget;};
//...
float4 fogColor = tex2Dfetch(Otis_FogColorFromMouseSampler, int2(0, 0));
gives on the tex2Dfetch line:
error X3017: '__tex2Dfetch': cannot implicitly convert from 'const int2' to 'int4'
Using tex2D() works (with float coords)
Then:
What I want is this:
float4 PS_Otis_AFG_PerformGetFogColorFromFrameBuffer(float4 vpos: SV_Position, float2 texcoord: TEXCOORD) : SV_Target
{
if(Otis_MouseToggleKeyDown)
{
return tex2D(ReShade::BackBuffer, ReShade::MouseCoords * ReShade::PixelSize);
}
else
{
discard;
return float4(AFG_Color, 1.0); // should be ignored anyway.
}
}
// with:
pass Otis_AFG_PassFogColorFromMouse
{
VertexShader = ReShade::VS_PostProcess;
PixelShader = PS_Otis_AFG_PerformGetFogColorFromFrameBuffer;
RenderTarget = Otis_FogColorFromMouseTarget;
}
So I thought, maybe it's me being stupid, so I'll just use clip(-1); but when I use clip(-1) like:
clip(-1); // line 80
I get:
AdaptiveFog.fx(80, 1): error X3004: undeclared identifier 'clip'
AdaptiveFog.fx(80, 9): error X3000: syntax error: unexpected ';'
(edit) I see 'clip' isn't in the list of known intrinsic functions for reshade, is this because there's no equivalent under OpenGL? I can live without using clip, I just want to discard the complete rendering of the pixel so it will keep the previous value written to the texture. I recon this isn't possible?
The whole idea is that in a pixel shader pass ran after the pass specified above I read from the fog color sampler, which contains the last color on the mouse coordinates when the toggle key was pressed. This way I can pick a color from the scene and keep it after moving the mouse. However when I toggle the key off again, I go back to black, so it overwrites the pixel stored with a value, even though discard is used.
Last edit: 6 years 1 month ago by OtisInf.
Please Log in or Create an account to join the conversation.
- crosire
-
- Away
Less More
- Posts: 3996
6 years 1 month ago - 6 years 1 month ago #2 by crosire
Replied by crosire on topic Bugs in generated HLSL code (2.0.3f1)
gives on the tex2Dfetch line:That's a known issue and fixed already. I had a parameter mismatch of tex2Dfetch between the parser and HLSL code generator. Should be "tex2Dfetch(sampler2D, int4)" (where .w is the mipmap level), but it's "tex2Dfetch(sampler2D, int2)" in 2.0.3.
error X3017: '__tex2Dfetch': cannot implicitly convert from 'const int2' to 'int4'
So I want to NOT write to the render target if the key isn't ON, so discard the pixel. However discard here doesn't discard the pixel, it will write black (0,0,0,0) to the texture as if it's re-initialized each time.That's because each rendertarget is cleared to zero before it is bound. This was a requirement for SMAA to work.
not sure, but it looks like 'clip' isn't a known function? I'm using Dx11, on a 780, which IMHO is way inside the required shader model.ReShade FX has no "clip", because the exact same behavior can be achieved with a conditional discard, which is much clearer (it's obvious the shader code won't continue after a discard statement, not so much after a function call). Keep in mind, you don't write HLSL, even though it looks similar.
(edit) I see 'clip' isn't in the list of known intrinsic functions for reshade, is this because there's no equivalent under OpenGL? I can live without using clip, I just want to discard the complete rendering of the pixel so it will keep the previous value written to the texture. I recon this isn't possible?
Last edit: 6 years 1 month ago by crosire.
Please Log in or Create an account to join the conversation.
- OtisInf
-
Topic Author
- Offline
Less More
- Posts: 351
6 years 1 month ago #3 by OtisInf
I obviously miss something like a flag to set somewhere to make it not being cleared or something else. Any advice would be great 
Replied by OtisInf on topic Bugs in generated HLSL code (2.0.3f1)
Ok thanks, will work around this.gives on the tex2Dfetch line:That's a known issue and fixed already. I had a parameter mismatch of tex2Dfetch between the parser and HLSL code generator. Should be "tex2Dfetch(sampler2D, int4)" (where .w is the mipmap level), but it's "tex2Dfetch(sampler2D, int2)" in 2.0.3.
error X3017: '__tex2Dfetch': cannot implicitly convert from 'const int2' to 'int4'
I feared as much. However it's then a blur to me how I can preserve a value across frames. In another thread you gave the advice to use a 1x1 texture for this. However if that's cleared before it's bound, I don't know what else to use as the clearing before bound makes it lose the values I wanted to preserve across framesSo I want to NOT write to the render target if the key isn't ON, so discard the pixel. However discard here doesn't discard the pixel, it will write black (0,0,0,0) to the texture as if it's re-initialized each time.That's because each rendertarget is cleared to zero before it is bound. This was a requirement for SMAA to work.


Ah thanks for the explanation. Now if I can get the value to be preserved it would be great...not sure, but it looks like 'clip' isn't a known function? I'm using Dx11, on a 780, which IMHO is way inside the required shader model.ReShade FX has no "clip", because the exact same behavior can be achieved with a conditional discard, which is much clearer (it's obvious the shader code won't continue after a discard statement, not so much after a function call). Keep in mind, you don't write HLSL, even though it looks similar.
(edit) I see 'clip' isn't in the list of known intrinsic functions for reshade, is this because there's no equivalent under OpenGL? I can live without using clip, I just want to discard the complete rendering of the pixel so it will keep the previous value written to the texture. I recon this isn't possible?
Please Log in or Create an account to join the conversation.
- crosire
-
- Away
Less More
- Posts: 3996
6 years 1 month ago - 6 years 1 month ago #4 by crosire
Replied by crosire on topic Bugs in generated HLSL code (2.0.3f1)
I feared as much. However it's then a blur to me how I can preserve a value across frames. In another thread you gave the advice to use a 1x1 texture for this. However if that's cleared before it's bound, I don't know what else to use as the clearing before bound makes it lose the values I wanted to preserve across framesThe ugly workaround solution would be to create a second "backup" texture. So:I obviously miss something like a flag to set somewhere to make it not being cleared or something else. Any advice would be great
- pass 1 [RenderTarget = texture 2]: copy texture 1 to texture 2
- pass 2 [RenderTarget = texture 1]: instead of discard, read the data you want to preserve from texture 2
Last edit: 6 years 1 month ago by crosire.
The following user(s) said Thank You: OtisInf
Please Log in or Create an account to join the conversation.
- OtisInf
-
Topic Author
- Offline
Less More
- Posts: 351
6 years 1 month ago #5 by OtisInf
Heh thanks
Will try it out.
Replied by OtisInf on topic Bugs in generated HLSL code (2.0.3f1)
I feared as much. However it's then a blur to me how I can preserve a value across frames. In another thread you gave the advice to use a 1x1 texture for this. However if that's cleared before it's bound, I don't know what else to use as the clearing before bound makes it lose the values I wanted to preserve across framesThe ugly workaround solution would be to create a second "backup" texture. So:I obviously miss something like a flag to set somewhere to make it not being cleared or something else. Any advice would be great
- pass 1 [RenderTarget = texture 2]: copy texture 1 to texture 2
- pass 2 [RenderTarget = texture 1]: instead of discard, read the data you want to preserve from texture 2
Heh thanks

Please Log in or Create an account to join the conversation.
- OtisInf
-
Topic Author
- Offline
Less More
- Posts: 351
6 years 1 month ago #6 by OtisInf
Replied by OtisInf on topic [SOLVED] Bugs in generated HLSL code (2.0.3f1)
Works like a charm! 
Ok, so to recap this thread: problem 1 is already solved in a new build, problem 2 has a workaround, problem 3 (clip) is not supported. Solved

Ok, so to recap this thread: problem 1 is already solved in a new build, problem 2 has a workaround, problem 3 (clip) is not supported. Solved

Please Log in or Create an account to join the conversation.