Welcome, Guest.
Username: Password: Remember me

TOPIC: anyway to fix upside down depth buffer

anyway to fix upside down depth buffer 1 year 6 months ago #1

  • Kleio420
  • Kleio420's Avatar
  • Offline
  • Posts: 213
  • Thank you received: 28
I have a game that the buffer is flipped upside down is there a way to flip it through a shader or is this reshade specific ?
The administrator has disabled public write access.

anyway to fix upside down depth buffer 1 year 6 months ago #2

  • crosire
  • crosire's Avatar
  • Offline
  • Posts: 2437
  • Thank you received: 1384
Yeah, just flip the texture coordinates before sampling from it. Shader thing, not related to ReShade itself =).
Cheers, crosire =)
The administrator has disabled public write access.

anyway to fix upside down depth buffer 1 year 6 months ago #3

  • Kleio420
  • Kleio420's Avatar
  • Offline
  • Posts: 213
  • Thank you received: 28
crosire wrote:
Yeah, just flip the texture coordinates before sampling from it. Shader thing, not related to ReShade itself =).
what file is this located in i would like to try and do it myself before asking if anyone could change it for me
The administrator has disabled public write access.

anyway to fix upside down depth buffer 1 year 6 months ago #4

I would very much like to know that myself. Also, which are exactly the values to be flipped? I tried fooling around (monkey with a typewriter!) with all texcoord values I found in Util.h in the Common folder, but all I got was black screens. Or half-black screens, which was interesting :)
Last Edit: 1 year 6 months ago by SpinelessJelly.
The administrator has disabled public write access.

anyway to fix upside down depth buffer 1 year 6 months ago #5

  • crosire
  • crosire's Avatar
  • Offline
  • Posts: 2437
  • Thank you received: 1384
I don't know all the files, it's used multiple times across different source files. To flip you would want to invert the Y texture coordinate:
tex2D(depthtex, float2(texcoord.x, 1.0 - texcoord.y))
Cheers, crosire =)
Last Edit: 1 year 6 months ago by crosire.
The administrator has disabled public write access.
The following user(s) said Thank You: Kleio420, SpinelessJelly

anyway to fix upside down depth buffer 1 year 6 months ago #6

  • Kleio420
  • Kleio420's Avatar
  • Offline
  • Posts: 213
  • Thank you received: 28
havent been able to find anything like that if anyone who reads this doesnt mind helping pointing me to the right function
Last Edit: 1 year 6 months ago by Kleio420.
The administrator has disabled public write access.

anyway to fix upside down depth buffer 1 year 6 months ago #7

Soooo.... I think it worked? I tried this on Ziggurat (Unity engine, opengl32.dll, reshade 0.18.4). Fixed Cel shader by making ...\ReShade\CustomFX\Cel.h look like this
...
float3 normals(float2 texcoord) // get normal vector from depthmap
{
	float	deltax = linearlizeDepth(tex2D(RFX_depthColor, float2((texcoord.x + px), 1.0 - texcoord.y)).x) - linearlizeDepth(tex2D(RFX_depthColor, float2((texcoord.x - px), 1.0 - texcoord.y)).x),
		deltay = linearlizeDepth(tex2D(RFX_depthColor, float2(texcoord.x, ((1.0 - texcoord.y) + py))).x) - linearlizeDepth(tex2D(RFX_depthColor, float2(texcoord.x, ((1.0 - texcoord.y) - py))).x);	
...

Also Matso DOF seems to work correctly by changing ...\ReShade\Common\Util.h to
...
// Depth buffer access
	float depth = tex2D(RFX_depthColor, float2(texcoord.x, 1.0 - texcoord.y)).x;
...

Both effects shine through the weapon and HUD but I guess that's the game's problem, not Reshade's. Still haven't managed to flip AO but the indication looks valid, thanks again Crosire.
Last Edit: 1 year 6 months ago by SpinelessJelly.
The administrator has disabled public write access.
The following user(s) said Thank You: Kleio420

anyway to fix upside down depth buffer 1 year 6 months ago #8

  • Kleio420
  • Kleio420's Avatar
  • Offline
  • Posts: 213
  • Thank you received: 28
SpinelessJelly wrote:
Soooo.... I think it worked? I tried this on Ziggurat (Unity engine, opengl32.dll, reshade 0.18.4). Fixed Cel shader by making ...\ReShade\CustomFX\Cel.h look like this
...
float3 normals(float2 texcoord) // get normal vector from depthmap
{
	float	deltax = linearlizeDepth(tex2D(RFX_depthColor, float2((texcoord.x + px), 1.0 - texcoord.y)).x) - linearlizeDepth(tex2D(RFX_depthColor, float2((texcoord.x - px), 1.0 - texcoord.y)).x),
		deltay = linearlizeDepth(tex2D(RFX_depthColor, float2(texcoord.x, ((1.0 - texcoord.y) + py))).x) - linearlizeDepth(tex2D(RFX_depthColor, float2(texcoord.x, ((1.0 - texcoord.y) - py))).x);	
...

Also Matso DOF seems to work correctly by changing ...\ReShade\Common\Util.h to
...
// Depth buffer access
	float depth = tex2D(RFX_depthColor, float2(texcoord.x, 1.0 - texcoord.y)).x;
...

Both effects shine through the weapon and HUD but I guess that's the game's problem, not Reshade's. Still haven't managed to flip AO but the indication looks valid, thanks again Crosire.
wow thanks this worked at least for the dof, just being able to see where this is applied is super helpful for getting better at understanding the flow of things ty guys very much

i take it for ssao this is what needs changed
float depth = tex2D(RFX_depthColor, texcoord.xy).x;

from looking at it , looks like the entire ssao.h file needs to be changed to work with this
Last Edit: 1 year 6 months ago by Kleio420.
The administrator has disabled public write access.

anyway to fix upside down depth buffer 1 year 6 months ago #9

Yes, I tried changing that line but nothing happened (not true, managed to get some error messages). But maybe I got the syntax wrong, I'm pretty clueless code-wise and logic can only get you that far.
The administrator has disabled public write access.

anyway to fix upside down depth buffer 1 year 6 months ago #10

  • Kleio420
  • Kleio420's Avatar
  • Offline
  • Posts: 213
  • Thank you received: 28
SpinelessJelly wrote:
Yes, I tried changing that line but nothing happened (not true, managed to get some error messages). But maybe I got the syntax wrong, I'm pretty clueless code-wise and logic can only get you that far.
well from my understanding float depth = (varible, float2(texcoord.xy).x; is how every ssao example ive found on the web uses to get the depth if you look at the rest the same layout is used for specific functions which is why i think it needs either a setting to enable a full on global buffer flip in one line of code or editing the entire file to work with it. Since this seems to be hugely a unity engine issue maybe we can get a fix in a framework release. By no means am i saying im correct in what i just typed as im probably wrong hopefully there is enough interest in this to put a fix for it
The administrator has disabled public write access.

anyway to fix upside down depth buffer 1 year 5 months ago #11

DisplayDepth.h

#if (RFX_DisplayDepth == 1)

NAMESPACE_ENTER(RFX) 

void DisplayDepth(in float4 position : SV_Position, in float2 texcoord : TEXCOORD0, out float3 color : SV_Target)
{
	color.rgb = tex2D(RFX_depthTexColor,texcoord).rgb;
	
}

technique Depth_Tech < bool enabled = false; int toggle = RFX_DepthToggleKey;>
{
	pass
	{
		VertexShader = RFX_VS_PostProcess;
		PixelShader = DisplayDepth;
	}
}


NAMESPACE_LEAVE()

#endif

Where do I input ?

tex2D(depthtex, float2(texcoord.x, 1.0 - texcoord.y))
The administrator has disabled public write access.

anyway to fix upside down depth buffer 1 year 5 months ago #12

  • crosire
  • crosire's Avatar
  • Offline
  • Posts: 2437
  • Thank you received: 1384
strelokgunslinger wrote:
...
color.rgb = tex2D(RFX_depthTexColor,texcoord).rgb;
...

Where do I input ?

tex2D(depthtex, float2(texcoord.x, 1.0 - texcoord.y))
There aren't many lines which look similar =P. Just repalce it there.
Cheers, crosire =)
The administrator has disabled public write access.

anyway to fix upside down depth buffer 1 year 3 months ago #13

  • biliskner
  • biliskner's Avatar
  • Offline
  • Posts: 3
  • Thank you received: 1
crosire wrote:
strelokgunslinger wrote:
...
color.rgb = tex2D(RFX_depthTexColor,texcoord).rgb;
...

Where do I input ?

tex2D(depthtex, float2(texcoord.x, 1.0 - texcoord.y))
There aren't many lines which look similar =P. Just repalce it there.

Thanks crosire, this helped me alot with fixing my Unity depth buffer inversion
I have zero programming knowledge so I spent the better part of the last 24hrs reading up about HLSL just so I could change my reshade.fx file to try and add a more permanent fix.
What i found was that changing the texcoord values manually for each shader effect was very time consuming and difficult to find the right line.. so my fix was to try and add a toggle in common.cfg

I then change RFX_PS_StoreDepth directly
Warning: Spoiler! [ Click to expand ]


after a few more hours of headache where i messed up my graphics -_- I realised DisplayDepth uses RFX_depthTexColor which is saved.. but CEL and SMAA use RFX_depthColor which i THINK? is read directly from the buffer

Long story short.. i added a few more lines of code to affect anything using RFX_depthTexColor or RFX_depthColor. I hope this helps someone, please feel free to correct me if my code is messed up.

Code additions/changes follow
Warning: Spoiler! [ Click to expand ]
Last Edit: 1 year 3 months ago by biliskner. Reason: made declaring RFX_fixdepthTex conditional
The administrator has disabled public write access.
The following user(s) said Thank You: SpinelessJelly

anyway to fix upside down depth buffer 1 year 3 months ago #14

  • crosire
  • crosire's Avatar
  • Offline
  • Posts: 2437
  • Thank you received: 1384
Code looks correct, so if you have both the RFX_PS_FixDepth and RFX_PS_StoreDepth changes it should work in all cases.

The current handling is a little complicated, too many different textures and samplers to keep track of. That should be simplified in the next versions.
Cheers, crosire =)
The administrator has disabled public write access.

anyway to fix upside down depth buffer 1 year 3 months ago #15

  • biliskner
  • biliskner's Avatar
  • Offline
  • Posts: 3
  • Thank you received: 1
Thanks so much for looking through my code crosire!
Yeah learning about textures and samplers was a massive headache.. keeping track of them was another heh
The administrator has disabled public write access.