Access to previous frames

  • aufkrawall
  • Topic Author
More
9 years 3 months ago - 9 years 3 months ago #1 by aufkrawall Access to previous frames was created by aufkrawall
Hello,
once ReShade supports access to previous frames, would it be possible to implement SMAA 2xT in every game?
Would it work even with alternate frame rendering?
Can the same quality be achieved, compared to good solutions like in Ryse or CoD: AW?

I think it won't hurt to make this clear publicly, since the question already has arisen in another forum (3DC). :)
Thanks!
Last edit: 9 years 3 months ago by aufkrawall.

Please Log in or Create an account to join the conversation.

  • crosire
More
9 years 3 months ago #2 by crosire Replied by crosire on topic Access to previous frames
That's the plan. How well that works out? I don't know yet =)
The following user(s) said Thank You: aufkrawall

Please Log in or Create an account to join the conversation.

  • aufkrawall
  • Topic Author
More
9 years 3 months ago #3 by aufkrawall Replied by aufkrawall on topic Access to previous frames
Ok, wait and see.
Oh, this tension.. :silly:

Please Log in or Create an account to join the conversation.

  • crosire
More
9 years 3 months ago - 9 years 2 months ago #4 by crosire Replied by crosire on topic Access to previous frames
I just noticed that there is actually no need to add an option to get the very last frame (looks different from frames beyond that, but that's yet another story).
It's possible to get access to the last frame by simply copying the current frame to a texture at the end and then using that texture the next time. Something like this:
texture2D currTex : COLOR;
texture2D prevTex { Width = BUFFER_WIDTH; Height = BUFFER_HEIGHT; };
sampler2D currColor { Texture = currTex; };
sampler2D prevColor { Texture = prevTex; };

float4 PS_PostProcess(float4 vpos : SV_Position, float2 texcoord : TEXCOORD) : SV_Target
{
	float4 curr = tex2D(currColor, texcoord);
	float4 prev = tex2D(prevColor, texcoord);
	
	....

	return curr;
}
float4 PS_CopyFrame(float4 vpos : SV_Position, float2 texcoord : TEXCOORD) : SV_Target
{
	return tex2D(currColor, texcoord);
}

technique MyTechnique
{
	pass DoPostProcessing
	{
		VertexShader = ...;
		PixelShader = PS_PostProcess;
	}
	pass DoCopyFrameForPrevAccess
	{
		VertexShader = ...;
		PixelShader = PS_CopyFrame;
		RenderTarget = prevTex;
	}
}
Last edit: 9 years 2 months ago by crosire.

Please Log in or Create an account to join the conversation.

  • CeeJay.dk
More
9 years 2 months ago #5 by CeeJay.dk Replied by CeeJay.dk on topic Access to previous frames
Instead of another pass to copy the frame, couldn't you have PS_PostProcess output to multiple render targets?
The following user(s) said Thank You: crosire, Nekrik

Please Log in or Create an account to join the conversation.

  • crosire
More
9 years 2 months ago #6 by crosire Replied by crosire on topic Access to previous frames
True. Even better, then also works with multi-pass effects:
texture2D currTex : COLOR;
texture2D prevTex { Width = BUFFER_WIDTH; Height = BUFFER_HEIGHT; };
sampler2D currColor { Texture = currTex; };
sampler2D prevColor { Texture = prevTex; };

void PS_PostProcess(float4 vpos : SV_Position, float2 texcoord : TEXCOORD, out float4 color : SV_Target0, out float4 prevOut : SV_Target1) 
{
	color = tex2D(currColor, texcoord);
	float4 prev = tex2D(prevColor, texcoord);
	prevOut = color;
	
	....
}

technique MyTechnique
{
	pass
	{
		VertexShader = ...;
		PixelShader = PS_PostProcess;
		RenderTarget1 = prevTex;
	}
	...
}

Please Log in or Create an account to join the conversation.

  • Deriest
More
9 years 2 months ago #7 by Deriest Replied by Deriest on topic Access to previous frames
Would it be possible to get more than just the previous frame with something like:
texture texColor;
texture texPrev1;
texture texPrev2;
texture texPrev3;
...

sampler samplerColor
{
Texture = texColor;
};

sampler samplerPrev01
{
Texture = texPrev01;
};

sampler samplerPrev02
{
Texture = texPrev02;
};

sampler samplerPrev03
{
Texture = texPrev03;
};
...

float4 PS_Main...



float4 PS_Previous03(in float4 pos : SV_Position, in float2 texcoord : TEXCOORD0) : SV_Target
{
float4 color = tex2D(samplerPrev02, texcoord.xy);
return color;
}

float4 PS_Previous02(in float4 pos : SV_Position, in float2 texcoord : TEXCOORD0) : SV_Target
{
float4 color = tex2D(samplerPrev01, texcoord.xy);
return color;
}

float4 PS_Previous01(in float4 pos : SV_Position, in float2 texcoord : TEXCOORD0) : SV_Target
{
float4 color = tex2D(samplerColor, texcoord.xy);
return color;
}

technique MyTechnique
{

        passMain
	{
		VertexShader = ...;
		PixelShader = Main;
	}

	passPrev03
	{
		VertexShader = ...;
		PixelShader = PS_Previous03;
		RenderTarget = texPrev03;
	}

	passPrev02
	{
		VertexShader = ...;
		PixelShader = PS_Previous02;
		RenderTarget = texPrev02;
	}

	passPrev01
	{
		VertexShader = ...;
		PixelShader = PS_Previous01;
		RenderTarget = texPrev01;
	}
}

Could we sample our previous frame texture to create a previous previous frame? If that makes sense?
The following user(s) said Thank You: crosire

Please Log in or Create an account to join the conversation.

  • crosire
More
9 years 2 months ago - 9 years 2 months ago #8 by crosire Replied by crosire on topic Access to previous frames
This copies the current frame again and again.

EDIT: Nevermind. My reading skills failed on me. =)
Last edit: 9 years 2 months ago by crosire.

Please Log in or Create an account to join the conversation.

  • CeeJay.dk
More
9 years 2 months ago #9 by CeeJay.dk Replied by CeeJay.dk on topic Access to previous frames

crosire wrote: This copies the current frame again and again. Access to frames older than the previous one needs to be implented in ReShade =)


Are you sure? .. It seems like this would work as long as the passes are executed in the order he wrote them.

I'd still use a single pass for performance reasons and have that pass output to SV_Target0 , SV_Target1 and SV_Target2

The current frame goes to SV_Target0 and to SV_Target1
and in the beginning of the pass you read from the previous image and then you output that to SV_Target2

Seems like you could make up to the limit of render targets of previous frames this way.
Though I don't think actually you need that if you want to do eye adaptation .. one previous frame should be enough.
The following user(s) said Thank You: Asmodean

Please Log in or Create an account to join the conversation.

  • crosire
More
9 years 2 months ago - 9 years 2 months ago #10 by crosire Replied by crosire on topic Access to previous frames
You're right. It's actually a brilliant idea. I misread/misunderstood it first, but now after rereading, it makes a lot of sense, so sorry for any confusion I caused there earlier. =)
Last edit: 9 years 2 months ago by crosire.

Please Log in or Create an account to join the conversation.

  • Deriest
More
9 years 2 months ago #11 by Deriest Replied by Deriest on topic Access to previous frames
I thought it made sense but I didn't doubt you too much because of my lack of experience with this.

Please Log in or Create an account to join the conversation.

  • crosire
More
9 years 2 months ago #12 by crosire Replied by crosire on topic Access to previous frames

CeeJay.dk wrote: I'd still use a single pass for performance reasons and have that pass output to SV_Target0 , SV_Target1 and SV_Target2

One thing to note here is that there are a few differences between the APIs on how many simultanous rendertargets they allow and that's one of the things ReShade can't change, so to enable full support for all, you should probably go for a maximum of 4 rendertargets (SV_Target0-3), even though ReShade in theory supports up to 8:

  • Direct3D9: D3DCAPS9.NumSimultaneousRTs = usually at least 4 on modern cards
  • Direct3D10/11: D3D11_SIMULTANEOUS_RENDER_TARGET_COUNT = 8
  • OpenGL: GL_MAX_DRAW_BUFFERS = usually the same as for Direct3D10/11

Please Log in or Create an account to join the conversation.

  • Ganossa
More
9 years 2 months ago #13 by Ganossa Replied by Ganossa on topic Access to previous frames
I tried this approach in a quick sample (since I will need that kind of stuff for porting GEMFX) but I think it either does not work yet or I miss the required knowledge on ReShades language extensions. Has anyone tried if it really works?

Here is the sample code::
texture2D currTex : COLOR;
texture2D prevTex : SV_TARGET0;

sampler2D currColor
{
	Texture = currTex;
	AddressU = CLAMP;
	AddressV = CLAMP;
	MinFilter = LINEAR;
	MagFilter = LINEAR;
	MipFilter = NONE;
};
sampler2D prevColor
{
	Texture = prevTex;
	AddressU = CLAMP;
	AddressV = CLAMP;
	MinFilter = LINEAR;
	MagFilter = LINEAR;
	MipFilter = NONE;
};

void VS_PostProcess(in uint id : VERTEXID, out float4 pos : POSITION, out float2 texcoord : TEXCOORD)
{
	texcoord.x = (id == 2) ? 2.0 : 0.0;
	texcoord.y = (id == 1) ? 2.0 : 0.0;
	pos = float4(texcoord * float2(2.0, -2.0) + float2(-1.0, 1.0), 0.0, 1.0);
}

float4 PS_Prev(float4 vpos : VPOS, float2 texcoord : TEXCOORD) : SV_TARGET0
{
	float4 curr = tex2D(currColor, texcoord);
	return curr;
}

float4 PS_PrevOut(float4 vpos : VPOS, float2 texcoord : TEXCOORD) : COLOR
{
	float4 prev = tex2D(prevColor, texcoord);
	return prev;
}
technique SamplePrev < bool enabled = true; >
{
	pass Prev
	{
		VertexShader = VS_PostProcess;
		PixelShader = PS_Prev;
		RenderTarget = prevTex;
	}

	pass PrevOut
	{
		VertexShader = VS_PostProcess;
		PixelShader = PS_PrevOut;
	}
}
Note that I tried re-ordering the passes and different SV_TARGETs already.

Even when I sample over the current frame and write/render it to another texture, this texture will remain black.

It would also be helpful to get a language extension that allows to copy resources, so I do not need to render the same thing on different textures.

Please Log in or Create an account to join the conversation.

  • crosire
More
9 years 2 months ago - 9 years 2 months ago #14 by crosire Replied by crosire on topic Access to previous frames
You're writing to "prevTex", but "prevTex" is defined to be source of the backbuffer image, same as "currTex" (COLOR and SV_TARGET0 are interchangable).
To create a texture you can actually write to (the above is read only), you have to define it like this (as an example, "BUFFER_WIDTH" and "BUFFER_HEIGHT" are defines resolving to the backbuffer size), so ReShade knows what to do with it:
texture2D prevTex
{
	Width = BUFFER_WIDTH;
	Height = BUFFER_HEIGHT;
	Format = RGBA8;
}
Last edit: 9 years 2 months ago by crosire.
The following user(s) said Thank You: Ganossa

Please Log in or Create an account to join the conversation.

  • Ganossa
More
9 years 2 months ago #15 by Ganossa Replied by Ganossa on topic Access to previous frames
Thanks, +; and it worked :) also thanks to the quick steam talk and hope to see all those nice new changes soon :side:
The following user(s) said Thank You: crosire

Please Log in or Create an account to join the conversation.

  • aufkrawall
  • Topic Author
More
9 years 2 months ago #16 by aufkrawall Replied by aufkrawall on topic Access to previous frames
I wonder if something like "temporal Lumasharpen" would be possible?
This could be useful to mitigate unwanted motion blur effects, like the extreme motion blur that comes with Far Cry 4 temporal SMAA or unwanted motion blur in general, e.g. if can't be turned off.

LuciferHawk, may I ask how you rate the possible effectiveness of temporal SMAA, injected via ReShade? Do you think it can be as good as if it was implemented by the game itself?

Please Log in or Create an account to join the conversation.

  • JPulowski
More
9 years 2 months ago - 9 years 2 months ago #17 by JPulowski Replied by JPulowski on topic Access to previous frames

aufkrawall wrote: This could be useful to mitigate unwanted motion blur effects, like the extreme motion blur that comes with Far Cry 4 temporal SMAA or unwanted motion blur in general, e.g. if can't be turned off.

I don't understand. What do you mean by "temporal" lumasharpen? And how could it de-motion blur the image exactly?

aufkrawall wrote: Do you think it can be as good as if it was implemented by the game itself?

Theoretically speaking, the answer is yes. Basic SMAA could be also good as native SMAA as well. And actually it is possible to do so currently in few games which support depth-buffer access, by choosing depth edge detection instead of color/luma edge detection. It is not perfect, but still it allows to by-pass the HUD. In the future ReShade will have an automated generic UI detection feature. GeDoSaTO currently has one, you just write the corresponding hash codes of UI textures/elements then it by-passes them, but it is all manual. ReShade will do everything, automatically. :)
Last edit: 9 years 2 months ago by JPulowski.

Please Log in or Create an account to join the conversation.

  • aufkrawall
  • Topic Author
More
9 years 2 months ago - 9 years 2 months ago #18 by aufkrawall Replied by aufkrawall on topic Access to previous frames

JPulowski wrote: I don't understand. What do you mean by "temporal" lumasharpen? And how could it de-motion blur the image exactly?

I have no idea, that's why I ask.
Maybe you could do primitive stuff like enhancing contrast of pixels or increase sharpening when motion in many screen areas is detected.
I don't know if it ever could be useful in terms of quality. It will probably never look close to no motion blur.
But who knows what shader gurus would say.

JPulowski wrote: Theoretically speaking, the answer is yes. Basic SMAA could be also good as native SMAA as well. And actually it is possible to do so currently in few games which support depth-buffer access, by choosing depth edge detection instead of color/luma edge detection. It is not perfect, but still it allows to by-pass the HUD. In the future ReShade will have an automated generic UI detection feature. GeDoSaTO currently has one, you just write the corresponding hash codes of UI textures/elements then it by-passes them, but it is all manual. ReShade will do everything, automatically. :)

With previous ReShade and SweetFX 2.0 versions, I had no success with depth detection of SMAA, even though depth buffer detection of ReShade was working fine.
There was simply no AA applied to any pixel. I will give it another try with newer versions as soon as I have my system working again.
Last edit: 9 years 2 months ago by aufkrawall.

Please Log in or Create an account to join the conversation.

  • Ganossa
More
9 years 2 months ago #19 by Ganossa Replied by Ganossa on topic Access to previous frames
We already discussed the possibility of temporal SMAA. We know the algorithm, sample implementations and the basic concept but in the end (without velocity values) temporal SMAA will only be as good as our best pseudo motion blur implementation.

Please Log in or Create an account to join the conversation.

  • aufkrawall
  • Topic Author
More
9 years 2 months ago #20 by aufkrawall Replied by aufkrawall on topic Access to previous frames
Ok, thanks for making that clear.

Please Log in or Create an account to join the conversation.

We use cookies
We use cookies on our website. Some of them are essential for the operation of the forum. You can decide for yourself whether you want to allow cookies or not. Please note that if you reject them, you may not be able to use all the functionalities of the site.