Welcome, Guest.
Username: Password: Remember me

TOPIC: Ported Storybook Shader

Ported Storybook Shader 1 year 9 months ago #1

Not sure if this is the right place to put a WIP. It may work fine though.

Download

Questions:
What is the minimal, most vanilla way to make a one-pass shader? I got stuck on exactly what the vertex shader needed to do, and I had to define a multi-pass to get it to work.

Is there a proper way to sample from a .tga? I had to convert to .jpg else it looked way too yellow. Better yet, what is the suggested image format for sampling from an image?

What variable is for buffer width/height and what is for actual screen size width/height? The tutorial references buffer_width as screen width. I may be using terms wrong.
The administrator has disabled public write access.

Ported Storybook Shader 1 year 9 months ago #2

  • Nerd
  • Nerd's Avatar
  • Offline
  • Posts: 38
  • Thank you received: 5
Some images and videos of it. I'd guess .png is the ideal format since Master Effect comes with some .png's.


a.pomf.se/xxpayd.webm
a.pomf.se/kmovxu.webm
a.pomf.se/rdcjus.webm
a.pomf.se/aqnyyn.webm
The administrator has disabled public write access.
The following user(s) said Thank You: One More Try, yawehoo

Ported Storybook Shader 1 year 9 months ago #3

  • crosire
  • crosire's Avatar
  • Offline
  • Posts: 2453
  • Thank you received: 1393
One More Try wrote:
What is the minimal, most vanilla way to make a one-pass shader? I got stuck on exactly what the vertex shader needed to do, and I had to define a multi-pass to get it to work.

Is there a proper way to sample from a .tga? I had to convert to .jpg else it looked way too yellow. Better yet, what is the suggested image format for sampling from an image?

What variable is for buffer width/height and what is for actual screen size width/height? The tutorial references buffer_width as screen width. I may be using terms wrong.

The following is the most simple possible shader doing nothing but returning the input color:
Warning: Spoiler! [ Click to expand ]


There may be a bug in the TGA reader, which gets colors wrong. I'd use either BMP (stores image data the most simple way) or PNG (lossless too, but smaller, because compressable).

"BUFFER_WIDTH" and "BUFFER_HEIGHT" are two macros expanding to the backbuffer width and height, which equals the screen dimensions or whatever resolution is set in the game options if in fullscreen or the window size if in window mode.
Cheers, crosire =)
The administrator has disabled public write access.
The following user(s) said Thank You: One More Try

Ported Storybook Shader 1 year 9 months ago #4

Thanks a ton! That simplest example really helps me get a hold on things.

I don't want to spam you with questions, so I'll keep it down to just two more:
Buffer_width leads us to pixel size, so what gives us texel size? How do I calculate things off an internal rendering resolution (when it's not equal to screen size). I saw tex2dsize, wasn't sure about that - couldn't get it to work or find an example of using it. Poking around, it looks like there might not be a way to pull that information...?

Second. When there are black bars because the game's resolution doesn't equal the window/screen size, Reshade seems to treat it like part of the image it's modifying and fills it in. Seeing this with Dolphin. Are there any easy ways to correct this or would I have to make a pass to detect these areas and then correct the variables?
The administrator has disabled public write access.

Ported Storybook Shader 1 year 9 months ago #5

  • crosire
  • crosire's Avatar
  • Offline
  • Posts: 2453
  • Thank you received: 1393
Don't worry about too many questions. I'm here to answer, so just ask what you want to know =).

The backbuffer is the internal rendering target, so its resolution is what you get with those two macros. You're right about derivering the pixel size from there (it's the reciprocal): ReShade thus also defines "BUFFER_RCP_WIDTH" and "BUFFER_RCP_HEIGHT" to "1 / BUFFER_WIDTH" and "1 / BUFFER_HEIGHT" for convenience.
Which texel size do you want? The one of the backbuffer equals the pixel size, the one for other textures can be calculated manually, as their dimensions are known too (since required for creation).

Dolphin renders to the full window resolution, including the black bars, which is why there is no way for ReShade to get the size of just the image without them.
You could check the color value for pitch black and discard rendering if that's the case, though that would have some problems if the game is very dark.
Cheers, crosire =)
The administrator has disabled public write access.

Ported Storybook Shader 1 year 9 months ago #6

Ok, I think I understand. Dolphin let's you render at a huge resolution (->texel) then shrinks the image to screensize(->pixel). When you apply shaders in dolphin, I think you do it to the large resolution, before it gets shrunk to the screen. It sounds like Reshade doesn't hook that larger internal rendering size, but rather after it is shrunk to regular screen size. Is that right?

Yeah. I was thinking you could edge detect the black borders and get what coordinates they start at. If that's possible then maybe you could modify everything to keep them black, but more importantly not include them in how it samples (like sampling at a widescreen res can stretch my noise.img)? The more I think about it, the more complicated it sounds. There may be simpler workarounds.
The administrator has disabled public write access.

Ported Storybook Shader 1 year 9 months ago #7

  • crosire
  • crosire's Avatar
  • Offline
  • Posts: 2453
  • Thank you received: 1393
Oh, that explains our misunderstanding. Yeah, ReShade renders at the end of a frame, after the game/Dolphin finished, so it doesn't know about the internal ofscreen rendering.

Here's an idea for black border detection.
texture colorTex : COLOR;
sampler colorSamp { Texture = colorTex; };

// x = left or image without black borders, y = top, z = right, w = bottom
float4 CalculateViewport()
{
	float4 viewport = float4(0, 0, 1, 1);
	
	// Look through the pixels on x and y axis until one that is not black is found.
	while (dot(tex2D(colorSamp, float2(viewport.x, 0.5))) > 0 && viewport.x < 1)
	{
		viewport.x += BUFFER_RCP_WIDTH;
	}
	while (dot(tex2D(colorSamp, float2(viewport.y, 0.5))) > 0 && viewport.y < 1)
	{
		viewport.y += BUFFER_RCP_HEIGHT;
	}

	// Let's asume the black bars are mirrored across the screen
	viewport.z = 1 - viewport.x;
	viewport.w = 1 - viewport.y;

	// Return our gathered information. It's a rectangle containing the position and size of our image without black borders.
	return viewport;
}
Running this on each pixel is probably inefficient and takes away performance, so instead you could calculate it only once and store the result in a texture to use later:
texture viewportTex { Width = 1; Height = 1; Format = RGBA32F; };
sampler viewportSamp { Texture = viewportTex; MinFilter = POINT; MagFilter = POINT; };

float4 CalculateViewport(in float4 pos : SV_Position) : SV_Target
{
	...
	return viewport;
}

float4 SomeOtherShader(in float4 pos : SV_Position, in float2 texcoord : TEXCOORD) : SV_Target
{
	// Retrieve our viewport information from the texture we rendered it to below.
	float4 viewport = tex2D(viewportSamp, float2(0, 0));

	// Lets just use it to discard/abort rendering in the black borders and only do stuff on the actual image.
	if (texcoord.x < viewport.x || texcoord.y < viewport.y || texcoord.x > viewport.z || texcoord.y > viewport.w)
	{
		discard;
	}
	
	float4 color;

	...

	return color;
}

technique MyTechnique
{
	pass
	{
		// Render the result of the CalculateViewport function into a texture ("viewportTex") only once (at start of post processing).
		VertexShader = ...;
		PixelShader = CalculateViewport;
		RenderTarget = viewportTex;
	}
	pass
	{
		// Now we can execute other shaders and use the result stored inside "viewportTex".
		VertexShader = ...;
		PixelShader = SomeOtherShader;
	}
}
Cheers, crosire =)
Last Edit: 1 year 9 months ago by crosire.
The administrator has disabled public write access.
The following user(s) said Thank You: One More Try

Ported Storybook Shader 1 year 9 months ago #8

  • Nerd
  • Nerd's Avatar
  • Offline
  • Posts: 38
  • Thank you received: 5
I read your post on the Dolphin forums but I feel like Reshade discussions should be here so please forgive my cross-posting :lol:

This little tool is amazing! It's much better than the Dolphin shader tweaker, the reset key is a godsend.

Some suggestions I have(mainly because I'm too dense to figure it out) would be to make the shader it's own .fx so I can hook it into mastereffect's reshade.fx and for a toggle key(the intensity slider works for this actually). It seems like when I rename your shader and include it into reshade.fx the game becomes unstable with odd crashes. Maybe I'm doing something wrong though.

What kind of multi-pass shaders are you thinking about?
Last Edit: 1 year 9 months ago by Nerd.
The administrator has disabled public write access.

Ported Storybook Shader 1 year 9 months ago #9

So, I totally missed the page 2 indicator and thought there were no more replies ><

Thanks so much Crosire! I came up with the same idea to run it once, run only until it hits a color, and also in the middle of the y-axis where it's less likely to have any issues (if we assume the borders are only on the left and right). I was floundering in actual implementation, so you've really helped me there. I can practice multi pass shading with that.

/edit Wow!! Looking it over in more detail you really did all the work for me. Glad I attempted it myself too. I can compare to better understand how to get to where I wanted to be. One question. The inserted, sampled .png gets stretched when there are borders. I don't think this fixes it. At what point would I connect it to the viewport dimensions so it gets scaled properly?

@Nerd (He's talking about my program that uses sliders to edit my .fx file).
Thanks, I'm glad it's useful! I'll get around to posting a new version with that program at some point. I can also add a toggle on/off.

I have no idea how to add it to mastereffect, but I plan on figuring that out at some point. By what method are you hooking it?

Other than the border fix, not sure on other multipass. There are other single-pass things I found to play with and maybe cross add to dolphin.

//Edit 2:
" Render the result of the CalculateViewport function into a texture ("viewportTex") only once (at start of post processing)."
I know this has to happen so it doesn't lag everything. I don't see any code in your example, nor tutorial file to make this happen. Right now it runs continually as far as I can tell. Another issue is that startup can be very black, so offsetting this one pass to a later point could make it more accurate (I could always make the user reload the shader if they need it). Other than that, I have it working well.
Last Edit: 1 year 9 months ago by One More Try.
The administrator has disabled public write access.

Ported Storybook Shader 1 year 9 months ago #10

  • Nerd
  • Nerd's Avatar
  • Offline
  • Posts: 38
  • Thank you received: 5
A toggle bind would be nice instead of a button. I hook it in by adding " #include "renamed.fx" " at the bottom since that's what works for Sweet.fx or the Guass.fx found on these forums. It doesn't seem to work well for this though.
The administrator has disabled public write access.

Ported Storybook Shader 1 year 9 months ago #11

  • crosire
  • crosire's Avatar
  • Offline
  • Posts: 2453
  • Thank you received: 1393
One More Try wrote:
One question. The inserted, sampled .png gets stretched when there are borders. I don't think this fixes it. At what point would I connect it to the viewport dimensions so it gets scaled properly?
Just scale the texture coordinates by those viewport dimensions before sampling:
tex2D(MySampler, texcoord * viewport);

One More Try wrote:
I have no idea how to add it to mastereffect, but I plan on figuring that out at some point.
ReShade supports rendering multiple techniques, so just include this file into the MasterEffect shader, or the other way round. It will work. Not the best possible performance though.

One More Try wrote:
I know this has to happen so it doesn't lag everything. I don't see any code in your example, nor tutorial file to make this happen. Right now it runs continually as far as I can tell. Another issue is that startup can be very black, so offsetting this one pass to a later point could make it more accurate (I could always make the user reload the shader if they need it). Other than that, I have it working well.
With "only once" I meant "only once per frame", instead of per pixel. You could also run it only once globally:
// This technique is only executed during the first 2000 milliseconds it gets enabled, which in this case means 2000 milliseconds after game start, because "enabled" is set to true. It is toggled off afterwards. But the last result written into "viewportTex" is still available and you can continue to use it.
technique RunMeOnlyOnce < enabled = true; timeout = 2000; >
{
	pass
	{
		VertexShader = ...;
		PixelShader = CalculateViewport;
		RenderTarget = viewportTex;
	}
}

// This technique is executed every frame
technique MyTechnique < enabled = true; >
{
	pass
	{
		VertexShader = ...;
		PixelShader = SomeOtherShader;
	}
}
Cheers, crosire =)
Last Edit: 1 year 9 months ago by crosire.
The administrator has disabled public write access.

Ported Storybook Shader 11 months 3 weeks ago #12

  • Nerd
  • Nerd's Avatar
  • Offline
  • Posts: 38
  • Thank you received: 5
How would I go about adding this to the new framework so I can use and toggle it just like any other shader?
The administrator has disabled public write access.

Ported Storybook Shader 11 months 3 weeks ago #13

  • Ganossa
  • Ganossa's Avatar
  • Offline
  • 최정장군
  • Posts: 790
  • Thank you received: 838
For now, I would suggest to add your shader to the CustomFX suite.

Add your resources (shader files and textures) to the CustomFX suite folder.

Add an entry for your shader in the Pipeline.cfg such as "#include EFFECT (CustomFX, YouShaderFileName)".

Add a shader entry to the CustomFX.cfg (use the custom entry in that file as a reference). Make sure to have the "#define USE_YOURSHADERNAME 0 //[YourShaderFileName]" line.

Adjust your shader file content to fit the format of the Custom.fx file which can be found in ReShade/CustomFX. So make sure all the USE, #includes, toggles and similar are set.

Please submit your proposal to our github repository and it will very likely be in the next release :)

Does that help or do you need any more information?
The administrator has disabled public write access.

Ported Storybook Shader 11 months 3 weeks ago #14

  • Nerd
  • Nerd's Avatar
  • Offline
  • Posts: 38
  • Thank you received: 5
I definitely need more information. I tried following your instructions but I'm not sure what to put into the new Storyook.fx based on Custom.fx or what I should put in CustomFX.cfg to configure the settings. I'm no wizard!
The administrator has disabled public write access.

Ported Storybook Shader 11 months 3 weeks ago #15

  • Ganossa
  • Ganossa's Avatar
  • Offline
  • 최정장군
  • Posts: 790
  • Thank you received: 838
We are currently changing the framework layout to provide more author transparency. The previous suite/domain idea will be replaced with author "name" spaces. You can find the current progress on GitHub.

The Mediator was changed accordingly and will be named ReShade Assistant with the next release:
Author Transparency

Due to those changes, it should might become easier for you to add new shader to the framework.
In the GitHub changes, you can simply make a copy of YourName.cfg, YourName.undef, YourName folder and the "#include EFFECT(YourName, YourShaderName)" entry in pipeline.cfg.
Replace YourName/YourShader name in those files with your name and your shader name.
After that, you can simply copy your relevant shader content into the YourShaderName.fx file and place your relevant shader variables in YourShader.cfg.
:)
The administrator has disabled public write access.
The following user(s) said Thank You: Elimina

Ported Storybook Shader 11 months 3 weeks ago #16

  • Sh1nRa358
  • Sh1nRa358's Avatar
  • Offline
  • Posts: 113
  • Thank you received: 9
Hey, fix the stretch shader b4 next release please. Make it use percentage instead of actual pixels. as of now, you have to redo your values whenever you switch to a different resolution in order for it to work.
The administrator has disabled public write access.

Ported Storybook Shader 1 month 4 weeks ago #17

  • Nerd
  • Nerd's Avatar
  • Offline
  • Posts: 38
  • Thank you received: 5
This shader works in 3.0 but I don't know how to add sliders to control the settings. Can anyone help?
The administrator has disabled public write access.