Providing game data to ReShade

I want to be able to provide game data to ReShade so that shaders can make use of it. I'm wondering about how I could go about doing that, and if there are any good practices I should follow, such as some way of indicating to shaders of the presence of such data.

Right now, any game render target could be provided to ReShade, as well as the main camera information (view, position, whatever), motion vectors, normals and specular. 

The lighting and weather information, game tonemapping/color correction parameters, and ENB variables can be provided, perhaps as constant buffers?

The world cubemap (the sky + lod) should be possible to provide. For another project I want to cache light source information, so that might be possible alongside the shadow map. I don't know if any of that's useful though.

Information about how best to provide these bits of data would be appreciated. These are also D3D11 resources (the render targets) although I guess I can just capture them via ReShade if they aren't compatible.

What actually are the render effect arguments? I don't really understand why there are two RTVs. Can they be given HDR buffers?
 

To provide data to effects, you can use the "set_[...]()" methods of the "reshade::api::effect_runtime" class ( crosire.github.io/reshade-docs/structres...effect__runtime.html ). Typically one would register a callback with the "reshade::addon_even::reshade_begin_effects" event (which is called right before ReShade renders effects) and in that callback use the provided effect runtime pointer to set/update any effect variables one wants. In your case you are AFAIK already choosing to overwrite when ReShade renders effects somewhere else during the frame by calling "reshade::api::effect_runtime::render_effects()", so you could instead set/update any effect variables right before calling that directly, instead of going the event callback route (since "reshade::addon_event::reshade_begin_effects" would just be called from within the "render_effects()" call anyway).


There are various ways to choose which effect variables should be updated, e.g. find those with a fixed name, or with a specific annotation on them, ...
For uniform variables for example, in ReShade it's common practise to have them ask for specific data via a "source" annotation (e.g. "uniform MyVarName < source = "fill_this_with_x";" in the ReShade FX effect code), so in an add-on you would then enumerate all uniform variables, filter them by those which have that annotation set to "fill_this_with_x" and update those with the appropriate data, like so (see also crosire.github.io/reshade-docs/structres...9781b8f0a3f2eac4209a ):Alternatively to do something like ENB, where variables are expected to have a fixed name, can either search for a specific one in a specific effect file via "reshade::api::effect_runtime::find_uniform_variable()" or update them all:

Textures are handled a bit different, since ReShade FX has explicit language syntax to specify that a texture should come from an external source (via "texture MyVarName : MY_TEXTURE_SEMANTIC;", where "MY_TEXTURE_SEMANTIC" describes what that texture should be filled with, similar to "COLOR" and "DEPTH" ReShade already provides). As such there is a special method "reshade::api::effect_runtime::update_texture_bindings()" to automatically update all texture variables with a specific semantic to a texture of your choosing (see also crosire.github.io/reshade-docs/structres...62e2e9d5d5090e495cca ):

Now as for "reshade::api::effect_runtime::render_effects()", the RTVs provided to that function are what ReShade will consider the "default backbuffer" while rendering. Meaning any passes that do not have a render target specified in ReShade FX effect code will end up in those RTVs (the former when the "SRGBWriteEnable" pass state is false, the latter when it is true), plus ReShade will automatically copy the data in those RTVs and make it available to textures with the "COLOR" semantic (like the commonly used ReShade::BackBuffer specified in ReShade.fxh). In case of D3D11 "reshade::api::resource_view" is equivalent to "ID3D11RenderTargetView" in this case, so if you already have the appropriate render target pointer, can reinterpret_cast them and pass directly. The format of these RTVs doesn't matter, they can be given HDR buffers, yes, ReShade will deal with that. The only thing ReShade expects is that these have the same width and height as the swapchain, otherwise effects may behave wrong (since "BUFFER_WIDTH" and "BUFFER_HEIGHT" in ReShade FX are not updated, see also crosire.github.io/reshade-docs/structres...5dacc2fdc84b1eed47c9 ).


In general I really encourage checking out crosire.github.io/reshade-docs/structres...effect__runtime.html to get a feel for all the methods available to communicate with the ReShade effect runtime.

Oh, and the presence of some specific data can be communicated to ReShade FX effects e.g. by setting a uniform bool value to true when available and false when not (like the Generic Depth add-on does to tell effects whether a depth buffer was found via uniform variables with a "bufready_depth" annotation: github.com/crosire/reshade/blob/79cb97bb...neric_depth.cpp#L341 ), or even set a special preprocessor definition and have all effects recompiled with that definition using "reshade::api::effect_runtime::set_preprocessor_definition" ( crosire.github.io/reshade-docs/structres...5e5dab3d21731f9dd572 ).

Note: "update_texture_bindings" and "set_preprocessor_definition" are expensive operations (the former synchronizes CPU/GPU, the later recompiles all effects), so do not call them every frame, but only when things change!!

Wow, this is extremely in-depth and helpful. Thank you so much.

Similarly to the annotations you referenced, would it be possible to have annotations which a shader can use to specify where it's ideally executed? It looks like to do so, I'd have to takeover the rendering of specific shaders. It would be cool if that for any game with "extended" support, shaders could declare where they would prefer to be run, such as before transparency. My understanding is that to render at unique points, users have to disable the shader in the main reshade window and enable it with custom UI (i.e. groups)

The problem is that ReShade is not really designed to be able to run subsets of effects at different times. The user interface presents techniques in a big list, with the notion that they will be looped through and executed from top to bottom.
So an add-on that wants to group effects into groups that can be rendered independently at different points would already make the ReShade user interface for effect selection moot, and would require an interface to configure of its own. At that point it's easier to just replace the effect rendering loop in ReShade entirely and ignore whatever is configured in the main ReShade window, hence why there is only an API to render techniques individually ("reshade::api::effect_runtime::render_technique()", to find out which techniques exist would use "reshade::api::effect_runtime::enumerate_techniques()"), which can be used to replace what ReShade does.

reshade::api::resource_view shader_resource_view = (reshade::api::resource_view)(int64_t)&rt.RTV;
reshade::api::resource_view srgb_shader_resource_view = (reshade::api::resource_view)(int64_t)0;
runtime->update_texture_bindings(RTNames[n], shader_resource_view, srgb_shader_resource_view);
        
So far I've ended up with this. Reinterpret cast just does not work, the compiler complains. I'm assuming it is just a pointer.

I don't understand how I assign a namespace to the texture, or if it already exists.

Don't pass in a ID3D11RenderTargetView into "update_texture_bindings"! It specifically requires a SRV rather than a RTV.

I'm not sure what you mean by namespace. The string you pass to "update_texture_bindings" is the semantic by which you would reference the texture in an effect. E.g. "update_texture_bindings("BLUB", ...)" in add-on => "texture WhatEverVariableNameYouWant : BLUB;" in ReShade FX.

Thanks for the clarification.

.. I got confused about the RTVs vs SRVs , assuming ReShade just does something to the RTV to make a new SRV. I was testing with SRVs though, hence my other post.

 

I've been having issues with this. I know exactly the next shader that is going to be executed (not deferred or anything) and I can access all of the relevant events, but trying to draw does nothing.

My assumption is that ReShade backs up the current state, and will execute render_technique at any point. 

Right now I am doing runtime->render_technique(technique, cmd_list, rtv[0]) within on_bind_render_targets_and_depth_stencil.

I don't really understand command lists. Is it correct that I use the one provided by on_bind_render_targets_and_depth_stencil? Does it matter if I use the one there, or get_immediate_command_list()?

If none of these are the issue, then the issue is presumably that this game shader is copying from other textures, into the render target. In that case, I'm using the wrong render target and just need to use the input texture's render target

streamable.com/fdu280

Pre-transparency is now functional. I used runtime->get_command_queue()->get_immediate_command_list() and a function to search for an SRV's RTV.

Command lists are primarily relevant for D3D12/Vulkan, in D3D11 they refer to a device context, so in most cases the immediate device context will be used and therefore the command list pointers are usually equal to that of "get_immediate_command_list()", so can just use that.

ReShade does backup/restore state in "render_technique" in case of D3D11, so you are safe there (it does not do so in D3D12 due to massive overhead and having multiple command lists making it less of an issue).

Is there no way to use structs?

It seems a bit wasteful otherwise because I already have structs I could directly send over.
it looks like set_uniform_value_float might be able to? But I'm unsure how that would look in ReShade, i.e. if each variable can have a name and be size float4. Some are also matrixes (view proj).

Ideally I could directly provide the constant buffer which already exists.

There is not: ReShade FX does not support structs, because it has to support D3D9, where the concept of a constant buffer does not exist. So it's invidiual variables only (which ReShade will put into a constant buffer for you behind the scenes in case of D3D11).

I am looking at The Witcher 3 for a helper now. Strangely even the example code will just make ReShade's overlay disappear when enabling any shader. I presume this is down to as you describe, ReShade not exactly backing up state for D3D12. Is there any advice you can give me on what to look out for, for taking control of whatever is going on? For my own code, I blended together shader toggler and your code, so I can check for a specific shader hash at the same point you did your example.

Edit: On DX11 no matter what, there will be a black texture output and nothing else, although the ReShade UI will show. Using the example code.

ReShade only restores the bound pipeline in D3D12, everything else has to be handled in the add-on (descriptor layout, descriptor sets, render targets, ...). This means registering for all the appropriate "reshade::addon_event::bind_[...]" events to keep track of which were last set and then rebinding those after effects were rendered.

For a somewhat complete example doing this (though a bit of a headbender to understand by itself, so I hope the previous explanation of what needs to be done helps) I'd take a look at alex1333's add-on (which allows rendering effects after the game has rendered something with a specific shader identified with Otis' shader toggler): github.com/4lex4nder/ReshadeEffectShader...ng/src/Main.cpp#L604 .

The reason this is not done by ReShade is that keeping track of all state has more overhead than e.g. in D3D11, where the runtime does that already anyway and lets one query it, and I don't want users to have to always pay for that overhead unless it's needed.

But the best way to verify something is going wrong is always to enable the D3D debug layer (Win + R > dxcpl > add game executable to list then set debug layer to "Force On"; don't forget to disable again when finished with debugging) and log its output by either attaching Visual Studio or using someting like Sysinternals DebugView.

Thanks, I got it rendering under the UI by checking against a shader hash. Just applying to any shader will typically crash so I had to modify how hunting worked. Right now it crashes if there is a custom rendering step before it is loaded in-game, when the hash doesn't exist yet. I think I'll have to rewrite the whole thing so that it doesn't do anything superfluous and is easier to understand what it's doing, i.e. a generic D3D12 ReShade Helper.

I've added a utility to the add-one examples now, which does all the state tracking automatically: github.com/crosire/reshade/blob/main/exa...e/state_tracking.hpp

Simply add state_tracking.cpp/hpp to your code and call "register_state_tracking()" in DllMain. Then can get the current state of a command list at any time via "cmd_list->get_private_data<state_block>();" and e.g. re-apply that state by calling "state_block.apply(cmd_list);".

Feel free to take that a base and improve upon it (haven't tested it much, so might be missing some corner cases, e.g. saw that alex1333 also kept track of the pipeline layout to know exactly how many descriptor sets to bind again, ... but it's probably a bit easier to understand).

Thanks, this worked. I think it might crash with overlays though.

I released a witcher 3 helper using it, so we can see what users dig up in terms of issues with these kinds of edits.

One thing I've seen so far is that the depth buffer breaks in a very strange way, where displaydepth shader seems to zoom into the top left corner, and doesn't split the image into normals and depth (only zoomed into normals) when looking into the sun. I know almost nothing about graphics programming so I guess perhaps the depth buffer is being used in another command lists for occlusion tests for sunrays or something.

Additionally, the gamma of the image is different to what reshade is happy with. This seems to be the same issue that the fo4 helper had. For this helper, I wrote a little shader Remap.fx which automatically runs before and after render_effects which converts from linear to sRGB and back. No idea how this would impact HDR monitors. I don't really understand the difference between a raw rtv and an rtv with srgb.

Edit: Seems that depth only has an issue on D3D11. Not too big of a deal then.

Edit: Is there a way to render before a shader with your methods? When I tried before (d3d11, singlethreaded) I ran into issues with creating RTVs from the SRV.

For gamma to work correctly, need to provide both a sRGB and a non-sRGB render target view to "render_effects".

E.g. if the target texture is of format "r8g8b8a8", the first RTV for that texture has to be of format "r8g8b8a8_unorm" and the second RTV for that texture has to be of format "r8g8b8a8_unorm_srgb". Either the game already has these lying around somewhere, or one needs to create them (the add-on API contains the utility function "format_to_default_typed" which can be helpful to translate the format as needed):Don't recreate these RTVs every frame of course, ideally cache them somewhere and only recreate them if the target texture changes.
An RTV is just a piece of information that tells the driver how to interpret the data that is sitting inside a texture. So a RTV with a sRGB format means that the hardware should automatically apply a sRGB->linear conversion every time the texture is accessed through that RTV, whereas a RTV with a normal format would just pass it through as-is.

I don't see a reason why rendering before the game draws with a shader should not work, what issues did you run into?

[Btw., if you don't mind, I'd prefer if credits only refer to my pseudonym "crosire", rather than my real name ]