Use ReShade Add-on to modify the game frames?

  • SYX
  • Topic Author
More
3 years 5 months ago #1 by SYX Use ReShade Add-on to modify the game frames? was created by SYX
I want to use ReShade Add-on to modify the game frames. From "10-video_capture" project, I know how to get the game frames to a CPU buffer. But I don't know how send the modified CPU buffer back to the game (for rendering). Is there any examples?

Please Log in or Create an account to join the conversation.

  • crosire
More
3 years 5 months ago #2 by crosire Replied by crosire on topic Use ReShade Add-on to modify the game frames?
You really don't want to copy frames from GPU to CPU and then back to GPU again, it is going to be too slow. CPU and GPU run asynchronously from each other, meaning the CPU is generally at least one frame ahead of the GPU. Adding synchronization here because copies would kill performance (the video capture example is already rather slow because of this).

If you want to try anyway, copying data back to the GPU works the same way it was copied from the GPU: Add a "copy_resource" command from the CPU (aka host) resource to the GPU resource after having modified it contents (by mapping it with "map_texture_region" , modifying it through the pointer that returns and then unmapping it again with "unmap_texture_region" ).

Please Log in or Create an account to join the conversation.

  • SYX
  • Topic Author
More
3 years 5 months ago #3 by SYX Replied by SYX on topic Use ReShade Add-on to modify the game frames?
Thanks crosire!
I want to add some kind of CPU-based frame process and the performance is not a big concern.
I have tried your suggestions (copy_resource) and find it works well if the back_buffer is not multisampled. But when it is multisampled, I don't know how to copy the processed host_resource (non-multisampled) to back_buffer (multisampled). No matter use copy_resource or copy_texture_region. Do you have any suggestions?

Below is my code:
static void on_present(reshade::api::command_queue *queue, reshade::api::swapchain *swapchain, const reshade::api::rect *, const reshade::api::rect *, uint32_t, const reshade::api::rect *)
{
    reshade::api::resource back_buffer = swapchain->get_current_back_buffer();
......
    if (multisampled)
    {
        // mutisampling RT -> single sample DEFAULT surface
        cmd_list->barrier(back_buffer, reshade::api::resource_usage::present, reshade::api::resource_usage::resolve_source);
        cmd_list->barrier(host_resource_ss, reshade::api::resource_usage::cpu_access, reshade::api::resource_usage::resolve_dest);
        cmd_list->resolve_texture_region(back_buffer, 0, nullptr, host_resource_ss, 0, 0, 0, 0, desc.texture.format);
        cmd_list->barrier(host_resource_ss, reshade::api::resource_usage::resolve_dest, reshade::api::resource_usage::cpu_access);
        cmd_list->barrier(back_buffer, reshade::api::resource_usage::resolve_source, reshade::api::resource_usage::present);

        // single sample DEFAULT surface -> single sample STAGING surface
        cmd_list->barrier(host_resource_ss, reshade::api::resource_usage::cpu_access, reshade::api::resource_usage::copy_source);
        cmd_list->barrier(host_resource, reshade::api::resource_usage::cpu_access, reshade::api::resource_usage::copy_dest);
        cmd_list->copy_resource(host_resource_ss, host_resource);
        cmd_list->barrier(host_resource, reshade::api::resource_usage::copy_dest, reshade::api::resource_usage::cpu_access);
        cmd_list->barrier(host_resource_ss, reshade::api::resource_usage::copy_source, reshade::api::resource_usage::cpu_access);
    }
    else
    {
        // single sample RT -> single sample STAGING surface
        cmd_list->barrier(back_buffer, reshade::api::resource_usage::present, reshade::api::resource_usage::copy_source);
        cmd_list->barrier(host_resource, reshade::api::resource_usage::cpu_access, reshade::api::resource_usage::copy_dest);
        cmd_list->copy_resource(back_buffer, host_resource);
        cmd_list->barrier(host_resource, reshade::api::resource_usage::copy_dest, reshade::api::resource_usage::cpu_access);
        cmd_list->barrier(back_buffer, reshade::api::resource_usage::copy_source, reshade::api::resource_usage::present);
    }
    queue->flush_immediate_command_list();

    // some kind of CPU-based process, just an example
    reshade::api::subresource_data host_data;
    if (!device->map_texture_region(host_resource, 0, nullptr, reshade::api::map_access::read_write, &host_data))
    {
        reshade::log_message(1, "Failed to map texture!");
        return;
    }
    for (int y = 0; y < 16; ++y)
    {
        for (int x = 0; x < desc.texture.width; ++x)
        {
            const size_t host_data_index = y * host_data.row_pitch + x * 4;
            static_cast<uint8_t *>(host_data.data)[host_data_index + 0] = 255;
            static_cast<uint8_t *>(host_data.data)[host_data_index + 1] = 0;
            static_cast<uint8_t *>(host_data.data)[host_data_index + 2] = 0;
        }
    }
    device->unmap_texture_region(host_resource, 0);

    //update backbuffer
    reshade::api::command_list *const cmd_list2 = queue->get_immediate_command_list();
    if (multisampled)
    {
        cmd_list2->barrier(host_resource, reshade::api::resource_usage::cpu_access, reshade::api::resource_usage::copy_source);
        cmd_list2->barrier(host_resource_ss, reshade::api::resource_usage::cpu_access, reshade::api::resource_usage::copy_dest);
        cmd_list2->copy_resource(host_resource, host_resource_ss);
        cmd_list2->barrier(host_resource, reshade::api::resource_usage::copy_source, reshade::api::resource_usage::cpu_access);
        cmd_list2->barrier(host_resource_ss, reshade::api::resource_usage::copy_dest, reshade::api::resource_usage::cpu_access);

        cmd_list2->barrier(host_resource_ss, reshade::api::resource_usage::cpu_access, reshade::api::resource_usage::copy_source);
        cmd_list2->barrier(back_buffer, reshade::api::resource_usage::render_target, reshade::api::resource_usage::copy_dest);
        //cmd_list2->copy_resource(host_resource_ss, back_buffer);
        cmd_list2->copy_texture_region(host_resource_ss, 0, nullptr, back_buffer, 0, nullptr);
        cmd_list2->barrier(host_resource_ss, reshade::api::resource_usage::copy_source, reshade::api::resource_usage::cpu_access);
        cmd_list2->barrier(back_buffer, reshade::api::resource_usage::copy_dest, reshade::api::resource_usage::render_target);
    }
    else
    {
        cmd_list2->barrier(host_resource, reshade::api::resource_usage::cpu_access, reshade::api::resource_usage::copy_source);
        cmd_list2->barrier(back_buffer, reshade::api::resource_usage::render_target, reshade::api::resource_usage::copy_dest);
        cmd_list2->copy_resource(host_resource, back_buffer);
        cmd_list2->barrier(host_resource, reshade::api::resource_usage::copy_source, reshade::api::resource_usage::cpu_access);
        cmd_list2->barrier(back_buffer, reshade::api::resource_usage::copy_dest, reshade::api::resource_usage::render_target);
    }
    queue->flush_immediate_command_list();
......
}

Please Log in or Create an account to join the conversation.

  • crosire
More
3 years 5 months ago - 3 years 5 months ago #4 by crosire Replied by crosire on topic Use ReShade Add-on to modify the game frames?
Fair enough!
Copying from a non-multisampled resource into a multisampled resource directly isn't really possible I'm afraid (logically it doesn't work: a non-multisampled resource has a single sample/value per pixel, a multisampled resource has multiple samples/values per pixel, how would one invent all the other samples from that single sample when copying into it. Copying from a multisampled into a non-multisampled resource on the other hand works since one can calculate the average of the multiple samples and put that into a single sample, which is what "resolve_texture_region" is doing).
One could draw a fullscreen quad with the source resource on it and the target multisampled resource as render target, which would use the rasterizer pipeline to generate multiple samples again, but setting that up is a lot more work (creating the pipeline, descriptor sets, ...). And that still isn't a great solution since it would just set all samples of a pixel to the same value, which makes the point of MSAA moot. So I think you are better off focusing on the usecase where MSAA is disabled for now.

Btw., you can register for the "create_swapchain" event, change "resource_desc::texture::samples" to 1 in the passed in resource description and return "true" to effectively force disable MSAA in any game that attempts to enable it.
Last edit: 3 years 5 months ago by crosire.

Please Log in or Create an account to join the conversation.

  • SYX
  • Topic Author
More
3 years 5 months ago - 3 years 5 months ago #5 by SYX Replied by SYX on topic Use ReShade Add-on to modify the game frames?
Thanks crosire! I have tried to register "create_swapchain" and disable MSAA for all the games. This idea works for me. The MSAA issue can be bypassed.

I am curious how ReShade Shaders & Effects (like FakeHDR.fx) handles the MSAA games. I guess it uses a pixel shader to read and write the MSAA back buffer. Am I right? Is there anyway to do the similar thing in ReShade Addon?
 
Last edit: 3 years 5 months ago by SYX.

Please Log in or Create an account to join the conversation.

  • crosire
More
3 years 5 months ago - 3 years 5 months ago #6 by crosire Replied by crosire on topic Use ReShade Add-on to modify the game frames?
ReShade renders the result on a fullscreen quad (or technically a fullscreen triangle) into the MSAA render buffer, so the method I had mentioned. I don't have an example add-on to easily show how to do that, but the entire ReShade effect runtime is implemented on top of the same API abstraction available to add-ons, so can try and figure out how that works from the ReShade source code:
github.com/crosire/reshade/blob/main/source/runtime.cpp#L217 (creation of the required objects)
github.com/crosire/reshade/blob/main/source/runtime.cpp#L635 (the copy code, where "_back_buffer_resolved_srv" is a shader resource view of the source resource and "_back_buffer_targets[0]" is a render target resource view of the target resource)
github.com/crosire/reshade/blob/main/res...s/fullscreen_vs.hlsl (the vertex shader which is referenced in the creation code, after compiling it to DX bytecode with a HLSL compiler)
github.com/crosire/reshade/blob/main/res/shaders/copy_ps.hlsl (the pixel shader which is referenced in the creation code, after compiling it to DX bytecode with a HLSL compiler)
Last edit: 3 years 5 months ago by crosire.

Please Log in or Create an account to join the conversation.

  • SYX
  • Topic Author
More
3 years 5 months ago #7 by SYX Replied by SYX on topic Use ReShade Add-on to modify the game frames?
Hi, crosire
Thanks for the detailed info and the guidance. I am going to add some code to render the texture to the MSAA render target.
If meeting any problems, I will raise the questions here.

Please Log in or Create an account to join the conversation.