Hook special environments only

helifax
Topic Author

7 years 7 months ago #1 by helifax 2.0 was created by helifax

Hi guys,

I was wondering if there is a way to tell Reshade only to hook special environments.
For example I have an OpenGL app that is also creating a DirectX9 window. I want reshade only to hook the OpenGL context but not the DirectX9 or the other way around.

I see by default it hooks everything it can find but this adds the effects 2 times when I only want it one time.
Is it possible to do something like this, in a config file or something?

Thank you in advance,
Best Regards,
Helifax

Please Log in or Create an account to join the conversation.

crosire

7 years 7 months ago #2 by crosire Replied by crosire on topic 2.0

helifax wrote: I see by default it hooks everything it can find but this adds the effects 2 times when I only want it one time.
Is it possible to do something like this, in a config file or something?

Nope, not right now. Except if you manage to create the Direct3D 9 context (call Direct3DCreate9) before ReShade is loaded, in that case effects will only apply on the OpenGL context.

The following user(s) said Thank You: helifax

Please Log in or Create an account to join the conversation.

helifax
Topic Author

7 years 7 months ago #3 by helifax Replied by helifax on topic 2.0

Hi,

Big thanks for the quick answer.
Unfortunately my context is created the other way;))

OpenGL App starts and Hooks my wrapper.
In my wrapper I load all the openGL API functions that I am interested and then make the DirectX9 context which I use to actually show on the screen.
The game/app still renders in OpenGL though, but I am using the Nvidia OpenGL to DirectX interooperability layer to register the DX and OGL buffers together.
Using the DX9 context as present allows me to use Nvidia 3D Vision.
I made a wrapper that we use basically for stereoscopic 3D (geometry mode) for openGL apps:
3dsurroundgaming.com/OGL3DVision.html

I will try to see if I can load load Reshade after I create the DX9 context;)
Just in case this will not work, is it possible to add a the ability to ignore a certain context? (Like ignore OGL, DX9, DX11)

Big thank you again for the answer and help!
Best Regards,
Helifax

Please Log in or Create an account to join the conversation.

crosire

7 years 7 months ago - 7 years 7 months ago #4 by crosire Replied by crosire on topic 2.0

Instead of using ReShade via wrapping the OpenGL or D3D9 DLL, you can instead load it explicitly via "LoadLibrary("ReShade32.dll")" after the Direct3D 9 context was created. ReShade does the rest for you afterwards. This however requires the OpenGL context to be initialized with at least OpenGL 4.3 and no list sharing, if OpenGL context creation happens before ReShade is loaded (which would apparently be the case here).

Last edit: 7 years 7 months ago by crosire.

Please Log in or Create an account to join the conversation.

helifax
Topic Author

7 years 7 months ago - 7 years 7 months ago #5 by helifax Replied by helifax on topic 2.0

Hi,

I gave it a go;)
Unfortunately since OpenGL is created first and then DirectX, loading Reshade afterwards doesn't hook the openGL environment:( and nothing gets hooked:(
I need the original windowHandle + resolution and other params from the OpenGL window in order to be able to setup the DirectX one. I tried making the DirectX window before yet it fails:(

The log shows just this:
25/08/2016 19:55:35:491 [07880] | INFO | Initializing crosire's ReShade version '2.0.3.1120' built on '2016-04-03 19:20:33' loaded from "

\BKA\Reshade32.dll" to "

Last edit: 7 years 7 months ago by helifax.

Please Log in or Create an account to join the conversation.

crosire

7 years 7 months ago #6 by crosire Replied by crosire on topic 2.0

Could you post the log?

Ignoring certain rendering APIs should be possible, though it would require to get that information in DllMain. That makes this a little problematic, since file reading operations (to read a config file) should perferably not occur in DllMain (possible synchronization issues with loader lock).

Please Log in or Create an account to join the conversation.

helifax
Topic Author

7 years 7 months ago - 7 years 7 months ago #7 by helifax Replied by helifax on topic 2.0

Sorry didn't noticed you just posted:)
I updated the previous post with the log where I

1. On DllMain of OpenGL32.dll:
LoadLibrary(opengl32.dll); -> for API function addresses that I redirect

2. Much later after the original OGL context is fully set:
LoadLibrary(d3d9.dll) -> for context creation
LoadLibrary(Reshade32.dll);

Here is the log for when I do:

1. On DllMain of OpenGL32.dll:
LoadLibrary(Reshade32.dll);
LoadLibrary(opengl32.dll); -> for API function addresses that I redirect
2. Much later after the original OGL context is fully set:
LoadLibrary(d3d9.dll) -> for context creation (as I need some information like window handle + size from the main App)

LOG:
25/08/2016 19:57:56:195 [09264] | INFO | Initializing crosire's ReShade version '2.0.3.1120' built on '2016-04-03 19:20:33' loaded from "

\BKA\Reshade32.dll" to "

\BKA\BrokenAge.exe" ...
25/08/2016 19:57:56:228 [09264] | INFO | Registering hooks for "C:\WINDOWS\system32\d3d8.dll" ...
25/08/2016 19:57:56:228 [09264] | INFO | > Delayed.
25/08/2016 19:57:56:228 [09264] | INFO | Registering hooks for "C:\WINDOWS\system32\d3d9.dll" ...
25/08/2016 19:57:56:228 [09264] | INFO | > Delayed.
25/08/2016 19:57:56:228 [09264] | INFO | Registering hooks for "C:\WINDOWS\system32\d3d10.dll" ...
25/08/2016 19:57:56:228 [09264] | INFO | > Delayed.
25/08/2016 19:57:56:228 [09264] | INFO | Registering hooks for "C:\WINDOWS\system32\d3d10_1.dll" ...
25/08/2016 19:57:56:229 [09264] | INFO | > Delayed.
25/08/2016 19:57:56:229 [09264] | INFO | Registering hooks for "C:\WINDOWS\system32\d3d11.dll" ...
25/08/2016 19:57:56:229 [09264] | INFO | > Delayed.
25/08/2016 19:57:56:229 [09264] | INFO | Registering hooks for "C:\WINDOWS\system32\d3d12.dll" ...
25/08/2016 19:57:56:229 [09264] | INFO | > Delayed.
25/08/2016 19:57:56:229 [09264] | INFO | Registering hooks for "C:\WINDOWS\system32\dxgi.dll" ...
25/08/2016 19:57:56:229 [09264] | INFO | > Delayed.
25/08/2016 19:57:56:229 [09264] | INFO | Registering hooks for "C:\WINDOWS\system32\opengl32.dll" ...
25/08/2016 19:57:56:229 [09264] | INFO | > Delayed.
25/08/2016 19:57:56:229 [09264] | INFO | Registering hooks for "C:\WINDOWS\system32\user32.dll" ...
25/08/2016 19:57:56:229 [09264] | INFO | > Libraries loaded.
25/08/2016 19:57:56:230 [09264] | INFO | > Found 5 match(es). Installing ...
25/08/2016 19:57:56:267 [09264] | INFO | > Installed 5 hook(s).
25/08/2016 19:57:56:267 [09264] | INFO | Registering hooks for "C:\WINDOWS\system32\ws2_32.dll" ...
25/08/2016 19:57:56:267 [09264] | INFO | > Libraries loaded.
25/08/2016 19:57:56:267 [09264] | INFO | > Found 8 match(es). Installing ...
25/08/2016 19:57:56:329 [09264] | INFO | > Installed 8 hook(s).
25/08/2016 19:57:56:329 [09264] | INFO | Initialized.
25/08/2016 19:57:56:337 [09264] | INFO | Installing delayed hooks for "C:\WINDOWS\system32\opengl32.dll" (Just loaded via 'LoadLibraryA("C:\WINDOWS\system32\opengl32.dll")') ...
25/08/2016 19:57:56:337 [09264] | INFO | > Found 360 match(es). Installing ...
25/08/2016 19:57:59:294 [09264] | INFO | > Installed 360 hook(s).
25/08/2016 19:57:59:296 [09264] | INFO | Installing delayed hooks for "C:\WINDOWS\system32\dxgi.dll" (Just loaded via 'LoadLibraryW("C:\WINDOWS\system32\nvapi.dll")') ...
25/08/2016 19:57:59:296 [09264] | INFO | > Found 7 match(es). Installing ...
25/08/2016 19:57:59:354 [09264] | INFO | > Installed 7 hook(s).
25/08/2016 19:58:00:545 [09264] | INFO | Redirecting 'RegisterClassW(00AFFD4C)' ...
25/08/2016 19:58:00:545 [09264] | INFO | > Adding 'CS_OWNDC' window class style flag to 'SDLHelperWindowInputCatcher'.
25/08/2016 19:58:00:561 [09264] | INFO | Redirecting 'RegisterClassW(00AFFD00)' ...
25/08/2016 19:58:00:561 [09264] | INFO | > Adding 'CS_OWNDC' window class style flag to 'SDL_app'.
25/08/2016 19:58:00:694 [04596] | INFO | Redirecting 'RegisterClassExW(05D3FA20)' ...
25/08/2016 19:58:00:694 [04596] | INFO | > Adding 'CS_OWNDC' window class style flag to 'Message'.
25/08/2016 19:58:00:877 [09264] | INFO | Redirecting 'wglChoosePixelFormat(4A011CD9, 00AFF924)' ...
25/08/2016 19:58:00:937 [09264] | INFO | Redirecting 'wglSetPixelFormat(4A011CD9, 9, 00AFF924)' ...
25/08/2016 19:58:00:937 [09264] | INFO | Redirecting 'wglCreateContext(4A011CD9)' ...
25/08/2016 19:58:00:937 [09264] | INFO | Redirecting 'wglCreateLayerContext(4A011CD9, 0)' ...
25/08/2016 19:58:00:937 [09264] | INFO | Redirecting 'wglMakeCurrent(4A011CD9, 00010000)' ...
25/08/2016 19:58:01:205 [09264] | INFO | Starting input capture for window 00030B94 ...
25/08/2016 19:58:01:205 [09264] | INFO | > Switched to new runtime 03549F48.
25/08/2016 19:58:01:245 [09264] | INFO | Redirecting 'wglChoosePixelFormatARB(4A011CD9, 00AFF974, 00AFFAA8, 1, 00AFF954, 00AFF94C)' ...
25/08/2016 19:58:01:253 [09264] | INFO | Redirecting 'wglMakeCurrent(00000000, 00000000)' ...
25/08/2016 19:58:01:253 [09264] | INFO | > Cleaning up runtime 03549F48 ...
25/08/2016 19:58:01:253 [09264] | INFO | Redirecting 'wglDeleteContext(00010000)' ...
25/08/2016 19:58:01:253 [09264] | INFO | Redirecting 'wglSetPixelFormat(3C0105E9, 9, 00AFFA74)' ...
25/08/2016 19:58:01:261 [09264] | INFO | Redirecting 'wglCreateContext(3C0105E9)' ...
25/08/2016 19:58:01:261 [09264] | INFO | Redirecting 'wglCreateLayerContext(3C0105E9, 0)' ...
25/08/2016 19:58:01:261 [09264] | INFO | Redirecting 'wglMakeCurrent(3C0105E9, 00020000)' ...
25/08/2016 19:58:01:261 [09264] | INFO | Starting input capture for window 001C0BCE ...
25/08/2016 19:58:01:261 [09264] | INFO | > Switched to new runtime 033959C0.
25/08/2016 19:58:01:437 [09264] | INFO | Redirecting 'RegisterRawInputDevices(00AFFCD0, 1, 12)' ...
25/08/2016 19:58:01:759 [09264] | INFO | Redirecting 'CreateDXGIFactory({7B7166EC-21C7-44AE-B21A-C9AE321AE369}, 00AFF244)' ...
25/08/2016 19:58:01:765 [09264] | INFO | Installing delayed hooks for "C:\WINDOWS\system32\d3d9.dll" (Just loaded via 'LoadLibraryA("d3d9.dll")') ...
25/08/2016 19:58:01:765 [09264] | INFO | > Found 9 match(es). Installing ...
25/08/2016 19:58:01:848 [09264] | INFO | > Installed 9 hook(s).
25/08/2016 19:58:01:848 [09264] | INFO | Redirecting 'Direct3DCreate9Ex(32, 00AFF53C)' ...
25/08/2016 19:58:01:901 [09264] | INFO | Redirecting 'IDirect3D9::CreateDevice(035A9DC0, 0, 1, 001C0BCE, 0x54, 00AFF4D0, 00AFF5E4)' ...
25/08/2016 19:58:02:443 [09264] | INFO | Recreated runtime environment on runtime 0356B9C8.
25/08/2016 19:58:02:477 [09264] | INFO | Loading effect from "

\BKA\ReShade.fx" ...
25/08/2016 19:58:02:673 [09264] | WARN | > Successfully compiled effect with warnings:

\BKA\ReShade.fx(14, 2): preprocessor warning: Could not find application profile, falling back to default

25/08/2016 19:58:02:724 [09264] | INFO | > Resizing image data for texture 'texNoise' from 1920x1080 to 1014x718 ...

Thank you again!
Best Regards,
Helifax

Last edit: 7 years 7 months ago by helifax.

Please Log in or Create an account to join the conversation.

crosire

7 years 7 months ago - 7 years 7 months ago #8 by crosire Replied by crosire on topic 2.0

Ah, well, the game calls "wglMakeCurrent" only once after initialization. Then this won't work. ReShade needs to be loaded at that point.

Another workaround would be to zero out the "Direct3DCreate9" entry in the export table of ReShade32.dll. Not pretty, but would work for the moment. ReShade would then no longer recognize the function and skip hooking it. Do this right after ReShade was loaded and before d3d9.dll is loaded. Note: I didn't test the following code, it's just a hint at how this would be done.

HMODULE handleToReShadeDLL = ...; // Get this from LoadLibrary("ReShade32.dll")

const auto imagebase = reinterpret_cast<BYTE *>(handleToReShadeDLL);
const auto imageheader = reinterpret_cast<IMAGE_NT_HEADERS *>(imagebase +
	reinterpret_cast<IMAGE_DOS_HEADER *>(imagebase)->e_lfanew);
const auto exportdir = reinterpret_cast<IMAGE_EXPORT_DIRECTORY *>(imagebase +
	imageheader->OptionalHeader.DataDirectory[IMAGE_DIRECTORY_ENTRY_EXPORT].VirtualAddress);

for (DWORD i = 0; i < exportdir->NumberOfNames; i++)
{
	// Get address of export name in module
	char *name = reinterpret_cast<char *>(imagebase +
		reinterpret_cast<DWORD *>(imagebase + exportdir->AddressOfNames)[i]);

	// Check if we found the Direct3DCreate9 export
	if (strcmp(name, "Direct3DCreate9") == 0)
	{
		// Replace the name with an empty string so it is ignored.
		// Need to change memory protection here, since this section is not writable by default.
		DWORD protect = PAGE_READWRITE;
		VirtualProtect(name, 1, protect, &protect);
		*name = '\0';
		VirtualProtect(name, 1, protect, &protect);
		break;
	}
}

Last edit: 7 years 7 months ago by crosire.

The following user(s) said Thank You: helifax

Please Log in or Create an account to join the conversation.

helifax
Topic Author

7 years 7 months ago - 7 years 7 months ago #9 by helifax Replied by helifax on topic 2.0

Big thank you!

That definitely fixed that problem!
However now I noticed another issue:(
Because I never call "wglSwapBuffers" in the openGL context (instead I actually detour it and present it in DirectX) Reshade is never triggered:(
Do you think of any way I could apply the result of Reshade Shaders before "wglSwapBuffers" is called ?
I guess you can't really tell when the frame is finished unless the swapBuffers is called...

Which clearly says that what I am doing is wrong:))
I think I need to load the libraries in the other order:
Reshade -> myLib rather than myLib->Reshade.

Sorry to ask this, if is obvious, but is there a way to load a ProxyLib in Reshade? I don't think there is exactly based on what I've read;) Still never hurts to ask ^_^

BIG BIG BIG thank you again for all the help!!!!
Best Regards,
Helifax

Last edit: 7 years 7 months ago by helifax.

Please Log in or Create an account to join the conversation.

crosire

7 years 7 months ago - 7 years 7 months ago #10 by crosire Replied by crosire on topic 2.0

If "SwapBuffers" is never called, why is it a problem ReShade applies to both OpenGL and D3D9 in the first place? Effects will only apply on the D3D9 context then? Which if you use that one for presentation anyway, is a good thing?

ReShade would not be able to register the end of a frame in OpenGL without the "SwapBuffers" call. You could just call it on a hidden window and hope that the framebuffer contents are not discarded by the swap implementation, I guess (dangerous assumption, but we are already doing workaround here, so whatever

).

And nope, there is no way to tell ReShade to use another proxy due to the way the hooking system works.

Last edit: 7 years 7 months ago by crosire.

The following user(s) said Thank You: helifax

Please Log in or Create an account to join the conversation.

helifax
Topic Author

7 years 7 months ago #11 by helifax Replied by helifax on topic 2.0

Hehehe;))

Let me start with the beginning which somehow I see I missed:))
I got around this:
[img size=https://fo] forums.geforce.com/default/topic/959175/...ision/superdepth-3d/ [/img]

Which is a shader for Reshade;) It basically produces a side-by-side image for left and right view;)
In my wrapper I generate both framebuffers for left and right eyes (in DirectX9 since is a requirement for 3D Vision to work sadly.... There is no way to tell the driver to enable 3D Vision: monitor + emitter + glasses except in DirectX).
I also use the Nvidia OpenGL to DirectX interoop available in their driver;) (So I can transfer the OpenGL data across to DirectX buffers)
In order to render in 3D (geometry stereo 3D) I modify the whole pipeline but output normally in GL_FRAMEBUFFER_ZERO.
Then I hook the glSwapBuffers and I copy the content in my DirectX9 buffers which I present;) (for each eye).

This is basically more or less the whole thing to it;) (without the coding specific, complications and workarounds there

) )
Because of this, I can't hook Reshade on the DirectX renderer but I need it on the OpenGL one;) (In this case I wouldn't modify the render pipeline as that shader requires a non modified rendering^_^) But I need the SBS OpenGL output so I can Blit Left side in the Left DX buffer and Right side in the Right DX buffer

I haven't thought about calling glSwapBuffers on a hidden Window, I must admit ^_^

. In theory I believe that should work;)
Another thing I was thinking, would be to make a "merge" between my wrapper code and Reshade code so there is one code base that handles everything;)
My wrapper is only for OpenGL apps but I know the guys (DarkStarSword & bo3b) at 3DMigoto/HelixMod would also be interested;)
I know this is something very niche related to 3D Vision tech and not valuable to everyone, but it would still be interesting, if you are also interested of course:)
I also thing it would be an awesome idea and exercise as well

Since it is clear that companies in general don't really care that much about us "mortal consumers and beings"

)

I will definitely try the "hidden window approach" though;) As is sound very interesting;)
Also, please do let me know if you think we could make a merge of code between the 2/3 wrappers;) I find that idea very cool and it would also solve some of the issues others are reporting related to ProxyLibs;) (I know it will be harder to do than say, but I like a challenge ^_^

)
Big thank you again for everything!

Best Regards,
Helifax

Please Log in or Create an account to join the conversation.

crosire

7 years 7 months ago - 7 years 7 months ago #12 by crosire Replied by crosire on topic 2.0

ReShade is currently closed source, which makes a source merge rather impossible.

EDIT: Just read this on the GeForce forums:

Got it sorted in the end;))
You need both Reshade32.dll and a copy of it named d3d9.dll for it to work. weird;)
Big thank you!

You actually only need to rename the ReShade DLL to d3d9.dll. No copying. This is so the game loads the DLL somehow. That's the only thing that has to happen for ReShade to work: it has to be loaded into the game through some method (DLL hijacking, injection, LoadLibrary, doesn't matter).

Last edit: 7 years 7 months ago by crosire.

Please Log in or Create an account to join the conversation.

helifax
Topic Author

7 years 7 months ago #13 by helifax Replied by helifax on topic 2.0

Big thank you again!

Ah yes, that was when I was mucking around with Reshade 3.0 and at some point I had the impression that I needed both of them

)
I used Reshade in the past and it worked everytime just by renaming the dll to the API specific dll.
I corrected the post there to reflect that is a stupid comment.

Yes, I understand the code is closed source;) My code is still pretty much closed course as well, even if I released an older version as open source;)
Still, if you are willing we can do the merge on closed source without releasing it to the public, except the binary libraries:)

Again, I want to thank you for all your help and this awesome tool;)
Best Regards,
Helifax

Please Log in or Create an account to join the conversation.

helifax
Topic Author

7 years 7 months ago #14 by helifax Replied by helifax on topic 2.0

Hi again,

I was thinking if there is a possibility to zero out the original wglSwapBuffers() that reshade is hooking ?

What I am trying to say is:
- The app calls wglSwapBuffers()
- It gets detoured to custom_wglSwapBuffers() (part of reshade)
- Injection gets done here (I expect the post process is a Blit on FBO_ZERO?)
- Reshade then calls original_wglSwapBuffers() (from the true opengl32.dll) that flips the BACK to FRONT buffers.

What I am interested is in zeroing-out "original_wglSwapBuffers()" that reshade calls, so the Post-Process is still done as part of FBO_ZERO but is not presenting it.
This is based on the assumption that this is true ofc;) and everything gets done in the detoured "wglSwapBuffers()"

Is something like this possible? This would solve out the issue for me;) I tried rendering to a hidden window, but it didn't work out (for some reason)

Thank you in advance,
Best Regards,
Helifax

Please Log in or Create an account to join the conversation.

crosire

7 years 7 months ago - 7 years 7 months ago #15 by crosire Replied by crosire on topic 2.0

Sure. Find the "wglSwapBuffers" export in the ReShade DLL, find the last CALL instruction and replace it with a NOP. This is a little complex though, but doable.
Additional tipp: the CALL to the original "SwapBuffers" function happens right after an if block, so I'd look out for a conditional JUMP happening right before. Shouldn't be too hard to find, just look at it in a disassembler.

Alternativly you could try to manipulate the export table of opengl32.dll before ReShade is loaded, so the "wglSwapBuffers" entry points to a custom function in your code that does nothing. Should work, since you don't actually use "SwapBuffers" and it is definitly easier to implement than the first suggestion..

Last edit: 7 years 7 months ago by crosire.

The following user(s) said Thank You: helifax

Please Log in or Create an account to join the conversation.

helifax
Topic Author

7 years 7 months ago - 7 years 7 months ago #16 by helifax Replied by helifax on topic 2.0

Awesome that did it!!!!
Using the Reshade 2.0 repack: www.mediafire.com/download/ce5497vb8hl5c...+3D+v1.8.1+Repack.7z
I found out where the call to the original glSwapBuffers is called;)
Address: "Reshade32.dll"+85885. If I NOP this one, the FBO_ZERO is populated but the original OpenGL context is not used

This way I can capture what is in the FBO_ZERO and I can put it in my DX context and render it;)
The downside of this method is that if I change the Reshade32.dll file the address will change so I always need to keep an updated offset from the start of the library/function.
I guess I can use signature scanning to get the offset I am looking for though;) This should work as long as the layout of the function doesn't change:)) but I bet it already did with Reshade 3.0

In any case, it currently works like I expect it to work;) BIG BIG BIG thank you for helping me with this one;)

The second method would not work, as I am already doing that and is in the detoured wglSwapBuffers where I copy the content of FBO_ZERO and put it in the DirectX9 Buffers ^_^

Big thank you again!
Best Regards,
Helifax

Last edit: 7 years 7 months ago by helifax.

Please Log in or Create an account to join the conversation.

crosire

7 years 7 months ago #17 by crosire Replied by crosire on topic 2.0

Cheers. Also, the hooked functions are not likely to change much, so the offets should stay the same between releases.

Please Log in or Create an account to join the conversation.

helifax
Topic Author

7 years 7 months ago #18 by helifax Replied by helifax on topic 2.0

Big thank you again!

I made a post about managing to get everything to work together on the 3D VIsion Forums here:
forums.geforce.com/default/topic/682130/...ost/4960245/#4960245

I don't know if you are into Stereo 3D and the likes, but I am very very happy that I managed to make it work on the 3D Vision Hardware !!!
I want to THANK YOU AGAIN for all the help you gave! Without it I would have been sitting pondering still

)
Also I want to thank BlueSkyDefender for his awesome shader for your awesome Reshade tool:
reshade.me/forum/shader-presentation/212...pic-shader?start=100

I know, a lot of people take for granted all this stuff and mostly don't understand what is involved to make all of this happen;)
But I, that I know exactly what skill & knowledge & time it takes, I want to THANK YOU all again and never to be discouraged by other people;)

Best Regards,
Helifax

The following user(s) said Thank You: crosire

Please Log in or Create an account to join the conversation.