Reshade ( commit bad7be47ab) failing to capture Swapchain::Present()

More
6 months 2 weeks ago #1 by Sherief
This only happens in Batman: Arkham Knight (out of the things I've tried).

I'm building Reshade from source, pulling commit bad7be47ab from GitHub. I rename the DLL to d3d11.dll, and it can indeed capture all D3D device calls - but the swapchain present isn't being intercepted by Reshade, and so there's no UI displayed.

The same DLL works just fine with Wasteland 2 from GOG, so I suspect AK is up to some.. shenanigans.

Here's my log:
\Binaries\Win64\d3d11.dll" into "C:\Program Files (x86)\Steam\steamapps\common\Batman Arkham Knight\Binaries\Win64\BatmanAK.exe" ...
03:11:44:081 [19816] | INFO  | Registering hooks for "user32.dll" ...
03:11:44:081 [19816] | INFO  | > Libraries loaded.
03:11:44:082 [19816] | INFO  | > Found 13 match(es). Installing ...
03:11:44:113 [19816] | INFO  | Registering hooks for "ws2_32.dll" ...
03:11:44:113 [19816] | INFO  | > Libraries loaded.
03:11:44:113 [19816] | INFO  | > Found 8 match(es). Installing ...
03:11:44:146 [19816] | INFO  | Registering hooks for "C:\WINDOWS\system32\d3d9.dll" ...
03:11:44:148 [19816] | INFO  | > Libraries loaded.
03:11:44:148 [19816] | INFO  | > Found 9 match(es). Installing ...
03:11:44:179 [19816] | INFO  | Registering hooks for "C:\WINDOWS\system32\d3d10.dll" ...
03:11:44:179 [19816] | INFO  | > Delayed.
03:11:44:180 [19816] | INFO  | Registering hooks for "C:\WINDOWS\system32\d3d10_1.dll" ...
03:11:44:180 [19816] | INFO  | > Delayed.
03:11:44:180 [19816] | INFO  | Registering hooks for "C:\WINDOWS\system32\d3d11.dll" ...
03:11:44:180 [19816] | INFO  | > Delayed until first call to an exported function.
03:11:44:180 [19816] | INFO  | Registering hooks for "C:\WINDOWS\system32\d3d12.dll" ...
03:11:44:180 [19816] | INFO  | > Delayed.
03:11:44:180 [19816] | INFO  | Registering hooks for "C:\WINDOWS\system32\dxgi.dll" ...
03:11:44:180 [19816] | INFO  | > Libraries loaded.
03:11:44:180 [19816] | INFO  | > Found 7 match(es). Installing ...
03:11:44:212 [19816] | INFO  | Registering hooks for "C:\WINDOWS\system32\opengl32.dll" ...
03:11:44:212 [19816] | INFO  | > Delayed.
03:11:44:212 [19816] | INFO  | Initialized.
03:11:44:213 [19816] | INFO  | Redirecting D3D11CreateDevice(pAdapter = 000000000A2FBF30, DriverType = 0, Software = 0000000000000000, Flags = 0, pFeatureLevels = 00000000076551AC, FeatureLevels = 1, SDKVersion = 7, ppDevice = 00000000076550EC, pFeatureLevel = 0000000000147448, ppImmediateContext = 0000000064E04570) ...
03:11:44:213 [19816] | INFO  | > Passing on to D3D11CreateDeviceAndSwapChain:
03:11:44:213 [19816] | INFO  | Redirecting D3D11CreateDeviceAndSwapChain(pAdapter = 000000000A2FBF30, DriverType = 0, Software = 0000000000000000, Flags = 0, pFeatureLevels = 00000000076551AC, FeatureLevels = 1, SDKVersion = 7, pSwapChainDesc = 0000000000000000, ppSwapChain = 0000000000000000, ppDevice = 00000000076550EC, pFeatureLevel = 0000000000147448, ppImmediateContext = 0000000064E04570) ...
03:11:44:214 [19816] | INFO  | Installing export hooks for "C:\WINDOWS\system32\d3d11.dll" ...
03:11:44:214 [19816] | INFO  | > Found 2 match(es). Installing ...
03:11:44:285 [19816] | INFO  | > Using feature level 0xb000.
03:11:47:379 [27524] | INFO  | Redirecting ID3D11Device::CreateDeferredContext(this = 000000006C526C90, ContextFlags = 0, ppDeferredContext = 0000000064E04230) ...
03:11:47:388 [18280] | INFO  | Redirecting ID3D11Device::CreateDeferredContext(this = 000000006C526C90, ContextFlags = 0, ppDeferredContext = 0000000064E07970) ...
03:11:47:399 [28136] | INFO  | Redirecting ID3D11Device::CreateDeferredContext(this = 000000006C526C90, ContextFlags = 0, ppDeferredContext = 0000000064E074B0) ...
03:11:47:411 [12304] | INFO  | Redirecting ID3D11Device::CreateDeferredContext(this = 000000006C526C90, ContextFlags = 0, ppDeferredContext = 0000000064E06F70) ...
03:11:47:422 [15296] | INFO  | Redirecting ID3D11Device::CreateDeferredContext(this = 000000006C526C90, ContextFlags = 0, ppDeferredContext = 0000000064E06CB0) ...
03:12:55:627 [19816] | WARN  | Reference count for IDXGIDevice1 object 000000006C533D80 is inconsistent.
03:12:55:627 [19816] | WARN  | Reference count for ID3D11Device0 object 000000006C526C90 is inconsistent.
03:12:55:653 [19816] | INFO  | Exiting ...
03:12:55:653 [19816] | INFO  | Uninstalling 43 hook(s) ...
03:12:55:685 [19816] | INFO  | Finished exiting.

Let me know if you need any more info. I looked at whether something else is trying to hook the swapchain present, and so I disabled Steam's overlay, GeForce Experience, and Razer Synapse. Still no dice.
The following user(s) said Thank You: tOXicSilence

Please Log in or Create an account to join the conversation.

More
6 months 2 weeks ago - 6 months 2 weeks ago #2 by crosire
Install as dxgi.dll. This game loads DXGI before D3D11, so ReShade never got a chance to intercept DXGI here (since the game created the DXGI factory before d3d11.dll was loaded) and therefore cannot intercept the present call.
Last edit: 6 months 2 weeks ago by crosire.

Please Log in or Create an account to join the conversation.

More
6 months 2 weeks ago #3 by Sherief
It crashes when I install as dxgi.dll - and it's not easy to get a call stack since the DRM makes the process commit seppuku as soon as a debugger is attached. Here's the log:
04:44:47:542 [28800] | INFO  | Initializing crosire's ReShade version '0.0.0.2' (64-bit) built on '2020-02-29 00:48:20' loaded from "C:\Program Files (x86)\Steam\steamapps\common\Batman Arkham Knight\Binaries\Win64\dxgi.dll" into "C:\Program Files (x86)\Steam\steamapps\common\Batman Arkham Knight\Binaries\Win64\BatmanAK.exe" ...
04:44:47:680 [28800] | INFO  | Registering hooks for "user32.dll" ...
04:44:47:680 [28800] | INFO  | > Libraries loaded.
04:44:47:680 [28800] | INFO  | > Found 13 match(es). Installing ...
04:44:47:714 [28800] | INFO  | Registering hooks for "ws2_32.dll" ...
04:44:47:714 [28800] | INFO  | > Libraries loaded.
04:44:47:714 [28800] | INFO  | > Found 8 match(es). Installing ...
04:44:47:747 [28800] | INFO  | Registering hooks for "C:\WINDOWS\system32\d3d9.dll" ...
04:44:47:749 [28800] | INFO  | > Libraries loaded.
04:44:47:749 [28800] | INFO  | > Found 9 match(es). Installing ...
04:44:47:782 [28800] | INFO  | Registering hooks for "C:\WINDOWS\system32\d3d10.dll" ...
04:44:47:782 [28800] | INFO  | > Delayed.
04:44:47:782 [28800] | INFO  | Registering hooks for "C:\WINDOWS\system32\d3d10_1.dll" ...
04:44:47:782 [28800] | INFO  | > Delayed.
04:44:47:782 [28800] | INFO  | Registering hooks for "C:\WINDOWS\system32\d3d11.dll" ...
04:44:47:782 [28800] | INFO  | > Delayed.
04:44:47:782 [28800] | INFO  | Registering hooks for "C:\WINDOWS\system32\d3d12.dll" ...
04:44:47:782 [28800] | INFO  | > Delayed.
04:44:47:782 [28800] | INFO  | Registering hooks for "C:\WINDOWS\system32\dxgi.dll" ...
04:44:47:782 [28800] | INFO  | > Delayed until first call to an exported function.
04:44:47:782 [28800] | INFO  | Registering hooks for "C:\WINDOWS\system32\opengl32.dll" ...
04:44:47:783 [28800] | INFO  | > Delayed.
04:44:47:783 [28800] | INFO  | Initialized.
04:44:47:783 [28800] | INFO  | Redirecting CreateDXGIFactory(riid = {7B7166EC-21C7-44AE-B21A-C9AE321AE369}, ppFactory = 0000000000148B28) ...
04:44:47:783 [28800] | INFO  | > Passing on to CreateDXGIFactory1:
04:44:47:783 [28800] | INFO  | Redirecting CreateDXGIFactory1(riid = {7B7166EC-21C7-44AE-B21A-C9AE321AE369}, ppFactory = 0000000000148B28) ...
04:44:47:784 [28800] | INFO  | Installing export hooks for "C:\WINDOWS\system32\dxgi.dll" ...
04:44:47:784 [28800] | INFO  | > Found 7 match(es). Installing ...
04:44:48:976 [28800] | INFO  | Redirecting CreateDXGIFactory(riid = {7B7166EC-21C7-44AE-B21A-C9AE321AE369}, ppFactory = 000000000014A038) ...
04:44:48:976 [28800] | INFO  | > Passing on to CreateDXGIFactory1:
04:44:48:976 [28800] | INFO  | Redirecting CreateDXGIFactory1(riid = {7B7166EC-21C7-44AE-B21A-C9AE321AE369}, ppFactory = 000000000014A038) ...
04:44:49:028 [28800] | INFO  | Redirecting RegisterClassExW(lpWndClassEx = 0000000000149E40 { LaunchUnrealUWindowsClient }) ...
04:44:49:270 [16436] | INFO  | Redirecting RegisterClassW(lpWndClass = 00000000182AFDE0 { SplashScreenClass }) ...
04:44:49:270 [16436] | INFO  | > Adding 'CS_OWNDC' window class style flag to 'SplashScreenClass'.
04:44:55:079 [28800] | INFO  | Installing delayed hooks for "C:\WINDOWS\system32\d3d11.dll" (Just loaded via LoadLibrary("d3d11.dll")) ...
04:44:55:079 [28800] | INFO  | > Found 2 match(es). Installing ...
04:44:55:113 [28800] | INFO  | Redirecting D3D11CreateDevice(pAdapter = 000000000A32E250, DriverType = 0, Software = 0000000000000000, Flags = 0, pFeatureLevels = 00000000143CFA6C, FeatureLevels = 1, SDKVersion = 7, ppDevice = 00000000143CF9AC, pFeatureLevel = 0000000000147448, ppImmediateContext = 0000000065B34570) ...
04:44:55:113 [28800] | INFO  | > Passing on to D3D11CreateDeviceAndSwapChain:
04:44:55:113 [28800] | INFO  | Redirecting D3D11CreateDeviceAndSwapChain(pAdapter = 000000000A32E250, DriverType = 0, Software = 0000000000000000, Flags = 0, pFeatureLevels = 00000000143CFA6C, FeatureLevels = 1, SDKVersion = 7, pSwapChainDesc = 0000000000000000, ppSwapChain = 0000000000000000, ppDevice = 00000000143CF9AC, pFeatureLevel = 0000000000147448, ppImmediateContext = 0000000065B34570) ...
04:44:55:179 [28800] | INFO  | > Using feature level 0xb000.
04:44:56:809 [28800] | INFO  | Redirecting IDXGIFactory::CreateSwapChain(this = 0000000009A08A50, pDevice = 000000000A5B9910, pDesc = 00000000001477B0, ppSwapChain = 0000000065B345A8) ...
04:44:56:810 [28800] | INFO  | > Dumping swap chain description:
04:44:56:810 [28800] | INFO  |   +-----------------------------------------+-----------------------------------------+
04:44:56:810 [28800] | INFO  |   | Parameter                               | Value                                   |
04:44:56:810 [28800] | INFO  |   +-----------------------------------------+-----------------------------------------+
04:44:56:810 [28800] | INFO  |   | Width                                   | 1920                                    |
04:44:56:810 [28800] | INFO  |   | Height                                  | 1080                                    |
04:44:56:810 [28800] | INFO  |   | RefreshRate                             | 0                   0                   |
04:44:56:810 [28800] | INFO  |   | Format                                  | 28                                      |
04:44:56:810 [28800] | INFO  |   | ScanlineOrdering                        | 0                                       |
04:44:56:810 [28800] | INFO  |   | Scaling                                 | 0                                       |
04:44:56:810 [28800] | INFO  |   | SampleCount                             | 1                                       |
04:44:56:810 [28800] | INFO  |   | SampleQuality                           | 0                                       |
04:44:56:810 [28800] | INFO  |   | BufferUsage                             | 1072                                    |
04:44:56:810 [28800] | INFO  |   | BufferCount                             | 1                                       |
04:44:56:810 [28800] | INFO  |   | OutputWindow                            | 0000000000170996                        |
04:44:56:810 [28800] | INFO  |   | Windowed                                | TRUE                                    |
04:44:56:810 [28800] | INFO  |   | SwapEffect                              | 0                                       |
04:44:56:810 [28800] | INFO  |   | Flags                                   | 0x2                                     |
04:44:56:810 [28800] | INFO  |   +-----------------------------------------+-----------------------------------------+
04:44:57:150 [28800] | INFO  | Recreated runtime environment on runtime 00000000D63F5470.
04:44:58:047 [10236] | INFO  | Redirecting ID3D11Device::CreateDeferredContext(this = 000000000A4E99C0, ContextFlags = 0, ppDeferredContext = 0000000065B34230) ...
04:44:58:057 [30000] | INFO  | Redirecting ID3D11Device::CreateDeferredContext(this = 000000000A4E99C0, ContextFlags = 0, ppDeferredContext = 0000000065B37970) ...
04:44:58:068 [35324] | INFO  | Redirecting ID3D11Device::CreateDeferredContext(this = 000000000A4E99C0, ContextFlags = 0, ppDeferredContext = 0000000065B374B0) ...
04:44:58:079 [28504] | INFO  | Redirecting ID3D11Device::CreateDeferredContext(this = 000000000A4E99C0, ContextFlags = 0, ppDeferredContext = 0000000065B36F70) ...
04:44:58:090 [09436] | INFO  | Redirecting ID3D11Device::CreateDeferredContext(this = 000000000A4E99C0, ContextFlags = 0, ppDeferredContext = 0000000065B36CB0) ...
04:44:58:973 [07220] | INFO  | Redirecting D3D11CreateDevice(pAdapter = 00000000D9D626C0, DriverType = 0, Software = 0000000000000000, Flags = 0x80, pFeatureLevels = 000000003294EB00, FeatureLevels = 1, SDKVersion = 7, ppDevice = 000000003294EAB0, pFeatureLevel = 0000000000000000, ppImmediateContext = 0000000000000000) ...
04:44:58:973 [07220] | INFO  | > Passing on to D3D11CreateDeviceAndSwapChain:
04:44:58:973 [07220] | INFO  | Redirecting D3D11CreateDeviceAndSwapChain(pAdapter = 00000000D9D626C0, DriverType = 0, Software = 0000000000000000, Flags = 0x80, pFeatureLevels = 000000003294EB00, FeatureLevels = 1, SDKVersion = 7, pSwapChainDesc = 0000000000000000, ppSwapChain = 0000000000000000, ppDevice = 000000003294EAB0, pFeatureLevel = 0000000000000000, ppImmediateContext = 0000000000000000) ...
04:44:59:004 [07220] | INFO  | > Using feature level 0xb000.
04:45:08:684 [28800] | INFO  | Exiting ...
04:45:08:684 [28800] | INFO  | Uninstalling 47 hook(s) ...
04:45:08:717 [28800] | INFO  | Finished exiting.

Anything else I can try?

Please Log in or Create an account to join the conversation.

More
6 months 2 weeks ago #4 by crosire
Debug builds of ReShade will create crash dumps for every exception they encounter in the game directory (labeled "exception_XX.dmp"). That can be useful to figure out the crash reason.

Please Log in or Create an account to join the conversation.

More
6 months 2 weeks ago #5 by Sherief
Looks like it also crashes in Wasteland 2, which is much easier to debug than Arkham Knight, when used as dxgi.dll. Here's the stack trace:
d3d11.dll!NDXGI::CDevice::EnsureChildDevice(struct IDXGIAdapter *,class ATL::CComPtr<class NDXGI::CDevice> &,char const *)	Unknown
 	dxgi.dll!CDXGISwapChain::EnsureChildDeviceInternal(struct IDXGIAdapter *,enum DXGI_INTERNAL_CHILD_DEVICE_TYPE)	Unknown
 	dxgi.dll!CDXGISwapChain::PrepareWindowedBltPresent(bool,bool,unsigned int,class DXGITrack<long> &,unsigned __int64 &)	Unknown
 	dxgi.dll!CDXGISwapChain::PresentImplCore()	Unknown
 	dxgi.dll!CDXGISwapChain::PresentImpl()	Unknown
 	dxgi.dll!CDXGISwapChain::Present()	Unknown
>	dxgi.dll!DXGISwapChain::Present(unsigned int SyncInterval, unsigned int Flags) Line 281	C++
 	WL2.exe!00007ff680f3934b()	Unknown
 	WL2.exe!00007ff680f57ed5()	Unknown
 	WL2.exe!00007ff680f5b84f()	Unknown
 	WL2.exe!00007ff680f54c0a()	Unknown
 	WL2.exe!00007ff680b653b6()	Unknown
 	kernel32.dll!BaseThreadInitThunk()	Unknown
 	ntdll.dll!RtlUserThreadStart()	Unknown

But the Arkham Knight crash does show Steam's overlay DLL involved, despite me trying to kill it with fire from both global and per-game Steam options:
d3d11.dll!NDXGI::CDevice::EnsureChildDevice(struct IDXGIAdapter *,class ATL::CComPtr<class NDXGI::CDevice> &,char const *)	Unknown
 	dxgi.dll!CDXGISwapChain::EnsureChildDeviceInternal(struct IDXGIAdapter *,enum DXGI_INTERNAL_CHILD_DEVICE_TYPE)	Unknown
 	dxgi.dll!CDXGISwapChain::PrepareWindowedBltPresent(bool,bool,unsigned int,class DXGITrack<long> &,unsigned __int64 &)	Unknown
 	dxgi.dll!CDXGISwapChain::PresentImplCore()	Unknown
 	dxgi.dll!CDXGISwapChain::PresentImpl()	Unknown
 	dxgi.dll!CDXGISwapChain::Present()	Unknown
 	gameoverlayrenderer64.dll!00007ffbfc81ba53()	Unknown
>	dxgi.dll!DXGISwapChain::Present(unsigned int SyncInterval, unsigned int Flags) Line 281	C++
 	BatmanAK.exe!0000000145438d06()	Unknown
 	BatmanAK.exe!0000000145439620()	Unknown
 	BatmanAK.exe!0000000144f6e6fa()	Unknown
 	BatmanAK.exe!0000000144f70c57()	Unknown
 	BatmanAK.exe!0000000144f70d63()	Unknown
 	BatmanAK.exe!000000014516662a()	Unknown
 	BatmanAK.exe!0000000145166909()	Unknown
 	0000000000000060()	Unknown
 	00000000079592c0()	Unknown
 	0000057b69e13277()	Unknown
 	0000057b69e17da3()	Unknown

What do you think?..

Please Log in or Create an account to join the conversation.

More
6 months 2 weeks ago #6 by Sherief
BTW, what's the best way to share minidump files?..

Please Log in or Create an account to join the conversation.

More
6 months 2 weeks ago #7 by crosire
Is this happening on a desktop system or a laptop with an onboard chip + dedicated GPU?

I'd start by commenting out the D3D11 device hooking ( github.com/crosire/reshade/blob/d6d784b0.../d3d11/d3d11.cpp#L97 ) and see if that resolves the crash. That will prevent ReShade from working but at least give you a hint as to whether it's the device hook chain that is broken (which this very much looks like) or something else.

Sharing the dump files is probably not going to be all that helpful, since the call stack is the main thing to get and it's failing inside the D3D runtime.

Please Log in or Create an account to join the conversation.

More
5 months 4 weeks ago #8 by Sherief
Laptop with dual GPU, yup. I went IDAPro on d3d11.dll and it looks like the first swapchain present tries to create a device on the GPU that's physically connected to the display. Any ideas?

Please Log in or Create an account to join the conversation.

More
5 months 4 weeks ago - 5 months 4 weeks ago #9 by crosire
No clue. There were reports of issues with laptop systems before, so this is just the latest incarnation of those. I personally was never able to reproduce this though, probably because I only have an old pre-Windows 10 laptop and desktop systems are unaffected.
Maybe try to eliminate the Steam overlay as a factor by renaming the GameOverlayRenderer64.dll in the Steam directory.
Does it have any effect to disable D3D11 hooking?
Last edit: 5 months 4 weeks ago by crosire.

Please Log in or Create an account to join the conversation.

More
5 months 3 weeks ago #10 by Sherief
I fixed it! The solution is to detect whether you're being called by the D3D / DXGI runtime and _not_ return a proxy device!!

Please Log in or Create an account to join the conversation.

More
5 months 3 weeks ago #11 by crosire
Ah. Reminds me of github.com/crosire/reshade/commit/82c7c2...20a940cf52327f6f5d9c , where D2D rendering failed too, because apparently using interna of the DXGI device, so had to make sure the original device was passed to it.
I'm a bit surprised the DXGI/D3D runtime itself calls back into the exported create functions and not some internal one. But how'd you went about detecting where it's called from? Walking up the stack frame?

Please Log in or Create an account to join the conversation.