Welcome, Guest.
Username: Password: Remember me

TOPIC: Issue with create debug device using latest src

Issue with create debug device using latest src 3 months 3 weeks ago #1

Hi, I'm hacking away on a fork of Reshade, and I'm currently unable to create a debug device. I'm wondering if this functionality is busted, because debug builds are crashing immediately, and it's due to the call to

HRESULT hr = reshade::hooks::call(&D3D11CreateDeviceAndSwapChain)(pAdapter, DriverType, Software, Flags, pFeatureLevels, FeatureLevels, SDKVersion, nullptr, nullptr, ppDevice, &FeatureLevel, nullptr);

in d3d11.cpp, and the call stack looks like this
 	0000000000000000()	Unknown
 	d3d11_3SDKLayers.dll!00007ffb61f1c248()	Unknown
 	d3d11.dll!00007ffb6f277e7e()	Unknown
 	d3d11.dll!00007ffb6f277c9f()	Unknown
 	d3d11.dll!00007ffb6f2ad292()	Unknown
 	d3d11.dll!00007ffb6f2acf85()	Unknown
 	d3d11.dll!00007ffb6f2acd84()	Unknown
 	d3d11.dll!00007ffb6f26dfe4()	Unknown
 	d3d11.dll!00007ffb6f26720b()	Unknown
 	d3d11.dll!00007ffb6f265db9()	Unknown
 	d3d11.dll!00007ffb6f265a38()	Unknown
>	dxgi.dll!D3D11CreateDeviceAndSwapChain(IDXGIAdapter * pAdapter, D3D_DRIVER_TYPE DriverType, HINSTANCE__ * Software, unsigned int Flags, const D3D_FEATURE_LEVEL * pFeatureLevels, unsigned int FeatureLevels, unsigned int SDKVersion, const DXGI_SWAP_CHAIN_DESC * pSwapChainDesc, IDXGISwapChain * * ppSwapChain, ID3D11Device * * ppDevice, D3D_FEATURE_LEVEL * pFeatureLevel, ID3D11DeviceContext * * ppImmediateContext) Line 32	C++
 	dxgi.dll!D3D11CreateDevice(IDXGIAdapter * pAdapter, D3D_DRIVER_TYPE DriverType, HINSTANCE__ * Software, unsigned int Flags, const D3D_FEATURE_LEVEL * pFeatureLevels, unsigned int FeatureLevels, unsigned int SDKVersion, ID3D11Device * * ppDevice, D3D_FEATURE_LEVEL * pFeatureLevel, ID3D11DeviceContext * * ppImmediateContext) Line 19	C++
 	NvCamera64.dll!00007ffb4323fbea()	Unknown
 	NvCamera64.dll!00007ffb43126e99()	Unknown
 	NvCamera64.dll!00007ffb43127762()	Unknown
 	NvCamera64.dll!00007ffb430ff51f()	Unknown
 	[External Code]	
 	dxgi.dll!D3D11CreateDeviceAndSwapChain(IDXGIAdapter * pAdapter, D3D_DRIVER_TYPE DriverType, HINSTANCE__ * Software, unsigned int Flags, const D3D_FEATURE_LEVEL * pFeatureLevels, unsigned int FeatureLevels, unsigned int SDKVersion, const DXGI_SWAP_CHAIN_DESC * pSwapChainDesc, IDXGISwapChain * * ppSwapChain, ID3D11Device * * ppDevice, D3D_FEATURE_LEVEL * pFeatureLevel, ID3D11DeviceContext * * ppImmediateContext) Line 32	C++
 	dxgi.dll!D3D11CreateDevice(IDXGIAdapter * pAdapter, D3D_DRIVER_TYPE DriverType, HINSTANCE__ * Software, unsigned int Flags, const D3D_FEATURE_LEVEL * pFeatureLevels, unsigned int FeatureLevels, unsigned int SDKVersion, ID3D11Device * * ppDevice, D3D_FEATURE_LEVEL * pFeatureLevel, ID3D11DeviceContext * * ppImmediateContext) Line 19	C++

with an access violating trying to read from address 0.

it's curious that we're hitting D3D11CreateDeviceAndSwapChain twice in that callstack, a second time after passing through NvCamera64.dll ... which I think is nvidia ansel? I wonder if this has something to do with that. Maybe I'll try it on a non-ansel game.

EDIT:
Disabled ansel, and NvCamera64.dll dropped out of the callstack, now we only have one call to D3D11CreateDeviceAndSwapChain() but it still crashes at the same place in d3d11_3SDKLayers.dll. Hmmm.

EDIT 2:
The debug app seems to work with the debug layer. I'll compare and contrast the args.

EDIT 3:
Copying the args that the debug app uses doesn't help. Same crash result. Weird.

EDIT 4:
grabbing the symbols from the MS symbol server gives function names to the callstack. More to investigate:
>	d3d11_3SDKLayers.dll!NDebug::CDevice::LLOCompleteLayerConstruction(void)	Unknown
 	d3d11.dll!NOutermost::CDevice::FinalConstruct(struct NOutermost::CDevice::TConstructorArgs const &)	Unknown
 	d3d11.dll!TComObject<class NOutermost::CDevice>::TComObject<class NOutermost::CDevice>(void *,struct NOutermost::CDevice::TConstructorArgs const &,struct _GUID const &,void * *)	Unknown
 	d3d11.dll!TComObject<NOutermost::CDevice>::CreateInstance()	Unknown
 	d3d11.dll!D3D11CreateLayeredDevice()	Unknown
 	d3d11.dll!D3D11CoreCreateLayeredDevice()	Unknown
 	d3d11.dll!D3D11RegisterLayersAndCreateDevice()	Unknown
 	d3d11.dll!D3D11CoreCreateDevice()	Unknown
 	d3d11.dll!D3D11CreateDeviceAndSwapChainImpl()	Unknown
 	d3d11.dll!D3D11CreateDeviceAndSwapChain()	Unknown
 	dxgi.dll!D3D11CreateDeviceAndSwapChain(IDXGIAdapter * pAdapter, D3D_DRIVER_TYPE DriverType, HINSTANCE__ * Software, unsigned int Flags, const D3D_FEATURE_LEVEL * pFeatureLevels, unsigned int FeatureLevels, unsigned int SDKVersion, const DXGI_SWAP_CHAIN_DESC * pSwapChainDesc, IDXGISwapChain * * ppSwapChain, ID3D11Device * * ppDevice, D3D_FEATURE_LEVEL * pFeatureLevel, ID3D11DeviceContext * * ppImmediateContext) Line 39	C++
 	dxgi.dll!D3D11CreateDevice(IDXGIAdapter * pAdapter, D3D_DRIVER_TYPE DriverType, HINSTANCE__ * Software, unsigned int Flags, const D3D_FEATURE_LEVEL * pFeatureLevels, unsigned int FeatureLevels, unsigned int SDKVersion, ID3D11Device * * ppDevice, D3D_FEATURE_LEVEL * pFeatureLevel, ID3D11DeviceContext * * ppImmediateContext) Line 19	C++
Last Edit: 3 months 3 weeks ago by pyromuffin.
The administrator has disabled public write access.

Issue with create debug device using latest src 3 months 3 weeks ago #2

Just remove the "D3D11_CREATE_DEVICE_DEBUG" flag from the call in d3d11.cpp. I don't know why the debug layers cause problems for some, as I never had this problem myself, but you are not the first with this particular issue.
Cheers, crosire =)
The administrator has disabled public write access.

Issue with create debug device using latest src 3 months 3 weeks ago #3

i mean yeah, that does work, but i need to create a debug device, for you know, debugging :D

i’ll continue to investigate.
The administrator has disabled public write access.

Issue with create debug device using latest src 3 months 3 weeks ago #4

Digging through the disassembly at the crash site, and with a little help of a friend at MS ;) our hooked DXGI.dll fails to export a function called CompatValue, GetProcAddress returns null, and it crashes. I'm not an expert on DLL hooking, but it appears that we aren't exporting CompatValue. According to MSDN on GetModuleAddress: msdn.microsoft.com/en-us/library/windows...683199(v=vs.85).aspx
"if lpModuleName does not include a path and there is more than one loaded module with the same base name and extension, you cannot predict which module handle will be returned."
which might explain why sometimes it crashes, and sometimes it works. @crosire If you have an idea of how to just get our DLL to jump to that function in the regular DXGI.dll, I think that will do it. I don't know how to export a function when I don't know what the signature is, but I have barely looked into how the actual DLL hooking works. I mean, that's why I am using Reshade, and not writing my own app. :D

Thanks!

EDIT:
Changing the exports.def to export CompatValue and CompatString, and creating two functions:
HOOK_EXPORT void* WINAPI CompatValue()
{
	return reshade::hooks::call(&CompatValue);
}


HOOK_EXPORT void* WINAPI CompatString()
{
	return reshade::hooks::call(&CompatString);
}

makes things work. I can successfully create a debug device now. I can create a PR if you'd like.
Last Edit: 3 months 3 weeks ago by pyromuffin.
The administrator has disabled public write access.

Issue with create debug device using latest src 3 months 3 weeks ago #5

Interesting. I guess those exports are new to Windows 10? Never knowingly saw them in a dump of my Windows 7 DXGI files. Since ReShade already exports some of the other stubs found there. Or maybe I assumed they weren't used. But yeah, please create a pull request. Nice find!
Cheers, crosire =)
Last Edit: 3 months 3 weeks ago by crosire.
The administrator has disabled public write access.