Custom build from latest source - D3D9 broken

More
3 months 3 weeks ago #1 by Niko of Death
Custom build from latest source - D3D9 broken was created by Niko of Death
I compile reshade from the latest pushed source in order to have bleeding edge builds -  Obviously issues like this are to be expected, so if it's something you're already aware of apologies (I've held off on making this thread given the high activity on the repo, but the commits made seem to suggest that this issue isn't occurring for you in testing). Since the build I compiled on May 9th (The one before that was May 5th, and works as expected), D3D9 games have stopped working, in different ways. For example, Mass Effect crashes instantly, before the application is even visible, whereas The Witcher is able to display the opening FMVs, but has a black screen upon getting to the main menu (menu navigation seems to be possible, since moving the mouse brings the expected menu noises). However, both have the same error in d3d9.log (Mass Effect only once, The Witcher repeating infinitely): 
IDirect3DDevice9::CreateTexture failed with error code D3DERR_INVALIDCALL
. (and a few instances of 
IDirect3DDevice9::CreateCubeTexture failed with error code D3DERR_INVALIDCALL
. for the Witcher). Here's the d3d9.log for The Witcher: ghostbin.co/paste/9x2hjk (all that's after the last line in the actual log file is standard effects loading messages in between other instances of D3DERR_INVALIDCALL). This is on an AMD RX580 system, with an intel i5 3570k CPU.

Besides this, there are two other, lesser issues that I have come across with recent builds. Firstly (I believe this started with the same build that broke D3D9), in DX11 compute shaders seem to be broken, the expected performance hit occurs, but there's no visual presentation, and the GPU statistics show much less GPU time than is actually being taken. The logs don't show anything that stands out to me as a potential cause for this, with the only warnings in one I just tested with the build I just compiled being
Reference count for IDXGIDevice1 object 00B412A8 (00B4930C) is inconsistent (5700).
Reference count for ID3D11Device0 object 00AB9138 (00B49774) is inconsistent (5696).
The other, which started earlier (although I'm not sure exactly when - I believe it was when generic depth was implemented through the addon API), is even less severe, as it only affects a few specific games. The two I have encountered it with are max payne (using d3d8to9), and KOTOR 2 (using a special build with alpha to coverage forced). These games, which had working depth buffers with 4.9, now have blank depth buffers, both with the settings that worked before, and testing other settings. However, both are "Edge cases" which I was only able to get depth buffers for in the first place fairly recently (in particular, KOTOR 2 I was only getting many shaders working properly when using a forcealphatocoverage build, although trying 4.9 official it seems to have all shaders working properly), so I imagine it's an issue caused by some "hacky" method of getting depth buffers being broken/removed in the overhaul of depth code, rather than a failure of the proper implementation.

Please Log in or Create an account to join the conversation.

More
3 months 2 weeks ago #2 by crosire
Replied by crosire on topic Custom build from latest source - D3D9 broken
The D3D9 problem should be fixed by github.com/crosire/reshade/commit/e8cfb3...683b8437216bf16194b6 (I just wasn't running D3D9 apps for testing until yesterday, so this didn't occur).
Compute shaders seem to be working on D3D11 too with ToT, could you double check again?
The following user(s) said Thank You: Niko of Death

Please Log in or Create an account to join the conversation.

More
3 months 2 weeks ago #3 by Niko of Death
Replied by Niko of Death on topic Custom build from latest source - D3D9 broken
Just recompiled, can confirm D3D9 works now (I could've sworn I had pulled that commit when I last compiled, weird).
Double checked compute shaders - still not working. It could be an issue with the specific shader maybe (RTGI 0.22 - don't have 0.23 to test. 0.21 (non-CS) works properly). Is there a different shader I should use to test?

Please Log in or Create an account to join the conversation.

More
3 months 2 weeks ago - 3 months 2 weeks ago #4 by Niko of Death
Replied by Niko of Death on topic Custom build from latest source - D3D9 broken
Another issue I forgot to mention, with recent builds some games have started having severely messed up gamma (or some other form of washed out color) when reshade is hooked - regardless of whether any effects are applied.
The two games I have come across that experience this so far are Genshin Impact (imgsli.com/NTU5NzY) and Dragon Quest Heroes (imgsli.com/NTU5Nzc). It appears to be something limited too specific engines - testing Pathologic 2 (a unity game like Genshin Impact) it also appears, same with berserk and the band of the hawk (an omega force game like Dragon Quest Heroes), whereas I have not experienced it with any UE4 games for example.
Last edit: 3 months 2 weeks ago by Niko of Death.

Please Log in or Create an account to join the conversation.

More
3 months 2 weeks ago - 3 months 2 weeks ago #5 by crosire
Replied by crosire on topic Custom build from latest source - D3D9 broken
RTGI in D3D11 is probably fixed by github.com/crosire/reshade/commit/9bf2dc...209033b3e6031ed5a1c6.

As for the other thing: Sounds like a sRGB mismatch. Could you elaborate which render API (D3D9/11/...) this appears with and also post a log (contains the swapchain format which would help). I don't have those games so can't test directly.
Last edit: 3 months 2 weeks ago by crosire.

Please Log in or Create an account to join the conversation.

More
3 months 2 weeks ago #6 by Niko of Death
Replied by Niko of Death on topic Custom build from latest source - D3D9 broken
All of the games I've experienced it with are D3D11 64 bit.
Here are the logs:
Genshin Impact: ghostbin.com/paste/qKHzx
Pathologic 2: ghostbin.com/paste/LtnjQ
DQH: ghostbin.com/paste/PSJr5
Berserk: ghostbin.com/paste/HMKQh

Please Log in or Create an account to join the conversation.

More
3 months 2 weeks ago #7 by crosire

Please Log in or Create an account to join the conversation.

More
3 months 2 weeks ago #8 by Niko of Death
Replied by Niko of Death on topic Custom build from latest source - D3D9 broken
Trying to test but can't compile, getting
"'reshade::vulkan::device_impl::_instance': an object of const-qualified type must be initialized" reshade_api_device.cpp line 27

Please Log in or Create an account to join the conversation.

More
3 months 2 weeks ago #9 by crosire
Replied by crosire on topic Custom build from latest source - D3D9 broken
Ah, something slipped into a commit that I didn't want to commit yet. Is fixed now. May have to reset the Git repository, since I forced-pushed the fix to an earlier commit.

Please Log in or Create an account to join the conversation.

More
3 months 2 weeks ago #10 by Niko of Death
Replied by Niko of Death on topic Custom build from latest source - D3D9 broken
Alright, compiles now. The sRGB mismatch is fixed, but RTGI is still broken.

Please Log in or Create an account to join the conversation.

More
3 months 2 weeks ago #11 by Niko of Death
Replied by Niko of Death on topic Custom build from latest source - D3D9 broken
I've encountered two new problems:
In Genshin Impact (and perhaps other games, Genshin is just the one I've noticed it in), having the effects toggle enabled (regardless of whether any shaders are actually enabled) is causing texture corruption, that only goes away when disabling the toggle and moving the camera. Examples of the corruption: imgsli.com/NTY1MTU imgsli.com/NTY1MTY. Again, Genshin is 64 bit, DX11, Unity. Here's the log I got: ghostbin.com/paste/Onk4E.
It also seems that the opengl runtime is broken. I tested it in two polar opposite games, KOTOR 2 (32 bit, opengl 1.4). and Digimon Story Cyber Sleuth (64-bit - not certain which opengl version, but presumably at least opengl 3.3), both get failed to create imgui pipeline and failed to recreate opengl runtime environment errors, and reshade of course doesnt work in game. Kotor 2 log: ghostbin.com/paste/gm0R2 Digimon log: ghostbin.com/paste/EAodL KOTOR log: ghostbin.com/paste/gm0R2

Please Log in or Create an account to join the conversation.

More
3 months 2 weeks ago - 3 months 2 weeks ago #12 by crosire
Replied by crosire on topic Custom build from latest source - D3D9 broken
I've tried compute shader RTGI with ToT in GTA 5 (D3D11) and it's working there now. Don't have problems with OpenGL initialization anymore either.
Last edit: 3 months 2 weeks ago by crosire.

Please Log in or Create an account to join the conversation.

More
3 months 2 weeks ago #13 by Niko of Death
Replied by Niko of Death on topic Custom build from latest source - D3D9 broken
Can confirm CS RTGI is fixed on my end, as is openngl initialization.
However, opengl is severely broken on my end. Testing with KOTOR 2, 3D rendering is completely broken regardless of whether or not effects are enabled, and effects behaving abnormally on 2D elements including continuing to show the last 2D element on screen (e.g. the menu) when 3D elements are on screen, as seen here: imgsli.com/NTY2OTA (note that in the effects enabled screenshot, the UI elements in the corners are UImasked, and none of that main menu texture should be visible since its in game). Log file here: ghostbin.com/paste/HgBzM.
In Jedi Knight 2 Jedi Outcast, the screen is entirely black with effects enabled (displaydepth is the only one I could find that displayed normally), with no 2D elements remaining, but rendering is completely normal with effects disabled. Log file here: ghostbin.com/paste/czrYO.
Digimon Cyber Sleuth seems to work fine, but its difficult to test as reshade doesnt respond to any input (whether thats opening the overlay or enabling/disabling effects). Log file here: ghostbin.com/paste/fLPbN

Please Log in or Create an account to join the conversation.

More
3 months 2 weeks ago #14 by Niko of Death
Replied by Niko of Death on topic Custom build from latest source - D3D9 broken
Testing with the commits you added today, KOTOR 2 seems to be fixed (including the texture corruption issues from the other thread), however Jedi Knight 2 is still broken (I should also note that in addition to displaydepth, the lighting view and normal view debug modes in RTGI also work)
Latest log file: ghostbin.com/paste/A4GF6

Please Log in or Create an account to join the conversation.

More
3 months 1 week ago #15 by crosire
Replied by crosire on topic Custom build from latest source - D3D9 broken
Could you post the preset you are using for Jedi Knights 2? Tried in the demo with a bunch of different effects, but could not reproduce a black screen.

Please Log in or Create an account to join the conversation.

More
3 months 1 week ago #16 by Niko of Death
Replied by Niko of Death on topic Custom build from latest source - D3D9 broken
A preset wouldn't help afaik, as I'm getting it with any effect with any settings. Here's a preset anyway though: ghostbin.com/paste/wuUOz
Here's what I get when I take a screenshot with the before/after setting on: imgsli.com/NTcwNjk
Might be an AMD-specific issue?

Please Log in or Create an account to join the conversation.

More
3 months 1 week ago #17 by crosire
Replied by crosire on topic Custom build from latest source - D3D9 broken
Maybe. Little I can do without reproducing unfortunately. On a different note: Is the Genshin Impact issue still a thing?

Please Log in or Create an account to join the conversation.

More
3 months 1 week ago #18 by Niko of Death
Replied by Niko of Death on topic Custom build from latest source - D3D9 broken
Yes, still getting the genshin issue.

Please Log in or Create an account to join the conversation.

More
2 months 2 weeks ago #19 by Niko of Death
Replied by Niko of Death on topic Custom build from latest source - D3D9 broken
Found another new issue - crash on startup in Resident Evil HD Remaster. Here's a debug log with a fresh build: ghostbin.com/paste/s8m2I

Please Log in or Create an account to join the conversation.

More
2 months 3 days ago #20 by Niko of Death
Replied by Niko of Death on topic Custom build from latest source - D3D9 broken
REmake crash is fixed with latest commits.
The following user(s) said Thank You: crosire

Please Log in or Create an account to join the conversation.