Marty McFly's Ambient Obscurance (MXAO) with IL

Chaosrik wrote: Ok, thanks a lot. Really appreciate that. Unfortunately it didn't work, the buffer doesn't flip, it seems to become a mirror image that's reversed - something not quite right I'll keep trying.

@Crosire, @MartyMcFly -- are you able to take a look at SMAA? This seems kinda important given SMAA's primary use (and even better with when the depth buffer is available).

Yeah I made a mistake, Life is Strange: Before the Storm actually isn't flipped despite using Unity where most games have a upside-down depth buffer.

Kaldaien even has a screenshot showing the game behaving as such, I completely forgot about that:
steamcommunity.com/sharedfiles/filedetails/?id=1127405727

So the SMAA tweak I linked to and suggested wouldn't make a difference in this game and this issue must then be from something different.

EDIT: What does the settings panel of ReShade show for the preprocessor settings?

That's these from the ini file:
PreprocessorDefinitions=RESHADE_DEPTH_LINEARIZATION_FAR_PLANE=1000.0,RESHADE_DEPTH_INPUT_IS_UPSIDE_DOWN=0,RESHADE_DEPTH_INPUT_IS_REVERSED=0,RESHADE_DEPTH_INPUT_IS_LOGARITHMIC=0

ReShade 3.1.1 changed it so reversed is defaulted to 1 now due to many newer open world or even otherwise tend to use logarithmic depth by default so if this is set to 1 then setting it back to 0 and reloading ReShade might help.

ReShade.fxh for the shader files was also updated when 3.1.1 was released to match this change.

ReShade's code for this is here.
github.com/crosire/reshade/commit/7d3bed...0286e3c15adec1b79101

And the ReShade.fxh commit change is here.
github.com/crosire/reshade-shaders/commi...daf21808a9f15bfc0978

The in-game setting from Shift+F2 and opening the settings and adjusting this should take preference even if the default is 1 now instead of 0 though you could try to test if that helps.

The change works well in several newer games (In addition to often using logarithmic depth.) but for many older titles this probably just flips the depth buffer incorrectly and needs to be set to 0 again although if you already had a preset for the game chances are this is already set as such.
(I can see the shader settings on the previous page but not the ReShade settings which is where these preprocessor values are defined.)


EDIT: There's probably a better to check this too but I mostly use the depth shader itself and hotkey that and check what the game looks like, gray and black outlines is good whereas pure black or white usually means something is set incorrectly or at worst the buffer didn't work or didn't hook correctly either as a result of network traffic (Say when using ReShade in a MMO or just a online game.) or due to other issues so it's mostly done on a per-game basis and then setting these preprocessor values, reloading and seeing how that looks until it's normal.

For the SMAA shader it has a depth mode of it's own though it looks a bit different but I assume it works much the same, if it looks similar to the regular game that's good but if nothing is visible it doesn't have depth hooked up right.

RESHADE_DEPTH_LINEARIZATION_FAR_PLANE=1000.0,RESHADE_DEPTH_INPUT_IS_UPSIDE_DOWN=1,RESHADE_DEPTH_INPUT_IS_REVERSED=0,RESHADE_DEPTH_INPUT_IS_LOGARITHMIC=0,MXAO_ENABLE_IL=0,MXAO_SMOOTHNORMALS=1,MXAO_TWO_LAYER=0,MXAO_ENABLE_TSS=0,SMAA_PREDICATION=0

These are my settings. I did try different settings in case. I think SMAA must work differently with the depth buffer than other shaders (MXAO, DOF, etc). Maybe it uses the raw depth buffer which isn't flipped instead of linearized depth buffer which you can flip easily. I haven't got a clue when it comes to shaders other than they look a bit like c++ which I used to program in

After reviewing the original SMAA source code, it indeed appears to me as if it needs linearized depth buffer. The question is now, create a separate prelinearized depth texture (that responds to all reversed/flipped etc flags) or just redirect to linearized depth function? The latter would save video memory but would require all calculations for linearization to be done for each texture fetch...

Video memory is tricky, some games try to be conservative but others cache or utilize as much as they can (Shadow of War, Wolfenstein New Colossus if you up the settings.) and depending on the users configuration that could also have a impact on things, not everyone can afford a new GPU with 4+ GB VRAM especially what with today's pricing on hardware having risen sharply over the last two years or so.

The latter would depend on from the sounds of it a possible small performance hit but it also depends on how many of these texture fetches are required, if a game loads or unloads texture data more often that could be more severe.

EDIT: Right, API differences too and how DirectX 11 and above let's games cache or request memory which some games abuse heavily or downright break.
(Though it might not impact ReShade much and the API is probably intelligent enough even if some games have leaks and requests gigabytes of memory.)

I think I'd go with the latter, for ReShade atleast, simply because it's what tends to affect frametimes the least in my experience.

Marty McFly wrote: The latter would save video memory but would require all calculations for linearization to be done for each texture fetch...

Do those calculations on every fetch. Math is fast. Memory bandwidth is slow. So better use less textures for better cache utilization at the cost of a few more instructions that are hardly noticable.

Even if it means R32F texture read vs R16F (as I doubt full precision is required for SMAA)?

Hi folks. I'm having a bit of a strange problem. Running Total War Warhammer 2, MXAO 2.0 works just fine for me....but MXAO produces an entirely black screen, which is white with Debug mode on. Both are positioned at the top of the load order, didn't mess around with any settings, etc.

It looks as if the shader doesn't have access to the depth buffer at all...but that's weird since MXAO 2.0 does and works.

One weird wrinkle - I'm playing over steam streaming in my house. So I'm using the .DLL from Stormshadei since it bypasses the network activity check. That was compiled from reshade 3.07, not 3.1. Might that be making a difference? And, if not, any ideas on what might be up?

Thanks in advance, I can't tell you how much your work improves games

dwillner wrote: Hi folks. I'm having a bit of a strange problem. Running Total War Warhammer 2, MXAO 2.0 works just fine for me....but MXAO produces an entirely black screen, which is white with Debug mode on. Both are positioned at the top of the load order, didn't mess around with any settings, etc.

It looks as if the shader doesn't have access to the depth buffer at all...but that's weird since MXAO 2.0 does and works.

One weird wrinkle - I'm playing over steam streaming in my house. So I'm using the .DLL from Stormshadei since it bypasses the network activity check. That was compiled from reshade 3.07, not 3.1. Might that be making a difference? And, if not, any ideas on what might be up?
Thanks in advance, I can't tell you how much your work improves games

Disable 3D Portholes (and of course AA, if you haven't already), that should do the trick.

Marty McFly wrote: Even if it means R32F texture read vs R16F (as I doubt full precision is required for SMAA)?

Not that I want to push, especially because it's off-topic, but is there a fix for this -- from any shader guru?

*offers beer, virgins, and strawberry donuts to the shader gods*

*yes, strawberry donuts > all. Pfft!*

Uncle Crassius wrote: Disable 3D Portholes (and of course AA, if you haven't already), that should do the trick.

Unfortunately those aren't it - I'm already on 2D portholes and have in-game AA off .

A pastebin/github link to the lastest version can be added to the first post? Thanks!

dwillner wrote:

Uncle Crassius wrote: Disable 3D Portholes (and of course AA, if you haven't already), that should do the trick.

Unfortunately those aren't it - I'm already on 2D portholes and have in-game AA off .

Oh, bummer. Have you tried if it works if you're not streaming? Guess so since you're using another version, just want to make sure. Unfortunately I'm already out of ideas but maybe someone else might chime in if you provided more details.

MXAO 3.4.3 still has some issue in OpenGL:

C:\ReShade 3\Shaders\MXAO.fx(411, 6): error: internal shader compilation failed
Fragment info

---

0(60) : error C7011: implicit cast from "float" to "bool"
(0) : error C2003: incompatible options for link
C:\ReShade 3\Shaders\MXAO.fx(572, 1): error: program linking failed

I'm guessing ReShade FX isn't handling this cast right: github.com/crosire/reshade-shaders/blob/...Shaders/MXAO.fx#L417 . Try "|| GetCullingMask(MXAO) == 0". Although there is code in place to handle that, so it would surprise me. Maybe also try "|| (0.25 * 0.5 * MXAO_SAMPLE_RADIUS / (tex2D(sMXAO_DepthTex,MXAO.scaledcoord.xy).x + 2.0) * BUFFER_HEIGHT) < 1.0" a few lines above.

Hi, Marty!
When we can download a new version of MXAO?

Crystrex wrote: Hi, Marty!
When we can download a new version of MXAO?

Warning: Spoiler!

It's in the standard package now.

you can make SMAA is more better now ?

I'd like to keep SMAA talk outside the MXAO thread.