SMAA

Further investigation, and to answer my own query on why depth method was producing better results than color. This will be two posts due to link posts limit.

First, from Firewatch all with Reshade 4.2.1:

No AA


FXAA


SMAA color detect (default)


SMAA depth detect


You can really see here what a poor job SMAA does versus FXAA with its standard color detection method. It's almost exactly like the No AA shot.

However, using depth detection it's easily as good as FXAA. The problem here of course is all non-depth edges are missed. But why is color performing so poorly with such clear on the edges in this sample? The answer may lie in the next post.

Color method edge mask


Depth method edge mask


Color method weight mask


Depth method weight mask


So it would appear the problem occurs in the weighting applied for the color method. We can't, as far I can see, toggle weighting from the SMAA control panel, only detection. Making this available and with some stronger defaults may bring SMAA closer to quality of FXAA, without the whole-scene blurring.

The above is all without predication as well at the moment, however that is supposed to improve things if it were working properly. But since not every game has the depth buffer available for Reshade, even getting normal SMAA improved would be of great benefit.

Awesome! Thanks for doing this. Some small changes can be attributed to the fact that the SMAA in ReShade's repository is based on the most recent SMAA (Nov 2013), whereas the one in SweetFX was based on SMAA 2.8 (Aug 2013) and the lookup textures have changed between those. But looks like you confirmed my own findings.

On depth detection: SMAA actually uses a different threshold for depth detection compared to luma/color edge detection. It is configurable via a "SMAA_DEPTH_THRESHOLD" preprocessor define, which by default is set to "(0.1 * SMAA_THRESHOLD)". You can adjust this by adding a value in ReShade's global preprocessor definitions and e.g. settings it to "(0.8 * SMAA_THRESHOLD)" so more edges are let through (SMAA_THRESHOLD is an alias for the EdgeDetectionThreshold you can set in ReShade's UI).

Added a depth linearization pass which converts the depth input using the usual depth preprocessor settings to make it consistent with other shaders: github.com/crosire/reshade-shaders/commi...6b843a399339e77f2259 .

Going by the .FXH file and source code commentary it seems predication might be more of a optimization thing, not increasing the aliasing efficency.

/**
* Predicated thresholding allows to better preserve texture details and to
* improve performance, by decreasing the number of detected edges using an
* additional buffer like the light accumulation buffer, object ids or even the
* depth buffer (the depth buffer usage may be limited to indoor or short range
* scenes).
*
* It locally decreases the luma or color threshold if an edge is found in an
* additional buffer (so the global threshold can be higher).
*
* This method was developed by Playstation EDGE MLAA team, and used in
* Killzone 3, by using the light accumulation buffer. More information here:
* iryoku.com/aacourse/downloads/06-MLAA-on-PS3.pptx
*/
#ifndef SMAA_PREDICATION
#define SMAA_PREDICATION 0
#endif

But I don't fully understand shader code. (Now there's a understatement!) Thus it's possible the implementation in the .FX shader here might be more customized.

Nice comparisons too and nice to see a update to the shader again.


EDIT: Wonder if SMAA or FXAA were intended to be developed further, it does seem FXAA's author had plans for a 4.0 update which never happened and for SMAA not really sure how that went though I am glad what exists is available and does a pretty good job at a relatively low cost for improving on the aliasing issue in games.


EDIT:

On depth detection: SMAA actually uses a different threshold for depth detection compared to luma/color edge detection. It is configurable via a "SMAA_DEPTH_THRESHOLD" preprocessor define, which by default is set to "(0.1 * SMAA_THRESHOLD)". You can adjust this by adding a value in ReShade's global preprocessor definitions and e.g. settings it to "(0.8 * SMAA_THRESHOLD)" so more edges are let through (SMAA_THRESHOLD is an alias for the EdgeDetectionThreshold you can set in ReShade's UI).

Didn't ReShade 2.0's SMAA have a option for that directly? Ugh I'm no good with this but.

uniform float EdgeDetectionThreshold <
ui_type = "drag";
ui_min = 0.05; ui_max = 0.20; ui_step = 0.01;
ui_tooltip = "Edge detection threshold. If SMAA misses some edges try lowering this slightly.";
ui_label = "Edge Detection Threshold";
> = 0.05;


#define SMAA_THRESHOLD EdgeDetectionThreshold


So if these correspond to the .FXH

/**
* SMAA_THRESHOLD specifies the threshold or sensitivity to edges.
* Lowering this value you will be able to detect more edges at the expense of
* performance.
*
* Range: [0, 0.5]
* 0.1 is a reasonable value, and allows to catch most visible edges.
* 0.05 is a rather overkill value, that allows to catch 'em all.
*
* If temporal supersampling is used, 0.2 could be a reasonable value, as low
* contrast edges are properly filtered by just 2x.
*/
#ifndef SMAA_THRESHOLD
#define SMAA_THRESHOLD 0.1
#endif



Then linking the same for depth threshold should work in theory.


uniform float DepthEdgeDetectionThreshold <
ui_type = "drag";
ui_min = 0.01; ui_max = 0.10; ui_step = 0.01;
ui_tooltip = "Depth Edge detection threshold. If SMAA misses some edges try lowering this slightly.";
ui_label = "Depth Edge Detection Threshold";
> = 0.10;

#define SMAA_DEPTH_THRESHOLD DepthEdgeDetectionThreshold


Something like that perhaps, ugh I'm probably messing this up horribly hah.
(I also remember depth buffer detection having a much lower threshold to the range of not 0.01 but 0.001 or somewhere so maybe that would be better?)

Yeah that worked, somehow.

0.1
abload.de/img/div2_013ykn2.jpg

0.01
abload.de/img/div2_03pbkmk.jpg

0.001
abload.de/img/div2_02ltkgu.jpg


pastebin.com/uLYmqtfD
Just basically took the above and rammed it in where I thought it would be, using 64 sample steps and 20 for edge since I can't really see a performance impact but this is a higher-end GPU and the above paste is mostly just for comparison purposes.


EDIT: Division 2 because that's what I happened to be playing, using Easy Anti Cheat so ReShade 4.0.2 because whitelisting and taking forever.
Online game but depth detection works in the main menu and only the main menu, again whitelisting so no commenting out the depth code and running custom compile .dll with full depth detection but for a quick comparison this works. Somehow. No idea why and how I can't really do this sort of thing it basically just worked after taking the existing threshold and I guess I managed to hook into the depth define ha ha, oh what am I doing.

Going to have to split the post as this forum thinks posting is spam, what a load of crap.

Hello Reshade community hope everyone is good.

This looks brilliant, great read of the thread, tried some of the pastebins SMAA updates and may I ask should I be applying the updates to the 4.2 Reshade?

We noticed there's a new Reshade version but no mention of any SMAA fix? Has anything been updated in regards to SMAA in the new Reshade?

We have tested some of the SMAA pastebin updates on the 4.2 version and they look much better, but unfortunately the fonts are really blurry, looks like the original Morphological Anti-Aliasing which blurs fonts the exact same.

I uploaded some screenshot comparisons if you would kindly take a look at the screen shots, I marked them 2024 version' and 4.2 version'

www.dropbox.com/s/h5y62c6qpktw3qf/RESHADE%20PICS.rar?dl=0

Just look at the difference between the 2024 and the 4.2 versions, its looks like 4.2 has about 10% AA.

May I ask does anyone know how I could achieve the same look in the 2024 screenshots and port it over to 4.2? Tried every setting you can think off, makes no difference what so ever.

Thankyou for your time and hope to speak soon.

For fonts that's the corner or edge rounding side effect which I think with the updates and using defaults it's at 20 something and you would want to reduce it and with the ReShade menu you should be able to just adjust the value with the menu or text element on screen and see a immediate difference.

For solving it I only know of using SpecialK and it's fork of ReShade 3.0.8 and a ability to set a shader and pass over UI elements as the most likely use for this feature which full on shader tracking and other D3D11 advanced utilities would be well outside the scope of official ReShade and implications in multiplayer compared to the current method of just disabling the depth buffer access.

Turn it down to 0 would thus be my recommendation with this setting. Don't learn a entirely different program and it's side effects or other complications and do the easy thing just lowering the value or outright turning it off / zeroing it.

As to the above copy and paste that should be the latest from Github with a pretty simplistic addition to take the current precision slider and target the depth buffer with SMAA instead which could help improve anti aliasing with depth or the predication setting which combines color and depth but it's disabled by default as it can also reduce anti aliasing results if depth buffer access is not available.

I am not very skilled with ReShade shaders or the internals over on Github and the project itself nor do I know why the SMAA injector or ReShade 2.x against 3.x or newer differ even with posts here trying to clarify and clear it up and from my limited knowledge I mean the core shader is SMAA or it's source file and then a ReShade shader variant and exposing some user settings and extras but you're still doing a full-screen post-process effect on the entire screen yet somehow the effect varies. I find it curious but I'm not sure why that is due to lack of skill and knowledge on the subject.


Changes to depth buffer, precision and what not doesn't seem to do much either so why do they differ and I guess a in-depth comparison against ReShade 2.x and 3.x of both the ReShade core itself and the shader would be needed.

If a change to make the shader work for 2.x would be doable and it provides identical results then that would be something in ReShade itself I suppose but then what and where, seems strange though for what this is and again a post-process effect drawn over the frame or screen after other effects and elements like the UI.

Fascinating but weird though again I'm nowhere near an expert on either the code or the shaders. Interesting to read up on and see examples of it even so but well I can't do much though I wonder what others might be able to say on this subject and a possible solution if there is a issue here, somewhere.

I really don't know what is going on here. I use the SMAA fx version with my own settings, and it is very clear that SMAA is working. Here in Bit Trip you can see that even with 4K resolution you need SMAA to make the angles look better. And it does a wonderful job. I use 32 steps and 16 diagonal. No rounding.