SMAA question

Hi, I've been using SweetFX/reshade for quite some time now and most of the options are fairly straight forward to me, but the SMAA_DEPTH_THRESHOLD has always been troubling. I've just left it on default for the most part, but a couple of the Witcher 3 presets had it set higher.

I know it's tied together with the depth of a scene, but I have no clue on whether high values will produce better antialiased images (depth wise?) or is lower the better much like the SMAA_THRESHOLD. Also are the improvements really noticeable? I haven't been able to tell in Witcher 3, maybe the forest/tree layers look better?

Any insight would be greatly appreciated.

Thanks.

I found this while searching for an answer to another smaa question and from my readings I believe to have gleaned at least an answer to your question. I know I've read in several places that SMAA_DEPTH_THRESHOLD can be set extremely low and that unlike luma and color edge detection, depth detection is not prone to errors so it is best to leave it at the default low setting of 0.001whereas Crosire (reshade creator) in his notes mentions ideal range for color and luma is between 0.08 and 0.12. ... lower values for smaa_threshold can cause the shader to detect false edges. I'm assuming you meant #define SMAA_PREDICATION_THRESHOLD?

As for define SMAA_PREDICATION_SCALE and #define SMAA_PREDICATION_STRENGTH hopefully someone more knowledgeable will come along and enlighten us. and perhaps answer my question as well!

Which is: "#define SMAA_PREDICATION 0 //[0:1] //-Enables predication which uses BOTH the color and the depth texture for edge detection to more accurately detect edges"

Great! does that mean if I set #define SMAA_EDGE_DETECTION 1for luma detection and #define SMAA_PREDICATION 1, does that mean i can have all 3: luma, edge, and depth working together in a triforce of antialiasing? Would this not be the most effective settings to have?

Sorry for hijacking your thread but you can't be shy when seeking enlightenment

Thanks for the reply. I saw that post when I was searching through the forums. It kind of answered my question, but I'm still unsure on how it's utilized and how the value differences affect in game image quality. I think I might just leave it on 0.025 for now.

Golgotha wrote: Which is: "#define SMAA_PREDICATION 0 //[0:1] //-Enables predication which uses BOTH the color and the depth texture for edge detection to more accurately detect edges"

Great! does that mean if I set #define SMAA_EDGE_DETECTION 1for luma detection and #define SMAA_PREDICATION 1, does that mean i can have all 3: luma, edge, and depth working together in a triforce of antialiasing? Would this not be the most effective settings to have?

I vaguely remember asking CJ this same question a while ago. CJ can correct, but I think this is the case:

The description "Enables predication which uses BOTH the color and the depth texture" is a little misleading. It presumes you're using the default color detection method already, so enabling predication gives you color and depth. The option itself only enables depth. You can use depth and luma, or depth and color, not all three.

And, as far as I'm aware, color is the better method between that and luma. So to get the best edge detection, leave it on color and add depth -- if the depth buffer is available, which it often isn't -- as usual, enable the setting to view depth-buffer in-game and test it works before enabling Predication, otherwise you add unnecessary overhead for no return.

CJ should probably update that description, as I've seen a few profiles on sfxthelazy.net where the author has enabled luma and depth on the presumption they're using all three, when in fact the result is likely worse than if they left edge detection on color.

Thanks for the reply; another piece of the smaa puzzle is revealed! I find it kind of strange that I so rarely see predication enabled in presets and though it is probably the most used shader, its settings and how they fit into the performance vs quality equation seem to be shrouded in mystery. Whenever I have a question like this it seems I can always find out with a couple of google searches but I've come up empty on this one.

Actually I was about to write this big thing explaining what Ceejay.DK said about prediction and edge detection but I actually just found it buried in the SweetFX reddit

Here is direct copy and paste:

CeeJayDKSweetFX Developer[M] 1 point 4 months ago
You are correct Color edge detection is slightly better than Luma Edge detection (which only detects edges from a monochrome perspective).
Luma edge detection is a tiny bit faster on Nvidia cards but on AMD cards the color edge detection is actually the faster one.
I suggest Color edge detection which is also the default.
In SweetFX 2.0 depth edge detection is also supported in games that reshade provides a working depth buffer view for.
Depth buffer detection only picks up on edges to objects and not any edges in foliage (or other alpha textures) or in textures - which can be both good and bad depending on the game and your preference. But it has no or almost no false positives (where SMAA thinks there is an edge when there really isn't) and it does not process the UI, so text is unharmed.
And then there is the predication mode which is a hybrid of depth edge detection and color edge detection - it uses both to find edges, to ensure it caught (almost) all the edges.

Here is the actual thread: www.reddit.com/r/sweetfx/comments/2yfhw7/edge_detection/

Hopefully this helps you out a little:

And about the Threshold. Basically there is a point of diminishing returns for all those values. The Stock ones have been deemed good enough for most applications but you can increase all the values if are noticing missed spots. I would definitely use prediction first though if you are noticing missed lines and the like but you can increase the values too.

I use:

#define SMAA_THRESHOLD 0.08 //[0.05:0.20] //-Edge detection threshold. If SMAA misses some edges try lowering this slightly. I prefer between 0.08 and 0.12.
#define SMAA_MAX_SEARCH_STEPS 25 //[0:98] //-Determines the radius SMAA will search for aliased edges
#define SMAA_MAX_SEARCH_STEPS_DIAG 16 //[0:16] //-Determines the radius SMAA will search for diagonal aliased edges
#define SMAA_CORNER_ROUNDING 25 //[0:100] //-Determines the percent of antialiasing to apply to corners. 0 seems to affect fine text the least so it's the default.

Basically I put some AA on corners and increased the other values slightly. It effects text more but it's a non-issue in the game I applied it too.
I used to go all out and use the Ultra-settings which were like 0.05 threshhold max all other values (50 corner rounding) but I no longer do that because there is a point where it no longer makes a difference really.

And people don't use prediction in presets because:
1: they don't know what it does.
2: they see no improvement.
3: It doesn't work online due to no depth-buffer access.
4. (I think!) Sweetfx+Reshade cannot access Log-depth still so It has no access(and thus no effect) in games like GTA V and The Witcher 3.

Excellent thread that explained just about everything, except for providing the best quality values for 2 settings:
#define SMAA_PREDICATION_SCALE 2.0
#define SMAA_PREDICATION_STRENGTH 0.4

MonarchX wrote: Excellent thread that explained just about everything, except for providing the best quality values for 2 settings:
#define SMAA_PREDICATION_SCALE 2.0
#define SMAA_PREDICATION_STRENGTH 0.4

the shader file explains tons, and a google search of the word predication might hint at what its intended use is as to how these effect line detection you would need to run smaa in debug mode and change settings based on that unless you understand the code using debug is next best thing seeing is believing ;P