Highest Quality setting for SMAA Predication

Hi,

I am using the newest version of Reshade. Can anyone tell me what the highest quality SMAA prediction setting for Reshade is?

This is the setting I use right now (Prediction is default).

[SMAA.fx]
EdgeDetectionType=1
EdgeDetectionThreshold=0.050000
MaxSearchSteps=112
PredicationScale=0.2
MaxSearchStepsDiagonal=20
CornerRounding=100
PredicationStrength=0.400000
PredicationThreshold=0.01
DebugOutput=0

Just a reminder that predication requires depth-buffer access, and thus you'll need to use a shader like DisplayDepth.fx to both ensure depth access is possible, and also to correct it first if it's upside-down/back to front etc using the in-built preprocessor defintions under the settings tab (or directly from the ini):

RESHADE_DEPTH_INPUT_IS_UPSIDE_DOWN=0,
RESHADE_DEPTH_INPUT_IS_REVERSED=0,
RESHADE_DEPTH_INPUT_IS_LOGARITHMIC=0

I'm not sure if enabling predication when depth-buffer access is unavailable will degrade quality, certainly it will cause issues if you haven't set up the buffer correctly above.

Next for SMAA settings in general:

• EdgeDetectionThreshold, I think the author himself (or was it Ceejay?) said that anything lower than 0.10 probably isn't needed, 0.08 at the lowest. Lower than this and you'll lose details that are being anti-aliased that shouldn't be.
• Anything higher than 32 for MaxSearchSteps isn't likely to be noticeable, so you're incurring a performance penalty by going higher for no gain.
• Similarly for SearchStepsDiagonal, 16 is usually comprehensive.
• CornerRounding 100 will distort text quite badly. It is a personal preference, but usually when using rounding it's set at 10 or 20. I set it to 0.

doesn't prediction need the previous frame and depth buffer to work correctly ?Thought it was used for the temporal modes

How do I correct the depth acess please? imgur.com/a/oy13EZq

Kleio420 wrote: doesn't prediction need the previous frame and depth buffer to work correctly ?Thought it was used for the temporal modes

Hmm that's interesting, the update to SMAA on the Github page now has a description for predication possibly making the image worse but that's something I hadn't thought of as I assumed that more for the SMAA T2X or S2X modes. (And I suppose SMAA4X as well but it's using MSAA which is probably even more tricky than this temporal problem with previous frames and other data for a global injector to be able to utilize without game specific support.)

Come to think of it there's a update to the SMAA shader in some thread here too but I am unaware if it's needed with the current version of ReShade or if what it fixed is already dealt with via ReShade and improvements to depth buffer support and it's various options. (Shader update itself was depth buffer related I think?)

Good to know about the settings too, only seen a few games really being hit by a performance impact from ReShade when using less complex shader effects due to how it handles depth I think it was but that warrants some testing then in games such as Monster Hunter World and seeing how the settings affect performance there.
(Corner Rounding in particular, even for finer details and aliased edges the only thing I really notice with higher values is that it smudges text in menus.)
(Though I suppose it's the sample count that hits performance the most if predication is enabled and it comes down to the depth buffer being part of that.)

Although come to think of it the included source shader for SMAA also is really well commented and has descriptions for most of the settings although predication and a few others I believe are custom for ReShade's version and not part of the original code.

Using depth and either luma or color together, though on that note does depth even do much since in effect it doesn't appear to alter more than how the image looks like with the depth effect shader on although a recent post somewhere on this forum also mentioned it doesn't work in motion so that might be why.
(More things to eventually try and test I guess, quite a useful shader for post-process AA over having nothing at all and with a comparably small performance hit but still a ton of things I don't quite understand about it, or shaders in general I suppose but the effect of it is at least clear.)


EDIT: Ah yeah predication has it's own little list of settings now doesn't it. I don't think I've ever bothered with those but that could improve total aliasing coverage a bit further I'm assuming though there are limits to what SMAA 1x can to overall.
(Still a good option to have though and better than nothing at all for some titles not offering much of a choice for anti-aliasing what so ever.)

@Kleio420 it's predication, not prediction Though I don't know why it's called that.

It doesn't use the previous frame, it just uses the depth buffer to find edges to anti-alias that the other modes might miss. There's no temporal mode in SMAA1X.

@JBeckman the update to SMAA in these threads (and now in the Reshade github) was a fix for predication not working since the 3.X update, which Marty kindly fixed. Otherwise it should still be operating as Ceejay intended.

One thing to remember, which I'm not sure is documented, but back when Ceejay added predication he designed it to be used in combination with color for the best effect, so make sure both color AND predication are enabled. Don't set SMAA's mode to 'depth' as, as predication is already doing that.

@JBeckman
"Using depth and either luma or color together, though on that note does depth even do much since in effect it doesn't appear to alter more than how the image looks"

Using the depth buffer in predication is just another means to find edges which color or luma may miss. This is, presumably, why Ceejay intended it be used with in combination with color detection.

Finally, from memory, Ceejay said the default settings for predication are probably the best. But -- @Ceejay.dk is back now so he certainly comment on more of the above than I can