SMAA ignores Preprocessor

SMAA ignores ReShade's global preprocessor settings with regard to both Depth Edge detection and Depth Predication of Color and Luma detection as far as I can tell. I can't provide a screenshot at the moment, but essentially the Depth Buffer was upside down, so I set the appropriate preprocessor value. That said, SMAA's debug view shows no change while DisplayDepth showed it correctly, and SMAA didn't appear to be effecting the (correct) pixels at all, suggesting the error is not rooted in the debug view itself.

Now, I can fix this for Depth Edge only by myself - by flipping the texcoord along the y axis based on ReShade's preprocessor setting - but that doesn't work for Color and Luma with Predication, as the texcoord is shared, and breaks those modes entirely. Near as I can tell, fixing the issue would require major rewrite in either the way ReShade exposes the Depth Buffer to shaders (with preprocessor settings pre-applied to the buffer, not only in the linearized form) or in SMAA's original header (to have a separate texcoord definition for depth from the onset.)

While the second is obviously more achievable, I wanted to make sure I'm not simply overlooking something that could more easily address the issue.

Tested game: Pillars of Eternity (Force-ran in DX11 through Unity's command line, and Advanced Depth Detection to select correct buffer)
ReShade version: 4.2.1
Latest shader(s) revisions

Discussion going on about this (and related quality) right now here:
reshade.me/forum/shader-discussion/5203-smaa

But yes, it is broken in the SMAA.fx currently on github. Here is a fixed version: pastebin.com/KRjDzFgG

This is linked from this post here in the thread above: reshade.me/forum/shader-discussion/5203-smaa#33192

Since you clearly also like to play with predication, would be interested in your results with other games and adding your experience to the thread above.

Interesting. It will need to be updated to account for some of the changes crosire made (eliminating the extraneous sampler for example) but has anyone attempted to submit a pull request with these (or similar) changes since?

And as far as the variance in those SMAA implementations it probably has to do with a fix that was integrated into ReShade for SMAA in 2016 that never made it into the official SMAA repository on GitHub before iryoku moved on (which SweetFX and other implementations pulled from) where detection was done in an inconsistent manner.

My experience with predication is such that rather than relying on the UI minimums and maximums I lower the depth based threshold to the lowest non-destructive value manually and make the color detection extremely conservative as to preserve text edges and UI corners. Predication works as intended, but only when the depth threshold for it is pushed below the minimum UI allowed (for the games I've used it on.)

Fixed: github.com/crosire/reshade-shaders/commi...6b843a399339e77f2259 . You can update your shaders with the setup tool.