AA and Edge detection using Depth only

Hi everyone,
I've already tested SMAA with edge detect, using depth map, but it avoids too much edges, even with sensible settings. There are not enough edges detected, resulting in partial AA.

I like this idea and I've just looked for what Display Depth plugin can do. Using normal maps, I see that a depth based algorithm can detect a lot more edges than SMAA edge detection:


I can see on this image that every edge that I want to be AA are highlighted with high contrast lines.
Is it possible to modify FXAA to add an option for using normal maps (or something like or more accurate) to detect and apply AA?
It seems very accurate and, good side effect, it does not apply on game UI, so letters are not affected.

EDIT: I also found this topic: reshade.me/forum/shader-presentation/4635-mesh-edges
FXAA based on this edge detection would be nice too

What do you think? Thank you

Have you tried SMAA with pre-proccessor definitions ?

SMAA_PREDICATION 0 //[0 or 1] Enables predication which uses BOTH the color and the depth texture for edge detection.

It really depends on the game for results. But using this it combines the depth edges info with normal SMAA color edge detection. Results can be very good in some games. In others, especially those with ridiculous long render ranges it isnt always as good on the depth part, like a space game.

The SMAA predication gives access to new sliders. So it can take some experimenting to learn how they interact. Recommend be edge debug mode.

Hi,
I've already tested SMAA and the best result I've got was not enough edges detected. Only main character and some rare obvious edges were AA but other obvious edges were not. Depth is correctly detected as I'm using MXAO, DOF and Display Depth successfully, without any issue.
I'll try what you've said, but I don't expect miracle ^^

I think that the main problem is the actual edge detection and what I want to spot is that there are very good algo that can detect edges very well, as we can see in the normal map picture.

I think there isn't much to improve on. Post-processing has it limitations. Not sure how much you could tweak depth or normal map detections and make better edge anti-aliasing.

I would love more AA options. Maybe some wizard would make something magic I gonna try the FXAA hybrid one(Didnt saw it your post before).

I have big problems when there is to little resolution to resolve. Like playing my city manager game City Skylines. All those straight lines buildings have just becomes messy. Only thing that solves it is to get higher resolution. I mostly allergic to creepy crawlies, those pixel lines that pulsate. SMAA doesnt help there and FXAA works somewhat with blur.

But yeah, If just had SMAA 2x version, which requires MSAA. As I read it not possible with Reshade. But it works very well with games that has it implemented.

The thing you ask for would be super cool, wonder if it works well