(H)ybrid high-(Q)uality (A)nti-(A)liasing (HQAA)

It's an optional technique like HQAACAS. It'll show up in the effects list as HQAATemporalStabilizer in versions that have it.

Long compile times are pretty normal with recent versions of HQAA. I've occasionally even seen games crash due to the length of the hitch. If it's too widespread a problem I'll reduce the default number of FXAA scan iterations (the loops in FXAA are where the compiler tends to hiccup).

Finally, version 12.0 now available. I made some pretty major improvements to the edge detection process in both SMAA and FXAA (I love it when I get random brainwaves that actually work).

Edit: The UI did need a bit of work again. 12.0.1 now up, which mostly just does a spring cleaning of the UI options to improve readability and general setup flow. As an experiment, I also reversed the order of the logical checks in the FXAA while loops (the iteration counter now checks first instead of second) since if the compiler is looking for a hint about the possible maximum duration of the loop, it's only likely to check the first logical comparison used by the loop declaration. In theory, this might shorten compile times.

Still artifacts for me. Stabilizer make picture darker for some reason.
Compiling time is ok now, i think.

Wow, this has grown a LOT, I'm really happy to see it!

I'm especially excited about you working on a temporal solution, for me the main issue with aliasing is actually not the edges but the shimmering on movement, I find the high contrast blinking to be really distracting. I'd live happily with some amount of blurring if it could catch that blinking.

The ghosting I imagine could be subdued by edge/surface detection, in a similar way to how Bilateral Comic detects edges, surfaces and sky.
Filmic Anamorph Sharpen also does something interesting with an edge mask, reducing sharpening centered around depth edges.

Perhaps you could use the a single edge/surface mask to further prevent CAS from reinforcing mesh-edge aliasing and to help filter your temporal component from surfaces and sky/long depth.

The temporal component is very basic for now - it's a literal weighted past-frame blend with no masking whatsoever. It does have some beneficial effect controlling flickering noise in the results though, so we'll see where it goes. I'd really like to get the problems with the shader on older geforces sorted out, so my attention's still pretty divided between new features and trying to fix that.

I've just had another stab at trying to make it work on older geforces (v12.1, now available). Compatibility mode now disables all alpha channel blending as I suspect the problems on those cards are being caused by it somehow. Let me know whether this works or not.

I still have same artifacts even with COMPATIBILITY MODE =1
also errors (at first only on hqaacas and temporal stabilizer, but then on all techniques)

Don't mind those, they're just compiler warnings because the new compatibility mode changes both the output and the actual function type from float4 to float3 to drop the alpha channel. Since the compiler can see a path to a float4 output from a float3 subroutine, it generates that message. In reality it will never happen because both are tied to the same preprocessor check.

I just posted version 12.2, it's another large-ish spring clean + bugfix + general attempt at fixing the problems on pascal Geforces and below. I've been reading through the Microsoft HLSL reference documentation and found a couple of ways my code might have been producing odd results in some cases.

It looks like this on a DX11 game using Unity engine. It looks exactly the same with the OpenGL game I showed previously. I too got a pascal card. 1080ti to be specific.

imgur.com/a/OJ6fXYQ

EDIT: It also makes red and green artifacts as if one of the debug options would be bleeding through even though they're all off. It doesn't really show with any sensible settings but is clear when cranking down the threshold and when the screen is dark.

imgur.com/a/PQHoYs1

Yup, I think we've collectively got the problem area identified as Geforces in the Pascal series and older. The specific engine+game combination doesn't seem to matter, it's the hardware architecture the shader doesn't seem to get along with. I'm really curious why there's a difference at this point - clearly the execution engine in older Geforces is handling something differently than most other SM3 compatible cards. If I can figure out what that difference actually IS, I should be able to fix it.

SM3 as in Shader Model 3.0? All cards supporting dx12 are at least shader model 5.0 compatible and AFAIK there are no modern GPUs that don't support the 3.0. Not sure what else that could mean?

i added latest version to shaders
cleared cache and stuff
and at first there was no artifacts, but until i checked "performance mode" after it, artifacts reappeared, and stays even when i unchecked "performance mode"

I experience these artifacts in Beamng.drive on my RTX 3060 laptop GPU since 10.0 update of HQAA. I tried clearing cache and using the compatibility mode, but it's not working. I have also noticed that when I select original buffer copy in debug it shows me the SMAA Blend Weights screen.
Here are my screenshots:
ibb.co/wzMNVkr
ibb.co/DQ6GDyr

Thanks for reporting, that gives me more info to go on. Expect another fix attempt soon.

Alright folks, the day of reckoning is here. I'm officially tired of my shader not behaving itself on other platforms. The 6700XT is coming out and the GTX 1050 Ti is going into my PC until I get the problem nailed down.

Edit: it looks so tiny...



I hope it doesn't vomit blood trying to drive my 1440p display.

So now I'm in troubleshooting mode for a different reason. ReShade is responding to all the default hotkeys except the one to open the overlay on my 1050 Ti, even after overriding the profile to change it. I tried loading a blank shading profile just in case HQAA was causing some kind of weird memory overrun, but that wasn't it.

Unfortunately, I can also report that my first impressions were that although performance tanked, HQAA appeared to be working. I might need to start collecting information about CPUs as well - maybe something is going differently on the CPU side on certain architectures. However extended testing is still pending as I need to solve this initial unresponsive ReShade overlay problem first.

Edit: fixt. The problem was just ye olde installed a new piece of hardware and didn't reboot the system. Modern drivers are supposed to be able to be loaded on the fly but we all know how well that sometimes goes. More testing now starting.

Testing Chart: GTX 1050 Ti + Ryzen 5600X

All game settings are maintained as same as 6700XT (read: unplayable on a 1050 Ti due to poor performance and should be heavily straining the card)

(Pass indicates no artifacts in the final output and all debug modes contain expected outputs)

Euro Truck Simulator 2 (DX11 64bit): Pass
Doom 3: BFG Edition (OpenGL 32bit): Pass
Witcher 3 Wild Hunt (DX11 64bit, original failure case): Pass
RAGE 2 (Vulkan, 64bit): Pass
Borderlands TPS (DX9 32bit): Pass*
Diablo III (DX9 64bit): Pass*
Satisfactory (DX12 64bit): Pass*
Subnautica (DX11 64bit): Pass

*Borderlands TPS displayed correct final output but odd behavior in the alpha normals texture. The texture is defined red-only (single channel) and when viewed in debug should definitely look red. In TPS it was strongly Cyan.

*Diablo III and Satisfactory displayed correct final output but odd behavior in the alpha normals texture. The texture was correctly in the red channel, however computed values did not appear to be correct to what was on screen (the texture was often set to a full 1.0 resulting in huge blocks of red with no definition of the scene).

Open to suggestions of different engine / API combinations that I should try. If I own the game I'll see if it works. Overall though the 1050 Ti is not helping, the final outputs are all clean even when there are oddities in the alpha normals (I'm still curious about that though - will look into it).

Edit - I stand corrected, the weird behavior in the alpha normals texture ended up being more revealing than I expected.

Version 13.0 going up, which is the product of a full-shader sanity check using the Geforce GTX 1050 Ti. There are a large number of changes, although for systems where previous versions had no issues, v13.0 will likely look very close to the same (its results may be just a touch more blurry due to removal of the inline sharpener).

Edit: updated to 13.0.1 correcting a minor bug in 13.0 (FXAA luma data was being generated too early in the technique).

Tested 13.0.1
Soulstorm have same artifacts. I've checked compatibility flag and artifacts disappeared, but reappeared after i checked "performance mode" in reshade

Also, I've tested in Wild Hunt, and it's much worse, it's NaN's again.

I have 2700x Ryzen, if it still relevant.

I have just noticed that when I use the load only enabled effects option in reshade the artifacts disappear, so they might be caused, at least for me,  by incompatibility with other shaders.