(H)ybrid high-(Q)uality (A)nti-(A)liasing (HQAA)

More
2 weeks 3 days ago #1 by lordbean
# HQAA

HQAA is a merging of FXAA, SMAA, and CAS balanced and configured to provide high visual quality anti-aliasing effect in cases where Temporal AA is not available or not desirable. It is configured to calculate the settings for each pass from two master user-controlled settings that keep the outputs of each pass balanced with each other, keeping the visual quality high whether you prefer to hold more detail in the scene or bias further towards aggressive anti-aliasing. Although it uses contrast-adaptive sharpening as its final pass, you are still free (and encouraged) to use another sharpening pass of your choice after HQAA in the shader list since the CAS pass in this shader is very low-strength (it is only designed to counteract the blur caused by the AA techniques).

HQAA requires the supporting header and texture files normally used by FXAA and SMAA. They are not included in the project because HQAA is built to work with the stock code in the respective headers for broad compatibility. Note however that HQAA configures the operating parameters of the shaders in a somewhat odd way compared to the stand-alone versions of FXAA and SMAA due to the need to minimize their overlap effects, and therefore may not be compatible with running alongside a separate pass of either AA method.
The following user(s) said Thank You: Covid19, Judge_K

Please Log in or Create an account to join the conversation.

More
1 week 6 days ago #2 by lordbean
Here's a series of before and after images taken from various games to show off the balancing act being performed by HQAA. I used the ReShade UI "Save before and after images" function to demonstrate that each game has its internal anti-aliasing disabled (yes, even DOOM Eternal - it can be forced off using a console command). All screenshots are taken using HQAA default settings with no other shaders loaded.

imgur.com/a/6EYjkgf

Please Log in or Create an account to join the conversation.

More
1 week 5 days ago #3 by Tojkar
What settings are you using? Somehow I couldn't get the very slight angles look as good as in your screenshots.

Please Log in or Create an account to join the conversation.

More
1 week 5 days ago #4 by lordbean
The defaults. The shots are 2560x1440 though, if you have a 1080p screen that might be why.

Please Log in or Create an account to join the conversation.

More
1 week 5 days ago #5 by Tojkar
Yeah, that could explain it. Very nice shader regardless, both this and the Ass AA. :D

Please Log in or Create an account to join the conversation.

More
1 week 5 days ago #6 by lordbean
Thanks, I'm quite happy with how it turned out :)

Ever since I discovered that TAA side effects give me headaches I've been on a quest to somehow squeeze more out of the fallback options. Short of actually trying to write a whole new algorithm (no idea where I'd even start) I think I've pushed the older methods about as far as they can go at this point.

Please Log in or Create an account to join the conversation.

More
1 week 4 days ago #7 by lordbean
v1.1 released. Improves overall aliasing correction compared to 1.0.

Please Log in or Create an account to join the conversation.

More
1 week 3 days ago #8 by lordbean
v1.2 update released. It primarily reduces considerably the amount of blurring caused by the shader, however overall AA effect was also slightly improved as a result due to reduction in false positive detection. Overall image quality is considerably improved in v1.2 as compared to v1.1.

Please Log in or Create an account to join the conversation.

More
1 week 2 days ago #9 by John Doe
Do we have an idea for this? I'm at a loss. It would certainly be interesting to compete my proven SMAA/CAS-configuration against your HQAA-solution in terms of quality and effectiveness.

18:38:14:345 [01040] | ERROR | Failed to load "C:\Program Files (x86)\Steam\steamapps\common\Tomb Raider\ReShade\Shaders\HQAA.fx":
C:\Program Files (x86)\Steam\steamapps\common\Tomb Raider\ReShade\Shaders\HQAA.fx(456, 393): preprocessor error: unterminated string literal
C:\Program Files (x86)\Steam\steamapps\common\Tomb Raider\ReShade\Shaders\HQAA.fx(2831, 156): preprocessor error: unterminated string literal

Warning: Spoiler!

Please Log in or Create an account to join the conversation.

More
1 week 2 days ago - 1 week 2 days ago #10 by lordbean
HQAA can crash or fail to load properly when traditional FXAA or SMAA is also being loaded in the shader list because HQAA sets up the preprocessor defines in ways that the normal versions of either find confusing. If HQAA and FXAA and/or SMAA (the .fx files) exist in the same place, you might need to enable "Load only enabled effects" to get HQAA to compile correctly.

Updated to v1.21, which implements checks against all known used preprocessor defines for normal FXAA and SMAA to hopefully prevent any issues from arising during compile. Appears to work on my own system, remains to be seen whether this is an effective fix for everyone.

Edit - it should be noted that although it should be possible to load HQAA alongside FXAA and SMAA, I STRONGLY advise against running any further morphological techniques in combination with HQAA. HQAA is already a combo shader that is squeezing 3 separate morphological passes into the same package and is very carefully calibrated in order to keep blurring down. If you try and run more anti-aliasing in addition to HQAA, you may end up with a rampant blurring problem in the final output.
Last edit: 1 week 2 days ago by lordbean.

Please Log in or Create an account to join the conversation.

More
1 week 1 day ago #11 by lordbean
Here's a quick preview of the level of quality to expect from HQAA. This is a comparison between HQAA using the shader defaults against the built-in Temporal AA in Borderlands 3.

imgsli.com/ODM3MTQ

Please Log in or Create an account to join the conversation.

More
1 week 13 hours ago #12 by John Doe
You're pretty swift with your updates. I appreciate that.

It took the afternoon to tinker with HQAA v1.21 and ASSMAA respectively. Since it's a compiler warning showing up, I even grabbed a former, knowingly fully functional system backup. ReShade runs smoothly in a standard minimum configuration on it, your required header files and textures included. qUINT_sharp for example - one of my favourite effects - turns out alright. All others do likewise.

Though, as soon as either of your shaders get involved, above mentioned error appears while loading on my machine. Is there a comprehensible way to debug these stated addresses in order to find our open but not closed string variables? Maybe it's as simple as my lengthy file pathes.

Let's call it a day.

Please Log in or Create an account to join the conversation.

More
1 week 4 hours ago - 1 week 3 hours ago #13 by lordbean
ASSMAA throws the same error? That's honestly quite confusing, I've never seen that one fail to compile before since it's still very similar to normal SMAA. It does suggest the culprit is something to do with the SMAA header file though - as an experiment, you could try setting up a shader library that only includes the anti-aliasing shaders and their support stuff, and see what happens. The other possibility that comes to mind is that if you've been using the same set of downloaded shaders for a long time, perhaps one or both of the actual header files for SMAA or FXAA is outdated compared to what gets pulled now, and the code in the old headers isn't compatible with HQAA.

I bought Tomb Raider on Steam to see if I can reproduce the problem on my own PC, but it seems I can't. It loads up without issues and completed the "benchmark" successfully.

This is from me playing No Man's Sky yesterday:
Warning: Spoiler!


And here's Satisfactory, an Unreal Engine 4 game... this one includes several on-the-fly reloads because I was playing around calibrating prod80's lift-gamma-gain shader:
Warning: Spoiler!


Tomb Raider:
Warning: Spoiler!
Last edit: 1 week 3 hours ago by lordbean.

Please Log in or Create an account to join the conversation.

More
6 days 15 hours ago #14 by John Doe
It's a mess, isn't it. Being a web-installer, ReShade pulls his packages from GitHub regularly. But three points have caused me to take a step back and reconsider:

- There's potential for sure in your shaders. We may be certain, some people have tried them with a good success. At the moment the error seems to be a singular problem of mine apparently.

- Your logs, in particular the third one, are eye openers. There's DirectX 12 active on your system.

- In ReShade's requirements, the compatibility in confirmed for Windows 7 upwards. However, it's also said: "[...] Some users reported issues with other versions than Windows 10, so this one is highly recommended. [...]" ReShade is compiled bleeding edge, I could bet on.

Though my gaming rig still shows up on ancient Windows 7 unfortunately. In this respect you could be right. Windows 10 on the other hand is something that will never happen here. I'm about to migrate this last machine of my park to a proper Linux anyway. As soon as it's done, we'll meet again and enjoy your work. Until then, thank you very much for your patience and support.

Please Log in or Create an account to join the conversation.

More
6 days 1 hour ago #15 by lordbean
You're quite welcome - if I had a Windows 7 test environment somewhere, I'd try and chase this further, but unfortunately I have no systems left that aren't running Windows 10. I was a slow adopter (we all know how buggy new M$ releases always are) but in the end I had to because I'm primarily a gamer, and, well... DirectX 12. I have to say though, despite some of the invasive phone-home bits in Windows 10, I have honestly come to consider it a superior OS to Windows 7. It's optimized surprisingly well and despite my misgivings, honestly seems to be more stable than Windows 7 was (which is saying quite a bit as I loved Windows 7).

Windows 11 isn't on my radar at the moment. Even if I was to run out and buy an Alder Lake platform I'd still be putting Windows 10 on it.

Please Log in or Create an account to join the conversation.

More
5 days 16 hours ago #16 by lordbean
v1.22 released. No major changes to AA effect, but side effect blurring is very slightly reduced compared to previous versions.

Please Log in or Create an account to join the conversation.

More
4 days 14 hours ago #17 by lordbean
v1.3 released. I have somehow (!) convinced the FXAA passes to be even less blurry, quite considerably this time. The final CAS pass has also been fairly weakened and so should play nicer with other post process sharpening techniques.

Please Log in or Create an account to join the conversation.