(H)ybrid high-(Q)uality (A)nti-(A)liasing (HQAA)

  • lordbean
  • Topic Author
More
2 years 5 months ago #1 by lordbean (H)ybrid high-(Q)uality (A)nti-(A)liasing (HQAA) was created by lordbean
# HQAA

HQAA is a merging of FXAA, SMAA, and CAS balanced and configured to provide high visual quality anti-aliasing effect in cases where Temporal AA is not available or not desirable. It is configured to calculate the settings for each pass from two master user-controlled settings that keep the outputs of each pass balanced with each other, keeping the visual quality high whether you prefer to hold more detail in the scene or bias further towards aggressive anti-aliasing. Although it uses contrast-adaptive sharpening as its final pass, you are still free (and encouraged) to use another sharpening pass of your choice after HQAA in the shader list since the CAS pass in this shader is very low-strength (it is only designed to counteract the blur caused by the AA techniques).

HQAA requires the supporting header and texture files normally used by FXAA and SMAA. They are not included in the project because HQAA is built to work with the stock code in the respective headers for broad compatibility. Note however that HQAA configures the operating parameters of the shaders in a somewhat odd way compared to the stand-alone versions of FXAA and SMAA due to the need to minimize their overlap effects, and therefore may not be compatible with running alongside a separate pass of either AA method.
The following user(s) said Thank You: SLAMNDAN, acrox999, SilentPrayerCG, yoyas56, Valentine8342, Viper_Joe, p6kocka, Kidoki, EFermi, YF and 11 other people also said thanks.

Please Log in or Create an account to join the conversation.

  • lordbean
  • Topic Author
More
2 years 4 months ago #2 by lordbean Replied by lordbean on topic (H)ybrid high-(Q)uality (A)nti-(A)liasing (HQAA)
Here's a series of before and after images taken from various games to show off the balancing act being performed by HQAA. I used the ReShade UI "Save before and after images" function to demonstrate that each game has its internal anti-aliasing disabled (yes, even DOOM Eternal - it can be forced off using a console command). All screenshots are taken using HQAA default settings with no other shaders loaded.

imgur.com/a/6EYjkgf
The following user(s) said Thank You: Viper_Joe

Please Log in or Create an account to join the conversation.

  • Tojkar
More
2 years 4 months ago #3 by Tojkar Replied by Tojkar on topic (H)ybrid high-(Q)uality (A)nti-(A)liasing (HQAA)
What settings are you using? Somehow I couldn't get the very slight angles look as good as in your screenshots.

Please Log in or Create an account to join the conversation.

  • lordbean
  • Topic Author
More
2 years 4 months ago #4 by lordbean Replied by lordbean on topic (H)ybrid high-(Q)uality (A)nti-(A)liasing (HQAA)
The defaults. The shots are 2560x1440 though, if you have a 1080p screen that might be why.

Please Log in or Create an account to join the conversation.

  • Tojkar
More
2 years 4 months ago #5 by Tojkar Replied by Tojkar on topic (H)ybrid high-(Q)uality (A)nti-(A)liasing (HQAA)
Yeah, that could explain it. Very nice shader regardless, both this and the Ass AA. :D

Please Log in or Create an account to join the conversation.

  • lordbean
  • Topic Author
More
2 years 4 months ago #6 by lordbean Replied by lordbean on topic (H)ybrid high-(Q)uality (A)nti-(A)liasing (HQAA)
Thanks, I'm quite happy with how it turned out :)

Ever since I discovered that TAA side effects give me headaches I've been on a quest to somehow squeeze more out of the fallback options. Short of actually trying to write a whole new algorithm (no idea where I'd even start) I think I've pushed the older methods about as far as they can go at this point.

Please Log in or Create an account to join the conversation.

  • lordbean
  • Topic Author
More
2 years 4 months ago #7 by lordbean Replied by lordbean on topic (H)ybrid high-(Q)uality (A)nti-(A)liasing (HQAA)
v1.1 released. Improves overall aliasing correction compared to 1.0.

Please Log in or Create an account to join the conversation.

  • lordbean
  • Topic Author
More
2 years 4 months ago #8 by lordbean Replied by lordbean on topic (H)ybrid high-(Q)uality (A)nti-(A)liasing (HQAA)
v1.2 update released. It primarily reduces considerably the amount of blurring caused by the shader, however overall AA effect was also slightly improved as a result due to reduction in false positive detection. Overall image quality is considerably improved in v1.2 as compared to v1.1.

Please Log in or Create an account to join the conversation.

  • John Doe
More
2 years 4 months ago #9 by John Doe Replied by John Doe on topic (H)ybrid high-(Q)uality (A)nti-(A)liasing (HQAA)
Do we have an idea for this? I'm at a loss. It would certainly be interesting to compete my proven SMAA/CAS-configuration against your HQAA-solution in terms of quality and effectiveness.

18:38:14:345 [01040] | ERROR | Failed to load "C:\Program Files (x86)\Steam\steamapps\common\Tomb Raider\ReShade\Shaders\HQAA.fx":
C:\Program Files (x86)\Steam\steamapps\common\Tomb Raider\ReShade\Shaders\HQAA.fx(456, 393): preprocessor error: unterminated string literal
C:\Program Files (x86)\Steam\steamapps\common\Tomb Raider\ReShade\Shaders\HQAA.fx(2831, 156): preprocessor error: unterminated string literal

Warning: Spoiler!

Please Log in or Create an account to join the conversation.

  • lordbean
  • Topic Author
More
2 years 4 months ago - 2 years 4 months ago #10 by lordbean Replied by lordbean on topic (H)ybrid high-(Q)uality (A)nti-(A)liasing (HQAA)
HQAA can crash or fail to load properly when traditional FXAA or SMAA is also being loaded in the shader list because HQAA sets up the preprocessor defines in ways that the normal versions of either find confusing. If HQAA and FXAA and/or SMAA (the .fx files) exist in the same place, you might need to enable "Load only enabled effects" to get HQAA to compile correctly.

Updated to v1.21, which implements checks against all known used preprocessor defines for normal FXAA and SMAA to hopefully prevent any issues from arising during compile. Appears to work on my own system, remains to be seen whether this is an effective fix for everyone.

Edit - it should be noted that although it should be possible to load HQAA alongside FXAA and SMAA, I STRONGLY advise against running any further morphological techniques in combination with HQAA. HQAA is already a combo shader that is squeezing 3 separate morphological passes into the same package and is very carefully calibrated in order to keep blurring down. If you try and run more anti-aliasing in addition to HQAA, you may end up with a rampant blurring problem in the final output.
Last edit: 2 years 4 months ago by lordbean.

Please Log in or Create an account to join the conversation.

  • lordbean
  • Topic Author
More
2 years 4 months ago #11 by lordbean Replied by lordbean on topic (H)ybrid high-(Q)uality (A)nti-(A)liasing (HQAA)
Here's a quick preview of the level of quality to expect from HQAA. This is a comparison between HQAA using the shader defaults against the built-in Temporal AA in Borderlands 3.

imgsli.com/ODM3MTQ

Please Log in or Create an account to join the conversation.

  • John Doe
More
2 years 4 months ago #12 by John Doe Replied by John Doe on topic (H)ybrid high-(Q)uality (A)nti-(A)liasing (HQAA)
You're pretty swift with your updates. I appreciate that.

It took the afternoon to tinker with HQAA v1.21 and ASSMAA respectively. Since it's a compiler warning showing up, I even grabbed a former, knowingly fully functional system backup. ReShade runs smoothly in a standard minimum configuration on it, your required header files and textures included. qUINT_sharp for example - one of my favourite effects - turns out alright. All others do likewise.

Though, as soon as either of your shaders get involved, above mentioned error appears while loading on my machine. Is there a comprehensible way to debug these stated addresses in order to find our open but not closed string variables? Maybe it's as simple as my lengthy file pathes.

Let's call it a day.
[img

Please Log in or Create an account to join the conversation.

  • lordbean
  • Topic Author
More
2 years 4 months ago - 2 years 4 months ago #13 by lordbean Replied by lordbean on topic (H)ybrid high-(Q)uality (A)nti-(A)liasing (HQAA)
ASSMAA throws the same error? That's honestly quite confusing, I've never seen that one fail to compile before since it's still very similar to normal SMAA. It does suggest the culprit is something to do with the SMAA header file though - as an experiment, you could try setting up a shader library that only includes the anti-aliasing shaders and their support stuff, and see what happens. The other possibility that comes to mind is that if you've been using the same set of downloaded shaders for a long time, perhaps one or both of the actual header files for SMAA or FXAA is outdated compared to what gets pulled now, and the code in the old headers isn't compatible with HQAA.

I bought Tomb Raider on Steam to see if I can reproduce the problem on my own PC, but it seems I can't. It loads up without issues and completed the "benchmark" successfully.

This is from me playing No Man's Sky yesterday:
Warning: Spoiler!


And here's Satisfactory, an Unreal Engine 4 game... this one includes several on-the-fly reloads because I was playing around calibrating prod80's lift-gamma-gain shader:
Warning: Spoiler!


Tomb Raider:
Warning: Spoiler!
Last edit: 2 years 4 months ago by lordbean.

Please Log in or Create an account to join the conversation.

  • John Doe
More
2 years 4 months ago #14 by John Doe Replied by John Doe on topic (H)ybrid high-(Q)uality (A)nti-(A)liasing (HQAA)
It's a mess, isn't it. Being a web-installer, ReShade pulls his packages from GitHub regularly. But three points have caused me to take a step back and reconsider:

- There's potential for sure in your shaders. We may be certain, some people have tried them with a good success. At the moment the error seems to be a singular problem of mine apparently.

- Your logs, in particular the third one, are eye openers. There's DirectX 12 active on your system.

- In ReShade's requirements, the compatibility in confirmed for Windows 7 upwards. However, it's also said: "[...] Some users reported issues with other versions than Windows 10, so this one is highly recommended. [...]" ReShade is compiled bleeding edge, I could bet on.

Though my gaming rig still shows up on ancient Windows 7 unfortunately. In this respect you could be right. Windows 10 on the other hand is something that will never happen here. I'm about to migrate this last machine of my park to a proper Linux anyway. As soon as it's done, we'll meet again and enjoy your work. Until then, thank you very much for your patience and support.

Please Log in or Create an account to join the conversation.

  • lordbean
  • Topic Author
More
2 years 4 months ago #15 by lordbean Replied by lordbean on topic (H)ybrid high-(Q)uality (A)nti-(A)liasing (HQAA)
You're quite welcome - if I had a Windows 7 test environment somewhere, I'd try and chase this further, but unfortunately I have no systems left that aren't running Windows 10. I was a slow adopter (we all know how buggy new M$ releases always are) but in the end I had to because I'm primarily a gamer, and, well... DirectX 12. I have to say though, despite some of the invasive phone-home bits in Windows 10, I have honestly come to consider it a superior OS to Windows 7. It's optimized surprisingly well and despite my misgivings, honestly seems to be more stable than Windows 7 was (which is saying quite a bit as I loved Windows 7).

Windows 11 isn't on my radar at the moment. Even if I was to run out and buy an Alder Lake platform I'd still be putting Windows 10 on it.

Please Log in or Create an account to join the conversation.

  • lordbean
  • Topic Author
More
2 years 4 months ago #16 by lordbean Replied by lordbean on topic (H)ybrid high-(Q)uality (A)nti-(A)liasing (HQAA)
v1.22 released. No major changes to AA effect, but side effect blurring is very slightly reduced compared to previous versions.

Please Log in or Create an account to join the conversation.

  • lordbean
  • Topic Author
More
2 years 4 months ago #17 by lordbean Replied by lordbean on topic (H)ybrid high-(Q)uality (A)nti-(A)liasing (HQAA)
v1.3 released. I have somehow (!) convinced the FXAA passes to be even less blurry, quite considerably this time. The final CAS pass has also been fairly weakened and so should play nicer with other post process sharpening techniques.

Please Log in or Create an account to join the conversation.

  • lordbean
  • Topic Author
More
2 years 4 months ago - 2 years 4 months ago #18 by lordbean Replied by lordbean on topic (H)ybrid high-(Q)uality (A)nti-(A)liasing (HQAA)
HQAA v1.5 is now available, and is likely to be the last major update the shader will receive as I don't think there's much (if any) room left to squeeze SMAA and FXAA any further. 1.5 only causes very minimal blurring even compared to 1.41 and has a significantly higher success rate at detecting and correcting real aliasing. HQAAv1.5 should be able to produce image quality roughly on par with a reasonably well-calibrated temporal AA pass in most games (though a fine-tuned TAA routine is still likely to be better).

Comparison Showcase
Last edit: 2 years 4 months ago by lordbean.
The following user(s) said Thank You: Spacellary

Please Log in or Create an account to join the conversation.

  • Tojkar
More
2 years 4 months ago #19 by Tojkar Replied by Tojkar on topic (H)ybrid high-(Q)uality (A)nti-(A)liasing (HQAA)
One question about a slightly off-topic thing. What is that talk about fine-tuned Godly TAA (pun intended)? All I've ever seen is ungodly amount of aberrations(also a pun intended) that happens when slowly turning the viewpoint. I've never seen a good or even adequate implementation on any game or managed to make good settings myself with the TAA shader in Reshade.

As for on-topic things.. HQAA has become a my default shader for AA. Getting better with each update (although haven't tested the latest yet).
The following user(s) said Thank You: lordbean

Please Log in or Create an account to join the conversation.

  • lordbean
  • Topic Author
More
2 years 4 months ago #20 by lordbean Replied by lordbean on topic (H)ybrid high-(Q)uality (A)nti-(A)liasing (HQAA)

One question about a slightly off-topic thing. What is that talk about fine-tuned Godly TAA (pun intended)? All I've ever seen is ungodly amount of aberrations(also a pun intended) that happens when slowly turning the viewpoint.
 
TAA is actually a very difficult routine to balance so that it produces desirable output without also causing a lot of side effects (which you've apparently noticed). The best example of a game with good TAA calibration I can point to is Borderlands 3 - it rarely ever causes shimmering or visible ghosting in that game. Somewhat oddly, Fallout 4's TAA implementation is probably the runner-up in my mind, despite being an older title - it has a bit of visible ghosting during fast movement but rarely ever shimmers.
The following user(s) said Thank You: Tojkar

Please Log in or Create an account to join the conversation.

We use cookies
We use cookies on our website. Some of them are essential for the operation of the forum. You can decide for yourself whether you want to allow cookies or not. Please note that if you reject them, you may not be able to use all the functionalities of the site.
Ok Decline
More information