(H)ybrid high-(Q)uality (A)nti-(A)liasing (HQAA)

  • lordbean
  • Topic Author
More
2 years 2 months ago #201 by lordbean Replied by lordbean on topic (H)ybrid high-(Q)uality (A)nti-(A)liasing (HQAA)
Debanding as in debanding - it looks for color banding in the image and tries to fix it, just like Deband.fx.

The point of having the optional features present in HQAA is that it saves some performance to be able to run commonly-used techniques from a single pass inside a shader that's already running anyway. It isn't really part of AA, yes, but that's why it's an optional feature. Everything in HQAA that isn't explicitly part of anti-aliasing can still be completely removed from the compiled shader by setting HQAA_ENABLE_OPTIONAL_TECHNIQUES to 0.

Please Log in or Create an account to join the conversation.

  • lordbean
  • Topic Author
More
2 years 2 months ago #202 by lordbean Replied by lordbean on topic (H)ybrid high-(Q)uality (A)nti-(A)liasing (HQAA)
Standing version is currently 17.2.9. Since my last version update posted here, I've made some performance optimizations (mostly in the optional effects pass although there were a couple in the anti-aliasing code) and have additionally devised a fairly effective method of minimizing contrast washout when using the brightness gain optional feature (available as a checkbox in the config for it).

Please Log in or Create an account to join the conversation.

  • lordbean
  • Topic Author
More
2 years 2 months ago #203 by lordbean Replied by lordbean on topic (H)ybrid high-(Q)uality (A)nti-(A)liasing (HQAA)
v17.2.12 is now current. Optional debanding feature has been slightly better-optimized and now scans for banding using 3 different random angles per pixel. Optional brightness booster's Contrast Washout Correction feature (mentioned above) has also been significantly improved by normalizing the saturation of the result pixel as well as the luma, and has been very successful at minimizing the appearance of contrast loss (or "airy" look) in the tests I have done.

Please Log in or Create an account to join the conversation.

  • lordbean
  • Topic Author
More
2 years 2 months ago #204 by lordbean Replied by lordbean on topic (H)ybrid high-(Q)uality (A)nti-(A)liasing (HQAA)
version 18.0 beta available (change to the experimental branch on github if interested). I overhauled the whole shader to remove every wrapper function that wasn't really necessary. This means there's less overall code, and it should theoretically be somewhat faster than v17. I've got it working properly on my own PC, but since this is a major overhaul, I can't guarantee it won't cause artifacts in some situations. If you try it and experience a problem with it, I'd like to hear about it so that I can get it debugged ASAP.

Please Log in or Create an account to join the conversation.

  • lordbean
  • Topic Author
More
2 years 2 months ago #205 by lordbean Replied by lordbean on topic (H)ybrid high-(Q)uality (A)nti-(A)liasing (HQAA)
v18 beta is likely close to ready for full release. I've just completed a full-shader optimization audit and it's still running normally on my PC. Unfortunately I'm currently unable to test whether HDR support is working correctly (away from my home setup). If you've used version 18 beta with an HDR game, I'd like to know whether it worked correctly or not.

Please Log in or Create an account to join the conversation.

  • Illuzio
More
2 years 2 months ago - 2 years 2 months ago #206 by Illuzio Replied by Illuzio on topic (H)ybrid high-(Q)uality (A)nti-(A)liasing (HQAA)
Thanks for creating this shader, it's really good !
I have a problem in the game Dishonored, I can't use it (I checked and the problem exists also on older version of HQAA).

Here is the error message:
imgur.com/FZwG8AW
(Dishonored is DirectX9, 32 bits)
Last edit: 2 years 2 months ago by Illuzio.

Please Log in or Create an account to join the conversation.

  • lordbean
  • Topic Author
More
2 years 2 months ago - 2 years 2 months ago #207 by lordbean Replied by lordbean on topic (H)ybrid high-(Q)uality (A)nti-(A)liasing (HQAA)
I'll install Dishonored to check it on my own system, but to my knowledge there should not be an issue with HQAA in this title. It works in Borderlands 2 and Borderlands TPS, which are also 32-bit DX9 titles.

What could be happening, however, is an incompatibility with another shader in your list. If you start Dishonored with HQAA being the only shader in the library (just make a new folder and drop the other ones into it temporarily to try this), does it still throw an error? If not, you can also try taking the opportunity to disable all the additional features in HQAA you don't use, then try putting your normal shaders back into the list and see if there's still a problem.

Update: I can confirm HQAA is compatible with Dishonored using both legacy 4.9.1 and 5.0.2 DLLs. HQAA is an almost piggishly large shader though (due to integration of several other medium-large techniques as a single one) and this isn't the first time I've seen a weird incompatibility pop up with some other shader.

ReShade 4.9.1

ReShade 5.0.2
Last edit: 2 years 2 months ago by lordbean.

Please Log in or Create an account to join the conversation.

  • lordbean
  • Topic Author
More
2 years 2 months ago #208 by lordbean Replied by lordbean on topic (H)ybrid high-(Q)uality (A)nti-(A)liasing (HQAA)
HQAA v18.0 is now released (no longer beta) and I have just updated to 18.0.1 which adds a couple of compatibility checks in the texture declarations running on a hunch what might be wrong in Illuzio's case.

Please Log in or Create an account to join the conversation.

  • EFermi
More
2 years 2 months ago - 2 years 2 months ago #209 by EFermi Replied by EFermi on topic (H)ybrid high-(Q)uality (A)nti-(A)liasing (HQAA)
Transparency Antialiasing feature is still nowhere near even a"decent" level. 1-2-3 pixel-sized dots in the middle of textures are not affected.
 
Standart AA (multisampling, no TSAA):  imgur.com/2ricvLb  
HQAA (almost no difference from MSAA):  imgur.com/NNYfj3K  
FXAA (decent, not 100% coverage, but at least it gives some result, but I don't like it for misinterpreting edge pixels quite often, causing pixel-sized bleed-ins/outs):  imgur.com/l8wJZzU  

Of course it is understandable that it may be difficult to produce high quality results without messing up FXAA-style, but corrently transparence AA (TSAA) is virtually nonexistent.

A brief example of close to how I want it to look like:  imgur.com/VEkuNTV and  imgur.com/zOCnCpK
(Achieved using 8K resolution via GeDoSaTo + in-game FXAA for smoothness ( imgur.com/8TOlDpc ). With all honesty, is still looks like $h1t. Even FXAA did a better job. With 100% scale it is possible to still see pixel-sized aliased dots here and there, mostly because GeDoSaTo's lanczos scaler, but also because classic 2x normal grid supersampling is not the best AA mode ever anyway. If I had an nVidia GPU I could've switched on 8x transparency supersampling via nVidia Inspector for better demonstration, but since AMD cannot into proper TSAA in DX9, this is the best I can show for an example. Maybe should've chosen a better suited game for such demonstration, but whatever.)
Last edit: 2 years 2 months ago by EFermi.

Please Log in or Create an account to join the conversation.

  • EFermi
More
2 years 2 months ago #210 by EFermi Replied by EFermi on topic (H)ybrid high-(Q)uality (A)nti-(A)liasing (HQAA)
Here are some examples from Crysis 3:
8x SSAA (the goal for every AA algo to achieve and what I wish for):  imgur.com/VOfvrFc
SMAA T4x (despite having a temporal solution, it still fails to catch 'em all 1-pixel dots and lines):  imgur.com/DHhfIcx
SMAA 1x:  imgur.com/oDjZXrZ
HQAA (ultra preset, virtually no difference from SMAA in terms of transparency processing):  imgur.com/5eD2ZEQ

Please Log in or Create an account to join the conversation.

  • lordbean
  • Topic Author
More
2 years 2 months ago #211 by lordbean Replied by lordbean on topic (H)ybrid high-(Q)uality (A)nti-(A)liasing (HQAA)
SSAA is always going to produce better results than any other method. Period, end of story. The rough idea behind every anti-aliasing method in existence (both per-frame and temporal) is to try to replicate a supersampling effect without actually doing any supersampling - because supersampling is hugely expensive.

I think you might also have missed the part where I said I'm not an experienced programmer (something I've mentioned several times now). HQAA isn't a new method of doing anti-aliasing. It has modifications you can't find in the original routines, true enough, but at its heart it's still running SMAA followed FXAA.

What you seem to be asking for is honestly more along the lines of NVIDIA DLSS.
The following user(s) said Thank You: EFermi

Please Log in or Create an account to join the conversation.

  • Illuzio
More
2 years 2 months ago #212 by Illuzio Replied by Illuzio on topic (H)ybrid high-(Q)uality (A)nti-(A)liasing (HQAA)
I tried with HQAA (your new version) alone and without other shader, and the problem is the same. I wonder if it's not the depth buffer because Dishonored needs special settings.

Here is the error (same as last time):
imgur.com/KuqREI5
Depth settings:
imgur.com/0tw0jmS

Please Log in or Create an account to join the conversation.

  • lordbean
  • Topic Author
More
2 years 2 months ago - 2 years 2 months ago #213 by lordbean Replied by lordbean on topic (H)ybrid high-(Q)uality (A)nti-(A)liasing (HQAA)
Hmm. In theory depth buffer settings should not affect HQAA, because there is no use of it in the shader - let me see if I can reproduce that on my end. If not I can at least look at the batch of lines it seems to be stuck on and see if I can figure out a way to work around it.

Edit: I couldn't reproduce it with ReShade 5.0.2 (haven't tried 4.9.1 yet). Interestingly, the lines it's referring to are part of the optional debanding effect, but I don't see how a divide by zero is possible with the code being the way it is there. Try opening the preset file for the game (by default, ReshadePreset.ini in the game's folder) and place "HQAA_OPTIONAL_DEBAND=0" (without quotes) in the preprocessor definitions field right at the start of the file. If it still fails to compile, the actual error being thrown is probably a symptom rather than a cause.
Last edit: 2 years 2 months ago by lordbean.

Please Log in or Create an account to join the conversation.

  • lordbean
  • Topic Author
More
2 years 2 months ago - 2 years 2 months ago #214 by lordbean Replied by lordbean on topic (H)ybrid high-(Q)uality (A)nti-(A)liasing (HQAA)
On closer inspection I actually found a number of problems with how the debanding effect in HQAA ran compared to the technique in Deband.fx. I have pushed v18.0.5 which changes HQAA's debanding to run more similarly to the technique in the Deband.fx file. I'm not sure if this will have any effect on the problem you're experiencing, but at least the debanding feature should work better now.

18.0.6 pushed, which removes the optional debanding feature entirely. On further inspection of how the profiles were actually behaving followed by a lot of struggling with it, I've determined it doesn't fit well inside HQAA. There are probably more advanced debanding shaders out there anyway.
Last edit: 2 years 2 months ago by lordbean.

Please Log in or Create an account to join the conversation.

  • lordbean
  • Topic Author
More
2 years 2 months ago #215 by lordbean Replied by lordbean on topic (H)ybrid high-(Q)uality (A)nti-(A)liasing (HQAA)
Version 18.1 is now available. It doesn't change anything visually, but it separates the hysteresis calculations into a new pass that runs after FXAA. This could be slightly slower on average, but it should help HQAA to compile successfully on old or very low-end GPUs (the compiled number of instructions in the FXAA pixel shader when it was doing hysteresis was greater than the HLSL specification guaranteed minimum).

Please Log in or Create an account to join the conversation.

  • EFermi
More
2 years 2 months ago - 2 years 2 months ago #216 by EFermi Replied by EFermi on topic (H)ybrid high-(Q)uality (A)nti-(A)liasing (HQAA)
Well then the solution is simple - we must have a DLSS(-like) shader or implementator similar to what LosslesScaling or MagPie did for FSR. Even though DLSS itself is proprietary, there's XeSS from Intel and very likely much more I'm not aware of. 

Also, as I said, FXAA produced passable results on transparencies, it just wasn't accurate enough on edges, while HQAA as well as SMAA has the exact opposite issue - great on edges, sucks on transparencies. 
Last edit: 2 years 2 months ago by EFermi.

Please Log in or Create an account to join the conversation.

  • lordbean
  • Topic Author
More
2 years 2 months ago #217 by lordbean Replied by lordbean on topic (H)ybrid high-(Q)uality (A)nti-(A)liasing (HQAA)
Shrug, as I mentioned, my level of expertise doesn't currently include actually understanding the math being performed to fix the jagged edges. I've found a lot of ways to help the code detect edges, and even some clever-ish (I think) tricks to keep artifacting down, but the processes used by SMAA beyond the edge detection step are still rather beyond me. If you compare the SMAA routines in HQAA to those in SMAA.fxh with a trained eye, what you'll find is that the only ways I have modified the functions not related to edge detection are minor edits intended to coax a bit more performance out of it. I don't honestly even know if they work, because modern SMAA is already a really well-optimized bit of programming, and any changes I may have produced by tweaking its edge correction code weren't quantifiable in the frame times when I tried them - I couldn't see any statistically significant difference.

The upshot of this project has been that I'm starting to feel like I have a strong grasp of ReShade FX syntax. Even if there's a freak breakthrough in anti-aliasing mathematical methodology hiding in my brain somewhere (which I highly doubt, but hey, who knows), it wouldn't do anyone any good if I couldn't code it properly due to not understanding exactly what my code is doing.

Please Log in or Create an account to join the conversation.

  • Illuzio
More
2 years 2 months ago #218 by Illuzio Replied by Illuzio on topic (H)ybrid high-(Q)uality (A)nti-(A)liasing (HQAA)
I just tested your version 18.2.1 and everything works great in Dishonored! Thanks a lot !

Your shader has really good results on this game !

Please Log in or Create an account to join the conversation.

  • lordbean
  • Topic Author
More
2 years 2 months ago #219 by lordbean Replied by lordbean on topic (H)ybrid high-(Q)uality (A)nti-(A)liasing (HQAA)
You're welcome :) I'm doing my best to keep compatibility as good as possible since a lot of the idea behind HQAA is to offer an easy option for older games.

Please Log in or Create an account to join the conversation.

  • lordbean
  • Topic Author
More
2 years 2 months ago #220 by lordbean Replied by lordbean on topic (H)ybrid high-(Q)uality (A)nti-(A)liasing (HQAA)
HQAA v18.5 now available. I have rebuilt the hysteresis blending pass and added it as a fully user-configurable part of the shader (it's separate from the profiles and is displayed loose in the setup menu along with the profile selector, which is no longer in a category). This gives you more power to choose whether you want maximum possible AA effect or if you'd rather bias more towards keeping detail in the scene without needing to change anti-aliasing profiles to do it.

Please Log in or Create an account to join the conversation.

We use cookies
We use cookies on our website. Some of them are essential for the operation of the forum. You can decide for yourself whether you want to allow cookies or not. Please note that if you reject them, you may not be able to use all the functionalities of the site.
Ok Decline
More information