(H)ybrid high-(Q)uality (A)nti-(A)liasing (HQAA)

Have now pushed version 11.7, which adds no new features. It is focused entirely on fixing bugs in hopes of resolving the issues people are experiencing. If it was working for you before, it should be visually pretty much identical.

I've checked 11.6.3
and it seems per component flag doesn't change anything
but artifacts not black now for some reason

also when i switched it back and forth, and pressed "reload", sometimes artifacts disappears, and image seems ok and with aa

but it's until i again click "reload", and artifacts appear again

-
I've checked 11.7
and it's look like artifacts become worse, flag also don't change anything

PS, i can't post images
I've tried as before, but it said too many links
I've put everything in cloud, it says I'm spammer
😡

The board has a limit of 4 images per post, I believe. I've bumped into that before.

Regarding the shader, try resetting it to defaults, and then go into ReShade settings and click clear effects cache to make sure there arent any old pieces of code loading. I'll keep looking for anything in the shader code that might be wrong meanwhile.

Tried to reset, and clear cache, same result, sadly.

-
Maybe i doing something wrong? You mentioned HDR before, i didn't noticed any related option, it isn't hard requirement right? I have regular monitor.

Nope, I'm still convinced it's the shader. I think something about the way I handled some of the calculations in FXAA doesn't work well on some cards.

I just pushed v11.7.1, I thought I had separated my normalized luma scene from the backbuffer but the FXAA passes were leaking it back in through their early exit paths. I fixed that, and I made sure that the alpha channel in the full-run results has no connection to the normalized lumas as well.

Just tested this on an openGL game. The first image shows the problem and the second shows how the sharpening emphasises it.

imgur.com/a/qDyTHhN

Hmm, OpenGL... that's the one API I haven't tested yet on my own system.

Whatever's up with the shader currently, my own PC has been less than helpful trying to correct it. The AMD drivers must be somehow sanitizing the output from the shader...

I'm gonna plug it into Doom 3 BFG edition and see what it does.

Edit: ugh. Despite Doom 3 itself being a bit of a pain to get working under Windows 10, HQAA worked totally fine once I got the game sorted. For some reason I just can't get my 6700XT to duplicate a problem. I might have to start plugging random older GPUs into my system until I find one with issues.

I've checked latest version on my dad's PC, it have win 10 and 1050ti and yea, same problem. I've checked on Fallout 4.
ps
just in case checked Binding of Isaac, which is OpenGL, and shader produce same artifacts.
I think it's not about graphic API, it something about GPU, as i understand you use AMD GPU?

as i said before, it's remind me when my friend wrote GPU rendering system on C++ and OpenCL, there problems with shaders few times, when on my gpu it was some lines on textures that doesn't reproduced on his machine, i think he used AMD GPU too to make test, he eventually made some fixes, and told me that shader need to be fixed for Nvidia GPU
sadly i don't know more details, i mostly just tested thing, searching bugs

Thats useful information actually. I have a 1050 Ti lying around somewhere. If I can get a realtime analysis of the problem going it should be much easier to fix.

I have a nvidia 3090 card and have not experienced any issues in any games lately. dx9 and dx11.
However, everytime i update the shader i always reset the hqaa.fx settings just to be sure.

I still have the issue in other game, and I have GTX1080, maybe its something with Pascal family?

I'm starting to think along those lines, WSH303. Turing and above seems to not have a problem, Pascal and below seems to demonstrate artifacts. Now I just need to figure out why.

Pushed 11.8, I rolled the previous compatibility options into a single new one and added some new things to the list of what it does. One of the major changes in v10 and up was that I rebuilt a lot of the math in FXAA to request fused multiply-adds where it was compatible (via using the mad expression instead of directly performing the math). 11.8 reverts this back to mathematical expressions if compatibility mode is enabled.

tried 11.9 with compatibility mode, still artifacts

Roger that, trying again. lol

I WILL beat this problem

Edit: I noticed a possible issue with the fast sharpen being done by the result sharpener and corrected it in a small update. It doesn't seem to have a visual effect on my system, but since it changes the math a bit, it's at least worth trying.

Why dont you just disable sharpen, fxaa and smaa and let the user activate one effect at a time, it would be easier to find the culprit then.

I can rule out SMAA because I havent made any modifications to its blend calculation code, basically. You can already disable result sharpening using the preprocessor define, which only leaves FXAA as suspect if theres still bad output.

Edit: after doing some research for a while I discovered that although shader model 3 compatible GPUs all theoretically support unrolling large loops, the extent to which this support functions correctly can vary significantly (particularly going back in age). I've done a pretty significant rework of the flow control used for the FXAA loops and pushed as v11.10.

Any improvement from 11.10.2? As of 11.10.1 I added a new step for all AA results that computes new alpha channel values based on shifting the original pixel's alpha either up or down depending on the change in color channel values of the result compared to the original. The idea is basically that we perceive jaggies mostly by the contrast between the colors of different pixels, so sanitizing the alpha channel should have no impact on quality of the routine and should help guard against odd-looking outputs.

Version 11.11 now available, which introduces a new optional technique (HQAA Temporal Stabilizer). It uses past-frame blending (exact weight is specified by the user) to minimize artifacts produced by HQAA when operating at high sensitivity (such as text crawl).

Warning: will cause some ghosting, especially if you set past frame weight to a high value.

I've tested 11.10.2 yesterday, on Witcher 2 and The Evil Within.
It compiled very long, and in Witcher 2 there was no AA and no artifacts, like it didn't worked at all.
And in TEW it was same artifacts, maybe just little bit less noticeable than before, but still like outlines in pixels where AA should be.

I've tried 11.11 right now, in DOW Soulstorm, and long compiling and artifacts still in place.
I've tired to reset defaults and clear cache, and it start working! Normal AA, sharpen working.... until i pressed "performance mode" flag, after it artifacts returned
i didn't screenshot working state (i naively thought its finally start working properly)
i screenshots artifacts tho


it should mean something, it's not the first time when after fiddling with on/off reload and stuff, shader start working, until next reload.

also about temporal stabilizer, sounds cool, but where it in options? maybe i missing something, also did it need to fps be above 60? Like those attempts to make screen space TAA, that require high framerate to blends excess frames?