[SOLVED] FFXI: Ambient Occlusion problem, anything over version 4.5.3

There seems to be a major issue while using AO (Ambient Occlusion) with FFXI online, anything over version 4.5.3.

from 4.5.4 to 4.9.1, there seems to be some flickering issue while using AO, no matter which one: HBAO/MXAO/SSDO,etc.

I also tried version 5.0.2 and above, those are alot worse, infact, they don't work at all, even if it did, it would only for for a split second or two then it will just disappear on its own.

This was on both DX9 (With the use of the injection tool called: FFXI_d3dto9_proxy_v1.1.0.0 by Atom0ic), as well as DX11 (DG Voodoo 2).

Also, just another heads up, MSAA+SGSSAA within nvidia inspector works excellent under DX11 with DG Voodoo 2 (If using reshade version 4.5.3), but, soon as I want to use any form of AO within reshade, nothing appears, all the other reshade, such as bloom, HDR and whatnot, are working fine in conjuction with MSAA+SGSSAA.

Thanks alot.

EDIT:

Forgot to say this earlier:

While using Ambient Occlusion under DX11 (DG Voodoo 2), there is a very abnormal/massive performance drop, no matter the AO shaders, but in DX 9's case, there is none that I can see, just the normal performance drop from the AO shaders themselves.

This is intentional, it's the anticheat protection of ReShade kicking in, which is preventing use of depth effects in multiplayer games.
You'd have to use ReShade with add-on support (see download page), which has that disabled, but at your own risk (this will cause bans in multiplayer games; probably not in FFXI, but ¯\_(ツ)_/¯).

Thanks alot for the reply Cosire.

When you get a chance, take a look at my thread, lots of stuff:

forums.guru3d.com/threads/extreme-4-way-sli-tuning.425674/

EDIT:

For older version of reshade, is there a way to add full add-on support like the one you included with the newest version?

Thanks again.

I tried the that add-on like you told me to use, and kinda suspected, it does not work properly or even at all.

Quint_MXAO: Does not work

I also tried from earlier version of reshade, used MXAO.fx with the newest version of reshade along with that Add-on, just to see if it works, nope.

SSDO seems to kinda work ok but takes more hit then the previous version of reshade.

I also gave HBAO.fx (Downloaded) a try, it works very minute when compared to earlier version of reshade, and at one time, it stopped working entirely.

Everything was certainly worth giving a try, thanks again.

Are you getting the correct depth buffer selected in the D3D9/11 tab in the Reshade UI?

Can you confirm the depth buffer is working by using DisplayDepth.fx?

Sorry, I was doing some other testing, I'll do what your asking later on.

Also, I gave reshade 4.5.2, it seems to handle pretty nicely, small flickering issue with HBAO.fx but with Quint_MXAO.fx & MXAO.fx, seems to be working fine but there is one problem when using these two shaders:

It does not cover the whole screen, it only covers, what, like in what's in front of what you are seeing on the screen, in otherwords, pretty much straight forward, it only covers portion of the screen, so when rotating your camera, you can clearly see it.

Also, I tried both the DX8to9 injection tool and DX11 (DG Voodoo 2), I observed closely on the performance hit on the AO's in general, performance hit is about the same, seems DX11 offers better overall performance but the hit is about the same.

I'll do what you ask me to do later, will give heads up later, I am also doing some very extensive test on older reshade and see how the AO behaves, one reshade I tried, the latest of version 3, seems the AO is more accurate.

Thanks again.

The ReShade version is irrelevant here (apart from them having different heuristics, which may select a different depth buffer by default). Use the latest and get depth access set up correctly (via DisplayDepth effect and fiddeling with the settings of the "Generic Depth" add-on) and things will work.

Thanks for the help everyone, Everything is alot better now.

Although still from being perfect, I also notice that, the more you zoom out, the more you can see AO is taking into effect as well as better overall coverage, If your zooming all the way in, AO will turn off naturally if looking at an object/terrain in close range, not only that, depending on the angle your viewing in the game, some AO appears to be missing but will come back once it comes back to normal angle view.

Any flickering? There was small amounts, while using the HBAO.fx but for the most part, alot better now.

I am using the DX8to9 Injection tool, however, the DX11 (DG Voodoo2) seems to be working fine as well, seems the DX11 takes more hit then DX9 while using reshade 5.3.0.

So yes, in the end, huge difference.

I never miss with the Displaydepth or additional tabs within reshade until now, thanks again.

And yes, even though the AO is alot better now with basically small to no flickering, the Displaydepth.fx was still not perfect (Normal on the left, black screen on the right if set to 50/50), I set the settings to where the AO will show up properly, if I set the Displaydepth.fx to where it needs to be, AO is working, MXAO is working in extreme minute amounts.

Granted its an very old game, so still very impressed that even works.

Thanks Crosire & Martigen for your help.

Don't forget that DisplayDepth only shows you the depth buffer and allows you to set it up correctly -- if it needs to be reversed or flipped for example. But these settings don't save, it only shows you what they need to be.

To save them for a given game's profile, you then need to click 'Edit global preprocessor definitions' and set the environmental variables to match.

Note also when viewing with DisplayDepth the left side is the normal map, the right side is the depth buffer. It should not be all black or all white on the right, it's not set up correctly or the wrong depth buffer is being chosen if this is the case. Note also on the left, if you see horizontal lines in the geometry of the normal map, this is a sign you need to toggle RESHADE_DEPTH_INPUT_IS_LOGARITHMIC, setting it to 1 when you edit the preprocessor definitions.

I was trying to explain this earlier that, even though Displaydepth is showing correctly where it needs to be, (Normal on the left / Depth on the right) the AO in general is not even coming up at all, if it did, extreme small minute, now, If I turn on Displaydepth (Normal on the left / Black screen on the right) all the AO is working properly, I already saw some videos on Displaydepth, but for this game, for some odd reason or reasons, have to configure it abnormally to get proper AO results.

I have a question, is something like this, typical for very old games or not common at all?

Thanks again for the replies.

EDIT:

One thing I need to point out here:

Speaking about AO in general with this game, there are several parameters within the Registry for this game that has to exactly match, otherwise, AO won't work.

Under REGEDIT:

HKEY_LOCAL_MACHINE > SOFTWARE > WOW6432NODE > PlayOnlineUS > SquareEnix > FinalFantasy XI

There are 4 lines we need to look at:

0001: Screen resolution width in pixels
0002: Screen resolution height in pixels
0003: Background resolution width in pixels
0004: Background resolution height in pixels

Resolution in all, has to be exactly the same in order for AO to be working at all.
One thing I need to point out here is Windower4,

Upon launching POL.exe from Windower4, Lines 0003 & 0004 are now doubled from lines 0001 & 0002.

So what I did is did some registry changes, exported the registry file on my desktop, so every time I launch POL.exe from Windower4, right before actually logging in the game, I can double click on that exported registry file, so I don't have to keep going back and forth doing/typing regedit all the time, works with no problems.