Marty McFly's Ambient Obscurance (MXAO) with IL

Nerrel

6 years 6 months ago - 6 years 6 months ago #401 by Nerrel Replied by Nerrel on topic Marty McFly's Ambient Obscurance (MXAO) with IL

Really happy to see 3.0 release, but I'm getting an error that reads "warning X3206: Implicit truncation of vector type" followed by a list of errors when running this with Project64. MXAO 2.0 just worked perfectly fine with everything set up exactly the same... hopefully it's something that can be resolved.

Screengrab of error: c1.staticflickr.com/5/4473/37096546043_074c51c251_o.jpg

Last edit: 6 years 6 months ago by Nerrel.

The topic has been locked.

Marty McFly
Topic Author

6 years 6 months ago #402 by Marty McFly Replied by Marty McFly on topic Marty McFly's Ambient Obscurance (MXAO) with IL

ShoterXX: Whoopsie, it was late in the evening when I did that...
Nerrel: that's really odd. Is Project64 OpenGL? I ran MXAO just fine on both DX9 and 11 so this sounds like a ReShade bug, if it's really that.

The topic has been locked.

GP-Unity

6 years 6 months ago #403 by GP-Unity Replied by GP-Unity on topic Marty McFly's Ambient Obscurance (MXAO) with IL

This is working really well now

doing a lot of experimenting but great results so far. I adjust 'float3 lumcoeff = float3(0.2126, 0.7152, 0.0722);' if i want to refine colours a bit depending on game right?

The topic has been locked.

ShoterXX

6 years 6 months ago #404 by ShoterXX Replied by ShoterXX on topic Marty McFly's Ambient Obscurance (MXAO) with IL

Nah, you're not supposed to mess with that. That's the brightness that each color channel emits, used to calculate light emitted.

The topic has been locked.

Marty McFly
Topic Author

6 years 6 months ago - 6 years 6 months ago #405 by Marty McFly Replied by Marty McFly on topic Marty McFly's Ambient Obscurance (MXAO) with IL

Correct. If you want to calculate Luma, you could just average all 3 channels. As eye is most sensitive to green, a pure green pixel appears brighter than a pure blue or red one. These coefficients you see weight red green and blue in such a way that the pixels with same perceived brightness have the same calculated Luma value.

@everyone: would you be up for more blending modes? As there "can't" be a perfect one, I'd just add a few that come to my mind as just enforcing one that I view as best is inconsistent.

@Nerrel: forgot to say, this is really critical and I can't push an update on MXAO as long as this is around. I'll try to find an OpenGL app that works with ReShade...

Last edit: 6 years 6 months ago by Marty McFly.

The topic has been locked.

GP-Unity

6 years 6 months ago #406 by GP-Unity Replied by GP-Unity on topic Marty McFly's Ambient Obscurance (MXAO) with IL

More blending modes would be awesome

The topic has been locked.

Nerrel

6 years 6 months ago - 6 years 6 months ago #407 by Nerrel Replied by Nerrel on topic Marty McFly's Ambient Obscurance (MXAO) with IL

Marty McFly wrote: ShoterXX: Whoopsie, it was late in the evening when I did that...
Nerrel: that's really odd. Is Project64 OpenGL? I ran MXAO just fine on both DX9 and 11 so this sounds like a ReShade bug, if it's really that.

It depends on the graphics plugin. GlideN64 (the best plugin for enhancement in my opinion) is OpenGL, and it worked perfectly with MXAO 2.0 but required disabling framebuffer emulation to get depth buffer access... I might try to completely reinstall REshade using the newest version and MXAO 3.0 and see if that changes anything.

EDIT: Yep, same result after setting it up again. If it's useful, I can send you the Reshade log with the entire error string in it.

Last edit: 6 years 6 months ago by Nerrel.

The topic has been locked.

Marty McFly
Topic Author

6 years 6 months ago - 6 years 6 months ago #408 by Marty McFly Replied by Marty McFly on topic Marty McFly's Ambient Obscurance (MXAO) with IL

Okay, here's a new version:

pastebin.com/FPJGrvrb

Changes are a) 2 more blending modes! and b) I have a hunch what went wrong on Nerrel's end - got no OpenGL app to test. Attempted to workaround this, wouldn't be the first time for MXAO to break ReShade or the driver... If it still breaks, I need the log.

The blending modes: 0 is standard, 1 is what I made earlier, with a slight tweak (or none, didn't check) and 2 is another attempt at a more realistic and sophisticated blending mode. Mode 1 has the culprit that white IL onto objects = whiteish spot instead of object being brighter.

Last edit: 6 years 6 months ago by Marty McFly.

The following user(s) said Thank You: GP-Unity, Nerrel

The topic has been locked.

ShoterXX

6 years 6 months ago #409 by ShoterXX Replied by ShoterXX on topic Marty McFly's Ambient Obscurance (MXAO) with IL

DX9 doesn't seem to enjoy the switch-case.
Other than that, blend mode #2 seems to make IL a lot more meaningful and nicer looking overall.

The topic has been locked.

Nerrel

6 years 6 months ago - 6 years 6 months ago #410 by Nerrel Replied by Nerrel on topic Marty McFly's Ambient Obscurance (MXAO) with IL

So this time I get "Error C5215: Integer varying _param_MXAO_samples must be flat." I'm not sure how to post the log without overtaking the whole page, so I hope mega is Ok:
mega.nz/#!PNMRgRrT!OuEQ2KuP1R-11VWOHCEzKT2Z-lhW0bhlMDIvM4Okbmw

Last edit: 6 years 6 months ago by Nerrel.

The topic has been locked.

Marty McFly
Topic Author

6 years 6 months ago #411 by Marty McFly Replied by Marty McFly on topic Marty McFly's Ambient Obscurance (MXAO) with IL

Can't look at the entire log right now as I'm on the phone. This however indicates that we have a major problem:

www.opengl.org/discussion_boards/showthr...x-to-fragment-shader

Last post. I can't directly steer openGL code with ReShade.

The topic has been locked.

GP-Unity

6 years 6 months ago #412 by GP-Unity Replied by GP-Unity on topic Marty McFly's Ambient Obscurance (MXAO) with IL

Marty McFly wrote: The blending modes: 0 is standard, 1 is what I made earlier, with a slight tweak (or none, didn't check) and 2 is another attempt at a more realistic and sophisticated blending mode. Mode 1 has the culprit that white IL onto objects = whiteish spot instead of object being brighter.

I've spent a couple of hours testing out the different blending modes, and i'm surprised i like mode 1 the most despite its' whiteish' colouring. Might be because it has more lighting coverage. Each blend mode is really great though, i'm sure the best choice varies per game which i'll have to spend a lot of time testing.

The topic has been locked.

crosire

6 years 6 months ago #413 by crosire Replied by crosire on topic Marty McFly's Ambient Obscurance (MXAO) with IL

Marty McFly wrote: Last post. I can't directly steer openGL code with ReShade.

Change

struct MXAO_VSOUT
{
    float4 position : SV_Position;
    float2 texcoord : TEXCOORD0;
    float2 scaledcoord : TEXCOORD1;
    int samples : TEXCOORD2;
    int blursamples : TEXCOORD3;
    float3 uvtoviewADD : TEXCOORD4;
    float3 uvtoviewMUL : TEXCOORD5;
};

struct MXAO_VSOUT
{
    float4 position : SV_Position;
    float2 texcoord : TEXCOORD0;
    float2 scaledcoord : TEXCOORD1;
    nointerpolation int samples : TEXCOORD2;
    nointerpolation int blursamples : TEXCOORD3;
    float3 uvtoviewADD : TEXCOORD4;
    float3 uvtoviewMUL : TEXCOORD5;
};

The topic has been locked.

Marty McFly
Topic Author

6 years 6 months ago #414 by Marty McFly Replied by Marty McFly on topic Marty McFly's Ambient Obscurance (MXAO) with IL

Oh, I didn't know that such thing exists. So noninterpolation is the HLSL/ReShade FX pendant to flat in GLSL?

The topic has been locked.

crosire

6 years 6 months ago #415 by crosire Replied by crosire on topic Marty McFly's Ambient Obscurance (MXAO) with IL

Correct.

The topic has been locked.

Marty McFly
Topic Author

6 years 6 months ago - 6 years 6 months ago #416 by Marty McFly Replied by Marty McFly on topic Marty McFly's Ambient Obscurance (MXAO) with IL

Learn something new every day...thank you for stepping in.
Alright, so I added crosire's fix and replaced that switch to make the warnings go away on d3d9. Nerrel, please report if it works for you. I raised the version number to 3.1.0 to avoid confusion with ReShade itself.

pastebin.com/Btyfrq2U

scratch that, one of my beta testers says it still doesn't compile due to same error

Last edit: 6 years 6 months ago by Marty McFly.

The topic has been locked.

Nerrel

6 years 6 months ago #417 by Nerrel Replied by Nerrel on topic Marty McFly's Ambient Obscurance (MXAO) with IL

Yeah, same here. This one says Error C7561: OpenGL requires 'in/out/ with 'flat'.

What would cause this version to have OpenGL problems where 2.0 didn't?

The topic has been locked.

Marty McFly
Topic Author

6 years 6 months ago #418 by Marty McFly Replied by Marty McFly on topic Marty McFly's Ambient Obscurance (MXAO) with IL

3.0 has huge restructurements. The issue here is that I calculate some static but costy stuff in vertex shader so it won't be calculated for every pixel and pass it per to the pixel shader along with texcoord and the usual stuff. Here, integers seem to cause the problem. As Matsilagi provided me with an OpenGL game, I could find a fix which is really really stupid but works, switching the two integers to floats works...

pastebin.com/Gn3EUMyf

The following user(s) said Thank You: Nerrel

The topic has been locked.

crosire

6 years 6 months ago - 6 years 6 months ago #419 by crosire Replied by crosire on topic Marty McFly's Ambient Obscurance (MXAO) with IL

Nerrel wrote: Yeah, same here. This one says Error C7561: OpenGL requires 'in/out/ with 'flat'.

That's a bug in ReShade that has already been fixed in source code a while ago, but not released as a new version yet.

Last edit: 6 years 6 months ago by crosire.

The topic has been locked.

Nerrel

6 years 6 months ago #420 by Nerrel Replied by Nerrel on topic Marty McFly's Ambient Obscurance (MXAO) with IL

Yep, it's working now. It seems the sample radius setting is very different from 2.0; in order to match the visuals with a radius of around 3 in the old version I had to set to 9 in this version. It definitely seems sharper and more detailed now, like on the pegs on the rack on the left (2.0 on left, 3.1 on right).

The topic has been locked.