Rectangular Matrix Multiplication Bugged?

More
6 months 2 weeks ago #1 by akgunter
It looks like I've run into a bug with the mul() function while porting a shader from GLSL to ReShade. Although it works fine for square matrices, it doesn't for rectangular ones. Here's a quick and dirty shader that I think visualizes the problems:
#include "ReShade.fxh"

static const matrix <float, 4, 4> m1 = matrix <float, 4, 4>(
    1, 0, 0, 0,
    1, 0, 0, 0,
    1, 0, 0, 0,
    1, 0, 0, 0
);
static const matrix <float, 4, 4> m2 = matrix <float, 4, 4>(
    1, 0, 0, 0,
    0, 0, 0, 0,
    0, 0, 0, 0,
    0, 0, 0, 0
);
static const matrix <float, 4, 4> m3 = matrix <float, 4, 4>(
    0, 1, 1, 1,
    1, 1, 1, 1,
    1, 1, 1, 1,
    1, 1, 1, 1
);

static const matrix <float, 4, 3> n1 = matrix <float, 4, 3>(
    1, 0, 0,
    1, 0, 0,
    1, 0, 0,
    1, 0, 0
);
static const matrix <float, 4, 3> n2 = matrix <float, 4, 3>(
    1, 0, 0,
    0, 0, 0,
    0, 0, 0,
    0, 0, 0
);
static const matrix <float, 4, 3> n3 = matrix <float, 4, 3>(
    0, 1, 1,
    1, 1, 1,
    1, 1, 1,
    1, 1, 1
);
static const float4 u = float4(1, 1, 1, 1);
static const float3 v = float3(1, 1, 1);

void demoMulBug(
    float4 pos : SV_Position,
    float2 texcoord : TEXCOORD0,

    out float4 color : SV_Target
) {
    // Bar 1 - top of screen
    if (6 * pos.y < BUFFER_HEIGHT) {
        color = mul(u, m1) / 4;
    }
    else if (6 * pos.y < 2 * BUFFER_HEIGHT) {
        color = mul(u, m2);
    }
    else if (6 * pos.y < 3 * BUFFER_HEIGHT) {
        color = mul(u, m3) / 4;
    }
    else if (6 * pos.y < 4 * BUFFER_HEIGHT) {
        color = mul(u, n1) / 4;
    }
    else if (6 * pos.y < 5 * BUFFER_HEIGHT) {
        color = mul(u, n2);
    }
    // Bar 6 - bottom of screen
    else {
        color = mul(u, n3) / 4;
    }
}

This should output a set of six horizontal bars across the screen. Bars 1, 2, 4, and 5 should be RGB #FF0000, and bars 3 and 6 should be #BFFFFF. Additionally, if we comment out bars 1, 2, and 3, we should be able to change color to be a float3 because mul(u, n1) and co. should all output a float3.

Although we get the expected results for bars 1, 2, and 3, we do not for 4, 5, and 6. Bars 4, 5, and 6. instead show as #3F3F3F, #FFFFFF, and #000000. We also get compilation warnings "implicit truncation of vector type" for those three mul() calls. The results for bars 5 and 6 indicate that mul() is truncating n2 and n3 to just n2[0] and n3[0], possibly causing the warning as a side effect. If we comment out bars 1, 2, and 3 and change color to be a float3, we get a compiliation error "cannot convert these types (from float4 to float3)", indicating that mul() is outputting a float4 instead of the expected float3.

Please Log in or Create an account to join the conversation.

More
6 months 2 weeks ago - 6 months 2 weeks ago #2 by crosire
Replied by crosire on topic Rectangular Matrix Multiplication Bugged?
That's because all intrinsics in ReShade FX currently only work on square matrices, which is why these ones here are implicitly truncated. You can only use the "float2x2", "float3x3" and "float4x4" types for that reason ("float3x2" etc. will give you a compile error because they don't currently exist). The fact that declaration of rectangular matrices using the "matrix<>" syntax works is a bit of an oversight and untested.
So far square matrices were sufficient, so I didn't put time in expanding support to rectangular matrices. This can be done though if necessary.
Last edit: 6 months 2 weeks ago by crosire.
The following user(s) said Thank You: akgunter

Please Log in or Create an account to join the conversation.

More
6 months 2 weeks ago #3 by crosire
Replied by crosire on topic Rectangular Matrix Multiplication Bugged?
This is now implemented in github.com/crosire/reshade/commit/cc88ab...fcecb7788c3bb1a942a0. Checked with your test shader and that works now.
The following user(s) said Thank You: akgunter

Please Log in or Create an account to join the conversation.

More
6 months 2 weeks ago #4 by akgunter
Replied by akgunter on topic Rectangular Matrix Multiplication Bugged?
Wow, that was fast. You got that coded up before I'd even gotten out of bed.

I'll try to build that commit and see how things work out. Thank you!

Please Log in or Create an account to join the conversation.