Rectangular Matrix Multiplication Bugged?
- akgunter
- Topic Author
Less
More
2 years 9 months ago #1
by akgunter
Rectangular Matrix Multiplication Bugged? was created by akgunter
It looks like I've run into a bug with the mul() function while porting a shader from GLSL to ReShade. Although it works fine for square matrices, it doesn't for rectangular ones. Here's a quick and dirty shader that I think visualizes the problems:
This should output a set of six horizontal bars across the screen. Bars 1, 2, 4, and 5 should be RGB #FF0000, and bars 3 and 6 should be #BFFFFF. Additionally, if we comment out bars 1, 2, and 3, we should be able to change color to be a float3 because mul(u, n1) and co. should all output a float3.
Although we get the expected results for bars 1, 2, and 3, we do not for 4, 5, and 6. Bars 4, 5, and 6. instead show as #3F3F3F, #FFFFFF, and #000000. We also get compilation warnings "implicit truncation of vector type" for those three mul() calls. The results for bars 5 and 6 indicate that mul() is truncating n2 and n3 to just n2[0] and n3[0], possibly causing the warning as a side effect. If we comment out bars 1, 2, and 3 and change color to be a float3, we get a compiliation error "cannot convert these types (from float4 to float3)", indicating that mul() is outputting a float4 instead of the expected float3.
#include "ReShade.fxh"
static const matrix <float, 4, 4> m1 = matrix <float, 4, 4>(
1, 0, 0, 0,
1, 0, 0, 0,
1, 0, 0, 0,
1, 0, 0, 0
);
static const matrix <float, 4, 4> m2 = matrix <float, 4, 4>(
1, 0, 0, 0,
0, 0, 0, 0,
0, 0, 0, 0,
0, 0, 0, 0
);
static const matrix <float, 4, 4> m3 = matrix <float, 4, 4>(
0, 1, 1, 1,
1, 1, 1, 1,
1, 1, 1, 1,
1, 1, 1, 1
);
static const matrix <float, 4, 3> n1 = matrix <float, 4, 3>(
1, 0, 0,
1, 0, 0,
1, 0, 0,
1, 0, 0
);
static const matrix <float, 4, 3> n2 = matrix <float, 4, 3>(
1, 0, 0,
0, 0, 0,
0, 0, 0,
0, 0, 0
);
static const matrix <float, 4, 3> n3 = matrix <float, 4, 3>(
0, 1, 1,
1, 1, 1,
1, 1, 1,
1, 1, 1
);
static const float4 u = float4(1, 1, 1, 1);
static const float3 v = float3(1, 1, 1);
void demoMulBug(
float4 pos : SV_Position,
float2 texcoord : TEXCOORD0,
out float4 color : SV_Target
) {
// Bar 1 - top of screen
if (6 * pos.y < BUFFER_HEIGHT) {
color = mul(u, m1) / 4;
}
else if (6 * pos.y < 2 * BUFFER_HEIGHT) {
color = mul(u, m2);
}
else if (6 * pos.y < 3 * BUFFER_HEIGHT) {
color = mul(u, m3) / 4;
}
else if (6 * pos.y < 4 * BUFFER_HEIGHT) {
color = mul(u, n1) / 4;
}
else if (6 * pos.y < 5 * BUFFER_HEIGHT) {
color = mul(u, n2);
}
// Bar 6 - bottom of screen
else {
color = mul(u, n3) / 4;
}
}
This should output a set of six horizontal bars across the screen. Bars 1, 2, 4, and 5 should be RGB #FF0000, and bars 3 and 6 should be #BFFFFF. Additionally, if we comment out bars 1, 2, and 3, we should be able to change color to be a float3 because mul(u, n1) and co. should all output a float3.
Although we get the expected results for bars 1, 2, and 3, we do not for 4, 5, and 6. Bars 4, 5, and 6. instead show as #3F3F3F, #FFFFFF, and #000000. We also get compilation warnings "implicit truncation of vector type" for those three mul() calls. The results for bars 5 and 6 indicate that mul() is truncating n2 and n3 to just n2[0] and n3[0], possibly causing the warning as a side effect. If we comment out bars 1, 2, and 3 and change color to be a float3, we get a compiliation error "cannot convert these types (from float4 to float3)", indicating that mul() is outputting a float4 instead of the expected float3.
Please Log in or Create an account to join the conversation.
- crosire
Less
More
2 years 9 months ago - 2 years 9 months ago #2
by crosire
Replied by crosire on topic Rectangular Matrix Multiplication Bugged?
That's because all intrinsics in ReShade FX currently only work on square matrices, which is why these ones here are implicitly truncated. You can only use the "float2x2", "float3x3" and "float4x4" types for that reason ("float3x2" etc. will give you a compile error because they don't currently exist). The fact that declaration of rectangular matrices using the "matrix<>" syntax works is a bit of an oversight and untested.
So far square matrices were sufficient, so I didn't put time in expanding support to rectangular matrices. This can be done though if necessary.
So far square matrices were sufficient, so I didn't put time in expanding support to rectangular matrices. This can be done though if necessary.
Last edit: 2 years 9 months ago by crosire.
The following user(s) said Thank You: akgunter
Please Log in or Create an account to join the conversation.
- crosire
Less
More
2 years 9 months ago #3
by crosire
Replied by crosire on topic Rectangular Matrix Multiplication Bugged?
This is now implemented in
github.com/crosire/reshade/commit/cc88ab...fcecb7788c3bb1a942a0
. Checked with your test shader and that works now.
The following user(s) said Thank You: akgunter
Please Log in or Create an account to join the conversation.
- akgunter
- Topic Author
Less
More
2 years 9 months ago #4
by akgunter
Replied by akgunter on topic Rectangular Matrix Multiplication Bugged?
Wow, that was fast. You got that coded up before I'd even gotten out of bed.
I'll try to build that commit and see how things work out. Thank you!
I'll try to build that commit and see how things work out. Thank you!
Please Log in or Create an account to join the conversation.