It looks like I've run into a bug with the mul() function while porting a shader from GLSL to ReShade. Although it works fine for square matrices, it doesn't for rectangular ones. Here's a quick and dirty shader that I think visualizes the problems:
#include "ReShade.fxh"
static const matrix <float, 4, 4> m1 = matrix <float, 4, 4>(
1, 0, 0, 0,
1, 0, 0, 0,
1, 0, 0, 0,
1, 0, 0, 0
);
static const matrix <float, 4, 4> m2 = matrix <float, 4, 4>(
1, 0, 0, 0,
0, 0, 0, 0,
0, 0, 0, 0,
0, 0, 0, 0
);
static const matrix <float, 4, 4> m3 = matrix <float, 4, 4>(
0, 1, 1, 1,
1, 1, 1, 1,
1, 1, 1, 1,
1, 1, 1, 1
);
static const matrix <float, 4, 3> n1 = matrix <float, 4, 3>(
1, 0, 0,
1, 0, 0,
1, 0, 0,
1, 0, 0
);
static const matrix <float, 4, 3> n2 = matrix <float, 4, 3>(
1, 0, 0,
0, 0, 0,
0, 0, 0,
0, 0, 0
);
static const matrix <float, 4, 3> n3 = matrix <float, 4, 3>(
0, 1, 1,
1, 1, 1,
1, 1, 1,
1, 1, 1
);
static const float4 u = float4(1, 1, 1, 1);
static const float3 v = float3(1, 1, 1);
void demoMulBug(
float4 pos : SV_Position,
float2 texcoord : TEXCOORD0,
out float4 color : SV_Target
) {
// Bar 1 - top of screen
if (6 * pos.y < BUFFER_HEIGHT) {
color = mul(u, m1) / 4;
}
else if (6 * pos.y < 2 * BUFFER_HEIGHT) {
color = mul(u, m2);
}
else if (6 * pos.y < 3 * BUFFER_HEIGHT) {
color = mul(u, m3) / 4;
}
else if (6 * pos.y < 4 * BUFFER_HEIGHT) {
color = mul(u, n1) / 4;
}
else if (6 * pos.y < 5 * BUFFER_HEIGHT) {
color = mul(u, n2);
}
// Bar 6 - bottom of screen
else {
color = mul(u, n3) / 4;
}
}
This should output a set of six horizontal bars across the screen. Bars 1, 2, 4, and 5 should be RGB #FF0000, and bars 3 and 6 should be #BFFFFF. Additionally, if we comment out bars 1, 2, and 3, we should be able to change color to be a float3 because mul(u, n1) and co. should all output a float3.
Although we get the expected results for bars 1, 2, and 3, we do not for 4, 5, and 6. Bars 4, 5, and 6. instead show as #3F3F3F, #FFFFFF, and #000000. We also get compilation warnings "implicit truncation of vector type" for those three mul() calls. The results for bars 5 and 6 indicate that mul() is truncating n2 and n3 to just n2[0] and n3[0], possibly causing the warning as a side effect. If we comment out bars 1, 2, and 3 and change color to be a float3, we get a compiliation error "cannot convert these types (from float4 to float3)", indicating that mul() is outputting a float4 instead of the expected float3.
Topic Author
