How to convert RGB full range to limited range?

  • hoftago
  • Topic Author
More
2 years 1 month ago #1 by hoftago How to convert RGB full range to limited range? was created by hoftago
Hi all,

The steam deck outputs an incorrect signal when it comes to video range. Even though the Deck outputs full RGB, monitors recognise a limited video range signal. The result is crushed blacks and clipped highlights on your display.

This is a major problem for displays that do not support manually setting the video range. I have one such display, or rather, a video capture device.

Using reshade, it should be possible to convert the full range RGB signal to limited range. This should fix the above problem with the steam deck.

As far as I can tell, there might be two options that could be used:
  • LUT
  • Color Matrix (although if I understand correctly, this does not work for this intended purpose)

Now this should be easy if a LUT was available. But sadly, I can't find it.
So now I'm stuck at a possible solution with no next step forward.

Does anyone know the solution or where I might find one to this problem? 

Please Log in or Create an account to join the conversation.

  • hoftago
  • Topic Author
More
2 years 1 month ago - 2 years 1 month ago #2 by hoftago Replied by hoftago on topic How to convert RGB full range to limited range?
Ended up solving this myself by creating my own shader.

For anyone looking for the solution, you can use the shader found here: github.com/JosVerheij/reshade-shaders . It simply converts 0-255 to colors to values within the 16-235 range.
/*
Convert Full RGB to Limited RGB within a Full RGB container.

0-255 within 0-255 container > 16-235 within 0-255 container.
*/

// Full range: 0 - 255
// Limited range: 16 - 235

// Note that RGBA8 uses a range of 0.0-1.0, so we also need to divide by 255.

// RGB_Clamp: The minimum RGB value
// i.e. black is RGB(0, 0, 0) in full range, which should become RGB(16, 16, 16) in limited range.
// Clamp = 16
static const float3 RGB_Clamp =
float3(16.0/255.0, 16.0/255.0, 16.0/255.0);

// RGB_Gain: the factor to apply to the full range RGB value
// i.e. 255 should become 235.
// We have to take the clamp value into account such that 235 must be 235 - 16 = 219
static const float RGB_Gain =
float(219.0/255.0);


#include "ReShade.fxh"

// Shader
void FullToLimitedPass(float4 pos : SV_Position, float2 texcoord : TexCoord, out float3 color : SV_Target)
{

color = tex2D(ReShade::BackBuffer, texcoord).rgb;

color = color * RGB_Gain + RGB_Clamp;

color = saturate(color);
}

technique RGB_YUV_RGB
{
pass
{
VertexShader = PostProcessVS;
PixelShader = FullToLimitedPass;
}
}
Last edit: 2 years 1 month ago by hoftago.

Please Log in or Create an account to join the conversation.