Determining Brightest/Darkest Pixel?
- t1234
- Topic Author
Less
More
I've been tinkering with this psuedo cel shading shader that I downloaded from this thread:
reshade.me/forum/shader-suggestions/4196...similar-to-retroarch
. The way I understand it works is that for pixels with a luminosity within a certain threshold have the luminosity raised to pure white (or pure black if within the lower threshold). I want to know if there's a way to add a pass to the shader that goes over the image and checks what the luminosity of the brightest pixel in the image is and the luminosity of the darkest pixel, and set those as upper and lower limits respectively.
(Shader settings are set to extreme values for demonstration purposes)
Demonstration images:
Default Game Image without shader:
Screenshot with shader strength set to maximum to show how shader does its work by overlaying white or black (if I understand correctly):
Screenshot with maximum shader thresholds and shader strength:
By default the shader raises highlights to pure white rather than the actual max white value of the current frame, causing blown out highlights. And similar for black values.
My work so far:
I found that by editing cHSL.z line 155:WhtMax being a new variable in the user config menu with values between 0.0 and 1.0.
This allowed me to change the maximum luminosity value. To actually see the effects I also had to subtract the difference from the WhtCutoff value. Otherwise, the shader would only be looking at values above the new max white value, see that they're above the WhtCutoff, and then not do anything.
Additionally, I changed cHSL.y so that it takes into account saturation like how the black values are already handled by the shader so that the pixel maintains its non-white hue.
Default Game:
Shader with Default Max White:
Notice the overly bright highlights on the sleeves for instance.
Manual Max White Adjustment:
This adjustment fixes the overly bright highlights, but these settings won't look right in different brighter scenes. Ideally, the max white value could be dynamically and automatically adjusted.
I also did something similar for the darker colors but I won't bog you down with the details.
The issue however is that manually editing the value too high or too low can make the highlights and such brighter or darker than the default look of the game which can cause clipping on the default extreme white or black values.
What I would like is for the shader to automatically dynamically adjust for the minimum and maximum black and white values that change depending on the scene instead of having the user edit the values.
Stuff that hasn't worked:
Shaders like levels.fx or lightroom end up affecting the image as a whole when changing the white level while keeping relative brightness levels the same which is isn't what I want. Using lightroom to change the peak highlights seems to negatively affect other colors lower on the luminosity scale too.
Summary:
I want to find a way for the shader to automatically find the default highest bright value and adjust accordingly so that the image highlights are not raised above the default max bright value (causing blown out highlights) and are also not lowered below the maximum default (and causing clipping). If someone can show me how to figure it out for the max luminosity value, I can probably figure out how to adapt it for the minimum luminosity value (the black levels).
Proposed Solution:
I believe a way this can be done is to have the shader iterate through every individual pixel, convert the RGB values to HSV to figure out the luminosity (like how the shader already does to check if it's in the luminosity thresholds), and then compare that luminosity value to the current WhtMax value. If the current pixel's luminosity is higher than the current WhtMax value, it sets the MaxWht value to the current pixel's luminosity. Alternatively, to adapt it for the minimum BlkMin, I would just compare the current pixel's luminosity to the current BlkMin and if it's lower, set BlkMin to the current pixel's luminosity. This process would be done on each frame to account for changing scenes.
However, figuring out how to tell reshade to iterate through each pixel and check the luminosity values is beyond my knowledge and ability. I'm also not sure if it's possible or will have acceptable performance.
Can anyone assist me?
(Shader settings are set to extreme values for demonstration purposes)
Demonstration images:
Default Game Image without shader:
Screenshot with shader strength set to maximum to show how shader does its work by overlaying white or black (if I understand correctly):
Screenshot with maximum shader thresholds and shader strength:
By default the shader raises highlights to pure white rather than the actual max white value of the current frame, causing blown out highlights. And similar for black values.
My work so far:
I found that by editing cHSL.z line 155:
if (cHSL.z > (WhtCutoff - (1.0 - WhtMax))) { cHSL.y *= SatModify; cHSL.z = WhtMax; }
This allowed me to change the maximum luminosity value. To actually see the effects I also had to subtract the difference from the WhtCutoff value. Otherwise, the shader would only be looking at values above the new max white value, see that they're above the WhtCutoff, and then not do anything.
Additionally, I changed cHSL.y so that it takes into account saturation like how the black values are already handled by the shader so that the pixel maintains its non-white hue.
Default Game:
Shader with Default Max White:
Notice the overly bright highlights on the sleeves for instance.
Manual Max White Adjustment:
This adjustment fixes the overly bright highlights, but these settings won't look right in different brighter scenes. Ideally, the max white value could be dynamically and automatically adjusted.
I also did something similar for the darker colors but I won't bog you down with the details.
The issue however is that manually editing the value too high or too low can make the highlights and such brighter or darker than the default look of the game which can cause clipping on the default extreme white or black values.
What I would like is for the shader to automatically dynamically adjust for the minimum and maximum black and white values that change depending on the scene instead of having the user edit the values.
Stuff that hasn't worked:
Shaders like levels.fx or lightroom end up affecting the image as a whole when changing the white level while keeping relative brightness levels the same which is isn't what I want. Using lightroom to change the peak highlights seems to negatively affect other colors lower on the luminosity scale too.
Summary:
I want to find a way for the shader to automatically find the default highest bright value and adjust accordingly so that the image highlights are not raised above the default max bright value (causing blown out highlights) and are also not lowered below the maximum default (and causing clipping). If someone can show me how to figure it out for the max luminosity value, I can probably figure out how to adapt it for the minimum luminosity value (the black levels).
Proposed Solution:
I believe a way this can be done is to have the shader iterate through every individual pixel, convert the RGB values to HSV to figure out the luminosity (like how the shader already does to check if it's in the luminosity thresholds), and then compare that luminosity value to the current WhtMax value. If the current pixel's luminosity is higher than the current WhtMax value, it sets the MaxWht value to the current pixel's luminosity. Alternatively, to adapt it for the minimum BlkMin, I would just compare the current pixel's luminosity to the current BlkMin and if it's lower, set BlkMin to the current pixel's luminosity. This process would be done on each frame to account for changing scenes.
However, figuring out how to tell reshade to iterate through each pixel and check the luminosity values is beyond my knowledge and ability. I'm also not sure if it's possible or will have acceptable performance.
Can anyone assist me?
Please Log in or Create an account to join the conversation.
- t1234
- Topic Author
Less
More
3 years 2 weeks ago - 3 years 2 weeks ago #2
by t1234
Replied by t1234 on topic Determining Brightest/Darkest Pixel?
I discovered a solution with Prod80's Correct Contract Shader, code of which I repurposed. Here are the results. All MMJShader images ingnore pixels that have a luminance above 0.97 or below 0.03 as per the default settings.
The default game image without Shader can be seen in the opening post.
Screens can be seen on Imgur since this forum won't let me post them all and trying to embed them makes the forum consider my post spam.
imgur.com/a/Pcpw4pD
In the end, it is of my belief that my method which takes into account dynamic nature of the overall whitepoint and blackpoint (and thus the distance between them) performs a better job of flattening the image and ensuring the presence of banding essential to the cel shaded look. However, due to the limitations of only being able to analyze the whitepoint and blackpoint of the image as a whole, parts of the image can end up being overly darkened or overly brightened and different parts of the image will end up within one or fewer of the shades than desired (i.e. one object will only have 2 bands instead of the the 3 desired bands since it's luminance values are within those 2 overall image bands).
I'll be sharing my revised shader on github soon if anyone is interested in playing around with it. There are a lot of different possible settings combinations outside of the ones I posted.
The default game image without Shader can be seen in the opening post.
Screens can be seen on Imgur since this forum won't let me post them all and trying to embed them makes the forum consider my post spam.
imgur.com/a/Pcpw4pD
In the end, it is of my belief that my method which takes into account dynamic nature of the overall whitepoint and blackpoint (and thus the distance between them) performs a better job of flattening the image and ensuring the presence of banding essential to the cel shaded look. However, due to the limitations of only being able to analyze the whitepoint and blackpoint of the image as a whole, parts of the image can end up being overly darkened or overly brightened and different parts of the image will end up within one or fewer of the shades than desired (i.e. one object will only have 2 bands instead of the the 3 desired bands since it's luminance values are within those 2 overall image bands).
I'll be sharing my revised shader on github soon if anyone is interested in playing around with it. There are a lot of different possible settings combinations outside of the ones I posted.
Last edit: 3 years 2 weeks ago by t1234.
Please Log in or Create an account to join the conversation.
- Daemonjax
Less
More
2 years 11 months ago #3
by Daemonjax
Replied by Daemonjax on topic Determining Brightest/Darkest Pixel?
Wouldn't you have to sample every pixel on the screen to determine the brightest? That'd be expensive.
Please Log in or Create an account to join the conversation.
- Daemonjax
Less
More
2 years 10 months ago - 2 years 10 months ago #4
by Daemonjax
Replied by Daemonjax on topic Determining Brightest/Darkest Pixel?
Check out this old thread I found where crosire describes a way to run an expensive operation once-per-frame and pass that info on to other techniques (instead once-per-pixel-per-frame) by using a 1x1 texture to write a float4 to:
reshade.me/forum/shader-presentation/230...ted-storybook-shader
I'm unsure if it really really only runs once per frame, but I'd imagine Crosire knows what he's talking about.
You'd make a nested for loop that taps each pixel in the backbuffer and checks it against the brightest lum value found so far. Finally it would return the value which is written to the texture. In a later pass you'd retrieve that float4 value and use it in your calculations.
To get the lum of the pixel... there's a couple different formulas, but the cheapest that would look right would be multiplying the rgb values by constant float3 and adding them up.
Something like this -- assuming you just need the lum value of the brightest pixel on screen -- if instead you need the xy location of it, both, or maybe even the actual rgb values, then it would be slightly different but similar:
reshade.me/forum/shader-presentation/230...ted-storybook-shader
I'm unsure if it really really only runs once per frame, but I'd imagine Crosire knows what he's talking about.
You'd make a nested for loop that taps each pixel in the backbuffer and checks it against the brightest lum value found so far. Finally it would return the value which is written to the texture. In a later pass you'd retrieve that float4 value and use it in your calculations.
To get the lum of the pixel... there's a couple different formulas, but the cheapest that would look right would be multiplying the rgb values by constant float3 and adding them up.
Something like this -- assuming you just need the lum value of the brightest pixel on screen -- if instead you need the xy location of it, both, or maybe even the actual rgb values, then it would be slightly different but similar:
#include "ReShade.fxh"
calcBrightestPixelPS()
{
float brightest = 0;
float2 screen = float2(0,0);
float3 temp1;
float temp2;
[loop] loop over screen.x
{
[loop] loop over screen.y
{
temp1 = tex2d(ReShade::BackBuffer, screenXY).rgb;
temp2 = 0.299f * temp1.r + 0.587f * temp1.g + 0.114f * temp1.b;
[flatten] if (temp2 > brightest) brightest = temp2;
}
}
}
return float4(brightest, 0, 0, 0);
Last edit: 2 years 10 months ago by Daemonjax.
Please Log in or Create an account to join the conversation.
- Tojkar
Less
More
Pardon me for getting intimate with the dead but I've done everything else already to find an answer for my question.
I've seen two versions for this shader and in fact, I had both of them at one point but I lost them when my HDD died. One for DX9 and one for more modern. I have no idea who made the DX9 port but for the life of me I cannot find it anywhere. Not here nor in Github.
If anyone could throw me the DX9 version or tell where to find it, I'd be greatful.
I've seen two versions for this shader and in fact, I had both of them at one point but I lost them when my HDD died. One for DX9 and one for more modern. I have no idea who made the DX9 port but for the life of me I cannot find it anywhere. Not here nor in Github.
If anyone could throw me the DX9 version or tell where to find it, I'd be greatful.
Please Log in or Create an account to join the conversation.