Is Dithering applied AFTER TuningPalette 3DLUT?

  • MonarchX
  • Topic Author
More
8 years 2 months ago #1 by MonarchX Is Dithering applied AFTER TuningPalette 3DLUT? was created by MonarchX
I am curious to know whether ReShade dithering and/or grain effects from either SweetFX or CustomFX, get applied before or after TuningPalette 3DLUT application? In this case, "3DLUT" or "3D" Look Up Table is the term that calibration community uses to describe a complex LUT that affects & corrects not just grayscale alone, but also color gamut/colorspace, in games (and/or in movies rendered by madVR). 1DLUT is the term used to describe a simple LUT that corrects only the grayscale. Such LUT's are also part of ICC profiles, specially "vcgt" portion of such profiles and only 1DLUT's/"vcgt" portion of ICC profiles can be applied in games.

ReShade has the TuningPalette shader/effect, which can handle up to 64 x 64 x 64 3DLUT's. Normally the load order of dithering and 3DLUT's would not make much difference when custom, non-calibration-related LUT's / 3DLUT's, are created. That is because such LUT's affect grayscale at an even/equivalent rate/interval and no grayscale gradient banding can be observed.

However, when a calibration 1LUT/ICC profile or even 3DLUT (from DisplayCAL that was also known as dispcalGUI) is made, grayscale gradient banding is usually seen. Banding is a result of slight inaccuracies between adjacent grayscale levels and when grayscale gradient is presented, banding can be easily seen on 8bit displays (which contain only 256 shades of gray), but they are not as obvious on 10bit+ displays (which, AFAIK, contain 1024 shades of gray). MadVR video renderer (for film playback), which also supports identical-to-ReShade 3DLUT's, successfully reduces banding by applying madVR's dithering methods, including Ordered Dithering (which ReShade supports) after 3DLUT application. Applying dithering BEFORE TuningPalette would not have a smoothing effect on grayscale banding caused by 3DLUT's or 1DLUT's.

Does ReShade apply 3DLUT before or after applying dithering? It should definitely do it after applying dithering to also reduce grayscale gradient banding. I know madVR also uses internal 16bit processing of 3DLUT's to provide even further gradation improvement, but 16bit internal processing for ReShade TuningPalette may be too much and too heavy on GPU.

What about other effects, like Grain / FilmGrain from either CustomFX and / or SweetFX?


Off-topic Side-Question:
//>Core<\\
#define RFX_InitialStorage 0 //[0:1] //-Set this to 0 if you !only! use the SweetFX suite and want some additional fps.
#define RFX_DepthBufferCalc 0 //[0:1] //-Set this to 0 if run an application that does not allow depth buffer access or you use no depth dependent effects and want some additional fps.

That above is recommended setting for better perofmrnace when shaders / features that do not use/rely/need either Initial Storage or Depth Buffer Calculation are in use. Could someone provide a list of features that do use either one of them? Out of those, which ones use/need/rely on Initial Storage and which one use/need/rely on Depth Buffer Calculations?

Please Log in or Create an account to join the conversation.

  • crosire
More
8 years 2 months ago #2 by crosire Replied by crosire on topic Is Dithering applied AFTER TuningPalette 3DLUT?
You are free to set that order yourself in Pipeline.cfg.

Please Log in or Create an account to join the conversation.

  • MonarchX
  • Topic Author
More
8 years 2 months ago #3 by MonarchX Replied by MonarchX on topic Is Dithering applied AFTER TuningPalette 3DLUT?
OK, but I don't want to mess anything up... I mean there are some important things that work nicely. I think LumaSharpen gets applied BEFORE FXAA so that it doesn't cause addition aliasing. I am not sure what to touch and what not to touch...

What about my side-question? The side-question posted above was:
//>Core<\\
#define RFX_InitialStorage 0 //[0:1] //-Set this to 0 if you !only! use the SweetFX suite and want some additional fps.
#define RFX_DepthBufferCalc 0 //[0:1] //-Set this to 0 if run an application that does not allow depth buffer access or you use no depth dependent effects and want some additional fps.

That above is recommended setting for better perofmrnace when shaders / features that do not use/rely/need either Initial Storage or Depth Buffer Calculation are in use. Could someone provide a list of features that do use either one of them? Out of those, which ones use/need/rely on Initial Storage and which one use/need/rely on Depth Buffer Calculations?

Please Log in or Create an account to join the conversation.

  • MonarchX
  • Topic Author
More
8 years 1 month ago #4 by MonarchX Replied by MonarchX on topic Is Dithering applied AFTER TuningPalette 3DLUT?
IMHO, the Pipeline.cfg needs editing to have different defaults. I think it makes no sense to apply ReShade Shared before most of the shaders listed below it. For example, Dithering, Grain / FilmGrain from ReShade SweetFX are sharpened by LumeSharpen and AdaptiveSharpen if ReShade Shared is not loaded after them. That creates a highly unwanted effect of sharpened Dithering and Grain, making them too sharp and too visible. Of course CustomFX TuningPalette 3DLUT should be applied as the first (top-most) effect, unless there is a specific reason to keep something else at the top. As I mentioned, applying Dithering and Grain on top of 3DLUT's reduces banding caused by calibration, although not everyone use TuningPalette 3DLUT's for calibration.

Please Log in or Create an account to join the conversation.

We use cookies
We use cookies on our website. Some of them are essential for the operation of the forum. You can decide for yourself whether you want to allow cookies or not. Please note that if you reject them, you may not be able to use all the functionalities of the site.