Welcome, Guest.
Username: Password: Remember me

TOPIC: UIDetect

UIDetect 4 months 2 weeks ago #1

Update: UIDetect is now part of the offical ReShade shaders. For the latest updates visit my repository github.com/brussell1/Shaders
Example Video (here DOF, CA and Moviebars get toggled when UI becomes visible)

This shader can be used to toggle effects depending on the visibility of UI elements. Unlike UIMask, that uses a mask to decide which area to spare, this one automatically turns off effects for the whole screen. Hence it's mainly useful for (hudless) first person games, where one wants to use post processing like DOF, CA or AO, which however shouldn't be active when certain UI elements are displayed (e.g. inventory, map, dialoque boxes, main menu, options etc.).
Unfortunately there is no algorithm that magically detects those UI elements, so we have to manually define them by choosing a number of distinctive pixels.
In short the UI detection logic is like this:

IF ((UI1_Pixel1_Detected = true) AND (UI1_Pixel2_Detected = true) AND ... ) THEN {UI1_Detected = true}
IF ((UI1_Detected = true) OR (UI2_Detected = true) OR ... ) THEN {UI_Deteced = true}
IF (UI_Detected = true) THEN {EFFECTS = false}

And the workflow to get the pixels should be as follows:
-Take a screenshot without any effects when UI layer is visible
-open screenshot in an image processing tool (I use gimp)
-look for a static and opaque area in the UI layer that is usually out of reach for user actions like the mouse cursor, tooltips etc. (usually somewhere in the corner of the screen)
-use a color picker tool and choose two, three or more pixels, which are near to each other but differ greatly in color and brightness, and note the pixels coordinates and RGB values
(thus choose pixels that do not likely occur in non-UI game situations, so that effects couldn't get toggled accidently when there is no UI visible)
-add pixel coordinates and RGB values to UIDetect.fxh (see below)

The shader consists of two files (copy and it save with the according name):

UIDetect.fxh
//Game: COD4:MW , Resolution: 1920x1080

#define PIXELNUMBER 8

static const float3 UIPixelCoord[PIXELNUMBER]=
{
    float3(562,121,1),  //TAB - Mission details
    float3(614,121,1),
    float3(1589,106,2), //ESC - Menu
    float3(1695,106,2),
    float3(272,40,3),   //Options,Controls
    float3(272,33,3),
    float3(1238,174,4), //Main Menu
    float3(1273,174,4),
};

static const float3 UIPixelRGB[PIXELNUMBER]=
{
    float3(255,204,102),
    float3(255,204,102),
    float3(255,204,102),
    float3(255,204,102),
    float3(255,204,102),
    float3(255,204,102),
    float3(255,255,250),
    float3(255,255,249),
};
You have to manually edit this file (sorry, no GUI mode possible here) and add the follwing information (change existing example code):
-PIXELNUMBER is the total number of pixels you wish to use for UI detection
-UIPixelCoord is an array with PIXELNUMBER as index that defines the UI pixels screen space coordinates and UI layer number;
-UIPixelRGB is an array with PIXELNUMBER as index that defines the UI pixels RGB values

UIDetect.fx
//++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
// ReShade effect file
// UIDetect by brussell
//++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

#include "ReShade.fxh"

texture texColor_Orig { Width = BUFFER_WIDTH; Height = BUFFER_HEIGHT; };
sampler Color_Orig { Texture = texColor_Orig; };

texture texUIDetect { Width = 1; Height = 1; Format = R8; };
sampler UIDetect { Texture = texUIDetect; };

float PS_UIDetect(float4 pos : SV_Position, float2 texcoord : TEXCOORD) : SV_Target {
    #include "UIDetect.fxh"

    float4 color = tex2D(ReShade::BackBuffer, texcoord);
    float diff;
    float ui_detected = 0;
    float uilayer = 1;
    float nextuilayer = 0;

    for (int i=0; i < PIXELNUMBER; i++)
    {
        [branch]
        if (UIPixelCoord[i].z - uilayer == 0){
            if (nextuilayer == 0){
                diff = pow(dot(tex2Dlod(ReShade::BackBuffer, float4(UIPixelCoord[i].xy * ReShade::PixelSize.xy, 0, 0)).xyz - UIPixelRGB[i].xyz / 255.0, 0.333), 2);
                if (diff < 0.00001) {
                    ui_detected = 1;
                }else{
                    ui_detected = 0;
                    nextuilayer = 1;
                }
            }
        }else{
            if (ui_detected == 1){ return ui_detected; }
            uilayer += 1;
            nextuilayer = 0;
            i -= 1;
        }
    }
    return ui_detected;
}

float4 PS_StoreColor(float4 pos : SV_Position, float2 texcoord : TEXCOORD) : SV_Target {
    return  tex2D(ReShade::BackBuffer, texcoord);
}

float4 PS_RestoreColor(float4 pos : SV_Position, float2 texcoord : TEXCOORD) : SV_Target {
    float4 color = tex2D(UIDetect, float2(0,0)).x == 1.0 ? tex2D(Color_Orig, texcoord) : tex2D(ReShade::BackBuffer, texcoord);
    return color;
}

technique UIDetect {
    pass {
        VertexShader = PostProcessVS;
        PixelShader = PS_UIDetect;
        RenderTarget = texUIDetect;
    }
}

technique UIDetect_Before {
    pass {
        VertexShader = PostProcessVS;
        PixelShader = PS_StoreColor;
        RenderTarget = texColor_Orig;
    }
}

technique UIDetect_After {
    pass {
        VertexShader = PostProcessVS;
        PixelShader = PS_RestoreColor;
    }
}
The main shader code. No manual edits needed.

Effect ordering:
-UIDetectmust be first in pipeline (needs unaltered backbuffer)
...effects that affect UI
-UIDetect_Beforeplace before effects that shouldn't affect UI
...effects that should be turned off when UI is visible
-UIDetect_Afterplace after effects that shouldn't affect UI
...effects that affect UI

Drawbacks:
-pixels are only valid for one game resolution off course
-pixel values can change with different hardware anti aliasing settings
Last Edit: 2 months 1 day ago by brussell.
The administrator has disabled public write access.
The following user(s) said Thank You: crosire, Wicked Sick, Qsimil

brussells humble shaders: UIDetect 4 months 2 weeks ago #2

You definitly want "UIPixelCoord" and "UIPixelRGB" to be "static const float3", or else they will be interpreted as uniforms instead of constants, which is considerably slower.
Cheers, crosire =)
The administrator has disabled public write access.
The following user(s) said Thank You: Pondural, brussell, Qsimil, Fu-Bama

brussells humble shaders: UIDetect 4 months 1 week ago #3

I've added an example video to my initial post, in case anybody thinks tl;dr
The administrator has disabled public write access.

UIDetect 4 months 1 week ago #4

Your shader is great, thank you for your work!

I had an idea that might interest you!
All PC video game interfaces that are played on the keyboard/mouse have a common point when displayed on the screen (and which is identical for all resolutions) "A cursor appears".

Here's my idea: your shader should be able to detect the presence of the cursor on the screen, it compares the pixels of the cursor (relative to an image of it in the folder "textures") and it makes this comparison on the position of the cursor in the screen.
(The shader must be able to detect several different cursors because there are games that use more than one cursor).

With this system, you can even create a shader to display a custom cursor when the interface appears!

There is already a shader that can replace the cursor (but it's still visible.), it can maybe help you, you can find it here.

(Excuse my bad English, I'm french.)
Last Edit: 4 months 1 week ago by Illuzio.
The administrator has disabled public write access.

UIDetect 2 months 1 week ago #5

@brussell:
thank you very for your shader! Want to know Can use the mouse to get the coordinates of the ui element and the corresponding RGB value sequence instead of the eyedropper when the effect sequence is not started? tanks!
Last Edit: 2 months 1 week ago by hunt1hunt.
The administrator has disabled public write access.

UIDetect 2 months 1 week ago #6

@Illuzio
I've tried to use the mouse pointer for ui detection recently, but it's not really reliable. Every game behaves differently.

@hunt1hunt
Currently not, but I'll look into it!
The administrator has disabled public write access.

UIDetect 3 weeks 5 days ago #7

I tried everything for past 3 hours tonight to get this shader working and have been completely unable to hide any UI in ESO from DoF.
The administrator has disabled public write access.

UIDetect 3 weeks 22 hours ago #8

After messing around with this some more, I think the problem is because the UI is semi-transparent.

Any work around for this?
The administrator has disabled public write access.

UIDetect 3 weeks 18 hours ago #9

No. But you could try it with an UI mod like this one: www.esoui.com/downloads/info1678-CleanerHUD.html
The administrator has disabled public write access.