Welcome, Guest.
Username: Password: Remember me

TOPIC: Edge Smoothing

Edge Smoothing 10 months 1 day ago #1

Hi, everyone.
I was wondering if someone could make an edge softening shader, like a gaussian blur filter limited to the edges.

I'm not a fan of rough edges. it feels like smooth edges seems to give more cinematic look to the scene.
Not to mention all those games that uses too aggressive sharpening filters, causing ringing artifacts around the edges:



An example of ringing artifacts: Notice those white haloes around edges.

Here's a simulation of the effect I'd like to achieve, made with photoshop.

I applied gaussian blur to the whole image and then I made a mask using with the photoshop's "find edges" filter, so that the textures (especially the snow and the threes, in this case) are not affected by the blurriness.

Also, the subtle ringing artifact is gone.
Original Shot




Edge Smoothing




Edge Mask



I know that Reshade's FXAA filter could be a solution, but the problem is that it messes with already anti-aliased images:
(In-game TAA is on)

Last Edit: 3 weeks 1 day ago by Duran.te.
The administrator has disabled public write access.
The following user(s) said Thank You: BeTa, acknowledge, Viper_Joe, Dom324

Soft Edges / Oversharp Fix 10 months 1 day ago #2

Well, funnily I've just implemented a way to reduce halos in my sharpening shader. It's nothing sophisticated, but I'm quite satified with the results. Basically I'm just adding sharpness depending on pixel brightness, thus reducing edges in high-contrast areas (like skies).

Here's a comparison (left: original, middle: less sharpness for bright pixels, right: standard sharpness)


Of course due to the uneven sharpening some details can look slightly odd (text etc.), but imo it can achive better results than by just clamping the unsharp mask.
Maybe I will post the code here soon. Although it may lack context, here is the important line:
fine *= (1-saturate(pow(abs(color.xyz), 1 / (fSharpen_DynamicCurve + 0.001))));
Last Edit: 10 months 1 day ago by brussell.
The administrator has disabled public write access.
The following user(s) said Thank You: BeTa, jas01, Viper_Joe, Duran.te

Soft Edges / Oversharp Fix 10 months 23 hours ago #3

Check out this topic reshade.me/forum/shader-presentation/395...n?limitstart=0#26156

Created sharpening filter that addresses just that issue using depth high-pass mask and it's already in default repository under FilmicAnamorphSharpen.fx



Depth pass is required for it to work, but you can also try using "Sharpen clamping" setting without depth pass to reduce halos.
If you're looking for more performance-wise version without depth masking, visit my Shaders repository (link under my profile information at the bootom of this post), look for FilmicSharpen.fx
Last Edit: 10 months 23 hours ago by Fu-Bama.
The administrator has disabled public write access.
The following user(s) said Thank You: Duran.te

Soft Edges / Oversharp Fix 10 months 5 hours ago #4

This certainly solves the first problem. However, as I mentioned above, there's nothing to do with already - oversharpened images, in that case an edge softener shader would help a lot.
The administrator has disabled public write access.

Edge Smoothing 4 months 3 weeks ago #5

Bump.

Topic updated, added reference shots.
Could someone help this case?
The administrator has disabled public write access.

Edge Smoothing 4 months 1 week ago #6

You can try using gaussain blur shader or NFAA.fx from this shader pack. Could work in a few cases. I'm unable to give you any better solution here.
Last Edit: 4 months 1 week ago by jas01. Reason: ...
The administrator has disabled public write access.

Edge Smoothing 4 months 5 days ago #7

Unfortunately, since NFAA is an antialiasing shader, it causes some artifacts with in-game antialiasing on (as you can see in the last screen I posted).
It would also be nice if values like intensity and radius could be controlled with dedicated sliders, rather than just the threshold one.
Thanks for help, anyway. :)
The administrator has disabled public write access.
The following user(s) said Thank You: jas01

Edge Smoothing 4 months 5 days ago #8

A lot of modern games have a sharpening effect enabled. Try look for sharpening settings in the game's ini files.
Last Edit: 4 months 4 days ago by Insomnia.
The administrator has disabled public write access.

Edge Smoothing 4 months 2 days ago #9

Your screenshots look truly great (except that blur and slightly altered colors on the backpack), I can imagine number of games where this could be used.
The administrator has disabled public write access.

Edge Smoothing 4 months 1 day ago #10

Combination of Texture LOD -0.250(driver controlled LOD is off), AMD Sharpening filter (i used a version ported to Reshade) and NFAA is working really well for me
oh and i use Deband to smooth out some of those banding problems, you can see it really well in the skies in some games
Last Edit: 4 months 1 day ago by S3r1ous.
The administrator has disabled public write access.

Edge Smoothing 2 months 3 weeks ago #11

For anyone interested: Pirate_FXAA from PirateKitty's shaders collection works pretty well as an edge smoothing filter, since it seems to not causing artifact on thin lines, like texts: it just smooths them (Guess it's using an older FXAA version).

You'll find it on PirateKitty's GitHub repository: github.com/Heathen/Pirate-Shaders
The administrator has disabled public write access.
The following user(s) said Thank You: jas01, Viper_Joe

Edge Smoothing 3 weeks 1 day ago #12

Just a little concept i made up tonight. might make a proper version with depth+color detection and some settings if you are interested :)
/**
 * Smooth Edges
 * by Jakob W. ~ j4712
 * made for Duran.te 
 * NOT FINISHED JUST A CONCEPT FOR NOT USING DEPTH FOR THIS TASK WITHOUT DESTROYING INGAME AA
 */
 
#include "ReShade.fxh"
#include "ReShadeUI.fxh"

texture TexColorBuffer : COLOR;

sampler SamplerColor
{
	Texture = TexColorBuffer;
	AddressU = MIRROR;
	AddressV = MIRROR;
};

texture blurredColor
{
	Width = BUFFER_WIDTH;
	Height = BUFFER_HEIGHT;
};

texture edgesTex
{
	Width = BUFFER_WIDTH;
	Height = BUFFER_HEIGHT;
};

sampler blurredSampler
{
	Texture = blurredColor;
};

sampler edgesSampler
{
	Texture = edgesTex;
};

float rgbLuma(float3 color)
{
	return float3( color.r * 0.2126,  color.g * 0.7152,  color.b * 0.0722);
}

float3 BlurringPass(float4 vpos : SV_Position, float2 texcoord : TexCoord) : SV_Target
{
	float kernel[9] = { 1, 2, 1,
						2, 4, 2,
						1, 2, 1 };
						
	float3 color = float3(0.f,0.f,0.f);

	int kernalSumm = 0;
	
	for (int y = 0; y < 3; y++)
	{
		for (int x = 0; x < 3; x++)
		{
			int index = y * 3 + x;
			
			float3 pixelColor = tex2D(ReShade::BackBuffer, texcoord + (float2(x-1, y-1) * ReShade::PixelSize)).rgb;
			color += pixelColor * kernel[index];
			kernalSumm += kernel[index];
		}
	}
	
	color /= kernalSumm;
	
	return color;
}

float3 DetectSobelPass(float4 vpos : SV_Position, float2 texcoord : TexCoord) : SV_Target
{

	float kernel_x[9] = { 	-1.f, 0.f, 1.f,
							-2.f, 0.f, 2.f,
							-1.f, 0.f, 1.f 	};
							
	float kernel_y[9] = { 	1.f, 2.f, 1.f,
							0.f, 0.f, 0.f,
							-1.f, -2.f, -1.f };

	float3 color = float3(0.f, 0.f, 0.f);
		
	float edge_x = 0.f;
	float edge_y = 0.f;
	float kernelSumm_x = -1.f;
	float kernelSumm_y = -1.f;
	
	for (int y = 0; y < 3; y++)
	{
		for (int x = 0; x < 3; x++)
		{
			int index = y * 3 + x;
			float brightness = rgbLuma(tex2D(blurredSampler, texcoord + (float2(x-1, y-1) * ReShade::PixelSize)).rgb);
								 
			edge_x +=  (brightness * kernel_x[index]);	
			edge_y +=  (brightness * kernel_y[index]);	
			
			kernelSumm_x += kernel_x[index];				
			kernelSumm_y += kernel_y[index];
		}			
	}
	
	edge_x = ((edge_x / kernelSumm_x) / 2.f);
	edge_y = ((edge_y / kernelSumm_y) / 2.f);
	
	color.r = edge_x;
	color.g = edge_y;
	
	
	
	return sqrt((color.r * color.r) + (color.g * color.g));
}

float3 EffectPass(float4 vpos : SV_Position, float2 texcoord : TexCoord) : SV_Target
{
	float3 oColor = tex2D(SamplerColor, texcoord).rgb;
	float3 bColor = tex2D(blurredSampler, texcoord).rgb;
	float  edgeStrength = 0;
	
	float kernel[9] = { 1, 2, 1,
						2, 4, 2,
						1, 2, 1 };

	int kernalSumm = 0;
	
	for (int y = 0; y < 3; y++)
	{
		for (int x = 0; x < 3; x++)
		{
			int index = y * 3 + x;
			
			edgeStrength += tex2D(edgesSampler, texcoord + (float2(x-1, y-1) * ReShade::PixelSize)).r * kernel[index];
			kernalSumm += kernel[index];
		}
	}
	
	edgeStrength /= kernalSumm;
	
	if (edgeStrength < 0.075)
		edgeStrength = 0;
	edgeStrength = saturate(edgeStrength * 5);
	
	return lerp(oColor, bColor, edgeStrength);
}



technique SoftEdges
{
	pass BlurPass
	{
		VertexShader = PostProcessVS;
		PixelShader = BlurringPass;
		RenderTarget = blurredColor;
	}
	
	pass DetectSobel 
	{
		VertexShader = PostProcessVS;
		PixelShader = DetectSobelPass;
		RenderTarget = edgesTex;
	}	
	
	pass Effect 
	{
		VertexShader = PostProcessVS;
		PixelShader = EffectPass;	
	}
}
i know its messy and that there is a lot of code that cut be cut short. i didnt do any optimization
Last Edit: 3 weeks 1 day ago by j4712.
The administrator has disabled public write access.
The following user(s) said Thank You: Viper_Joe, BFG9000, Duran.te

Edge Smoothing 3 weeks 1 day ago #13

You're kiddin? It looks so neat!
I'd almost given up hope of seeing a custom shader made just for this.
A depth+color detection version with adjustable settings for stuff like threshold, radius and intensity would be so awesome, but I don't want to pressure you about that.

Thanks a lot man, you rock ;)
The administrator has disabled public write access.
The following user(s) said Thank You: j4712