New shader: cinematic depth of field

OtisInf wrote: About the AA: no idea. It might be the slight blur on the arrow might cause the AA applied on the arrow to get ruined, showing the actual pixels again. (Hence engines apply AA after DoF) In reshade the dof is applied after the AA which I think could lead to these artifacts. I have no other explanation.

Issue 2, that's indeed something that's not really solvable. It does bleed background into the foreground to create the blurred outline so it looks soft. In theory the best way to do this is to blur the area behind the object, but those pixels aren't known, so this background reconstruction is faked by pulling pixels from the background area around the object which are hoped to look the same. It sometimes looks a tad odd at places, but it's not something I can solve.

Interestingly, I suddenly couldn't replicate that aliasing issue! No idea why or how, since none of my settings changed during play sessions, but hey: if it resolved itself, let's not question it.
And yeah, I figured the second issue would be unsolvable. I've fiddled with faking defocusing in images before, and I realised it's nigh impossible without reconstructing the background to some degree. I guess a fix would be to make the near-plane blur outwards rather than along the edges, but that's not realistic, is it.

As for the third issue, I realised it pretty much happens in all situations where there's some amount of far-plane gradient, especially in the immediate far-plane. It's very clear in this screenshot, where you can see a huge mess of edges as it transitions to background:

Judging from the debug shots I'd say it's pretty much the same thing you've highlighted with the whole CoC stuff.

It occurs in other games too, here's an instance of it happening in NMS:


Anyways, it looks like you already have some ideas for solutions, so best of luck with those as well. Looking forward to seeing it fixed, as it really messes up screenshots that play with any semblance of depth.

Another instance of it happening:

I have no solutions, I only know it's more visible when the farplane blur is high and the aperture is high too, meaning the CoC's are relatively small, but blur is made huge due to the fareplane blur being high (> 1.5) Like I said: keep farplane blur first at 1 and tweak the blur first with lenslength and aperture.

At the moment I have no solution how to fix it if you use a high farplane blur and high aperture.

With aperture set to 1, which for the most part does fix the issue, but who doesn't like some far-plane gradients?:

Aperture set to 1 is a bit excessive, but it can clearly be mitigated by a lower aperture, e.g. 2-3.

Something else: I noticed the shader has difficulty blurring thin near-plane objects as clearly seen in the flower of the last screenshot. I believe a similar issue was discussed in that UE SIGGRAPH Life of a Bokeh presentation, slides 98-101?

Yep. the 'solution' they give there doesn't apply to my shader, as I don't use tiles, but a blurred CoC, which is used for opacity. If you enable nearplane debug you'll see the blurred CoC is very vague, i.o.w. has no real influence. The solution is to use tiles, but I have no 100% reliable implementation of that (I tried a tiled approach before fixing the approach I had, and the tiled approach didn't give me the results I wanted, also because essential info how to exactly blend the pixels during the blur is unknown. No presentation gives that info.) (using tiles then leads to this problem they're having too btw). The approach I'm using (a blurred CoC) works in a lot of cases, but can lead to problems in some cases.

The tiled approach basically simply blurs all near plane pixels in the tile with the same CoC, then based on depth you calculate an alpha and blend pixels together which should result in a smooth transition. I couldn't get that to work. The alpha values apparently were still off, so as this is a hobby project and it started to get annoying I abandoned that. Perhaps I'll implement that one day.

(edit) now that I'm writing this, there might be a solution, but not sure if that's going to work in all cases with thin lines in near plane: use max(blurredCoC, realCoC) for nearplane pixels, which could mitigate the low CoC from the blurred variant. It still won't blur a lot into the farplane tho, so don't get your hopes up.

Oh, interesting. I actually had my aperture already set at 2.7 for all of the screenshots, but also with a pretty high focal length at 95mm. Thought it'd be a bit more "realistic", since I tend to use high zoom/focal lengths in my screenshots. I did however leave my FP Max at 2.5, so that's probably why. Will update with some screenshots in a while.

altokitty wrote: Oh, interesting. I actually had my aperture already set at 2.7 for all of the screenshots, but also with a pretty high focal length at 95mm. Thought it'd be a bit more "realistic", since I tend to use high zoom/focal lengths in my screenshots. I did however leave my FP Max at 2.5, so that's probably why. Will update with some screenshots in a while.

Yeah it's a problem, and I hope to address it. I actually started to understand what Jimenez meant in his paper, and it now all makes sense, I just have to read up some more to be able to apply it. Will try some prototypes today, see what it brings.

Tile based near-plane blur bleed now works great (in most places), also with branches.

Still lots of work to do (as it's totally not optimized). Also fixed the tree leaf mess, that now also looks great. It can still use some better weighting tho but there's progress at least

(yes I know the tile edges are visible at places, still looking for a way to blend it better... as multiple objects in the near plane with different CoC still have a problem)

Ooo, can't wait to play with that!

By the way, I took a look at your screenshots page on your website and I must say, those are gorgeous. I really like the use of multiple aspect ratios, something I should really play with more since I also use SWRE. I want to ask, how're the Dramatic Portraits (isolated B&W subject & black background) done? I read you use Lightroom to make it B&W, but how's the overly dramatic lighting and background separation achieved?

altokitty wrote: Ooo, can't wait to play with that!

Still needs to solve a few things (which currently look ugly) so it's not ready yet

By the way, I took a look at your screenshots page on your website and I must say, those are gorgeous. I really like the use of multiple aspect ratios, something I should really play with more since I also use SWRE. I want to ask, how're the Dramatic Portraits (isolated B&W subject & black background) done? I read you use Lightroom to make it B&W, but how's the overly dramatic lighting and background separation achieved?

Thanks Lower brightness, increase contrast, play with the black/white curves. Most B&W are postsprocessed in faststone image viewer, which allows for lowering brightness and increasing contrast at the same time, giving that effect. Vignetting helps too, and desaturation and then increasing e.g. red or blue or green separately could give a better B&W image (not in lightroom shader, but in faststone)

Dunno if you already implemented this (I'm on my phone so it's not exactly convenient to read through everything), but you can achieve smooth focus transitions using two textures with lerping.

Check out the code for Arcane Bloom, Magic Bloom or Light DoF, the former two implement this for smooth adaptation, the latter for the very thing you want to do.

I haven't implemented it yet, I'm currently wrapping up better highlight implementation and have to optimize the blur passes with a pre-calculated ring buffer before release. If I've time I'll add it, it's not hard to do indeed

OK! Here's a new version. (v1.1). PR: github.com/crosire/reshade-shaders/pull/109 (with screenshots)

edit: Merged

What's new?

• Now using half-res for better performance (5 rings, no undersampling artifacts, 1.89ms @ 1200p)
• New highlight implementation to keep highlight bokeh, and optional old one still available
• Better near plane bleed which deals properly with e.g. branches and edges
• Auto-focus transition delay
• Better weight for far plane so DoF is now more evenly applied in far plane, avoiding ugly edges around e.g. leafs
• Lots of small optimizations and tweaks.

I think I've addressed most complaints/issues and feature requests (except the CA one, I think that's not going to happen... )

I'm very happy with the end result as it is now. Still a lot of passes, I think I can merge the upsampler with the near plane blur, maybe later. Its performance is now pretty good, 1.89ms in AC:Odyssey on a 1070 @ 1200p, but that always changes depending on what you're using it with of course. (I've seen scenes with 3.2ms of consumption, bumping the scene up to 4K then roughly results in 7ms.)

Enjoy!

Couple of bug fixes for Cinematic DOF in this PR: github.com/crosire/reshade-shaders/pull/111

- Near plane blur is now resolution independent
- Small tweak in luma constant usage (less constructor calls), pre-blur luma weight adjustment for out-of-focus areas
- Fixes bug in highlight bleeding due to pre-blur not masking enough.
- Small tweak in pre blur to mitigate highlight bleed in near-focus areas
- Average CoC calculation forgot central pixel
- Combiner now uses full res for far plane if far plane blur is set to 0 so you can create screenshots with just near plane blur.

(edit) Merged

Bug fix for near plane blur, it fixes artifacts which could occur in some TAA using games (which use depth buffer jitter), and it also corrects blur around corners of narrow elements. github.com/crosire/reshade-shaders/pull/114 (merged)

I had CoC averaging implemented (averaged positive and negative CoCs together, yeah it sounds good on paper but really isn't ) and didn't notice an issue with it in my test setup (UE4 demo) but noticed in some TAA using games with very small jitter in the depth buffer that the edge tiles in the near plane could get very big negative CoC values and therefore cause significant blurring around the edges but not elsewhere in the near plane. That of course looks awful so I removed that pass altogether as the net effect of it wasn't that big anyway (I mainly implemented it to mitigate depth buffer jitter in TAA setups)

Change is merged in main repo and OtisFX repo.

Is it possible to completely remove near plane blur?


Take a look at the picture [in full size] and you will see that even with near blur set to 0, out of focus near objects suffer from pixelation/blur. Is it possible to completely eliminate this thing?

If it's possible, then it will be perfect. This way I can use DOF even for actual gameplay instead of just taking screenshots.

Sure, shouldn't be too hard.

OtisInf wrote: Sure, shouldn't be too hard.

That's nice to know.

Sami 1999 wrote:

OtisInf wrote: Sure, shouldn't be too hard.

That's nice to know.

Having added the code, looking at your screenshot I don't think it will help you there. The problem is likely that the UI that's blurred isn't in the depth buffer with a depth close to the camera: UIs are often 2D planes rendered on top of the scene and not doing any depth testing (as they always are in front of the scene), and rendered after the whole scene is already rendered. This means that a pixel in the UI will be seen as a pixel at the depth of the pixel it's covering (e.g. in your shot a pixel in the scene far away).

The only solution is to add a mask for UI elements. I'm not going to add that. (so the issue in the characters in the team members on the right in your shot).

The aliasing on the left is caused by the half-res buffers used. I don't have a bicubic upscaler implemented yet (it's a simple bilinear one) so the staircases you see will sometimes happen. It might be mitigated with the change I made now for near plane blur == 0 -> original fragment.

OtisInf wrote:
The only solution is to add a mask for UI elements. I'm not going to add that. (so the issue in the characters in the team members on the right in your shot).

The aliasing on the left is caused by the half-res buffers used. I don't have a bicubic upscaler implemented yet (it's a simple bilinear one) so the staircases you see will sometimes happen. It might be mitigated with the change I made now for near plane blur == 0 -> original fragment.

I don't mind the blurring of UI. I will sort that out later.

Right now, being able to fix the pixelation is enough for me. By the way, even if bilinear/ bicubic upscaler is implemented, the objects will still look blurry. Isn't it possible to completely bypass DOF completely on near objects so that nothing is blurred/pixelated? Or maybe it's possible to use full res buffer but I guess that will cause severe performance penalty?


Edit: I've tested your latest edit of the shader and looks like it solved the pixelation problem, at least in PES 08. I will see if it's the same on other games [Most probably they will be alright too]. Here's some screenshot with your latest version: