Welcome, Guest.
Username: Password: Remember me

TOPIC: HBAO+ support ?

HBAO+ support ? 11 months 3 weeks ago #1

Hi everyone, I wanted to PM Crosire but I couldn't find the PM button :/

I've been playing around with the idea of integrating HBAO+ library into whatever injector is available. I guess it wouldn't be very useful to NVIDIA users but for us AMD users it would be convenient to have that feature at hand. Note that It would only work for DX11 though.

Reshade source-code not being available I had to dig back my old messy outdated injector and modify it. Well the library is actually really really easy to integrate. You do have to drop the HBAO+ library into the same directory as the targeted game. I actually got it to work on a color texture. It does work but as you can guess that's certainly not the proper texture to feed the shader with of course. You do need the depth texture.

And the problem is that I can't seem to be able to write depth into my own custom depth texture. I'm not even getting a full white texture that would indicate the depth values are not linearized but a black texture, which means... that I'm lost. I mean it could be a lot of things. I can't even use the D3D11 debug runtime since that project still use the old DX SDK from june 2010 which doesn't work anymore for Windows 8/10

I don't know if it's ok to pretend we're in gamedev forums but here's the depth texture/DSV/SRV creation code :

	td.Width = pDesc.BufferDesc.Width;
	td.Height = pDesc.BufferDesc.Height;
	td.MipLevels = 1;
	td.ArraySize = 1;
	td.SampleDesc.Count = 1;
	td.SampleDesc.Quality = 0;
	td.Usage = D3D11_USAGE_DEFAULT;
	td.CPUAccessFlags = 0;
	td.MiscFlags = 0;
	hr = pDevice->CreateTexture2D(&td, 0, &_depthMap);

	ZeroMemory(&dsvd, sizeof(dsvd));
	dsvd.Format = DXGI_FORMAT_D32_FLOAT;
	dsvd.ViewDimension = D3D11_DSV_DIMENSION_TEXTURE2D;
	dsvd.Texture2D.MipSlice = 0;
	hr = pDevice->CreateDepthStencilView(_depthMap, &dsvd, &_dmapDSV);

	srvd.Format = DXGI_FORMAT_R32_FLOAT;
	srvd.ViewDimension = D3D11_SRV_DIMENSION_TEXTURE2D;
	srvd.Texture2D.MipLevels = td.MipLevels;
	srvd.Texture2D.MostDetailedMip = 0;
	hr = pDevice->CreateShaderResourceView(_depthMap, &srvd, &_dmapSRV);

I'm injecting that depthstencil view into a hooked OMSetRenderTargets() :

void ID3D11devcontext::OMSetRenderTargets(UINT NumViews, ID3D11RenderTargetView *const *ppRenderTargetViews, ID3D11DepthStencilView *pDepthStencilView) {
	if (pDepthStencilView) {
		if (DepthMap::get().getdmapDSV()) {
			return pdevContext->OMSetRenderTargets(NumViews, ppRenderTargetViews, DepthMap::get().getdmapDSV()); // DepthMap::get().getdmapDSV() is an accessor to _dmapDSV created above
		else {
			return pdevContext->OMSetRenderTargets(NumViews, ppRenderTargetViews, pDepthStencilView);
	else {
		return pdevContext->OMSetRenderTargets(NumViews, ppRenderTargetViews, pDepthStencilView);

At this stage I'm hoping the "host game" has rendered scene and written depth into my own depth texture since I hacked into OMSetRenderTargets to force mine instead of the game's.
But when I try to output it to the screen it's all desperately black. Maybe it needs to be unbound first... but I think I've done everything described in that article ("Reading the Depth-Stencil Buffer as a Texture" section) right. So I'm confused.

So I was thinking if you guys and especially Crosire struggled in the past with that as well an insight would be warmly welcome.
(I forgot to mention that I'm using a DX11 NVIDIA sample that 100% works with Reshade depth access to make sure I get things right)
Last Edit: 11 months 3 weeks ago by Boulotaur2024.
The administrator has disabled public write access.
The following user(s) said Thank You: piltrafus

HBAO+ support ? 11 months 3 weeks ago #2

The "+" can already be done in reshade since it is capable of 4xMRT, the only thing it required in dx11 is texture array, which probably doesn't matters on performance wise compares to normal texture read. (don't know about d3d9.dll though?)
The administrator has disabled public write access.

HBAO+ support ? 11 months 3 weeks ago #3

  • crosire
  • crosire's Avatar
  • Online
  • Posts: 2313
  • Thank you received: 1316
Replacing the depthstencil view isn't as easy as it sounds. Games tend to use a whole range of different buffers for different usages, which means replacing them all with the same buffer doesn't work, they are too different for that (can be of different size, different format, cleared at various times between rendering etc.).
First thing is to find the one depthstencil view that contains the whole scene depth data (if it exists, sometimes even that data is splitted across multiple depthstencils, unfortunately). Next make sure to only replace exactly that depthstencil with one that is of a format which is readable from shaders. Also important is to carefully unbind it from the render pipeline before actually using it in a shader, since it's impossible to read and write to the same buffer at the same time.
Cheers, crosire =)
The administrator has disabled public write access.

HBAO+ support ? 11 months 3 weeks ago #4

  • Kleio420
  • Kleio420's Avatar
  • Offline
  • Posts: 204
  • Thank you received: 27
maybe i read the TOS wrong but if your using this in anyway that generates profit(donations) i was under the impression hbao+ was not allowed to be used. Please feel free to correct me in the most crappy way possible crosire/anyone else cause you like to do it , i just dont feel like logging in to check something like that
Last Edit: 11 months 3 weeks ago by Kleio420.
The administrator has disabled public write access.

HBAO+ support ? 11 months 3 weeks ago #5

  • Marty McFly
  • Marty McFly's Avatar
  • Offline
  • We've tried nothing and we're all out of ideas!
  • Posts: 591
  • Thank you received: 696
Well HBAO+ is closed source AFAIK and ReShade isn't supposed to hook dlls so it's probably not going to happen anyways. Plus some other AO implementations look better anyways.
The administrator has disabled public write access.
The following user(s) said Thank You: Kleio420