What is the cause of compile problems after 4.0?

luluco250
Topic Author

5 years 3 months ago #1 by luluco250 What is the cause of compile problems after 4.0? was created by luluco250

I've seen that many of my shaders, including the bloom ones, are getting this "error X4509: maximum sampler register index exceeded, target has 16 slots, manual bind to slot s16 failed" errors. I haven't worked on them for a while and have been out of touch with the scene in general, what changed in this version to cause this? I know the compiler has been rewritten (I think), so that must be part of the reason.

I'm still trying to figure out how to fix this, if someone could help I'd appreciate it.

The following user(s) said Thank You: Wicked Sick

Please Log in or Create an account to join the conversation.

crosire

5 years 3 months ago - 5 years 3 months ago #2 by crosire Replied by crosire on topic What is the cause of compile problems after 4.0?

That happens if you use more than the allowed number of samplers in your effect in some APIs. Nothing new with the new compiler there, except that ReShade 4 defines creates separate sampler bindings for each effect, so this is even less likely to occur than before. If you don't use that many samplers, then that's a bug. Which API does this occur with?

Last edit: 5 years 3 months ago by crosire.

Please Log in or Create an account to join the conversation.

luluco250
Topic Author

5 years 3 months ago #3 by luluco250 Replied by luluco250 on topic What is the cause of compile problems after 4.0?

crosire wrote: That happens if you use more than the allowed number of samplers in your effect in some APIs. Nothing new with the new compiler there, except that ReShade 4 defines creates separate sampler bindings for each effect, so this is even less likely to occur than before. If you don't use that many samplers, then that's a bug. Which API does this occur with?

It's happening to me with DirectX 11 in Deus Ex: Mankind Divided. I don't think it's only happening in this game, someone commented on my repo that the same problem was happening to them. Dunno if that's a DirectX 11-only problem, but I don't think it was happening in ReShade 3.x.x.

More specifically, it's Arcane Bloom, which was working fine before. I'm trying to figure out if I'm using too many samplers, but I shouldn't be, I don't think so.

Please Log in or Create an account to join the conversation.

luluco250
Topic Author

5 years 3 months ago #4 by luluco250 Replied by luluco250 on topic What is the cause of compile problems after 4.0?

Weird, it seems that if I disable the adaptation code the shader suddenly works, the ReShade in-game editor actually highlighted this code as being problematic:

Warning: Spoiler!

My guess is that the adaptation code adding more samplers is causing the problem, but it's still weird since this wasn't a problem in ReShade 3.x.x.

Please Log in or Create an account to join the conversation.

crosire

5 years 3 months ago - 5 years 3 months ago #5 by crosire Replied by crosire on topic What is the cause of compile problems after 4.0?

Your shader is using more than 16 samplers (bloom + alt bloom + backbuffer + adaption samplers) and ReShade 4 does not currently re-use sampler bindings for samplers with the same settings (ReShade 3 did).
Will try and come up with a solution.

Last edit: 5 years 3 months ago by crosire.

Please Log in or Create an account to join the conversation.

crosire

5 years 3 months ago #6 by crosire Replied by crosire on topic What is the cause of compile problems after 4.0?

Fixed for D3D10 and 11 in github.com/crosire/reshade/commit/aa0c37...07ae89056108f84ee157 . It will continue to not work in D3D9 though.

The following user(s) said Thank You: Wicked Sick

Please Log in or Create an account to join the conversation.

luluco250
Topic Author

5 years 3 months ago #7 by luluco250 Replied by luluco250 on topic What is the cause of compile problems after 4.0?

crosire wrote: Fixed for D3D10 and 11 in github.com/crosire/reshade/commit/aa0c37...07ae89056108f84ee157 . It will continue to not work in D3D9 though.

Wow thanks, but I'll try to slim down my sampler usage then, I think I can remove one bloom texture without it making much difference.

But still, it's good that DXGI behavior will work as it should, but I want to support D3D9 too so I'll have to optimize.

Again, thanks for the help!

The following user(s) said Thank You: Wicked Sick

Please Log in or Create an account to join the conversation.

Wicked Sick

5 years 3 months ago #8 by Wicked Sick Replied by Wicked Sick on topic What is the cause of compile problems after 4.0?

This topic made me happy. Thanks, guys.

Please Log in or Create an account to join the conversation.

luluco250
Topic Author

5 years 3 months ago - 5 years 3 months ago #9 by luluco250 Replied by luluco250 on topic What is the cause of compile problems after 4.0?

I'm toning down the sampler count on my shader by removing the ReShade.fxh dependency, that way I don't get two duplicate BackBuffer samplers and no DepthBuffer sampler either. I'm curious about the meaning of this "__RESHADE_FXC__" macro and if I should replicate it.

ReShade.fxh

Warning: Spoiler!

Last edit: 5 years 3 months ago by luluco250.

Please Log in or Create an account to join the conversation.

luluco250
Topic Author

5 years 3 months ago #10 by luluco250 Replied by luluco250 on topic What is the cause of compile problems after 4.0?

That did it, dropping the two samplers from ReShade.fxh worked, exactly 16 samplers now. I guess I could reuse one of the alt textures for a temporal effect I wanted to implement officially (I only did it on a personal build, MagicBloom3 I think). Gonna update the code at my repo. Might as well make a PR for reshade-shaders.

Please Log in or Create an account to join the conversation.

crosire

5 years 3 months ago #11 by crosire Replied by crosire on topic What is the cause of compile problems after 4.0?

luluco250 wrote: I'm curious about the meaning of this "__RESHADE_FXC__" macro and if I should replicate it.

It's for a standalone compiler, where the BUFFER_WIDTH and BUFFER_HEIGHT defines are not actual defines, but uniform variables. This is not currently in-use by ReShade itself.

Please Log in or Create an account to join the conversation.

luluco250
Topic Author

5 years 3 months ago #12 by luluco250 Replied by luluco250 on topic What is the cause of compile problems after 4.0?

crosire wrote: It's for a standalone compiler, where the BUFFER_WIDTH and BUFFER_HEIGHT defines are not actual defines, but uniform variables. This is not currently in-use by ReShade itself.

Oh I see, I guess I'll replicate it exactly as it is just to be safe then.

Please Log in or Create an account to join the conversation.

OtisInf

5 years 3 months ago #13 by OtisInf Replied by OtisInf on topic What is the cause of compile problems after 4.0?

Can't you use mipmaps in this situation? That would give you more headroom wrt the # of samplers. (Disc: I'm not a mipmap expert so it might be mipmaps can only be a power of 2)

Please Log in or Create an account to join the conversation.

luluco250
Topic Author

5 years 3 months ago #14 by luluco250 Replied by luluco250 on topic What is the cause of compile problems after 4.0?

OtisInf wrote: Can't you use mipmaps in this situation? That would give you more headroom wrt the # of samplers. (Disc: I'm not a mipmap expert so it might be mipmaps can only be a power of 2)

I've tested this before and the quality isn't great, mipmaps are too stretchy and blocky for high quality bloom.

The following user(s) said Thank You: Wicked Sick

Please Log in or Create an account to join the conversation.

OtisInf

5 years 3 months ago #15 by OtisInf Replied by OtisInf on topic What is the cause of compile problems after 4.0?

Not necessarily

Check this article: www.elopezr.com/the-rendering-of-rise-of-the-tomb-raider/ , scroll down to 'Bloom'. They use a nifty upscaling trick with mipmaps which gives great results. perhaps you can utilize it too. :)

The following user(s) said Thank You: Wicked Sick

Please Log in or Create an account to join the conversation.

luluco250
Topic Author

5 years 3 months ago #16 by luluco250 Replied by luluco250 on topic What is the cause of compile problems after 4.0?

I guess I could take a look again, boy am I tired of rewriting the same effect for the 6th time.

Please Log in or Create an account to join the conversation.