Performance tanks @ 4k resolution

Tested this with both games I have installed right now, Witcher 3 and GTA V. Works fine for my monitor (2560x1600) or if I loser my 4k tv to 1080p. Basically the game completely tanks after a minute or two.. also entering the menu and going back to the game seems to tank performance for both games. Drops to a crawl afterwards. Any ideas?

3x 980's
4790k
16gb ram

are my specs

I get something similar in Fallout 3 at times when I enable a couple of effects when running the game at higher screen resolutions, I'm assuming it's simply due to how complex some of these effects are and it's multiplied as you render more pixels on-screen, 4K or 3840x2160 is 4 times the res of 1920x1080 after all but I can't say if that means all shader effects will be 4x as demanding though I'd imagine effects like ambient occlusion and DOF can be real framerate killers as pixel count increases.

Also unsure if there's any good way to optimize it further, I guess that's something the actual shader creators will have to answer better.

EDIT: Oh and I imagine it's very dependent on the actual game too or rather how much said game is already pushing the GPU and CPU, Fallout 3 for example being very dependent on CPU performance or say Just Cause 3 which is currently suffering from a number of memory leaks and other optimization issues which probably causes all sorts of issues even without adding extra work from processing the ReShade shader effects.

It's weird because the game works fine for a few minutes.. then it just randomly tanks. Any suggestion on which features to disable to stop this from happening?

Check your GPU memory usage. This is just a though, but considering that a 4K resolution requires quite a huge backbuffer, it could be that your GPU memory fills up to the limit after a few minutes, which then takes a hit on performance. Shaders which make use of additional textures with the dimensions of the backbuffer only make this worse.

Where X is the number of additional backbuffer-sized texture created by all active effects (depth-based effects and bloom do this for instance). This includes working buffers, back buffer copies, depth buffer copies, etc.