I created an HUD mask for Majora's Mask and have found that it only partially blocks effects from interfering with the UI. With the RT shader, ambient occlusion is successfully blocked from the HUD but for some reason the shader still uses the HUD as a light source and illuminates the area with the hearts or buttons. If I go into the debug view, I can see that the HUD mask is working and cutting the HUD out. The same thing also happens with the SSR shader, with the HUD being reflected on the ground even though the mask is on.
This is with no UI mask applied and the RT shader cranked up fairly high. The AO on the fence clearly cuts through the HUD, the hearts and buttons cast light on the environment:
This is with the mask active. The AO no longer cuts through, but the blue light from the button still illuminates the wall:
This is what the RT debug view shows with the mask active:
People have suggested changing the effects order around but the only arrangement that allows the mask to work at all is with UImaskTop first, RT second, and UImaskBottom last. I'm not sure if this is a limitation of the UImask itself or not. Any advice?