Eye Tracking Support in ReShade [Tobii EyeX]

Hi!
I work at a company called Tobii. We develop eye trackers. www.tobii.com/xperience/
Eye tracker is a device that captures where exactly you look on the screen in real time.

Currently we entering consumer market with our peripheral (Tobii EyeX - $139) and integrated devices (MSI GT72S Dominator Tobii laptop) with a main focus on gaming.

Eye tracking can be used in games in various ways - object selection, interaction, aiming, camera control, ui adjustments, eye contacts, gaze triggers etc.
Of course we integrate these features in games when we have source access (Assassin's Creed Syndicate, Tom Clancy's The Division, The Hunter and other) and when we can make a mod like GTA V Eye Tracking Mod ( www.gta5-mods.com/scripts/gta-v-eye-tracking-mod ) or when we can feed in gaze data through joystick or freetrack input (Elite Dangerous, Arma 3, DayZ, ETS2 and other)

So i was thinking that it would be great to add eye tracking gaze coordinates as input to shaders in ReShade. This will let us do effects like dynamic depth of field or dynamic light adaptation for a big number of games. And probably many other. Like eye controlled flashlight, crazy peripheral vision effects.

We can assist and do all of the programming as well as give away some Tobii EyeX eye trackers for free to the developers.

Dynamic light adaptation (Assassin's Creed Syndicate) - 1:38


Dynamic depth of field (EyeX SDK) - 4:20


Tobii EyeX review (featuring MSI GT72S Dominator Tobii, Windows Hello, Assassin's Creed Syndicate, GTA V Eye Tracking Mod)

Hi,

The idea of dynamic depth of field and adaption via eye tracking does really sound great and should be possible with ReShade given support for tracker input was added. I took a quick glance at the C/C++ SDK you provide and it looks pretty straight forward to implement.

I tried out various virtual reality solutions in the past (experienced the Oculus Rift DK2 and the new HTC Vive so far), but never eye tracking, so this is new ground and definitly something I'd like to pursue (and that probably counts for everybody involved in the ReShade project). Especially since it could open up some really interesting new possibilities for ReShade shaders, as you pointed out.

If you like we can get into further contact via E-Mail ([email protected]).

Great that you got interested! I just sent you an email

So... THAT just happened...

Shows the popularity that ReShade is commanding these days - very cool.

I got the Steelseries Sentry (also made by Tobii)

I would really love ReShade even more (if that's even possible) if you got that stuff working
looks great!

Foveated rendering would be a great boon to people on lower-end setups. You could render most of the frame at 50% resolution, and render the gaze-tracked portion of the screen at full resolution. Similarly, for higher-end setups, you could enable supersampling or other heavy quality enhancement for the gaze-tracked portion.