Magical Prototype Lets You Play PS4 With Your Eyeballs

The author (left) tries out Sony’s gaze-tracking system at GDC 2014 in San Francisco. Photo: Josh Valcarcel/WIRED

Sony’s skunkworks team has a new method of game control that feels like magic — and it’s not the virtual reality headset.

Behind closed doors at its booth at Game Developers Conference last week, Sony let me test out some eyeball-tracking technology that its Magic Labs research and development group has been working on integrating with PlayStation 4.

The demo, a specially tweaked version of the new PlayStation 4 game Infamous: Second Son, lets you look around the world of the third-person action game just by moving your eyes. If you want to aim at a target and fire at it, all you have to do is focus your gaze at it.

The hardware itself, a thin yellow bar that looks like a thinner version of a Kinect camera and sits in front of the monitor, isn’t a Sony-created product. It’s created by the German company SensoMotoric Instruments, which says it has worked with Sony over the past few years on refining this technology for games.

Controlling Infamous: Second Son with Sony’s eye tracker. Photo: Josh Valcarcel/WIRED

I wasn’t even supposed to try out gaze tracking at GDC. I was at Sony’s booth to stick my face into Project Morpheus, the PlayStation 4′s VR display prototype. But the unit we were going to use actually broke down, leaving me without anything to do. Sony’s quick-thinking PR rep suggested I try out the gaze tracking demo while they tried to fix Morpheus.

Setup seemed pretty simple, although it was all handled by the Magic Labs developer who was manning the booth. The infrared cameras were pointed diagonally up at my face at an angle that allowed them to best view the movement of my eyeballs, not to mention each of my individual nose hairs.

The game started up. By moving my eyeballs left, right, up and down I could pan the camera around, as if I was using the right analog stick on the gamepad. It felt a little weird, but only for a few seconds. I soon realized that if I wanted to center something in the camera, all I had to do was keep my focus on it.

The crazy thing was, eventually I forgot that I was using eye tracking at all. Focusing on things that we’re interested in is so instinctual, so immediate that it was as if my brain was controlling the camera. It worked too well, felt too much like magic. I was really loving it.

Then it got better. I could aim my superpowers using eye tracking, the Magic Labs rep said, if I just looked at whatever I wanted to shoot. I decided to fire a blast of energy at a speed limit sign. I glanced at it, pressed the fire button and obliterated it. My very gaze had become an instrument of death.

Should we expect one of these to be sitting in front of our TV sets, jockeying for position with the Kinect, the PlayStation Camera and the Wii sensor bar? Not anytime soon, I’d wager.

Put aside the fact that these things are probably pretty expensive (although SensoMotoric doesn’t seem to list a price anywhere on its website). There are still some limitations. SensoMotoric says the technology currently works at its best only on monitor sizes up to 28 inches. And it only works if it’s between 50 and 75 centimeters away from your peepers.

So gaze tracking isn’t quite ready for prime time, at least in the console gaming world. But I’d love it if a consumer model became a reality—even an expensive reality—just so more people can experience how awesome it is to use.