Star Citizen’s facial recognition tech is horrifyingly realistic
Don’t speak, it knows just what you’re saying
Star Citizen’s most ambitious new feature is something called Face Over Internet Protocol (FOIP), which uses motion capture to mirror players’ expressions and facial movements in-game. A demo shown to Polygon at Gamescom proves that not only is this technology impressive, but it’s also ... fairly disturbing.
Cloud Imperium Games’ CEO Chris Roberts debuted the tech to the masses during a presentation held tonight in Germany, and he previewed it for us personally behind closed doors earlier in the week. Produced in collaboration with Faceware Technologies, FOIP uses webcams to detect and animate players’ actual faces within the game. It’s an advanced feature that Roberts referred to as novel in a press release.
“For the first time we’ll be able to deliver the full range of human emotion, not just voice,” he said. Our players’ facial expressions will be translated onto their avatars’ face. Combine that with a player’s voice correctly positioned in the virtual world, and you have the most lifelike player-to-player communication ever.”
It’s lifelike, yes, but that’s not always for the better. What we saw dipped heavily into the uncanny valley. Avatars at first had deadened expressions as their players stared vacantly ahead into the stream. Once aware that we were watching them, though, they began to move their jaws around, blinking their eyes, opening their mouths and showing us the full range of motion.
It was nightmarish, to be frank. To watch virtual lips flap in real time as people across the room from us had stilted conversations was a unique, and uniquely strange, experience. We wouldn’t think to dismiss the Faceware tech, to be clear; it’s just that this level of animation suggests that Star Citizen, a game already suffering from a bloated array of features, could use some serious paring down.
When FOIP is combined with the similar voice over IP, Star Citizen becomes even more bizarre. We watched a conversation happen between two players as they crossed paths on a gigantic spaceship, each avatar flapping their lips emotionlessly at each other. They were perfectly synchronized and matching the words players were speaking, to Faceware’s credit. But we couldn’t help but stare slightly aghast at what was happening.
For people who expect Star Citizen to be the most realistic fictional space drama ever, FOIP is a magnificent step in that direction. But by “magnificent,” we mean on a purely technological basis. The reason uncanny valley exists is because creators aspire for something bordering on the real. Approximating reality is impossible without all of its essential ingredients, however, and some of those just aren’t easily captured through tech — like emotion, or nuance, or subtlety.
The real-time facial recognition tech will make it into the game’s alpha in a later update, although a video above shows what it looks like. To aid players in getting their “believable digital faces” into Star Citizen, Cloud Imperium Games will sell a special, 60 fps-capable webcam at a later date as well.
When will FOIP come to Star Citizen? Not until version 3.1 of the game ships.
