Horizon Forbidden West’s Aloy is a clone in a post-apocalyptic future who uses a bow and spear to fight giant robot animals and corrupted AI programs. It’s a neat premise for what happens to be a very cool sci-fi action-RPG series. But what if Aloy was also controlled by an AI program and could patiently chat with you during her adventures, answering all of your inane questions like “where can I find salmon bones to upgrade my pouches?” That’s the type of thing Sony appears to be experimenting with in a recently leaked GPT demo that teases one of the more horrifying possibilities for gaming’s future.
As first reported by The Verge, the tech was briefly demoed in a leaked internal video that appeared on YouTube and showed Sharwin Raghoebardajal, a software engineering director at Sony, having a back-and-forth conversation with the PlayStation character using various layers of AI including OpenAI’s Whisper machine learning model, GPT-4, and Llama 3. It was run on PC but apparently can work on PS5 as well.
Sony’s own internal Emotional Voice Synthesis (EVS) system is apparently used to generate Aloy’s voice responses, and its Mockingbird technology is used to crudely animate her lips. The result sounds cold and robotic and the lip movements are somewhere between a bad dub and Will Smith eating spaghetti. As far as a rough tech demo goes, however, it’s both silly and deeply chilling. “This is just a glimpse of what is possible,” Raghoebardajal said during the video which has since been removed by a copyright strike company representing Sony.
Sony’s experiment has already drawn heavy criticism from some fans and game developers on social media. “I was lead to feel very confident that Sony wouldn’t do anything like this lol,” wrote Alanah Pearce, who previously worked on narrative at Sony Santa Monica. “Finally saw the Aloy NPC and I have so many issues with it I don’t know where to start,” wrote Anna Megill, a writer on the follow-up to Cyberpunk 2077 at CD Projekt Red. “I’d rather speak to the heap of twisted limbs that Miyazaki called an insult to life itself.”
Companies have been exploring generative AI and large language model (LLM) applications in gaming for years now. So far this has culminated in chatty but soulless NPCs, stilted scripts, and hallucinated gameplay footage, the uses for which aren’t immediately self-evident. Also lots of apparent grifters online are asking ChatGPT to rapidly code very basic video games for them. It feels like we’re still a long way from those technologies dramatically upending gaming and everything else, as some of their biggest advocates claim they will, but it’s also not hard to see how that could happen at some point and how much that might suck.
Many video game performers, including Ashly Burch who voices Aloy in the Horizon games, have been on strike for better AI protections for over a year now. The fear is that companies will steamroll talent with bad contracts that let them do whatever they want with the performances captured, including feeding them into algorithms to fuel additional AI-generated performances for free. Actors would not only lose out on pay but creative control over which of their performances are used and how.
It can seem like a theoretical fear until you watch something like Sony’s own experimentation with an Aloy chatbot. It’s not hard to imagine a future in which, instead of a robotic Aloy fumbling her lips to answer basic questions about the game, it’s a near-perfectly-animated character ad-libbing responses in a voice indistinguishable from Burch’s. Instead of being an avatar for an authored story by other people, Aloy simply becomes another vessel keeping players engaged through endlessly convincing but completely empty conversations.
In that version of gaming’s future, the ultimately compelling version of the character is one aimed exclusively at boosting PlayStation player metrics like daily active users and session length. It might not be the type of game that wins awards but it could be the one, like algorithmic feeds on Instagram and TikTok, that keeps people endlessly logged on. You know, a kernel of the entire thing that went haywire and ruined the planet in the Horizon games, and which Aloy has spent hundreds of hours fighting to fix.
.