So far, the implementation of generative AI has been rather limited in the gaming space. One might argue that the so-called next-gen artificial intelligence can be used in coding and creating assets for games, but we haven’t seen many practical cases where LLMs and generative artificial intelligence models can be beneficial for gamers.
Or so I thought until I came across the Mantella mod from genius developer Art-from-the machine. If you haven’t heard of it, it’s a mod that lets you use ChatGPT, LLama, and a host of other LLMs in tandem with Whisper, a speech-to-text tool, and text-to-speech generators like Piper, xVASynth, and XTTS.
As a die-hard fan of Skyrim, from the moment I laid eyes on the mod, I knew I had to give it a spin. And after spending more hours than I’d like to admit, I can confirm that Mantella has surpassed all the other mods (including the cool VR add-ons) I’ve ever used for Skyrim.
Related
AI has somehow managed to lower and raise a skill bar at the same time.
Installing Mantella was rather cumbersome
By the time I was done, my Downloads folder was chock-full of mods and zip files
With a mod of its caliber, I never expected Mantella to be simple to install. While it’s not as difficult to set up as, say, configuring a Pterodactyl server, you’ll have to spend an afternoon downloading all the plugins and mods that Mantella relies on. If you’re used to modding Skyrim, you’ll find some familiar names, like the SKSE plugins and their Address Library. Oh, and there are a whopping 70+ mods for specific characters, followers, and enemies, in addition to generic NPC voices that you’ll want to download for maximum immersion.
Thankfully, the instructions on the official Mantella website are fairly straightforward, and Vortex helped me keep track of most of the mods I’d downloaded. That said, I did encounter an issue early on when Mantella displayed an error every time I tried speaking to a character. After snooping around forums and Discord servers, I switched the speech-to-text tool to xVASynth. And with that, it was time to begin my journey as an annoyingly chatty Dragonborn.
How Mantella works
Mantella appears as a set of Illusion spells that let you engage in normal conversations with NPCs
When you first install Mantella, you’ll notice that the conversation mod brings some magic spells to your inventory. Available right from the beginning, the Mantella spells can be cast without mana. And yes, I said spells, as you’ll typically use two of them: one for initiating conversations, and another for ending them.
Casting the Mantella spell on an NPC will cause them to halt in their tracks and start listening for input. Depending on the AI models you’ve used, the NPCs may halt for a short time (usually a few seconds), before responding to your queries, threats, or statements in kind. For reference, I used the online version of the free liquid/lfm-40b LLM for the majority of my tests, and the NPCs took anywhere between two-to-six seconds to respond.
Behind the scenes, Mantella runs the trifecta of the speech-to-text, text generators, and text-to-speech tools to analyze your input and create the appropriate responses. Of course, most of the named NPCs (and even some of the generic ones, for that matter), should have their own unique reactions to your input. That’s because Mantella uses a multitude of variables, including their background and your past actions. And this is precisely what makes Mantella so enjoyable…
How Mantella feels
With the right setup, you’ll have a hard time believing you’re talking to chatbots
When I first attempted to run Mantella and its battalion of mods in my system, I was rather skeptical about the AI NPCs aspect due to all the limited capabilities of generative LLMs and AI voice generators. While the pre-scripted lines are undeniably better, using the lip sync mods alongside the robust text-to-speech conversion tools results in nearly organic dialog options. Although I was unable to change their affiliations and ideologies using words, it’s possible to get some truly fun dialog options thanks to AI.
For example, an Imperial-aligned NPC can’t be made to like the Stormcloaks, and you can’t force the opposing factions to make peace without additions. But there are a lot of cool things you can pull off with Mantella. You can provoke amicable strangers to attack you, request bards to tell you about high-level monsters, prod followers about their tales of heroism, or even pose philosophical questions and expect reasonable answers. If you want, you can even ask more than one NPC to jump into the conversation.
Asking AI NPCs the real questions…During the course of my tests, one of the NPCs accidentally called me Player, instead of using the name on my profile. As such, this article wouldn’t be complete if I didn’t give the NPCs some existential dread. While they didn’t cower in fear at the thought of being inside a game, I did manage to aggravate a couple of NPCs after repeated attempts to get them to admit that Skyrim is just a virtual world!
Mantella: It changes the way you experience Skyrim
After my catastrophic failure when creating a short film with AI a few weeks ago, I had my expectations fairly low going into this mod. While complicated prompts and wacky conversational scenarios did throw the AIs into a loop a couple of times, I’m extremely impressed with what Mantella is capable of – to the point where I’m low-key looking forward to all the cool things creative game devs can pull off using the power of generative AI.
Related
Say what you will about Photoshop, but there’s no beating Adobe’s premium app when it comes to generative fill