We view AI as a sliding scale. There will always be a human

What the report says

Services provider Side has signed a memorandum of understanding with Danish AI firm Modl.ai.The MOU is binding for six months and will see the two companies working together on an AI-enabled quality assurance system.Side – formerly PTW – is a long-time player in the services space. Meanwhile, Modl.ai is a newer entrant to the games market. Founded in 2018, the company raised $1.7 million in seed funding two years later."The MOU effectively brings the best of what Side has and the best of what Modl.ai has together in a unique solution," Side's chief commercial officer Martin McBride says.

"One of the crown jewel, holy grail experiences for the games industry right now has been using AI technology to better augment QA services. No one's really found a great solution for that. We think we have."Previous AI-enabled QA tech – including that created by Modl.ai – involved developers having to send over entire builds for quality assurance companies for the tech to test. Modl.ai's solution used to be "integration-based" with a plugin and SDK and required developers to provide source code. Unsurprisingly, this created too much friction and "became difficult over time", according to the company's founder Christoffer Holmgard.This new system, Holmgard says, doesn't require access to any of the internals."We take the product as you would give it to a QA tester, as you would eventually give it to the consumer," he says.

"You put it on a very similar or even the same device. And then our solution plays the game by looking at the screen, simulating key processes, generating a data set of everything that happens, and then hands it over to a second AI system that has a look at it and identifies bugs, raises it up to the human level and asks whether this is something we should act on."The first step in the process is what Modl.ai calls building "game knowledge". Developers can upload assets from their project and talk to the AI to build up their 'understanding' of the game."All of this is in natural language," Holmgard says.

Key details

"We just chat with the system about our game, but we're also adding in images, or we might upload our game design documents and so on."After that, developers define what they want to test. This can vary from "classic test cases" to creating new things that are more "suitable for the AI system"."This is where you need some human expertise here to adapt to the prompting and the language to the AI agent to make it do the right things," Holmgard explains.From there, the system generates bug reports based on their session – or run, as Modl.ai calls it – with data produced."We get all the texts, and we get some video snippets that we call evidence along with all the log data and everything," Holmgard says.

"This goes to a human QA specialist, an expert. They review it, and then they decide what happens with it for the rest of our production."One thing that an AI might find tricky to figure out is whether something in a game is a bug or a feature. Holmgard says that feedback from QA testers will, in part, help the AI system understand the nuances of game design."Some of it comes from the prompting. Some of it comes from that game knowledge that you've built up around the game ahead of time," he says.

"And then a good part of it also comes from feedback over time from the human testers. And so they will provide feedback to the bug reports and say: 'This was a bug; this was not a bug; this was not a bug for this reason' and so, over time, the system gets attuned."In fact, Side envisions a future where QA staff move into being AI engineers. In doing so, this will change where quality assurance jobs are done globally."One of the cool parts about programs like this is that we get to move our people up the value chain," McBride explains.

More from the announcement

"So instead of just doing testing and having a career at Side being a tester and moving up to being a senior tester and all the way up, they can become an AI engineer. And so it allows our people to get paid more money. It allows us to have new career paths. And it allows us to potentially bring jobs back to countries and areas where these jobs historically have gone away due to price points and labour concerns.