I let an AI agent run my user interviews

User testing is always worth it. The hard part is usually finding the time. You write the questions, schedule the sessions, run each one, and then go back through all of it to write up what you learned. So it's easy to keep putting off.

With ElevenAgents, that's changed for me. I set up an agent that runs the interviews. I send people a link, they talk to it when it suits them, and I keep designing. The answers come back already sorted.

I'm upfront that it's an AI agent, and people have been comfortable with that. They often ask it to switch to their own language partway through, like Portuguese or Spanish, and it just does.

Writing the script

I set the agent up through the ElevenLabs MCP in Claude Code and talked the whole thing through out loud.

The most helpful step was attaching our previous research. Claude could see what we'd already learned, so instead of starting from a blank page we worked out what was still unanswered and which questions would get at it. Then it wrote the prompt and created the agent directly through the MCP. After that, a few more prompts were enough to fine-tune it.

Every interview opens the same way. This is the first message you hear in the video when the call starts, with the feature redacted:

The interview is designed to take about six minutes, though the complete ones ran closer to ten. It starts by asking how people expect the feature to work before they've seen anything, so their expectations aren't shaped by the design. Only then does it ask them to open a link to the design and react to it.

What makes it feel natural

Mostly it comes down to how it takes turns, which makes it feel like a conversation rather than a survey. It asks one question, then stops and waits for the answer. It doesn't jump in at the first pause, and when someone finishes, it acknowledges briefly with a "Got it" instead of repeating their words back. If an answer is vague, it follows up once or twice, then moves on.

It also stays neutral. It never hints that the design is good, and never explains how it "really" works when someone misunderstands it. Their confusion is the finding. Here's the start of the agent's instructions:

Voice needed a few extra rules. The agent can't see the participant's screen, so when someone mentions part of the design, it asks which screen they mean or what it says. People also go quiet while they look around, so it's told never to ask "Are you still there?"

After the call

When an interview ends, the agent fills in a set of fields with the answers: what people expected, their first reaction, what confused them, what was missing, and how likely they'd be to use it and why. It also checks whether all the core questions got a real answer, so I can quickly spot the interviews that came up short.

After a few days, once the interviews had wrapped up, I pulled everything into Claude through the same MCP and we went through the results together. The overall design held up well, and we made some changes to the copy.

It's great for getting a quick sense of how people read a simple screen. For a more complex experience that people need to actually try, you'd still want a deeper study. But for early checks like this one, I can test a lot more often than I used to, without stopping what I'm working on.

Thanks to Jack McDermott, whose interview agent got me curious to try this myself. He's written about his setup too.