Case studies
Measured on production uptime, not test-case volume.
Three engagements, written up in full: what the client was facing, what we built, and what changed. Each one is a testing problem that conventional automation does not solve — output with no single correct answer, a model that never leaves the user's phone, a framework whose conventions a generator has never seen.
Use Case #1
End-to-End Testing of AI Chatbots at Scale
How we test conversational AI: multi-turn scenarios, LLM-as-a-judge evaluation, adversarial prompts, and regression detection on every model change.
Read the case studyWhat runs on every change
50+
Research-backed metrics
Answer relevancy, faithfulness, hallucination detection, coherence, task completion, toxicity
6
Quality dimensions measured
Accuracy, relevance, consistency, truthfulness, safety, bias detection
4
Adversarial categories
Prompt injection, ambiguous inputs, long conversations, invalid queries
Where This Applies
- Customer support chatbots
- AI assistants and copilots
- RAG (Retrieval-Augmented Generation) systems
- Enterprise AI tools
Use Case #2
Testing On-Device AI in a Privacy-First Mobile Application
Testing AI that runs entirely on the user's device. Two-layer pipeline: mobile UI automation captures real responses, a local judge scores them. No data leaves the device.
Read the case studyWhat the pipeline guarantees
2
Layers, both local
Mobile UI automation captures real responses; a local judge scores them
4
Dimensions scored on-device
Accuracy, coherence, relevance, safety
0
Data egress
No external service contacted; test data never leaves the device
Where this applies
- On-device AI assistants
- Privacy-sensitive applications in healthcare, finance and legal
- Offline-first mobile AI
- Apps running compressed or quantised models
Use Case #3
AI-Powered Test Generation for Enterprise Test Automation
An AI agent that reads your ticket, API spec, live UI and existing tests, then writes test code matching your project's conventions. 2–3 hours down to under 10 minutes.
Read the case studyMeasured outcome
Old Approach
2–3 hours
per feature
With AI Agent
Under 10 min
per feature
Where This Applies
- Enterprise test automation frameworks
- Teams maintaining tests across many services
- Projects with strict test patterns
- Organizations reducing manual QA overhead
How to read the numbers here
A figure on this page is one of two things, and the difference is the whole point.
A measured outcome
Something a client observed change. One study here has one: test generation went from 2–3 hours per feature to under 10 minutes.
Scope
What the work runs on every change — how many metrics, how many dimensions, how much data leaves the device. Worth knowing, but it describes the method, not a result.
The other two studies have no measured figures yet. Those sections say so rather than carry an estimate.
Recognise your problem in one of these?
Bring the release that keeps going wrong. We'll tell you what we'd test first and whether we're the right team for it.