Research
“We are not querying like humans anymore”: OpenSearch’s Bianca Lewis on why AI agent mistakes look like successes
The New Stack · By Chris J. Preimesberger ·
The Brief
- An AI agent can call the wrong API, receive a valid response, and continue as if the task succeeded.
- Such failures can leave monitoring dashboards green, since they look like normal successful operations.
- OpenSearch's Bianca Lewis discusses the problem in a New Stack interview.
Why it matters
Traditional troubleshooting looks for errors and failed requests, so agent mistakes that return valid responses can go unnoticed. Developers running agents need observability that checks whether the agent did the right thing, not just whether the call succeeded.
Who’s affected
- Teams building or operating AI agents that call APIs
- Developers relying on dashboards and error-based monitoring
- #ai-agents
- #observability
- #opensearch
- #monitoring
- #api
This situation is more common than one might think: An AI agent calls the wrong API, gets a perfectly good response, and carries on as if everything went according to plan. Somewhere a dashboard is still green. For developers used to troubleshooting applications by looking for…