Ship It Conversations: The AI Hallucination That Looks Right, With Devansh of Irys on Legal AI, Three-Tier Error Isolation, Models That Change Under You, Overhyped Reinforcement Learning, and Optimizing for Maintenance Over Generation

In this episode

This is a guest conversation episode of Ship It Weekly, separate from the weekly news recaps. In this Ship It Conversations episode, I talk with Devansh about why the trickiest legal AI hallucination is a real case used the wrong way, how his team splits their system into three tiers to find failures fast, and why he'd rather optimize for maintenance than for generation. Devansh is the Founder and Head of AI at Irys. He builds reasoning and verification systems for legal AI, a tough place for hallucinations and confident wrong answers. He also publishes open source AI research on Substack. We start with why AI for professionals breaks down. Devansh thinks models are getting good at doing a l…