Notes
Notes on AI products and reliable agent workflows
Short, public notes from product work and Atlas. I write about the artifacts I can inspect: workflow traces, eval results, review paths, side effects, and recovery.
Microsoft internal details and private Atlas records stay out of these notes. Examples and diagrams use public-safe, simplified shapes.
Archive
- Product Development Was Never Just About Coding Product thinking — Agentic development compresses software creation, but customer understanding, trust, onboarding, workflow change, and adoption remain product work.
- Self-improving agents need evidence, not just autonomy Atlas note — Bad answers, bad actions, and false green status need behavioral evidence that can become evals.
- AI Features Are Not the Same as AI Leverage Product thinking — Faster drafts help; leverage begins when a product helps a team see and carry the judgment call.
- Evals should test the workflow, not the demo Atlas note — Good tests include state, files, side effects, and the user's real input shape.
- When more powerful tools make agents worse Atlas note — More capability can reduce reliability when agents lack routing discipline.
- When an agent says it used a tool, that is not evidence Atlas note — The execution trace matters more than a narrative claim that a tool ran.
- AI agents debug symptoms before systems Atlas note — Check the system boundary before accepting a plausible local diagnosis.
- If an AI can write data, it needs a recovery path Atlas note — Undo, audit, and repair belong in the product surface when AI can write data.