AI Engineer, Evaluation

Internal deployment & tooling · Evidence for this classification:

Partners, Khosla Ventures, Coatue, DST Global, and the board-members of 20+ F500s. What We Are Looking For At Distyl, we build AI systems using Evaluation-Driven Development—an approach where evaluation is not an afterthought, but the primary mechanism for iterating, improving, and trusting AI behavior in production. AI Evaluation Engineers focus on designing and implementing the evaluation systems that drive this process. They are hands-on engineers who write production Python code, build evaluation pipelines, and use structured signals to guide system design, prompt iteration, and deployment decisions for real customer-facing AI systems. This role is for engineers who believe that AI systems only improve when measurement is tightly coupled to development—and who want to apply that philosophy directly to systems that matter. Key Responsibilities Design and implement evaluation