Software Engineer - Baseten Inference Stack

ABOUT BASETEN Baseten powers mission-critical inference for the world's most dynamic AI companies, like Cursor, Notion, OpenEvidence, Abridge, Clay, Gamma, and Writer. By uniting applied AI research, flexible infrastructure, and seamless developer tooling, we enable companies operating at the frontier of AI to bring cutting-edge models into production. We're growing quickly and recently raised our $1.5B Series F, led by Altimeter Capital, Conviction Partners, and Spark Capital. Join us and help build the platform engineers turn to ship AI products. THE ROLE Baseten’s Inference Stack team builds the distributed runtime that powers large-scale LLM inference across our platform. We operate at the intersection of distributed systems, model performance, infrastructure, and developer experience. We enable customers to deploy and operate cutting-edge LLM models with industry-leading performance, scalability, reliability, and ease of use. As a Software Engineer on the Inference Stack team, you’ll work across the stack - from the developer experience customers use to deploy models, the libraries used for features like tool calling and reasoning, all the way down to the systems we use to orchestrate deployments in Kubernetes and route traffic efficiently. This is an ideal role for engineers who enjoy owning systems in production, solving hard integration problems, and making complex [... source excerpt omitted ...] ynamo/ https://www.baseten.co/blog/how-baseten-multi-cloud-capacity-management-mcm-powers-cloud-self-hosted-and-hybr/#comparing-deployment-options-cloud-vs-self-hosted-vs-hybrid RESPONSIBILITIES Develop infrastructure and orchestration systems for deploying and managing large-scale distributed LLM inference Work across the stack, from customer-facing features to low-level infrastructure components Build platform capabilities related to routing, autoscaling, scheduling, observability, and runtime management Improve the reliability, scalability, and usability of our inference stack Collaborate closely with Model Performance engineers to make new inference optimizations broadly av [... source excerpt omitted ...] reliability, operational simplicity, and developer experience Own projects end-to-end: from architecture and implementation through deployment, monitoring, and iteration based on customer feedback REQUIREMENTS Bachelor's, Master's, or Ph.D. in Computer Science, Engineering, or a related field Strong background in distributed systems, backend infrastructure, or platform engineering Experience building and operating production systems where reliability, latency, and scale are first-class concerns Strong sense of developer experience: you think about how systems are used, not just how they work Motivated and willing to learn new languages, frameworks, and systems as needed A