Inference Optimization Engineer
AI summary of the role
This role is for an Inference Optimization Engineer on the Performance Labs team at Modular (a Qualcomm company), focused on optimizing LLM inference performance on Modular Cloud across GPU and ASIC architectures.
What you’ll do
- Build the optimization platform that drives inference performance of LLMs served on Modular Cloud to state-of-the-art levels across the latest GPU and ASIC architectures.
- Shape the technical direction of Modular Cloud, delivering LLM performance on the Pareto frontier for agentic use cases.
- Partner with GTM to deliver highly customized LLM inference tuned to specific customer use cases and collaborate across engineering to drive full-stack optimizations.
- Translate insights from customer engagements into technical direction for engineering teams.
What you’ll bring
- 5+ years of experience in distributed systems or performance engineering.
- Track record of building durable, reusable software tools and libraries adopted across teams.
- Sound judgment in evaluating technical tradeoffs and setting priorities.
- Strong communication and technical leadership skills.
Technologies
GPU kernel programming · inference engine · distributed inference · Kubernetes · cloud native · LLM architectures · inference optimization · ASIC · distributed systems · performance engineering
Source and classification
Internal deployment & tooling · Evidence for this classification:
performance from day one, then keeps getting better. As we learn the shape and patterns of each customer's workload, the platform adapts and improves performance automatically over time. The Performance Labs team builds the infrastructure that makes this possible at scale. We continuously apply the latest optimizations across kernels, the inference engine, and distributed systems so that customer workloads stay on the Pareto frontier of cost and performance. We get there through deep workload insights, a scalable platform, and close collaboration with engineering and product teams. In this role you will dig into real customer inference workloads, profile them end to end, and apply the optimizations across kernels, engine, and distributed systems that push each workload toward the Pareto frontier. You will build the tooling and platform that turns one off performance wins into a
More from the job description
About Modular At Modular, a Qualcomm company, we’re on a mission to revolutionize AI infrastructure by systematically rebuilding the AI software stack from the ground up. Our team, made up of industry leaders and experts, is building cutting-edge, modular infrastructure that simplifies AI development and deployment. By rethinking the complexities of AI systems, we’re empowering everyone to unlock AI’s full potential and tackle some of the world’s most pressing challenges. If you’re passionate about shaping the future of AI and creating tools that make a real difference in people’s lives, we want you on our team. You can read about our culture and careers to understand how we work and what we value. About the role: At Modular, we optimize inference from kernel to cloud on one unified stack. We are building a differentiated cloud platform that delivers state of the art inference performance from day one, then keeps getting better. As we learn the shape and patterns of each customer's workload, the platform adapts and improves performance automatically over time. The Performance Labs team builds the infrastructure that makes this possible at scale. We continuously apply the latest optimizations across kernels, the inference engine, and distributed systems so that customer workloads stay on the Pareto frontier of cost and performance. We get there through deep workload insight [... source excerpt omitted ...] tomer inference workloads, profile them end to end, and apply the optimizations across kernels, engine, and distributed systems that push each workload toward the Pareto frontier. You will build the tooling and platform that turns one off performance wins into a repeatable, automated optimization loop, and you will work directly with engineering, product, and GTM to bring those gains to customers in production. LOCATION: Candidates based in the US or Canada are welcome to apply. You can work in our office in Los Altos, CA or remotely from home. Onboarding for new hires is conducted in-person in our Los Altos, CA office. What you will do: Build the optimization platform that dr [... source excerpt omitted ...] areto frontier for agentic use cases and keeping it there as the landscape evolves. Partner closely with the GTM team to deliver highly customized LLM inference tuned to specific customer use cases, and collaborate across engineering to drive optimizations spanning the full stack, from GPU kernels to cloud infrastructure. Translate insights from customer engagements into technical direction for engineering teams. Publish blog posts on innovative approaches to LLM inference optimization that shape industry wide best practices. What you bring to the table: 5+ years of experience in distributed systems or performance engineering. A track record of building durable, reusable sof
Employer postings · Data from · Sources