Director of Product Management
AI summary of the role
Director of Product Management owning the product strategy and evolution of FriendliAI's AI inference platform, spanning model serving, deployment, orchestration, APIs, and developer-facing capabilities.
What you’ll do
- Own product vision, strategy, and roadmap for the AI inference platform, covering model serving, deployment, orchestration, APIs, and developer capabilities.
- Define product direction across the inference stack, balancing performance, scalability, reliability, usability, and cost.
- Partner with engineering and research on routing, batching, scheduling, resource allocation, model optimization, and inference performance.
- Drive strategy around inference metrics: latency, time-to-first-token, throughput, GPU utilization, cost per inference, and availability.
What you’ll bring
- 10+ years of product management experience, with significant experience owning technically complex platforms or infrastructure products.
- Proven experience leading product strategy for AI/ML infrastructure, cloud platforms, distributed systems, developer platforms, or related technical products.
- Deep understanding of AI inference and LLM serving architectures (prefill vs. decode, KV cache, token streaming, batching, routing, GPU utilization).
- Strong technical fluency to work directly with senior engineering and research teams on complex distributed systems.
Technologies
AI inference · LLM serving · GPU · vLLM · TensorRT-LLM · SGLang · KV cache · token streaming · batching · routing
About FriendliAI
Generative AI inference platform selling fast, cost-optimized LLM/agent serving (managed endpoints + on-prem engine) to enterprises.
Seed · 50–200 people
Source and classification
Internal deployment & tooling · Evidence for this classification:
About the job FriendliAI is looking for a Director of Product Management to own the product strategy and evolution of our AI inference platform. As Director of Product Management, you will define how FriendliAI’s inference platform serves models, scales across demanding workloads, and delivers measurable gains in performance, reliability, and cost. AI inference is an unusually complex product problem. Models, GPUs, networking, scheduling, serving systems, and developer workflows must work together under real production constraints. Customers expect low latency and high throughput as workloads constantly change, and the platform must make this complexity feel simple to the engineers building on top of it. This is a hands-on product leadership role for a technically strong product leader who understands AI infrastructure, works deeply with engineering and research, and translates
More from the job description
About the job FriendliAI is looking for a Director of Product Management to own the product strategy and evolution of our AI inference platform. As Director of Product Management, you will define how FriendliAI’s inference platform serves models, scales across demanding workloads, and delivers measurable gains in performance, reliability, and cost. AI inference is an unusually complex product problem. Models, GPUs, networking, scheduling, serving systems, and developer workflows must work together under real production constraints. Customers expect low latency and high throughput as workloads constantly change, and the platform must make this complexity feel simple to the engineers building on top of it. This is a hands-on product leadership role for a technically strong product leader who understands AI infrastructure, works deeply with engineering and research, and translates complex systems into products customers want to use. You will own product direction across the inference platform while helping build and lead the product organization as FriendliAI scales. Key Responsibilities Product Strategy & Platform Own the product vision, strategy, and roadmap for FriendliAI’s AI inference platform, spanning model serving, deployment, orchestration, APIs, and developer-facing capabilities. Define product direction across the inference stack, balancing performance, scalabili [... source excerpt omitted ...] s rapidly evolving AI workloads, including LLMs, multimodal models, and agentic applications. Translate complex technical capabilities into clear product value and differentiated customer experiences. Inference & Customer Experience Develop a deep understanding of AI inference workloads and the challenges faced by ML engineers, AI platform teams, and infrastructure organizations. Partner closely with engineering and research to shape capabilities across routing, batching, scheduling, resource allocation, model optimization, and inference performance. Drive product strategy around critical inference metrics including latency, time-to-first-token, throughput, GPU utilization, [... source excerpt omitted ...] th enterprise customers and technical users to understand production workloads, validate product direction, and identify high-impact opportunities. Partner with Sales, Solutions, Customer Success, and Marketing to translate customer needs into product priorities and strengthen FriendliAI’s market position. Product Leadership Lead product initiatives end-to-end, from discovery and problem definition through prioritization, execution, launch, and iteration. Lead and mentor Product Managers and Product Designers, establishing clear ownership and a high bar for product thinking and execution. Establish product planning, prioritization, and measurement practices that help the org
Employer postings · Data from · Sources