Skip to content
NEXTMOVEFDE careers · United States

AI Engineer, Model Quality and Performance

AI summary of the role

You will own model quality and performance for Cerebras' inference offerings, defining evaluation criteria and building AI-driven systems to measure quality at scale.

No longer in the current catalog. Last included 2026-09-09. Check the employer’s posting for availability.

What you’ll do

  • Design eval suites with AI agents in the loop, curating a mix of advanced, basic, long-context, and customer-use-case-specific evals.
  • Build custom evals for target customers by orchestrating AI agents to mine trajectories from their workloads and synthesize representative eval sets.
  • Automate eval execution end-to-end with AI-driven pipelines on top of Docker, Git, and CI.
  • Build automations to forecast and benchmark model performance on Cerebras for top customers.

What you’ll bring

  • Experience building AI agents with Claude (or equivalent) as a force multiplier.
  • Strong math/stats background.
  • Comfort with Docker, Git, and the standard automation stack.
  • A taste for tooling design; shipped something a non-engineer used without complaining.

Technologies

Claude · Docker · Git · CI · EvalScope · lm-eval-harness · FPGAs · GPUs

About Cerebras Systems

Builds wafer-scale AI processors, CS-3 systems, and cloud inference services that deliver ultra-fast training and inference without conventional multi-GPU orchestration overhead.

Growth

Source and classification

Internal deployment & tooling · Evidence for this classification:

GPU-based hyperscale cloud inference services. This order of magnitude increase in speed is transforming the user experience of AI applications, unlocking real-time iteration and increasing intelligence via additional agentic computation. About The Role You'll own model quality and performance for Cerebras' inference offerings. You will define what "good" looks like across the models we serve, building AI-driven systems to measure it at scale, and translating those signals into artifacts our customers and product team actually use. You'll use AI agents to spin up custom eval suites per customer use case, mine trajectories for representative test data, automate the repetitive parts of release qual, and help build performance datasets and benchmarking workflows for customer use cases. We want someone whose first instinct is "how do I get an AI agent to do this on a loop." You'll sit
More from the job description

Cerebras Systems builds the world's largest AI chip, 56 times larger than GPUs. Our novel wafer-scale architecture provides the AI compute power of dozens of GPUs on a single chip, with the programming simplicity of a single device. This approach allows Cerebras to deliver industry-leading training and inference speeds and empowers machine learning users to effortlessly run large-scale ML applications, without the hassle of managing hundreds of GPUs or TPUs. Cerebras' current customers include top model labs, global enterprises, and cutting-edge AI-native startups. OpenAI recently announced a multi-year partnership with Cerebras, to deploy 750 megawatts of scale, transforming key workloads with ultra high-speed inference. Thanks to the groundbreaking wafer-scale architecture, Cerebras Inference offers the fastest Generative AI inference solution in the world, over 10 times faster than GPU-based hyperscale cloud inference services. This order of magnitude increase in speed is transforming the user experience of AI applications, unlocking real-time iteration and increasing intelligence via additional agentic computation. About The Role You'll own model quality and performance for Cerebras' inference offerings. You will define what "good" looks like across the models we serve, building AI-driven systems to measure it at scale, and translating those signals into artifacts our c [... source excerpt omitted ...] ustomer use case, mine trajectories for representative test data, automate the repetitive parts of release qual, and help build performance datasets and benchmarking workflows for customer use cases. We want someone whose first instinct is "how do I get an AI agent to do this on a loop." You'll sit between engineering, product, and customer-facing teams. What You'll Do Design eval suites with AI agents in the loop. For every model release, curate a thoughtful mix of advanced, basic, long-context, and customer-use-case-specific evals. Use Claude to generate, validate, and prune candidate test cases at speed. Build custom evals for target customers by orchestrating AI agents to [... source excerpt omitted ...] I). The goal is a system that runs itself between releases, not a script you re-run by hand. Build automations to forecast and benchmark model performance on Cerebras for our top customers, including modeling how fast customer-specific workloads will run in production. Build product-quality tooling that synthesizes quality + performance data into a single, easy-to-use view. Skills & Qualifications Experience building AI agents. You ship real systems with Claude (or equivalent) as a force multiplier. You've built things that would have been infeasible solo without AI agents in the loop. Strong math/stats background.. Comfort with Docker, Git, and the standard automation stac

How jobs are selected

Employer postings · Data from · Sources