Skip to content
NEXTMOVEFDE careers · United States
ngrok

Full-stack Software Engineer III/Senior, AI Gateway

AI summary of the role

Build and evolve the AI Gateway at ngrok, owning the AI-specific control plane that classifies, enforces policies, tracks token usage, and makes AI traffic observable.

What you’ll do

  • Build and evolve the AI Gateway components that classify and handle AI traffic in real time on the request path
  • Design and implement AI Gateway Traffic Policy Objects—rate limits, usage caps, and access rules specific to AI workloads
  • Build and maintain systems that accurately measure AI usage (requests, tokens, metadata) for customer visibility
  • Expose clear, trustworthy signals around AI traffic: what was allowed or blocked, which policies applied, and usage accumulation

What you’ll bring

  • Comfortable in a statically typed, compiled language such as Go, Rust, C++, or Java (bonus for Go)
  • Experience working with AI/LLMs and understanding their edge cases
  • Experience defining system behavior, not just plumbing
  • Experience with retries, limits, and costs in production systems

Technologies

Go · TypeScript · Kubernetes · Terraform · Helm · Buildkite · Postgres · Kafka · Protobuf · React

About ngrok

All-in-one cloud networking platform that secures, transforms, and routes traffic to services anywhere via one gateway — replacing nginx, NLBs, VPNs, and model routers.

Series A · 50–100 people

Source and classification

Production engineering · Evidence for this classification:

toad) We have a designated Chief Emoji Officer - they are vital to our success! We like software that’s serious and culture that’s not Still reading? Good. There's more below worth your time. About the AI Gateway Team Our AI Gateway team builds the systems that define how AI traffic is identified, controlled, and understood as it passes through ngrok. We own the AI-specific control plane at the gateway layer: policies, usage tracking, and enforcement that sit directly on live customer traffic. Our systems must behave correctly under real-world conditions—traffic spikes, unexpected model behavior, misconfigured policies, and customers asking, “Why was this blocked?” or “Where did my tokens go?” What You’ll Actually Do Build and evolve the AI Gateway: You’ll work on the AI-aware gateway components that classify and handle AI traffic in real time. This code runs directly in the
How jobs are selected

Employer postings · Data from · Sources