Staff + Senior Software Engineer, Inference

What this role involves

The Inference team at Anthropic designs, builds, and maintains the distributed systems that serve Claude to millions of users worldwide. The role focuses on maximizing compute efficiency across diverse AI accelerators and enabling breakthrough research with high-performance inference infrastructure. Responsibilities include intelligent request routing, autoscaling, and integrating new AI hardware platforms.

Skills this posting asks for

• distributed systems
• machine learning systems
• load balancing
• request routing
• traffic management
• llm inference optimization
• batching
• caching
• kubernetes
• aws
• gcp
• azure
• python
• rust
• observability
• autoscaling
• deployment pipelines
• ai accelerators
• cloud infrastructure
• pair programming

Requirements

• Level: staff

From the employer’s posting

About Anthropic Anthropic’s mission is to create reliable, interpretable, and steerable AI systems. We want AI to be safe and beneficial for our users and for society as a whole. Our team is a quickly growing group of committed researchers, en…