Software Engineer, AI Evaluation

What this role involves

Nuna is building an AI health coach for chronic disease management. This role focuses on building the evaluation system for the AI agents, including harnesses, datasets, judges, and release gates. The engineer will own the evals end to end, working with data science and clinical partners to ensure safety and quality.

Skills this posting asks for

• ai evaluation
• llm-as-judge
• red-teaming
• adversarial testing
• synthetic scenario generation
• multi-turn evaluation
• agentic evaluation
• production systems
• tooling
• statistics
• experimentation

Requirements

• Level: senior

From the employer’s posting

Chronic disease isn't managed in a clinic. It is managed at home, in relationships, in the everyday. What's on the dinner table, what gets said, and who notices when someone's struggling. For the 130 million Americans managing a chronic condition, the healthcare system has offered the same answer f…