Summary written by AI from the employer’s posting. The full original text is further down.
About the role
Design and train classifiers to detect unsafe agent actions in real-time
Deploy and operate classifiers in critical path of internal agents and RL training
Build datasets, labelling pipelines, and evaluations to measure and improve classifiers
Partner with red teaming colleagues to stress-test and harden classifiers
Work with teams to fix false positives and maintain protection
Contribute to team's direction and decision-making
Requirements
- Familiarity with research on AI control, scalable oversight, agent safety, and reinforcement learning
- Experience with LLM inference or serving infrastructure, or integrating new components into training infrastructure
- Experience leading ambiguous technical projects and setting direction for other engineers or researchers
- Experience evaluating large language models, developing classifiers, or operating high-scale production systems
- Experience running low-latency, high-availability services and building detection systems
- Background in security, such as sandboxing or detection and response
- Sincere interest in developing safe and beneficial AI systems
Benefits and perks
- Comprehensive health, dental, and vision insurance
- Inclusive fertility benefits via Carrot Fertility
- 22 weeks of paid parental leave
- Flexible paid time off and absence policies
- $500/month flexible wellness and time saver stipend
Skills
Full job description
Description
- Our team builds, tests, deploys, and monitors the classifiers that block internal agents from taking dangerous actions
- These classifiers run across RL, evals, and internal deployments, and are designed to catch things like sandbox escape attempts or exfiltration of sensitive data
- When a classifier flags an action, it can block the action before the tool call runs, end the task, and alert a human
- This is one layer of a defence-in-depth with several layers that reinforce each other: what our classifiers flag feeds into offline monitoring, and the findings from that go back into better classifiers and safer environment design
- The work combines research and production engineering:
- The research problem is building high recall and high precision classifiers to catch rare, novel, and sometimes adversarial behaviour
- There are also several narrow research questions to answer - for example, how to deploy safety classifiers inside training environments without incentivising models to evade them
- Some research also builds on the multi-stage designs described in our work on Claude Code auto mode and Constitutional Classifiers
- The engineering problem is shipping these classifiers into the critical path of research and product work
- These classifiers need to cover a huge volume of traffic while operating within cost, latency and availability constraints
- Day to day, much of our work is done with Claude: you’ll direct agents to run experiments and ship code, and review what they produce
- That moves the hard part of the job towards deciding what to do and in what order. Additionally, organisation, communication, and prioritisation skills matter a great deal as our classifiers sit in the path of many other teams’ work
- Design, train, and evaluate classifiers that detect unsafe agent actions in real time, owning pieces such as prompted models, trained classifiers, or probes on model internals
- Deploy, tune and operate these classifiers in the critical path of internal agents, RL training, and evaluations, and own their accuracy, reliability, latency, cost, and failure behaviour
- Build the datasets, labelling pipelines, and evaluations that allow us to measure and improve our classifiers and harm taxonomies
- Partner with red teaming colleagues to stress-test our classifiers, and harden them against what they find
- Work with the teams whose agents, environments, and evaluations we run on, so that false positives get fixed quickly without weakening protection
- Contribute to the team’s direction, including what to build, what to partner on, and where to invest
Benefits
- Comprehensive health, dental, and vision insurance for you and your dependents
- Inclusive fertility benefits via Carrot Fertility
- 22 weeks of paid parental leave
- Flexible paid time off and absence policies
- Mental health support for you and your dependents
- Competitive salary and equity packages
- Optional equity donation matching at a 1:1 ratio, up to 25% of your equity grant
- Retirement plans with competitive matching
- Life and income protection plans
- $500/month flexible wellness and time saver stipend
- Commuter benefits
- Annual education stipend
- Home office stipends
- Relocation support for those moving for Anthropic
- Daily meals and snacks in the office
- Ability to organise and prioritise several parallel streams of work, and to communicate clearly with the teams that work affects
- Sincere interest in developing safe and beneficial AI systems
- Ability to critically review code and experiment results, including work produced by AI agents
- Familiarity with research on AI control, scalable oversight, agent safety, or reinforcement learning
- Background in security, such as sandboxing or detection and response
- Experience with LLM inference or serving infrastructure, or with integrating new components into training infrastructure
- Experience leading ambiguous technical projects and setting direction for other engineers or researchers
- Experience evaluating large language models, developing classifiers, or operating high-scale production systems where requirements change quickly
- Experience running low-latency, high-availability services that other systems depend on, including being on call for them
- Experience building detection systems where true positives are rare and false positives are costly, such as abuse, fraud, spam, or intrusion detection
- Experience fine-tuning language models, or working with probes and other methods that read model internals
- Minimum years of experience: Years of experience required will correlate with the internal job level requirements for the position
- Required field of study: A field relevant to the role as demonstrated through coursework, training, or professional experience
- Minimum education: Bachelor’s degree or an equivalent combination of education, training, and/or experience
- We encourage you to apply even if you do not believe you meet every single qualification. Not all strong candidates will meet every single qualification as listed
Opens Anthropic’s own application page in a new tab.