As a Safeguards Policy Analyst, you will be responsible for building and executing enforcement workflows for our products and services, with a focus on detecting and mitigating potential harmful use. In this role, you will have the unique opportunity to function both as a policy owner and develop the enforcement strategy for a suite of policies. As a member of the user Integrity and Authenticity team, your initial focus will be on improving on the current policies and expanding integrity and authenticity enforcement workflows. This role may later expand to include broader areas and methods of harm reduction. Safety is core to our mission and you’ll help shape policy enforcement so that our users can safely interact with and build on top of our products in a harmless, helpful and honest way.
*Important context for this role: In this position you may be exposed to and engage with explicit content spanning a range of topics, including those of a sexual, violent, or psychologically disturbing nature.
Responsibilities
- Design and architect automated enforcement systems and review workflows that scale effectively while maintaining high accuracy
- Partner with Product, Engineering, and Data Science teams to optimize detection models for policy violations and automated enforcement systems
- Review flagged content to drive enforcement and policy improvements
- Work with external experts to gather feedback on policy, product interventions, and harm mitigations
- Enforce usage policies with a focus on detecting and mitigating potential harmful use of AI systems
- Support the Safeguards policy design team by providing detailed feedback on policy gaps based on real enforcement scenarios
- Keep up to date with emerging AI policy enforcement best practices, and use these to inform our decision-making and workflows
You may be a good fit if you have:
- Experience establishing and scaling policy enforcement, and review workflows
- Written and improved policies for tech products and platforms
- Excellent written and verbal communication skills, with the ability to explain complex policy topics to various audiences
- Used SQL and/or other data analysis tools to draw insights from large datasets
- Identified emerging risks and threat actors, and provided feedback to a diverse sets of stakeholders, such as Product, Policy, Engineering, and Legal teams
- Worked with generative AI products, including writing effective prompts for content review and enforcement
- Navigated and thrived in a fast-paced and dynamic environment
- An understanding of the challenges that exist in implementing product policies at scale, including in the content moderation space
- Maintained strong collaboration with team members while navigating rapidly evolving priorities and workstreams
- Experience as a trust & safety professional or subject matter expert working in one or more of the following focus areas: elections, influence operations, or fraud and abuse
Deadline to apply: None. Applications will be reviewed on a rolling basis.
