CONSTITUTIONAL-AI

Constitutional AI AI safety + governance framework.

Definition

Constitutional AI (CAI) is the AI safety training methodology developed by Anthropic (San Francisco AI safety company founded 2021 by Dario + Daniela Amodei + ex-OpenAI researchers, ~$15B+ valuation 2024 + ~$8B+ funding incl Google + Amazon investments) described in 'Constitutional AI' paper December 2022, Reinforcement Learning from AI Feedback RLAIF methodology (vs RLHF Reinforcement Learning from Human Feedback) where AI model self-supervises against constitutional principles (ethics + safety + helpfulness + honesty + harmlessness), foundational for Anthropic Claude AI assistant series training. Framework + standard + guidance objectives AI safety + governance + risk management + transparency + fairness + accountability + explainability + multiple AI ethics principles adoption multi-stakeholder process + voluntary adoption industry + emerging regulation alignment EU AI Act + US Executive Orders + multiple national jurisdictions + private sector adoption Microsoft + Google + Meta + OpenAI + Anthropic + other major AI labs + corporate compliance teams 2020s+.

Origin

Anthropic founded 2021 ; Constitutional AI paper decembre 2022 ; Claude AI assistants Claude 2 (2023) + Claude 3 family (mars 2024 + Sonnet/Haiku/Opus) + Claude 3.5 Sonnet (juin 2024) all trained with CAI methodology evolution.

Example in context

Anthropic trains Claude 3.5 Sonnet using Constitutional AI methodology: (1) Helpful base model trained via traditional RLHF ; (2) CAI Phase 1 self-critique: AI generates responses to potentially harmful prompts + critiques responses using constitutional principles + revises to better adhere to principles ; (3) CAI Phase 2 self-improvement: trained on revised responses; result AI model that better refuses harmful requests + provides helpful explanations + maintains honesty + reduces sycophancy + better safety properties than RLHF-only models.

Related terms

• Model Card — complementary AI documentation.