CONSTITUTIONAL-AI

Constitutional AI AI safety + governance framework.

Définition

Constitutional AI (CAI) est l'AI safety training methodology developed par Anthropic (San Francisco AI safety company founded 2021 par Dario + Daniela Amodei + ex-OpenAI researchers, ~$15B+ valuation 2024 + ~$8B+ funding incl Google + Amazon investments) decrit dans paper 'Constitutional AI' decembre 2022, methodology Reinforcement Learning from AI Feedback RLAIF (vs RLHF Reinforcement Learning from Human Feedback) ou AI model self-supervises against constitutional principles (ethics + safety + helpfulness + honesty + harmlessness), foundational pour Anthropic Claude AI assistant series training. Framework + standard + guidance objectifs AI safety + governance + risk management + transparency + fairness + accountability + explainability + multiple AI ethics principles adoption multi-stakeholder process + voluntary adoption industry + emerging regulation alignment EU AI Act + US Executive Orders + multiple national jurisdictions + private sector adoption Microsoft + Google + Meta + OpenAI + Anthropic + autres major AI labs + corporate compliance teams 2020s+.

Origine

Anthropic founded 2021 ; Constitutional AI paper decembre 2022 ; Claude AI assistants Claude 2 (2023) + Claude 3 family (mars 2024 + Sonnet/Haiku/Opus) + Claude 3.5 Sonnet (juin 2024) all trained with CAI methodology evolution.

Exemple en contexte

Anthropic trains Claude 3.5 Sonnet using Constitutional AI methodology : (1) Helpful base model trained via RLHF traditional ; (2) CAI Phase 1 self-critique : AI generates responses to potentially harmful prompts + critiques responses using constitutional principles + revises to better adhere to principles ; (3) CAI Phase 2 self-improvement : trained on revised responses; result AI model that better refuses harmful requests + provides helpful explanations + maintains honesty + reduces sycophancy + better safety properties than RLHF-only models.

Termes liés

• Model Card — complementary AI documentation.