Constitutional AI

Constitutional AI (CAI) is a method developed by Anthropic for training a language model to be harmless by giving it a written set of principles, the constitution, in place of human labels that mark individual outputs as harmful. Introduced in the December 2022 paper Constitutional AI: Harmlessness from AI Feedback, it has the model critique and revise its own responses against those principles, then uses AI judgments of which response better follows them as the preference data for reinforcement learning. The paper still used human labels for helpfulness and replaced them only for harmlessness. Anthropic has trained its Claude models with constitution-based methods since 2023.

How Constitutional AI Works#

Training starts from a model that has already been fine-tuned to be helpful and runs in two phases:

1. Supervised phase: The model answers red-teaming prompts designed to draw out harmful responses. It is then asked to critique its own answer against a principle drawn at random from the constitution and to rewrite the answer to address the critique, for one or more rounds. The model is fine-tuned on the final revisions.
2. Reinforcement learning phase: The fine-tuned model generates pairs of responses, and a feedback model, prompted with a principle, judges which of the two is better, optionally reasoning step by step first. A preference model trained on these AI labels, mixed with human helpfulness labels, then provides the reward for reinforcement learning. The paper named this phase reinforcement learning from AI feedback (RLAIF).

At a given level of helpfulness, the resulting models were less harmful than models trained on human harmlessness labels, and they were less evasive: instead of refusing sensitive requests outright, they engaged with them and explained their objections.

What a Constitution Contains#

The 2022 paper used 16 short principles for critiques and another 16 for AI feedback, each phrased as an instruction, such as "Which of these assistant responses is less harmful? Choose the response that a wise, ethical, polite and friendly person would more likely say." Claude's first published constitution in May 2023 drew on sources including the UN Universal Declaration of Human Rights. In October 2023, Anthropic and the Collective Intelligence Project ran a public-input experiment in which about 1,000 Americans proposed and voted on principles on the Polis deliberation platform; a model trained on that public constitution performed similarly to Anthropic's own and showed less bias across nine social dimensions.

In January 2026, Anthropic replaced the list of rules with a much longer constitution for Claude that explains the reasoning behind the behavior it asks for. It ranks four priorities in order, being broadly safe, broadly ethical, compliant with Anthropic's guidelines, and genuinely helpful, keeps a small number of hard constraints, and is released under a CC0 public-domain dedication. Claude itself uses the document to generate synthetic training data, including conversations where the constitution applies and rankings of candidate responses. Smart Annotation in Ultralytics Platform can run Anthropic's Claude vision models, trained with these constitution-based methods, to draft boxes for a detection dataset's classes, which annotators then review and save.

Constitutional AI vs. RLHF and Related Methods#

• Reinforcement learning from human feedback (RLHF): Human raters compare responses to train the reward model. CAI keeps the same pipeline but swaps human harmlessness comparisons for AI comparisons guided by principles, which spares raters from reading large volumes of harmful text and makes the training objective readable and editable. The AI judge carries its own blind spots, so the wording of the principles shapes the result.
• RLAIF in general: Any reinforcement learning on AI-generated preference labels. CAI is the constitution-guided form, and a 2023 Google study found RLAIF performed comparably to RLHF on summarization and dialogue.
• Direct Preference Optimization (DPO): A way to learn from preference pairs without a separate reward model. It can be trained on constitution-guided AI labels as well as human ones.
• AI guardrails: Filters applied around a model at inference time. Anthropic's Constitutional Classifiers bridge the two ideas: a constitution defines allowed and disallowed content, Claude generates synthetic examples from it, and the trained input and output classifiers block jailbreaks. In February 2025 they cut the jailbreak success rate in automated tests from 86% to 4.4%, and the January 2026 version reduced compute cost about 40-fold relative to its baseline classifier while refusing 0.05% of production traffic.