The Chief Digital and Artificial Intelligence Office of the Defense Department has announced it will award Anthropic, Google, OpenAI, and xAI contracts worth up to $200 million each "to develop agentic AI workflows across a variety of mission areas" and "increase the ability of these companies to understand and address critical national security needs." While the Defense Department's corporate welfare is par for the course, the ideological constitutions and ambiguous alignment of some of these companies' models are concerning for any governmental use.
OpenAI uses reinforcement learning from human feedback, which uses a reward model and human input to minimize "untruthful, toxic, [and] harmful sentiments" from ChatGPT. IBM explains that the benefit of this alignment strategy is that it does not rely on a nonexistent "straightforward mathematical or logical formula [to] define subjective human values." Google also uses this method to align its large language model Gemini.
Anthropic's model, Claude, does not rely on reinforcement learning but on a constitution, which Anthropic published in May 2023. Claude's constitution provides it with "explicit values…rather than values determined implicitly via large-scale human feedback." Anthropic explains that its constitutional alignment avoids problems that the human feedback model suffers from, such as subjecting contractors to disturbing and increasingly abstruse outputs.
