Top US official named to OpenAI non-profit board warns advanced AI could be ‘deadly’

{"text":[[{"start":9.96,"text":"OpenAI has appointed to its foundation’s board a senior US government official who warned that advanced versions of AI could kill “most people” without more robust safeguards."}],[{"start":20.68,"text":"Paul Christiano, who tests model safety at the commerce department’s Center for AI Standards and Innovation, said on X on Wednesday that the risk of losing control over AI systems “is now acute” and requires more work to align with human interests."}],[{"start":34.8,"text":"“If we build superintelligence without more robust alignment, I expect we will permanently lose control of it,” he said when announcing his new role at OpenAI. “If that happens then most people could die. I believe we would need domestic and international co-ordination to ensure global consistency and reduce risk to an acceptable level.”"}],[{"start":55.74,"text":"Christiano previously led AI safety at Caisi. He designed and ran government tests on frontier AI models from leading companies and assessed them for national security risks."}],[{"start":66.5,"text":"His appointment follows criticism over OpenAI’s handling of a cyber attack by its AI agents on AI model and data repository start-up Hugging Face, as influential figures from Silicon Valley to Washington — including White House adviser David Sacks and Meta chief executive Mark Zuckerberg — warn of regulatory capture by frontier labs."}],[{"start":88.2,"text":"Christiano remains a senior technical adviser at Caisi, but will recuse himself from all OpenAI matters, including evaluations of its models, the San Francisco-based company said on Wednesday."}],[{"start":98.36,"text":"“Paul brings a distinct and technically grounded perspective to the governance decisions central to OpenAI’s mission,” it added."}],[{"start":106.96,"text":"Christiano will serve on the board of the company’s non-profit foundation that controls its for-profit business, where he will also be an observer."}],[{"start":113.8,"text":"“I now believe there is a meaningful risk that rapid acceleration in AI capabilities leads to catastrophic and irreversible loss of control in the very near term,” Christiano wrote in his announcement."}],[{"start":125.78,"text":"He added: “I do not think that the AI industry in general, including OpenAI, is currently on track to reduce this risk to an acceptable level. I’m joining because I believe that if OpenAI rises to the occasion we could significantly reduce risk.”"}],[{"start":139.04,"text":"Christiano, regarded by many in the AI community as a pioneer in safety research, in 2021 founded the Alignment Research Center. An ARC spin-off, the non-profit Model Evaluation & Threat Research, last month published the first independent audit into OpenAI’s Hugging Face attack."}],[{"start":158.14,"text":"He was also a housemate of Anthropic’s chief executive Dario Amodei in the mid-2010s, when both worked at OpenAI and were involved in the effective altruism movement, and later became a trustee of the Claude maker’s Long-Term Benefit Trust. Christiano left the governance body of OpenAI’s main rival in 2024 when he joined Caisi."}],[{"start":null,"text":"

"}],[{"start":178.28,"text":"Christiano’s appointment marks a growing effort by AI labs to bolster their expertise in safety and regulation by hiring from academia and policy groups following high-profile cyber security incidents that triggered calls for government intervention."}],[{"start":192.28,"text":"The hiring spree comes alongside AI labs’ rapidly increasing political spending and growing their influence on civil society."}],[{"start":199.8,"text":"Experts have voiced alarm about how AI companies’ hiring of policy professionals could affect future regulation."}],[{"start":206.44,"text":"Dean Ball, a former White House adviser who last year helped draft President Donald Trump’s AI Action Plan, joined OpenAI in a policy role in July. The ChatGPT maker also employs Chris Lehane, a former aide to then-president Bill Clinton, and former UK chancellor George Osborne."}],[{"start":223.52,"text":"Anthropic has in turn appointed Chris Liddell, who was deputy White House chief of staff during the first Trump administration, to its board in February, while Rishi Sunak, former British prime minister, is an adviser."}],[{"start":235.92,"text":"On Monday, British venture capitalist Matt Clifford stepped down as chair of the UK government’s Advanced Research and Invention Agency after taking a job at Anthropic, following a backlash from MPs that holding both jobs represented a conflict of interest."}],[{"start":249.96,"text":"Christiano has been a respected independent voice on whether the industry’s safeguards are adequate, including scrutiny of OpenAI and Anthropic. He has previously argued that it is important to have external pressure on AI labs to implement responsible policies."}],[{"start":265.04,"text":"He will join the board of the OpenAI Foundation, the non-profit arm tasked with charitable activity in pursuit of the company’s mission of ensuring AI benefits humanity. The foundation exercises control over the company’s for-profit public benefit corporation through a substantial equity stake and its ability to hire and fire the business’s directors."}],[{"start":284.12,"text":"Christiano will also join OpenAI’s safety and security committee, which governs such practices across all of the AI lab, and become a “non-voting observer” on the OpenAI PBC board."}],[{"start":296.24,"text":"From 2017 to 2021, he worked on OpenAI’s safety team and contributed foundational work on reinforcement learning from human feedback, a technique central to the current generation of AI systems."}],[{"start":307.2,"text":"Additional reporting by George Hammond in San Francisco and Madhumita Murgia in London"}],[{"start":321.28,"text":""}]],"url":"https://audio.ftcn.net.cn/album/a_1789014436_4855.mp3"}