AI - Computers - Crusades

AGI — a theoretical AI that can do many of the same tasks as humans can — could come within a decade. College students, including from elite universities, are abandoning school now to work full-time on preventing it from turning on humanity.

When Alice Blair enrolled in the Massachusetts Institute of Technology as a freshman in 2023, she was excited to take computer science courses and meet other people who cared about making sure artificial intelligence is developed in a way that’s good for humanity.

Now she’s taking a permanent leave of absence, terrified that the emergence of “artificial general intelligence,” a hypothetical AI that can perform a variety of tasks as well as people, could doom the human race.

“I was concerned I might not be alive to graduate because of AGI,” said Blair, who is from Berkeley, California. “I think in a large majority of the scenarios, because of the way we are working towards AGI, we get human extinction.” She’s lined up a contract gig as a technical writer at the Center for AI Safety, a nonprofit focused on AI safety research, where she helps with newsletters and research papers. Blair doesn’t plan to head back to MIT. “I predict that my future lies out in the real world,” she said.

Blair’s not the only student afraid of the potentially devastating impact that AI will have on the future of humanity if it becomes sentient and decides that people are more trouble than they’re worth. “Extinction-level” risk is possible given how fast AI is being developed, according to a 2024 U.S. Department of State-commissioned report. Efforts to build AI with safeguards to prevent this from happening have exploded in the last few years, both from billionaire-funded nonprofits like the Center for AI Safety and companies like Anthropic.

A lot of researchers disagree with that premise—“human extinction seems to be very very unlikely,” New York University professor emeritus Gary Marcus, who studies the intersection of psychology and AI, told Forbes. “But working on AI safety is noble, and very little current work has provided answers.”

Now, the field of AI safety and its promise to prevent the worst effects of AI is motivating young people to drop out of school.

Physics and computer science major Adam Kaufman left Harvard University last fall to work full-time at Redwood Research, a nonprofit examining deceptive AI systems that could act against human interests.

“I’m quite worried about the risks and think that the most important thing to work on is mitigating them,” said Kaufman. “Somewhat more selfishly, I just think it’s really interesting. I work with the smartest people I’ve ever met on super important problems.”

He’s not alone. His brother, roommate and girlfriend have also taken leave from Harvard for similar reasons. The three of them currently work for OpenAI.

Other students are terrified of AGI, but less because it could destroy the human race and more because it could wreck their career before it’s even begun. Half of 326 Harvard students surveyed by the school’s undergraduate association and AI safety club were worried about AI’s impact on their job prospects.

“If your career is about to be automated by the end of the decade, then every year spent in college is one year subtracted from your short career,” said Nikola Jurković, who graduated from Harvard this May and served as the AI safety group’s AGI preparedness lead. “I personally think AGI is maybe four years away and full automation of the economy is maybe five or six years away.”

The post continues to part 2.

- Author: Victoria Feng, Forbes