“We’re not only sacrificing understanding, we’re also sacrificing safety.”

“When these tools are just chatbots, it can be charming if they make mistakes. But now we’re putting them in charge of reward systems.”

“A lot of junior research projects just got wiped out overnight.”

“One or two papers even had errors and were retracted.”

“We don’t have a clear picture of what’s going on, and no one from OpenAI is helping us.”

“And then there were, of course, parts of it that were, I don’t know, crazy and almost cult-like. But it had maybe the good dimension of a cult where there was one big thing they knew that the other people did not. And I think even though there are ways I’d like to be very critical of them today, there were ways they got that one big thing right.”

“And then has a cult ever started a company as successful as Anthropic? You know, I don’t know what the most success, I mean, like even a billion dollar company seems like bigger than a crazed cult could build. So if you have a cult that’s built a company that’s worth probably close to two trillion at the IPO, it would probably be the, I don’t know, sixth, seventh largest company by market cap in the world built in like six, seven years. Has a cult ever done something like that? So, yes.”

“So I want to be critical of them. And they’re things I’m uncomfortable and critical about. But the first thing I would try to do is always, you always want to steel man your opponents. You want to try to, you know, try to understand them as well as they understand themselves or even make their arguments better. And the way I would steel man the effective altruists is, first off, they got one enormously big thing right.”