On Monday, Mark Zuckerberg published a 6,500-word manifesto about the future of artificial intelligence, focused on the possibilities of what he calls “personal superintelligence” systems built by Meta AI. The essay is not entirely new. An earlier version ran in The Wall Street Journal two weeks ago, and Zuckerberg has touched on similar themes during Meta earnings calls. But this is likely the most detailed version he has shared with the public.

The manifesto has been described in some corners as an anti-doomer essay, but that label does not quite fit. It is less a rebuttal of AI pessimism and more a personal explanation of why Zuckerberg is excited about how AI will transform society. Yet even as he paints a picture of abundant superintelligence improving daily life, he keeps reminding readers of the reasons many people remain nervous.

AI is an exciting topic, but a large portion of the public views it as creepy and unpleasant. That makes it important to understand why Zuckerberg’s framing is not helping the industry. The essay is a reminder of how much trust in technology companies has eroded, and how difficult it will be to rebuild.

A history of distrust

Social media remains a sensitive subject in the technology industry. Facebook, as a product, and Zuckerberg, as a person, are both unpopular with the American public. A recent survey found that 64% of Americans believe social media has been harmful to democracy, and a similar share believe the platforms should be more heavily regulated. Those numbers cut evenly across party lines.

The legal environment has also become more hostile. Just this past weekend, a court fined Meta $567 million over findings that its platforms were harmful to children. The decision added to a long list of regulatory and public relations problems for the company. The overall mood is bad, and Zuckerberg’s latest essay does little to improve it.

This history matters because the fallout from social media is one of the central reasons AI anxiety is so high. Fairly or not, the public does not trust tech executives to ensure that new technologies have a positive impact on society. Instead of acknowledging that reality and trying to earn back trust, Zuckerberg’s essay repeatedly demonstrates how the trust was lost in the first place.

Hazy generalities about intelligence

A large part of the essay is devoted to high-level claims about intelligence that sound a lot like the vague promises Zuckerberg and Twitter co-founder Jack Dorsey used to make about free speech. One passage reads: “As everyone gains more powerful tools, each person will become more capable of shaping the future, not less. … People and institutions with competing interests naturally check and balance each other to lead towards positive outcomes. The best and most realistic path to building a positive AI future is by delivering superintelligence to everyone.”

That is a comforting thought in theory, but history provides many examples of competing interests leading to terrible outcomes. The more striking issue is that this abstract philosophy is being presented as a serious conclusion rather than as a description of a specific product called “personal intelligence” that Meta is bringing to market. For anyone already inclined to distrust Zuckerberg, the essay reads less like a vision statement and more like an attempt to convince himself that nothing can go wrong.

Education and the homework problem

That feeling grows stronger when Zuckerberg moves to concrete examples. Many of them are alarmingly out of touch with the way AI tools are actually being used. On education, he writes: “Everyone will have a personalized tutor and coach with a PhD in every subject and unlimited patience to help you learn anything you want. Students will have extra help in areas they need it that is currently only available to those whose parents can pay. Adults will have a superintelligent learning assistant that knows exactly how to teach you new job skills, new languages, new hobbies, or anything else you’re interested in.”

The product he describes already exists in the form of consumer chatbots like ChatGPT, Claude, and Gemini. These tools can be genuinely helpful for exploring a topic. But in education, the main use is often to avoid learning. Students use the chatbot to finish homework, write essays, and complete assignments. Because robust watermarking systems are not yet in place, teachers have no reliable way to know whether a submitted essay was generated by AI.

This is a relatively low-stakes example of AI harm, but it is happening right now. It illustrates a basic principle: a technology designed for good can produce unintended consequences that make things worse. When one of the most powerful people in the world refuses to acknowledge even that simple possibility, it makes people understandably nervous.

Legal AI and the complexity risk

Zuckerberg also offers an example of AI improving the legal system. He asks readers to imagine a world where only one person has a superintelligent lawyer. That person would have an unfair advantage in court, even if they were wrong on the merits. Then he says: “But now imagine everyone has a superintelligent lawyer. In this case, justice would be carried out much more fairly and efficiently than it is today when there is often an imbalance in skills and resources in litigation.”

It is a loaded example. Access to legal AI could indeed democratize legal help. But it could also add complexity to an already bureaucratic system. It might unleash a wave of vexatious litigants flooding the courts with the legal equivalent of spam. The uncertainty is profound, and the fact that Zuckerberg does not seem worried only increases concern for others.

The freemium dream and surge pricing reality

Some parts of the manifesto touch on Meta’s existing business model. Zuckerberg describes a commitment to a freemium approach: “Everyone will have free or affordable access to these tools. … We will offer free versions that will be accessible to billions of people. For those who want to pay to use more compute, there will be a dynamic auction mechanism that will guarantee that everyone gets the lowest price possible for the intelligence and compute they’re using while also ensuring the capacity is used for whatever people collectively find most valuable. This will ensure the benefits of superintelligence are distributed widely.”

In broad strokes, the argument is reasonable. Access is an important issue, and a freemium model provides more access than requiring everyone to pay upfront. Dynamic markets for spot compute already exist, so the concept is not radical. But there is a reason every consumer AI product insulates users from the spot price of the compute they consume. Surge pricing is a terrible user experience, especially when the tool is something people rely on for their jobs.

It is possible Zuckerberg knows this and has no intention of setting token prices through dynamic auctions. But after reading the essay, it is hard to be sure. That ambiguity is itself a problem for a company trying to build consumer trust.

What trust-building looks like

There are better ways to talk about AI. Leaders at other AI companies, including Sam Altman and Dario Amodei, have at times communicated directly with the public in a more effective manner. They acknowledge the dangers of AI, emphasize the precautions they are taking, and try to convince listeners that they can be trusted. That is exactly what Zuckerberg resists in his manifesto.

Of course, communication can only do so much. Safeguards sometimes fail, and the inherent danger of the enterprise becomes clear. But without sustained trust-building, the AI industry may not be able to survive those failures. Zuckerberg’s manifesto, with its smooth promises and avoidance of real-world messiness, shows why the public remains skeptical.

Source: TechCrunch News