I’ve been very interested lately in artificial intelligence (AI) and our current fears and hopes about it, and have done a lot of reading on the topic. From the title, this book looked like it might be more of the same. But actually this is the one book I’ve read that goes deeper into the underlying problem behind not only AI, but also many of the other themes and obsessions occupying the attention of our high-tech oligarchs.
The basic story here is surprisingly easy to grasp. A small circle of grown-up boys, fascinated by the science fiction stories of the mid-twentieth century they read as children, have become the leaders of new technology empires, often based on devices and applications they invented. In the process, they became so fabulously wealthy that they can now exert disproportionate influence on policy and funding for research and development of new technologies that echo the science fiction stories they loved in their youth.
An unfortunately common collateral development was that as they became richer and more powerful, these men increasingly began to fall victim to “the engineer’s fallacy” – the idea that because they had invented or designed something remarkable, and had become vastly wealthy using their bright intellects, engineering skills, social connections and astonishingly good luck, that they were also the smartest people in every domain of knowledge and policy, even ones in which they had no training or relevant experience.
It is true that these “bright boys” have brought much of mid-twentieth century science fiction into being. Our computers, our global electronic networks, our smartphones, our software, our healthcare, our transportation systems – all these science and technology miracles, and many more, have been brought into being in our lifetimes by some of these men, in ways that do resemble science fiction stories of the past century.
And certainly there will be more amazing technology developments and scientific leaps ahead for humanity, presuming that the second Trump administration doesn’t destroy our ability to do science and educate our young to prepare them for the future. The problem is that although many astonishing breakthroughs have occurred, that doesn’t mean that every 20th century science fiction plot can come true, or will. Some of them are just fantasies, and will remain so.
Becker does a wonderful job inventorying the wrong turns and bad ideas now being pursued by a familiar cast of characters: Elon Musk, Peter Thiel, Mark Zuckerberg, Jeff Bezos, Sam Altman, Marc Andressen, and others in their Silicon Valley circles who are less well known to us. In this absorbing walk through the tech bros’ grandest dreams, the author lays bare the shaky scientific foundations on which their hyped-up plans rest, their problematic ethical and philosophical roots, and the personal quirks of the leaders who have been involved in promoting them.
Early on, there is an excellent discussion of the rise of “effective altruism” (EA). This was originally conceived of as an approach to life and charity that suggested that giving should be based on a moral imperative to do the greatest good for the greatest number of people per dollar. Many of the current generation of wealthy tech leaders were enthusiasts. It sounded morally very defensible in theory.
But then the tech and finance types who found it so appealing for its rational-sounding ethics began to come up with other complementary ideas, like “long termism” – the notion that effective giving should be based on the number of lives saved across all future time.
When combined with outlandish beliefs about space colonization, life extension, eugenics and other unproven technologies and science, this quickly led EA enthusiasts in high tech to rationalize focusing their investments on those areas where there was a great deal of money to be made, while feeling ethically excused from dealing with any contemporary problem areas or suffering. From there, the author traces how EA and its offspring tendencies have devolved into various factions and cults.
Becker particularly caught my attention with his history of K. Eric Drexler, the MIT-trained cross-disciplinary engineer and scientist who wrote Engines of Creation in 1986, the book which launched the nanotechnology revolution. (I met Drexler while he was promoting his book and ideas back then, and even became enthusiastic about his theories for a time, after hearing him speak at a conference of space colonization enthusiasts).
Drexler’s thrilling idea was that we would soon be able to utterly transform the world, create inexhaustible wealth, fix all diseases, cure aging, and create colonies in space, using tiny robot engines to build and transform materials at the molecular level. This was nanotechnology as he defined and explained it, and it sounded like magic come true.
Drexler had some brilliant insights, based on an idea originally voiced by the famous physicist Richard Feynmann. As the first and greatest promoter of this new field he conceptualized – “nanotechnology” – Drexler really did set off a “race to the bottom” in the late 1980s and 1990s for research and development in many different disciplines, most notably in medicine, genetics and materials sciences. Large amounts of money would be allocated for nanotechnology R&D based on his ideas and his persuasive promotion of them, and many important scientific advances did happen as a result.
The problem was, as Becker explains, that Drexler’s actual plan – building invisibly small smart robots that could manufacture substances and products molecule by molecule – has still never happened. It doesn’t appear to be practical for a variety of physical and financial reasons. After a while, despite the lack of progress in achieving the specific nanotech engineering vision he promoted, Drexler’s dreams for the nanotechnology golden age, and his group of followers, increasingly took on the appearance of other tech cults or religions. As with EA and AI, this led inevitably to schism and apostasy among his true believers.
Becker goes on to dismantle other fantasies and wishful thinking of his wealthy tech subjects. He shows why the belief in artificial general intelligence (AGI) in the very near future appears to be an article of faith in AI business circles, but not one that can be supported by either the current large language models (LLM) used to train AI, or Moore’s Law (which appears to be reaching its end).
As he explains in detail, the belief in infinite technical acceleration in AI R&D breakthroughs, and in fast-doubling computing power, is already beginning to crash against physical, technical and financial limitations imposed by the real world, and must inevitably do so.
The author points out that exponential growth cannot continue forever in any system, and it won’t. But without that kind of rapid growth in computing power, the common theory in the AI industry that scaling up existing AI models will inevitably lead to AGI simply collapses. It is a wan hope and a marketing ploy, not a realistic prediction.
Becker does a similarly effective job demolishing the idea of space colonization, which billionaires like Musk and Bezos believe is inevitable, essential to humanity’s survival, and likely to occur very soon. Becker claims (and I agree) that space colonization could happen only over a very long time, likely centuries or millenia, or more likely never, and for very good and obvious reasons.
(For an amusing and more extensive book-length analysis of the many barriers to space colonization, read A City on Mars: Can We Settle Space, Should We Settle Space, and Have We Really Thought This Through? (2023), by Kelly and Zach Weinersmith, which I also enjoyed reading recently).
Becker notes that the surface of Mars contains an abundant substance, perchlorate, that is poisonous to humans, a fact discovered by the robotic Mars landers. Cosmic radiation and lack of earth-equivalent gravity are also already known to degrade the human body in space. All known planetary bodies in the solar system except Earth cannot sustain human life for many reasons – no air, wrong temperatures, too much radiation, too much or too little gravity, poisonous environments and so on. There are simply limits to what even clever engineering can overcome.
Space colonization would also require safely transporting a huge number of people, probably hundreds of thousands or millions (needed to sustain a healthy population with sufficient genetic diversity) across vast distances and time periods, by some means of mass space travel not yet invented, and where necessities of colonization such as human reproduction are unproven, unlikely or impossible. Even attempting reproduction in space would involve ethically indefensible experimentation on any future human children who might be created.
And then there is the most obvious other question. How many actual humans on Earth would risk their life and future to attempt to live on a deadly planet Mars, or in a manufactured deep space colony? The obvious answer is “not very many”.
In short, space colonization won’t happen in any near-term foreseeable future, but spending billions on it can be much more fun, exciting and profitable than admitting to or worrying about solving the very real and immediate problems humanity faces here on its home planet.
Becker goes on to describe and then puncture more sacred fantasies of the ultra-rich tech oligarchs. Infinite life extension. Elaborate bunkers and preparations for surviving social collapse. Uploading human consciousness to the cloud, and the predicted impending Singularity to be caused by AI.
Becker notes (as others have also observed) that the idea of the Singularity, so popular in AI circles, bears more resemblance to ancient religious ideas like the Rapture or Armageddon, in its vision of a mysterious godlike power taking control of humanity’s fate and giving everyone eternal life in “heaven”, than it does to any scientifically-based projection of our likely futures.
Becker ties many of these tech bro aspirations back to the universal desire to escape death. It must be demoralizing for many of these powerful men to have so conquered this material world, yet to face the same aging process and ultimate demise as the rest of us. And they’re not going quietly into that night. They have their sci-fi visions of how to escape it, and the money to pursue those visions, no matter how outlandish and hopeless their plans may be.
And that, as Becker finally concludes in the last two pages of this essential book, is the root of the problem these few mega-rich, mostly white male tech leaders now pose for humanity.
Like the rest of us, they are free to pursue their dreams, make mistakes, have foolish ideas and hopes, and try to bring to life the fantastic stories they loved in their childhoods, regardless of how impossible or meaningless their goals might seem to others. But unlike the rest of us, they have the money and power to make their vain pursuits everyone else’s problem, and to waste huge financial and social resources that might be far better applied to fixing humanity’s real and immediate challenges.
The underlying problem here, Becker concludes, is billionaires, especially tech billionaires with childish beliefs based on sci-fi fantasies. If it weren’t for their power and wealth, he suggests, modern society wouldn’t be taking most of their grandiose but absurd ideas as seriously as we do.
After reading this very entertaining and well-researched book, and its thorough debunking of the science and logic behind these high tech moguls’ expensive hobbies, fetishes and plans, I would hope most readers would agree with his conclusions. That’s an important recognition for all of us as we consider any future hype, business decisions and political machinations of this small but massively influential power elite.
They may have won life’s financial lottery, but they aren’t necessarily wiser or more well-meaning than the rest of us. We shouldn’t believe they are, just because they’re fabulously rich, and can harness a massive hype machine and political influence to promote their ideas.
Adam Becker has written the definitive expose’ on the Silicon Valley tech elite, and their flawed visions of the future. Highly recommended.
Another enlightening review! Thank you!