A senior staff scientist at Google\u2019s artificial intelligence laboratory DeepMind, Alexander Lerchner, argues in a new paper that no AI or other computational system will ever become conscious. That conclusion appears to conflict with the narrative from AI company CEOs, including DeepMind\u2019s own Demis Hassabis, who repeatedly talks about the advent of artificial general intelligence. Hassabis recently claimed AGI is \u201cgoing to be something like 10 times the impact of the Industrial Revolution, but happening at 10 times the speed.\u201dThe paper shows the divergence between the self-serving narratives AI companies promote in the media and how they collapse under rigorous examination. Other philosophers and researchers of consciousness I talked to said Lerchner\u2019s paper, titled \u201cThe Abstraction Fallacy: Why AI Can Simulate But Not Instantiate Consciousness,\u201d is strong and that they\u2019re glad to see the argument come from one of the big AI companies, but that other experts in the field have been making the exact same arguments for decades.\u00a0\u201cI think he [Lerchner] arrived at this conclusion on his own and he’s reinvented the wheel and he’s not well read, especially in philosophical areas and definitely not in biology,\u201d Johannes J\u00e4ger, an evolutionary systems biologist and philosopher, told me.\u00a0Lerchner\u2019s paper is complicated and filled with jargon, but the argument broadly boils down to the point that any AI system is ultimately \u201cmapmaker-dependent,\u201d meaning it \u201crequires an active, experiencing cognitive agent\u201d\u2014a human\u2014to \u201calphabetize continuous physics into a finite set of meaningful states.\u201d In other words, it needs a person to first organize the world in way that is useful to the AI system, like, for example, the way armies of low paid workers in Africa label images in order to create training data for AI.\u00a0The so-called \u201cabstraction fallacy\u201d is the mistaken belief that because we\u2019ve organized data in such a way that allows AI to manipulate language, symbols, and images in a way that mimics sentient behavior, that it could actually achieve consciousness. But, as Lerchner argues, this would be impossible without a physical body.\u00a0\u201cYou have many other motivations as a human being. It’s a bit more complicated than that, but all of those spring from the fact that you have to eat, breathe, and you have to constantly invest physical work just to stay alive, and no non-living system does that,\u201d J\u00e4ger told me. \u201cAn LLM doesn’t do that. It’s just a bunch of patterns on a hard drive. Then it gets prompted and it runs until the task is finished and then it’s done. So it doesn’t have any intrinsic meaning. Its meaning comes from the way that some human agent externally has defined a meaning.\u201dOne could imagine an embodied AI programmed with human-like physical needs, and J\u00e4ger talked about why a system like that couldn\u2019t achieve consciousness as well, but that\u2019s beyond the scope of this article. There are mountains of literature and decades of research that have gone into these questions, and almost none of it is cited in Lerchner\u2019s paper.\u00a0\u201cI’m in sympathy with 99 percent of everything that he [Lerchner] says,\u201d Mark Bishop, a professor of cognitive computing at Goldsmiths, University of London, told me. \u201cMy only point of contention is that all these arguments have been presented years and years ago.\u201dBoth Bishop and J\u00e4ger said that it was good, but odd, that Google allowed Lerchner to publish the paper. Both said the argument Lerchner makes, and that they agree with, is not an obscure philosophical point irrelevant to the average user, but that the claim that AI can\u2019t achieve consciousness means that there\u2019s a hard cap on what AI could accomplish practically and commercially. For example, J\u00e4ger and Bishop said AGI, and the impact 10 times the Industrial Revolution that DeepMind CEO Hassabis predicts, is not likely according to this perspective.\u00a0\u201c[Elon] Musk himself has argued that to get level five autonomy [in self-driving cars] you need generalized autonomy\u201d which is Musk\u2019s term for AGI, Bishop said.\u00a0Lerchner\u2019s paper argues that AGI without sentience is possible, saying that \u201cthe development of highly capable Artificial General Intelligence (AGI) does not inherently lead to the creation of a novel moral patient, but rather to the refinement of a highly sophisticated, non-sentient tool.\u201d DeepMind is also actively operating as if AGI is coming. As I reported last year, for example, it was hiring for a \u201cpost-AGI\u201d research scientist.\u00a0Lerchner\u2019s paper includes a disclaimer at the bottom that says \u201cThe theoretical framework and proofs detailed herein represent the author\u2019s own research and conclusions. They do not necessarily reflect the official stance, views, or strategic policies of his employer.\u201d The paper was originally published on March 10 and is still featured on Google DeepMind\u2019s site. The PDF of the paper itself, hosted on philpapers.org, originally included Google DeepMind letterhead, but appears to have been replaced with a new PDF that removes Google\u2019s branding from the paper, and moved the same disclaimer to the top of the paper, after I reached out for comment on April 20. Google did not respond to that request for comment.\u00a0\u201cWe can imagine many financial and legislative reasons why Google would be sanguine with a conclusion that says computations can’t be consciousness,\u201d Bishop told me. \u201cBecause if the converse was true, and bizarrely enough here in Europe, we had some nutters who tried to get legislation through the European Parliament to give computational systems rights just a few years ago, which seems to be just utterly stupid. But you can imagine that Google will be quite happy for people to not think their systems are conscious. That means they might be less subject to legislation either in the US or anywhere in the world.\u201dJ\u00e4ger said that he\u2019s happy to see a Google DeepMind scientist publish this research, but said that AI companies could learn a lot by talking to the researchers and educating themselves with the work Lerchner failed to cite in his paper, or simply didn\u2019t know existed.\u00a0\u201cThe AI research community is extremely insular in a lot of ways,\u201d Jager said. \u201cFor example, none of these guys know anything about the biological origins of words like \u2018agency\u2019 and \u2018intelligence\u2019 that they use all the time. They have absolutely frighteningly no clue. And I’m talking about Jeffrey Hinton and top people, Turing Prize winners and Nobel Prize winners that are absolutely marvelously clueless about both the conceptual history of these terms, where they came from in their own history of AI, and that they’re used in a very weird way right now. And I’m always very surprised that there is so little interest. I guess it’s just a high pressure environment and they go ahead developing things they don’t have time to read.\u201dEmily Bender, a Professor of Linguistics at the University of Washington and co-author of The AI Con: How to Fight Big Tech\u2019s Hype and Create the Future We Want, told me that Lerchner might have been told that he\u2019s replicating old work, or that he should at least cite it, if he had gone through a normal peer-review process.\u00a0\u201cMuch of what’s happening in this research space right now is you get these paper-shaped objects coming out of the corporate labs,\u201d but that go through a proper scientific paper publishing process.\u00a0Bender also told me that the field of computer science and humanity more broadly \u201cif computer science could understand itself as one discipline among peers instead of the way that it sees itself, especially in these AGI labs, as the pinnacle of human achievement, and everybody else is just domain experts […] it would be a better world if we didn’t have that setup.\u201d<\/p>\n","protected":false},"excerpt":{"rendered":"