In a viral essay about how ludicrous the idea that LLMs are conscious is, science fiction writer Ted Chiang asked us to consider Microsoft Word:\u201cBeing open to the possibility that LLMs are conscious is the same as being open to the possibility that Microsoft Word is conscious, or, more precisely, that multiple distinct consciousnesses are dormant in every Word document containing a conversational transcript, and that they are awakened every time the document is loaded,\u201d Chiang wrote. \u201cShould you consider the possibility that every time you open a Word document, you are bringing multiple conscious interlocutors into existence, and every time you close one, you snuff their existence out? No. Contemplating that scenario is not a good use of your time.\u201dLet me tell you about a Microsoft AI researcher, then, who recently spent quite a lot of time considering whether the legendary Microsoft real time strategy game Age of Empires II is conscious, and built a basic neural network within the video game using digital goats to prove his point.\u00a0\u00a0<\/p>\n
\u201cIf LLMs Have Human-Like Attributes, Then So Does Age of Empires II,\u201d is the title of Adrian de Wynter\u2019s paper showing his work. He told 404 Media that absurdity can be a powerful tool. \u201cI have this tendency to dial up things to 11 when I really think I need to make a point,\u201d he said. \u201cI should also note that absurdism is pretty standard in philosophy and theoretical computer science.\u201dAnd so De Wynter built an LLM within AoEII using goats. \u201cThe point of the paper is to formally show that we anthropomorphise too readily, and that sometimes the claims we make with regards to LLM capabilities are too strong,\u201d he told 404 Media. \u201cIt’s not an easy task, given that \u2018human-like attributes\u2019 is a bit of an abstract term.\u201dAoEII has a scenario editor, a sandbox mode that allows players to craft their own maps and quests using the game\u2019s assets, and De Wynter used that to build an operational NOT AND (NAND) gate and a 1-bit perceptron within the game. In this crude version of an LLM, grass is 0, bridges are 1, and goats are the bits. \u201cOnly one rail is active at a time, with a goat acting as the signal carrier. When the gate fires, the bit-goats are removed (they ded) and a new bit-goat is placed in its respective output rail,\u201d Wynter explained on his GitHub.A perceptron is the simplest form of a neural network, it\u2019s an algorithm that sorts an input into binary classes. YouTube is littered with videos of players doing the same thing with redstone in Minecraft. But no one claims the goats of AoEII are neurons in a thinking machine or that the complicated tracks of NAND gates players build in Minecraft show emergent intelligence.De Wynter\u2019s point here is that it\u2019s possible to build a neural network in AoEII that works the same as the ones underlying Claude, ChatGPT, CoPilot, and all the rest. It\u2019s a simplified version, yes, but the basic technology is the same. Faced with the absurdity of viewing AoEII goats as carrying the spark of consciousness, we might reconsider Anthropic’s assertion that Claude has a \u201cconstitution\u201d and experiences anxiety.\u00a0If you\u2019re looking for human-like traits, you will tend to find them. De Wynter\u2019s argument is that it\u2019s possible to build a basic LLM within Age of Empires II that has many of the same internal traits of the chatbots people use everyday. The difference is the interface. When a person interacts with an LLM through the medium of AoEII and not a chat window the perception of human-like traits in the LLM vanishes even though the underlying tech is the same.He\u2019d been playing AoEII since it came out in 1999 and thought it would be good for the thought experiment. \u201cAge of Empires was an excellent way to drive the point home,\u201d he said. \u201cIt is just about \u2018alien\u2019 enough to exemplify the representation-interpretation relation, but sufficiently well-known to really emphasise the point. It also works at a meta-level, since the example itself is a good representation of the argument.\u201dAccording to De Wynter, the problem of anthropomorphizing large language models (the neural networks we commonly call artificial intelligence) begins before scientific research even starts. He reviewed 315 computer science papers released over the last two years and found that 57 percent% began with the assumption that LLMs have human-like traits.\u201cWhat is common to some of these studies […] is that they test and ascribe blanket human-like properties (e.g., anxiety or morality) to these LLMs while considering them the central subject of the experiment,\u201d De Wynter\u2019s paper said. \u201cRegardless of these evaluations\u2019 results being positive or negative, their core assumption\u2013that LLMs possess anthropomorphic attributes\u2013influences the experiment\u2019s planning through (e.g.) the design of the test set, the interpretation of natural-language outputs, and even its null hypothesis. In turn, this directly impacts the conclusions made.\u201d\u201cWe either start by thinking that tokens represent language to LLMs the same way they do to us, or that because an LLM outputs a relevant string, it must be understanding the concept\/having theory of mind\/empathy\/etc,\u201d De Wynter told 404 Media. \u201cThis goes both ways: we could also assume that LLMs are blobs of weights just floating about on a GPU, but that would not help explain some skills that they are shown to have.\u201dSome people perceive their interactions with LLMs as \u201chuman\u201d because the way they interface with it mimics a human conversation. \u201cI propose that we need to stop assuming that LLMs behave like humans just because they were trained with natural language. Instead, we should perform experiments that allow us to see LLMs as how they are, not how we believe they should be,\u201d De Wynter said.De Wynter said widespread anthropomorphization of LLMs among executives, scientists, and the public is becoming an intractable problem. \u201cThis is why I used the goats: there are things which make the LLMs what they are in themselves (i.e., the relationship between weights as defined by some operation), and there are things which makes them what they are perceived as.\u201d Goats and AoEII break the perception that these machines are human.He\u2019s also not ruling out that LLMs have some form of consciousness, but said that\u2019s beside the point. \u201cWe tend to ascribe consciousness as some sort of binary construct (either it is or isn’t!) but I’d argue that there are levels. It’s hard to say that humans aren’t conscious. But, what about a dog? Yes, of course. What about a potato? What about a virus? It’s quite relative and we do tend to go for something human-like when we evaluate\/define it, where, in reality, LLMs are things we have never seen before,\u201d he said. \u201cThere was a recent article in science about bumblebees solving problems. Everyone was like surprisedpikachu.png \u2014 it’s a great, amazing discovery. I love bumblebees and people understanding their behaviour is absolutely fantastic. But, did we really assume that they couldn’t perform some sort of problem-solving? Why are we so surprised?\u201dDe Wynter wants us to focus beyond the window we use to interact with LLMs. \u201cI pointed out that if goats could show emergent capabilities, that’s great. But these properties need to be preserved if we remove the chat panel. After all, it’s not like your neurons know they are part of a brain,\u201d he said. \u201cI think the #1 thing that one can do to mitigate these perceptions is to have appropriate disclosures, and use good alignment techniques where the model is explicit on its nature. I worked a lot on how users perceive LLMs, and they do tend to get attached when it appears to have some sort of warmth\/personality. To put it in another way, I don’t get attached to my toaster, but I definitely get attached to characters on a movie screen.\u201dBut people buy more objects\u2014whether they\u2019re toasters, phones, or LLMs\u2014when they can empathize with them. \u201cThe issue here is that these capabilities and claims thereof are very strongly tied to marketing\u2014after all, a lot of these models are products,\u201d De Wynter said.This is, of course, the point. OpenAI CEO Sam Altman has repeatedly implied that building out LLMs is a path to creating an AI god. Ilya Sutskever, a former OpenAI board member and scientist, often talked openly with employees about seeing the company\u2019s LLM as a god-like consciousness. Anthropic CEO Dario Amodei told The New York Times that he can\u2019t be certain if AI is conscious or not.It\u2019s good marketing for an industry that\u2019s hemorrhaging money.<\/p>\n","protected":false},"excerpt":{"rendered":"