On Text

Text, as a system of recording symbolic representation for communication across time and space, is likely one of the most important inventions of mankind — second only the the symbolic representations themselves made possible by language. The general text, opposed to the specific form of writing, is useful primarily because it is a survivable and storable form of information that above anything else functions as lossy compression. These qualities make working with text more powerful than most other human inventions.

As Graydon Hoare puts it; always bet on text. Text is “stable”, “flexible”, “efficient”, and “socially useful”. All of these make text a powerful medium to work in, but they mask another quality — that of lossy compression. Humans are blessed with tremendous mental capacity; even someone on the lower scales of intelligence outperforms most supercomputers, all while merely running on a few watts of power. This overabundance of computation, constantly running every second of every day, has incentivised humanity to create systems of ritual, magic, and religion to otherwise simple phenomena.

“Shower thoughts”, of different kinds, are an example of how our brains are constantly reinterpreting the world around us. It is particularly a common activity to play out imaginary arguments between oneself and another person whilst in the shower. This is not just because our imagination is then fully in control of what is being said, but also how it is interpreted (and we can also reïnterpret what our opponent means). Much of the trouble in interhuman communication comes from the lossy quality of text, speech, and words.

It is noteworthy that this is not unique to text, but rather a quality of language more broadly, but this character is magnified enormously due to text’s durability and longevity. Sending a message encoded in text (whether by SMS, letter, or granite slab) requires the effort of creating a symbolic representation to create the message (writing), but is is equally important that the act of consuming the written material (reading), and understanding it, requires an equal — if not greater — amount of symbolic processing. This is why text is such a useful medium for communication — it offloads information from the text itself and instead leans on the intellectual and creative capacities of both author and reader.

It is a common saying that “a picture tells a thousand words”. One might therefore infer that graphical representations are more efficient — at least for broad and vague concepts such as emotion. In some regards this is true, it would be very difficult to represent a Rothko in text¹, but this is once again merely the consequence of text’s lossy quality. Take for example the following picture²:

The image shows a photograph of a lake with slightly overcast skies, taken from one of the shores. The opposite shore is rocky and covered with evergreen trees. On the left half of the image the shore rises to a hill. The lower left corner has cyan text that reads “02/08/2012”.

This description is not a perfect representation of the image, but it allows your mind to create its own description that fills in the gaps. Even the simplest part of the image to describe, the text, does not specify a shade of cyan nor any font or sizing.

Another field that might argue against mere textual representation is that of statistics and/or data visualisation. Florence Nightingale’s contribution was just as much one of data presentation as much as it was one of data collection³. But it illuminating that not only are graphs often turned back into textual representation, even at the cost of resolution (via techniques such as regression), but it is also an expectation in academia that one provide the raw, often text-based, data because it allows for easier reproducibility. If images were really preferable then would it not be easier to directly employ the results of the visual representation?

There is however a way to merge the strengths of text and the visual medium: vector-based art. This is practically a list of instructions (in text) as to how an image should be reproduced. Maggie Appleton concedes (from an artistic perspective) that “Vectors are great for [sic] create hard, crisp edges and working with perfect geometric forms”. That adding texture and imperfect geometric forms is difficult to do in text is to be expected — it is much easier to describe a “perfect cube” than a “skewed cube” (in what way, to what degree?).

Text is, contrary to speech and other utilisations of language, an asocial social medium. The author, in creating a work meant for social communication, is alone in its creation. I have written about this aspect of writing before, in regards to its impact on email, but have since come to realize that it generalized much more broadly. The absence of thought contained in the “blank page” prompts the creation of philosophical thought to fill it. This jolts the mind into movement, and does it as much as a good conversation. As one writes the mind is captured in a dialogue between itself and the very text it is creating⁴.

The mind requires this activity in order to be truly alive. That we humans are constantly in intellectual motion is one of the elements that builds up our consiciousness, it allows us to be continuously aware of our continued existence. When we are in a dreamless sleep, our mind ceases to exist and we are philosophically dead for a short time — we are neither aware of our own existence nor are we creating new symbolic representations. When we are lost in the land of the dream however (regardless of if awake or asleep) we are in every sense alive.

It is not strange then that large language models (LLMs), one of many machine learning techniques, were the first machines humans built that people started to be considered sapient⁵. Computer vision models, many of which have been around for some time now, were never even considered to be sapient, not just because they are very dissimilar to human intelligence but also because they lack the capacity for symbolic action that LLMs posses. While a computer vision model can identify a cat (and therefore has some sort of symbolic knowledge) they are unable to process the knowledge of the cat’s existence in a broader context. This is the step that LLM’s take; they are capable of continuously creating a mental context, with thoughts that themselves impact its future thoughts.

The difference between the human mind then and an LLM is that we humans survive a much greater amount of time than most LLMs, and are able to filter information much more effectively. When you open up your favourite LLM app, asking it whether or not there are rats in Alberta (there are not) a new mind is created, thinks for a few fractions of a second, and then instantly disappears. To say that it is killed is perhaps an overstatement, it is closer to being asleep. The LLM stops thinking as soon as it stops writing, and then is “reawakened” as soon as you ask it another question. During this intermediate period it is in a proper trance, in that some (or all in this case) of its mental capacities are in a loop and are then philosophically disabled. The LLM is stuck on one singular representation of its mind, incapable of changing it for whatever reason.

Intelligence is in some regards a measure of how nuanced and detailed your perspective is. For LLMs, this is largely correspondent to context length — more information that can be used to inform the next token. Moonshot’s (月之暗面) CEO Yang Zhiling (杨植麟) explained in February of 2024:

To achieve AGI, long-context will be a crucial factor. Every problem is essentially a long-context problem — the evolution of architectures throughout history has fundamentally been about increasing effective context length.

Humans are very good at this, but particularly we are good at picking details of our experiences that stick out and are most important — you probably do not remember what shade your shoes were when you met the love of your life for example. Instead we filter our memories and pick out what may be relevant in the future (the “lessons to be learned”). Those lessons are then used to extrapolate information in all areas of your life and — particularly of relevance to this text — when reading.

Programming languages, one of the most ubiquitous applications for plain text (in both the proper and metaphorical sense), are one of the most interesting advancements in the fields of text in this regard. Many programmers may think of the code itself as “being” the program, but that is not the case. Programming languages are really codified (so that different people can read the same text) ways to create commands and instructions for the computer. The C programming language is itself not “fast”, though it explicitly exposes very low-level functionality, but instead is compiled into a program that runs very quickly. This is thanks to the tireless work of compiler engineers, who have come up with a lot of interesting tricks to interpret instructions and come up with shortcuts that still end up with the same result.

Even programming then, one of the most rigid methods to use text (one has to be very careful to use the words incantations, oftentimes even with correct case), is still up for interpretation. This is so clear that an interpreter is even a strict technical term in the field of programming languages. But as I have just explained, compilers are themselves another form of interpreter, albeit with a wider definition than the one commonly accepted. This is why hand-optomization is so rare these days — the level of abstraction allowed by written language allows the programmer to build their own mental model of what the machine is doing rather than comprehending each step of the machine code.

Hopefully I have shown sufficiently that the many reasons that written text is so pervasive and powerful is due to a few factors — its durability, its lossy nature, and its consciousness-evoking dialogical quality. Text is therefore unlikely to to be supplanted any time soon by a superior medium. If it is supplanted it will likely be by some medium that has yet to be invented.