AI Enables Fourth-order Observations
And they are very similar to second-order observations
Digital society is a second-order observer. Rating, rankings, likes, citations, and reviews — all these determine choices more than direct perception. We choose restaurants by their Google rating, select movies by their IMDB score, and strive to send our children to a higher-ranked university. Ratings and rankings reduce uncertainty at the price of increased anxiety. They are bombarded by critics but are highly resistant to them. The resistance is successful because what is perceived as their bugs are the features that make them so effective.1
When you watch a movie, you don’t just like it or not; you have a multidimensional, contextual experience. This is first-order observation.
When you want to choose a movie, you check its IMDB or Rotten Tomatoes rating — a single-dimensional, decontextualized metric. You observe how others observe. The movie is considered good because of its high rating. This is a second-order observation.
The quality of the rating does not influence the importance of the rating when making choices. When you want to choose a high-ranked university for your child, the rank will influence your decision not because you agree with it, but because you know it will influence the decisions of your child's future employers. That second-order observation is why the rankings work.
Enter AI, another second-order observer but a large-scale, powerful one. And the users of AI are, naturally, third-order observers.
While ratings reduce uncertainty by the perceived certainty of aggregated first-order observations, AI deals with uncertainty by matching it with generated and ostensibly tamed uncertainty. It is a new kind of machine. When the output of a traditional machine is surprising, we see that machine as broken.2 With LLM it's the opposite. They are broken when their output is not surprising. And only if it's the right amount of surprise. To apply Spencer-Brown over Shannon’s information theory, we can see it as re-entry of surprise into surprise. From the amount of surprise in a message to the amount of surprise in a surprise. If the surprise is not enough, the output is useless. The machine gives us something we already know. If the surprise is too much, we call it hallucination. Equally useless (but way more annoying).
While humans are first and higher-order observers, LLMs can't be first-order observers. They start as second-order observers. Their human users are third-order observers. Most AI users are oblivious of that. Others are reflexive. They observe themselves doing third-order observations by using AI, and that's a fourth-order observation.3
Those who stop there are passive fourth-order observers. However, some take advantage of this realization and start gaming the system. These are active fourth-order observers. They apply decoding strategies and reverse engineering to attempt to infer hidden parameters and manipulate the models. This creates power asymmetries.
When active fourth-order observers gain critical mass, their strategies backfire. It's like many drivers are being diverted by their GPSes to the same route to avoid a traffic jam, creating a bigger one. Only this time, with AI, there are way more loops, and it’s way more unpredictable. And that’s why such power asymmetries will tend to oscillate.
But the oscillation is present already in the observation order. How come? To see that we first need a way to represent these observations, and then some examples. Since the laws of thought are for first-order observations, we can use the laws of form for higher-order observations.4
A first order observation is making a distinction:
()
Observing the making of a distinction (observation), should be represented like this:
(())
This is the form of a second-order observation. Following the same logic, a third-order observation will look like this:
((()))
And a fourth-order observation will look like this:
(((())))
There are two laws of form, the law of calling and the law of crossing.5 Here we can apply the law of crossing to transform these forms. Then we get:
((())) = ()
(((())))=(())= .
Observations oscillate between the marked and the unmarked state.
The sign "=" means "can be confused with", again reminding of the observer.6
It turns out that a third-order observation can be confused with a first-order observation, and a fourth-order observation can be confused with a second-order observation.
When using an LLM to answer a question, this is a third-order observation. If we get what we already know, it's of little value. On the other hand, if we get something far-fetched, it's of little value as well. We ascribe this to LLMs’ lack of explainability or hallucinations.
When using a book to get the answer, that's a first-order observation.7 Again, output is valuable if it has the right amount of surprise. It's worthless if it tells what we already know. If the book is or has parts that are completely unrelated to what we know, it is again of little value. We either blame ourselves for being stupid and not understanding it or the writer for not explaining it well.
The realization that using AI is a third-order observation is a fourth-order observation. It can remain a reflection, or we can act on it. For example, a content creator will game the system by tailoring their output not just to human tastes but to the hidden rules embedded in AI algorithms.
These two modes are also present in second-order observation. Choosing a book based on rating differs from selecting a university according to ranking. The choice of a book depends on what others like ourselves prefer. For that reason, there are multiple ratings for creative works. Using our favorite niche listing is a second-order observation but not for observing others; it's for observing ourselves. The university rankings are only a few, and they serve a similar purpose as the active forth-order observation: to manipulate the system.
Both second- and fourth-order observations attract gamification.
For more arguments supporting the claim that ratings are second-order observation and their persistence being because of and not in spite of their shortcomings, see
Esposito, E., & Stark, D. (2019). What’s Observed in a Rating? Rankings as Orientation in the Face of Uncertainty. Theory, Culture & Society, 36(4), 3–26. https://doi.org/10.1177/0263276419826276
Ratings and rankings can be seen as second-order observations if the reference point is the object of evaluation. But also, and more importantly, when the reference point is the ratings themselves. Ratings and rankings work because we observe others observing them.
von Foerster, H. (2003). Perception of the Future and the Future of Perception. In H. von Foerster (Ed.), Understanding Understanding: Essays on Cybernetics and Cognition (pp. 199–210). Springer. https://doi.org/10.1007/0-387-21722-3_7
One may entertain the unlikely possibility that this was the real reason for Open Al's naming one of their models “4o.”
Aristotle and George Spencer-Brown. (n.d.). ResearchGate. https://doi.org/10.2139/ssrn.2073361
My 2015 article, From Distinction to Value and Back, can serve as a quick tutorial on the Laws of Form.
It can only be confused by someone, echoing “anything said is said by an observer” (attributed to Humberto Maturana by Heinz von Foerster in Cybernetics of Cybernetics)
This is true of the book itself because we experience it directly. If we refer to the book's object, then we can say we observe the author's observations.