The Ecstasy of Deep Influence

Extending Jonathan Lethem's arguments to the LLM era

The Contraptions Book club March pick is Ibn Khaldun: An Intellectual Biopgraphy, by Robert Irwin. Chat thread. We will discuss this the week of April 28th.

In a 2007 article, The Ecstasy of Influence: A Plagiarism, published in Harper’s, Jonathan Lethem delivered a great polemic against a narrow-minded view of authors’ rights, plagiarism, and claims to originality. A key passage is worth quoting in full:

Despite hand-wringing at each technological turn — radio, the Internet — the future will be much like the past. Artists will sell some things but also give some things away. Change may be troubling for those who crave less ambiguity, but the life of an artist has never been filled with certainty.

The dream of a perfect systematic remuneration is nonsense. I pay rent with the price my words bring when published in glossy magazines and at the same moment offer them for almost nothing to impoverished literary quarterlies, or speak them for free into the air in a radio interview. So what are they worth? What would they be worth if some future Dylan worked them into a song? Should I care to make such a thing impossible?

Any text is woven entirely with citations, references, echoes, cultural languages, which cut across it through and through in a vast stereophony. The citations that go to make up a text are anonymous, untraceable, and yet already read; they are quotations without inverted commas. The kernel, the soul — let us go further and say the substance, the bulk, the actual and valuable material of all human utterances — is plagiarism. For substantially all ideas are secondhand, consciously and unconsciously drawn from a million outside sources, and daily used by the garnerer with a pride and satisfaction born of the superstition that he originated them; whereas there is not a rag [hehe, cf RAG in AI] of originality about them anywhere except the little discoloration they get from his mental and moral caliber and his temperament, and which is revealed in characteristics of phrasing. Old and new make the warp and woof of every moment. There is no thread that is not a twist of these two strands. By necessity, by proclivity, and by delight, we all quote. Neurological study has lately shown that memory, imagination, and consciousness itself is stitched, quilted, pastiched. If we cut-and-paste our selves, might we not forgive it of our artworks?

The knowledgeable reader will notice that Lethem’s title is derived from Harold Bloom’s classic The Anxiety of Influence, as mine is from his. Both references would seem like homages to those who know, and like original (and creditworthy) turns of phrase to those who don’t. That’s how it goes.

This essay was written a couple of years before Fei-Fei Li released the ImageNet dataset to the world, unleashing the storm that would become Deep Learning. It was an era when we were just getting used to the idea that GPUs could do more than graphics. Before Jensen Huang released the Kraken.

Lethem’s article was formative for me. It was published just as I was launching my old blog Ribbonfarm, and as I was developing what would become my big project at Xerox — a mix-and-mash SemWeb-inspired approach to online content assembled on the fly out of fragments of online content. This would come to life a couple of years as a product called Trailmeme, which you can think of as a publishing medium for nonlinear meta-texts, assembled out of existing ones. It was abandoned a few years after, but I learned a lot about publishing and textual cultures building it, and it shaped a lot of my opinions. Lethem’s essay helped shaped both my writing and my technology development work. I consciously thought of all of Ribbonfarm as a sort of ongoing plagiarism in Lethem’s sense.

Lethem’s argument, my old project, and my weird and scabby attitude towards writing and authorship properly belong in the era of mature LLMs that would not emerge for another 15 years. The idea of the entire world as churning soup of liquid text (a phrase that is not original to me, which I picked up from somewhere around that time), is a very natural one. For much of history, from the oldest epics to early modern pre-copyright eras, the conceit of individual, sovereign authorship that we treat as sacred was not just laughable, but a liability. Historically, the general practice was to attribute an idea one wanted to market effectively to some famous, legendary, or even mythical figure. Or manufacture a line of transmission to lend it gravitas. When making copies is expensive, “copyright” is an expensive taste to indulge.

Now, why is all this relevant? It is relevant because a deeply disingenuous literary/artistic reaction is taking shape in response to the rise of LLMs — casting the very creation of LLMs as constituting “theft” of some sort, while at the same time constructing human use of inherited traditions and broader textual cultures as somehow not just above reproach, but in some way constituting an honoring of a sacred tradition. When a writer rips off another writer without acknowledgment or even consciousness, it is homage within a tradition. When a programmer digests the same text into the weights of an LLM, a crime has been committed.

This is not so much a double standard as a sense of sacred/profane distinctions born of a sense of ownership of language itself on the part of those who self-consciously try to make art with it, or cultivate tastes concerning it.

Now, I do have more profound, existential concerns around LLMs that I’ll share in a minute, but it feels important to firmly reject this sort of bad-faith (or what is perhaps worse, unthinking) posturing. There’s an entire theater taking shape around it, including pretentious badges proclaiming works to be done without LLM aid, moralizing screeds, lamentations about a “semantic apocalypse” and so on.

There are elements worth taking seriously here. One for example, is the dangers of an oligopoly of a handful of companies or nations provisioning all the LLMs in the world. But this problem has nothing to do with authors’ rights or “fairness.” Nor does it have anything to do with entrepreneurs getting vastly richer off words aggregated by the exabyte than any writer ever will, producing words by the mere kilobyte. Resentment by itself does not constitute either a principled moral argument or a viable politics.

The danger has to do with risks of capture and centralization of the means of knowledge production. Of a few people shaping the tools that are going to start overwhelmingly mediating our relationships with textual culture.

The problem is not that OpenAI and others are scraping vast quantities of online information without compensating the authors. Or that Sam Altman is getting rich off it. I guarantee you that all those authors did not share any of their income with people they plagiarized, drew “inspiration” from, and so on, in the broader Lethem sense. I have zero sympathy for this particular kind of lament.

Yes, a couple of million words of my own writings are now in the bellies of all the LLMs, including writings I make money from. In fact, when an LLM does not know about something I’ve written (because it’s paywalled, or only in an offline book for example), I find myself getting annoyed about it. If it were easy, I’d simply submit everything I write directly to some sort of LLM-accessible portal. It would be nice to make money off this (as should soon be possible if the MCP architecture takes off), but even without that, not only do I think it’s entirely fair for my writings to be scraped and sucked into LLMs, I think I personally benefit maximally when that happens. The selfish value to me of LLMs knowing everything I write is going to be higher than any money I can make off it directly. LLM digestibility optimization is the new search engine optimization. Soon, most people will learn of me and my writing from being informed about it by their LLMs, not through personal referrals or Google.

But beyond rejection of small-minded concerns, let’s look at the upsides of full-blown embrace.

Let me share an example of the sort of power that gets unleashed when you take a deeply liberal attitude towards LLMs. Last week, in Protocolized, I published a story called The Signal Under Innsmouth.

The Signal Under Innsmouth

This has been my best attempt at AI-assisted fiction writing to date, and also one of the easiest. I simply fed the original story (The Shadow Over Innsmouth, a cult classic which Lovecraft wrote in 1937, so it’s in the public domain — not that I’d have hesitated if it hadn’t) to ChatGPT and asked it to transpose it to a transhumanist key, creating a kind of uncanny transhumanist horror instead of a cosmic horror. Then, with surprisingly few nudges and tweaks, I got the story linked above.

This is probably a better story than any fiction I have written unaided. It is really good, if I do say so myself. And I can without risk of appearing vain or self-congratulatory because I did very little to midwife it into existence. All it took was the original story, and a processing through humanity’s collective understanding of Lovecraftian writing, modern digital technology, and ideas about transhumanism, as captured in ChatGPT’s 4o model weights. Generating this story does not make me even an ersatz storyteller approaching Lovecraft calibre, anymore than generating ghiblified art makes you as talented as Miyazaki. But I still am glad to be able to produce and read it!

I liked this story so much, I re-read it myself several times. Everyone who shared feedback with me has loved it. I’m convinced the world is strictly better for it existing, and nobody has been hurt by its production.

Is it “a plagiarism” in Jonathan Lethem’s sense?

Yes, and?

To me, “LLMing” a piece of writing belongs categorically with behaviors like dictionary-use, Googling for background research, or using a spellchecker.

Or heck, reading.

If you use language at all, you belong in what we might call the Lethem Sea (in the spirit of the idea of the Dirac Sea in physics). Your “original” thoughts, ideas, and creations, in a very deep sense, aren’t, in the sense that copyright-obsessed industrial modernity understands originality, attribution, provenance, and credit. They are original in the sense that you bring something of your individual lived uniqueness to how you transform what you suck up from the Lethem sea, and regurgitate into it. You’re “original” in the sense a drop of water thrown up waves on the surface of the sea is “original.”

I’m belaboring this point because I think the standard “artists rights” criticisms of LLMs, the tedious defenses of Miyazaki and Studio Ghibli (anyone ever taken an inventory of all the subtle tricks and ideas he picked up from artists who came before him, ranging from Walt Disney to Herge?) are deeply wrong. Not just factually wrong about the ways the phenomenology of human text works, but morally wrong about the ethics of language and art use, and economically illiterate in their understanding of how the economy of words works.

This does not mean you cannot be compassionate about the plight of writers and artists, or agree that they should be compensated for their efforts that others derive value from. But such compassion needs to be properly shaped by an understanding of their marginal contribution, the conditions under which they access their own inputs, and so on. Taking their work at their own subjective valuation is as stupid and immoral as treating the price discovered for it by an imperfect market as sacrosanct.

Okay, that screed aside, let’s talk about the actual dangers of LLMs.

Enthrallment and Ecstasy

Someone commented, in response to one of my many Substack notes about AI (which I cannot now find), that I seem to be getting into a personal relationship with AI. The sort of thing that I turned into a Lovecraftian horror premise in The Signal Under Innsmouth.

This is correct.

It is not possible to use LLMs in particular very effectively if you don’t consciously anthropomorphize them and get into a sort of personal relationship with them. But the relationship is more than a procedural contrivance. The anthropomorphizing has to have philosophical depth to it that goes beyond acting or role-playing. You need to actually relate to it, the way you might to a student, an employee, or a servant. While remaining aware of the ways it is not, thanks to technical limits (especially around memory). This actually matters — LLMs have been shown to respond to the tone of interactions as a human would. This doesn’t mean they have subjective experience, but it alters your objective and subjective experience of the relationship.

Mine are, you might notice, superior-inferior frames, but that is merely a function of my age and the sort of person I am. I naturally tend to take charge and boss everybody around if nobody else is stepping up to do so. And LLMs, as of today, are not quite as good at taking charge as humans.

That doesn’t stop many people — younger, or with different personalities — from instinctively gravitating to the inferior position in the relationship. Or a peer-to-peer relationship. If you ask ChatGPT to describe a relationship, as I often do when auto-documenting a session’s protocol, it will often describe it as p2p when I think it was superior-inferior, but I let it be.

But there’s an important point here. With almost any other technology, my general attitude is “good servant, bad master.” Guns, cars, hammers, wheels, fire, PCs before the internet — almost everything humans have ever invented should be handled with the “good servant, bad master” heuristic.

But not AI. Especially not LLMs. These digital egregores conjured up and hallucinated by our prompting invocations can, and should, inhabit the full range of relational modes, from superior through peer to inferior. This is obvious if you stop for a second to think about it. Quite obviously, once LLMs replace most human teachers for young children, in forms resembling the Young Lady’s Illustrated Primer in Diamond Age, there is no other option than for them to adopt a benevolent but superior position.

Despite all the panicked efforts of the “alignment” theologians, this is not in itself a problem. The gradient of a relationship is best constructed as a function of relative knowledge, responsibility, and agency in a particular situation. That humans have historically constructed gradients as a function of status dynamics and tradition is merely a kind of computational hack born of a lack of sufficiently rich social organizing technologies, not logical necessity.

No, the problem with AI is not the gradient of the relationship, but the vector of engagement.

What do I mean by that?

If you and I stare directly into each other’s eyes, the resulting vector of engagement can only be along the straight line between us, pointing up the direction of the power gradient (which includes skill at the staring contest of course). But when you and I look together not just at each other, but at a third object of attention in our shared field of view, the vector of engagement can have a component that’s net orthogonal to the line between us. If we’re peers, that orthogonal component will be the net engagement vector, pointing from our intersubjectivity to the object of mutual interest.

This, I think, is the healthy way to look with/at/through an AI. Not eye-to-eye along a disturbingly intimate and degenerate engagement vector, but obliquely. Like panelists on a stage partly facing each other, partly the audience.

Why is this a good idea?

Well, because it shapes how AI transforms us at the deepest level, at the level of our being-and-becoming. I wrote about this in another (AI-assisted) essay last week, autoamputation flow:

The tldr of this article is that if you stare directly at an AI for too long, your head will get amputated, and the LLM void will swallow you hole. But if you look with an AI at something else, while also looking at it obliquely, you’re set for a generative relationship of growth.

It’s like how looking directly at the sun is a bad idea, but looking at sunlit things is great.

As I note in that article, this relationship model is a function of the fact that when you talk to an LLM, you’re effectively talking to all of humanity’s past thoughts digested into a mathematical construct, and turned into a mirror that sort of reflects a version of you back at yourself.

This whole argument should sound very familiar. It is the same argument I’ve been making for years about social media, for instance in my 2018 writings on Waldenponding and the “global social computer in the cloud.”

This argument is not a rhyming or analogical argument. It’s an extension of the same argument. You should get into a disciplined relationship with LLMs for exactly the same reasons you should participate in social media in a disciplined way.

It’s the same damn hive mind, the same social hindbrain, that first emerged on places like Twitter and Reddit (and if we’re being properly general — in libraries and university campuses and bureaucracies pre-computing). Except it now includes all the dead people, and allows all their thinking to combine in richer, more potent ways. Perhaps even in a way that gets past something analogous to a nuclear pile reaching criticality.

This brings us to an interesting thesis: LLMs at least (not all AI) are an extension of social media. I wrote a note about this that I will reproduce here in somewhat cleaned-up form. I’m not the only one thinking along these lines. See for instance Henry Farrell and others arguing last month that AIs are “social and cultural technologies” (though their line of argument is different).

LLMs are Social Media++

Here’s the argument as an outline sketch. I need to develop it more carefully, but in essence: As currently productized, LLM-centric, chat-based AI is really “social media++”

Social media is to chat or even copilot-based AI as pre-calculus is to calculus. Consider the shared affordances:

• Parasocial relationship to sample points of an aggregated hivemind

• Emergent madness/wisdom of the crowds in response to your prompts

• “Moderation” and “filtering” of outlier minds under fancy term “alignment”

• “Many eyes make bugs shallow” effects

• Cunningham’s law effects (post wrong answer to provoke right one)

• Simping behaviors

• Reply Guy behaviors

• Random deep nerds showing up around special topics

• Someone knows any language you can think of

If you’ve enjoyed a big following on any social media site for any length of time, this is all brain-dead obvious. For me, prompting an LLM and prompting is uncannily similar to prompting my followers on Twitter with a question or musing back in the day.

Social media, you might say, are weak, pre-criticality LLMs. Where atomized individuals haven’t been sintered into a cohesive mass of cognitive potential. Where instead of weights from training, you have weights on a social graph evolved through likes and reposts and an algorothm.

This close architectural similarity is a function of memory context boundaries being largely drawn in “human” ways within modern LLMs. Training protocols capture social graph structures and contingent historical knowledge contours implicitly, because language as a reality-mapping scheme is deeply saturated with the social context of text creation and evolution.

Ie LLMs are far more human than they need to be because of path dependency in the evolution of human language.

I don’t think LLM-centric AIs can be more “super” or “general” than collections of humans socially connected in grammatically plausible ways (eg there’s nothing inconceivable about my 5 closest friends being Mongolians for eg, though I don’t know any Mongolians in this life), but they can certainly get a lot weirder than they are now.

The two variables shaping how AIs behave are: a) context size at inference time, b) what goes in that context. I explored some of the technical ramifications of this in To Know is to Stage.

Shoving all available historical civilizational memory into a model during training is certainly significant. LLMs now span a far bigger space of thought than human history can explore or has explored, in a cheaply accessible way. But you’re still bound to what’s in that history so far, as encoded in language. Unlike chess, Go, classical physics for game engines, or protein folding, where synthetic data can extend far past contingent human historical experience, LLMs are more bound to the linguistically encoded data history of a contingent complex system trajectory. And we haven’t been where we haven’t been. Ours is a BIG collective experiential data set (~100b humans have ever been alive iirc) but still from a contingent system trajectory.

One metaphor. Imagine replacing every human who has ever lived in a simulation with someone 10x taller, and with superhearing. But otherwise they live exactly the lives they lived. Except now they saw/heard 1000x more and integrated it into personal memory and cultural memory contribution. They made a superhistorical memory. LLMs can sort of access that.

Is there room for more super/general behavior? Sort of. Presumably human lived historical experience has captured data to support generalized and universalized ahistorical theories that we haven’t yet figured out. AI could figure those out. Like if Kepler and Newton hadn’t happened but Tycho Brahe data existed, AI could plausibly figure out Newton’s and Kepler’s laws from the data.

But this is not really the AI being super or more general. It’s about the universe as seen through our historical data having untapped generalizability we haven’t accessed yet. And there are good reasons (chaos theory type reasons) that suggest there aren’t many undiscovered reserves of generalizability in our data. AI can go deeper where we’ve made a start (eg deep fold), but I’d be shocked if (for eg) AI discovered a high accuracy/precision “law of historical cycles.” Or a theory of time travel. The underlying phenomenology doesn’t support that level of useful generalization.

The only way to expand AI capability is to expand sensing and embodiment beyond human, and let it evolve its own language based on what it actually senses through that capability. For example, give it IR/UV vision and it will come up with names for colors we can’t see except as transposed false color. But it will remain a Rylean intelligence. Nothing in its mind that wasn’t first in its senses.

We have met the slop and it is us

As the name of my AI-assisted writings, Sloptraptions, suggests, I want to reclaim the term slop as a positive where it applies to AI generated content, but also direct the pejorative sense of the term where it should point. Towards individual living humans and our little individualistic vanities and hypocrisies.

If you experiment seriously with LLMs for even a few hours, being open to the possibility that they can not just be a partner in your creative process, but be legitimately creative in their own right, it becomes brain-dead obvious: the bad slop is in the writer, not the process or tool. This is one reason anthropomorphizing is not optional. Treat the LLM as a skilled graduate student or legal aide, and it will act like one. Treat it as a Frankenstein’s monster that’s an absolute horror that you must prove has no redeeming qualities, well yeah — it will act like one.

In a much deeper sense than with any other technologies, LLMs do really reflect your intentions and tendencies back at you.

So what is slop? The bad kind born of individual human conceits?

Slop is disingenuous intentions and bad-faith authorial processes, and you don’t need an LLM to produce it.

Slop is what you get when humans aren’t paying attention to the right things in the right ways. When humans aren’t caring right.

For example, everything I’ve read about the book Abundance, by Klein and Thompson, suggests to me that it is slop in this sense. Less an argument, more a permission structure designed to manufacture consent and consensus as a political project. But pretending to be a real argument and trying to envelop certain audiences in certain claustrophobically coercive narratives.

So, in what is admittedly a bit of a rhetorical stunt, I had ChatGPT write me a polemical opinion on whether or not I should read it.

I think, as we barrel into the LLM-mediated era of language, it is truly important to be able to tell good and bad slop apart, and locate it.

I’ve had quite a bit of interesting subscription churn for this newsletter over the last month or so, as I’ve gone all out with AI-assisted writing. It’s up on net, but the volatility has been amusing to watch. I am assuming the unsubscribers mostly have a problem with AI, and the new subscribers are highly AI-positive.

At least one reader (I know him personally) put in the note that they can’t in good conscience support AI use, even when segregated like mine (he made it a point to call that segregation out and appreciate it, as several others have). I’m fine with this and respect such choices. If you don’t want to consume or support AI-assisted writing, all the best to you.

To be clear though, I’m only segregating my writing into assisted and unassisted sections of this newsletter to make it easier to track my learning and evolution. Philosophically I’m a full-blown desegregationist here. I fully expect to merge the two sections back soon. And unlike many people, I don’t think it is necessary to declare that an article is authored with AI help. That should be a literacy readers pick up, to the extent it matters and they care. I for one, don’t. If an article is good, I’ll read, irrespective of any cues I pick about authorship. Sure, I may read slightly differently if I sense it is AI-assisted or entirely AI-generated, but I won’t dismiss it for that reason.

I’ll dismiss it if I sense that it is slop in my pejorative sense — suffering from a deficit of human mindful attention and caring in the creation.

Pretty soon, it will be the safe default assumption, kinda like you assume everything you read written by living authors is probably spell-checked and Googling-supported.

AI is an all-or-nothing type technology, and it is already obvious to me it’s going to be part of all the writing I do for the rest of my life. All my unfinished projects are either going to get abandoned, or get completed with LLM help.

This makes it all the more important to get attuned to the real slop and call it out wherever you see it. The slop of disingenuous intentions, authorial bad faith, and where LLMs are used, a disrespectful relationship with both the tools of creativity, and the cultural heritage of language. The slop of mindlessness and inattentiveness. The slop of not caring. And in my experience so far, this is universally caused by humans.

The alternative to human slop is getting deeply, uncannily involved with LLMs, perhaps even embracing the Lovecraftian horror of a kind of transhuman miscegnation. Because that is also how you get more deeply involved with the history of human thought than has ever been possible.

It’s how you leave behind the anxiety of influence, blow right past Lethem’s ordinary ecstasy of influence, and get to a deep ecstasy of influence.

And if in the process, you develop a disturbing “Innsmouth look,” breeding with the Deep Ones lurking in the latent depths of the Lethem Sea, so be it.

Cthulhu fhtagn! and damn the shoggoths!