Technology incumbents have converged on a singular focus:

Using statistical models to spin mounds and mounds of text like fabric on a loom. Some of these applications can be useful. Others are… less so.

But this focus betrays a lack of imagination on the part of both established firms and investors. There’s so much more work that needs to be done in technology than just “AI,” and much of it is a consequence of the explosion in LLM-driven products.

As Frederik Pohl famously said:

“A good science fiction story should be able to predict not the automobile but the traffic jam.”

Similarly, a good investor or corporate strategist should be imagining the changing demands of an LLM-saturated world.

Engines of discovery

Since the initial burst of participation in the web, a central challenge faces both content publishers and web enthusiasts:

Human attention is finite, and the volume of web content far outstrips it.

The problem has only grown in scale, and this was before the rise of automated information generation. It’s going to get worse from here.

In the beginning, there was Yahoo: a team of human curators indexing and categorizing great content for the web. This was so valuable it made its founders rich. But quickly the web outgrew human-indexable scale. So the mantle of brokering finite attention passed to Google.

Google used signals from other humans as inputs to an algorithm that crawled, indexed and ranked web pages. Borrowing from academia, Larry Page’s insight was that the most valuable web pages would be the ones most cited, or linked to, from other pages.

Thus, googling became a byword for discovery: whatever you wanted to know was just a query away. Google was lauded for its minimalist presentation and incredible utility. If it was possible to find what you wanted, Google would give it you.

This was a brief, second golden age of the web. People built great web content and found themselves rewarded by the algorithm. It was possible to build entire businesses on this premise.

But quickly an industry emerged to game the system. While SEO allowed good actors to control their own fate, it also created opportunities for bad actors to poison search results altogether. Memorably, “Experts Exchange” polluted search results on every conceivable technical topic, paywalling the result. Mercifully, Stack Overflow emerged to bury these crooks and useful, free search results for technical problems returned to the land.

Still, it was a preview of the world ahead: content mills spinning endless crap, and an arms race between Google’s algorithm designers and SEO spammers.

Like Stack Overflow, Reddit emerged as a champion. Community-driven moderation distributed the load of controlling spam across a vast force of caretakers focused on every conceivable subject, from professions to technologies to hobbies to culture and breaking news. Between individual users reporting bad actors and mods setting and enforcing standards, Reddit became a place of healthy, useful content. Solving a problem on the web in recent years was usually accelerated by appending “reddit” to a Google search query.

Meanwhile, Google has poisoned its own search results. Pages that were once beloved for their clarity and simplicity are now contaminated by endless bullshit. Ads push real content further and further down the page. Dubious summaries shown at the search result level discourage viewers from clicking to publishers’ actual content.

And now Google is shoving poorly-conceived LLM features at every search.

Yet attention remains finite

In its quest to continue goosing its growth numbers, Google is strangling its utility and brand equity. The primary engine of web discovery is in decline, opening the door to new options.

For example, data shows that discovery engines like TikTok are eating into its mindshare. Younger folks would prefer to search TikTok for everything from basic information to local recommendations. It’s a sea change that was inevitable: A web that began with the constraints of 14.4kbps modems, favoring text-based exchanges, has transitioned to a web where 10 megabit broadband is not only abundant, but mobile.

The rise of video

Participation in the web once required knowledge of HTML and the vagaries of web hosting. The rise of Web 2.0 abstracted all of these things into simple signup flows and text input boxes.

But most of our interactions remained text-based. Status updates, blogging, comments—all of it was a world of text.

The inevitability of a ”pivot to video” was originally Facebook’s canard: they wanted a sea change that would favor them, and used their leverage on publishers for traffic to try and force it. If this sounds similar to their failed pivot to the metaverse, that’s because it is: these guys suck at timing the future.

The rise of video was not driven by traditional publishers, but by everyday people. As 2020 isolated us through the Covid 19 pandemic, every industry ground to a halt. Even culture. TV and film production shut down while the world figured out how to best contain this emerging crisis.

TikTok and other social media filled the breach. While 2020 was dark in lots of ways, it was also fertile ground for TikTok to transition out of its early adoption phase—driven as, often happens in social media, by teenagers—into a mass phenomenon. People of all ages and interests began watching and creating in earnest.

It was a great way to stay connected to humanity in an otherwise isolated period.

Why TikTok works

TikTok works because it carefully watches everything you do. It is an unparalleled engine of discovery, and one that Meta has failed to defeat. It is singularly effective, efficiently meting out attention to those who can earn it, catalyzing a cutting edge popular culture.

Creators agree that these signals inform how the algorithm constructs your interest graph:

• How much time you spend on a specific video
• Whether you interact with a video by sharing, liking, following the creator, or looking at their profile
• The songs on videos that you watch
• The hashtags on videos that you watch
• The phrases and hashtags that you search for
• Your general geographic location
• The interests and viewing habits of your friends and especially those you share content with

There are surely many, many more inputs to the algorithm, but that gives you a good idea for its ability to profile and understand both viewers and content.

The result of this is a “For You” feed that constantly surfaces fascinating, funny, engaging content that you want to see. You never know what you’re going to get.

TikTok, in short, has realized the dream of infinite channel flipping. Through a vast corps of content creators in every conceivable niche, it is able to provide you with interesting videos you’ll enjoy. Reliably.

Video as a counter to text spam

Because video contains layers of information content.

It can give you enormous detail about its creator. You can learn things about their identity, economic circumstances, where in the world they live.

By contrast to text which can obscure these things, video makes it easy to authenticate the source of the content. Viewers instantly, subconsciously analyze the credibility of the video they’re watching.

Is it over-produced and polished? Is it simple and home-spun? What’s in the background? Does it look like your home? Like an expensive home?

If the creator is talking about travel, are they actually in the location they’re describing? If they’re describing a recipe, does it actually look like the food would be nice to eat?

In other words, video is much more expensive to convincingly spam.

It’s possible that, given time, the same economics that allow LLMs to generate text spam will also come to video. But we’re not there yet. Synthesizing video from statistical models remains uncanny, especially if the cut lasts more than a few seconds.

This solves meaningful business problems, especially in an age where Google is in decline. An authentic entrepreneur with a great product can go far by making their case to the camera.

But there’s a problem:

All of that information content isn’t free. Hosting and transport of video remains ruinously expensive, which gives the platform owners like TikTok and Meta outsized control of what does and does not play. In some ways, this isn’t so different from the web of old: Google could determine your economic fate through its algorithm.

Still, the world is a complicated place. The interests of megascale technology businesses are often opposed to those of the masses.

Underground internet

The internet has long hosted an underground. Various neighborhoods of Usenet were open bazaars of pirated pornography and software.

Hotline planted the seed of the peer-to-peer sharing revolution, which evolved into Napster, Limewire and eventually BitTorrent.

The so-called Dark Web plays host to all kinds of commerce, from drug deals to certain kinds of freelance.

But as the everyday experience of the web has grown centralized under a handful of totalizing corporate interests, I suspect there will be greater appetite for a more accessible, usable underground internet. Corporate interests will mandate compliance with censorship, or even withdrawing from certain markets, as happened with PornHub’s withdrawing from states like Arkansas and Montana.

In a pressing and concrete example, multiple US states endure a theocratic assault on reproductive health and trans and queer identity. People living under these conditions have a legitimate reason to desire an internet that obscures their identity, circumvents government-mandated censorship, and connects them with resources like health care, mutual aid and simple community companionship.

The entangled rise of video and underground internet will be interesting to watch. It’s possible to imagine a future where P2P rises again, allowing something like an underground TikTok. Activists have reason to reach exactly those people most vulnerable, educating and encouraging their political action.

Which takes us to the last big trend.

Privacy software

There’s a porous border between ‘underground internet’ and privacy software.

This is well illustrated by Signal, which ships a reliable, secure, audited product for encrypted communications over the internet. It’s an enormous success story: a high quality product with mass adoption and no cost to the user.

It will be the first of many.

Last week Microsoft announced a technology that automatically screenshots your usage, scrapes the text into a SQLite database, and thus dramatically increases your computer’s privacy attack surface.

Part of what we get in an “AI” world is a vastly expanded capacity for surveillance, data mining and profiling individual behavior. It’s going to be rough stuff.

Even for above-board individuals and organizations, it opens the door to legal complications and threat actors who can access and manipulate the results of this digital dragnet.

An inevitable response? Software specifically designed to protect against this kind of behavior. It’s easy to imagine a future of virtual machines or even live operating systems on flash drives specifically designed to circumvent the sort of surveillance technology that Microsoft is so eagerly pioneering.

There’s just so much to do

The future is about so much more than “AI.” The traffic jams these cars will create are going to happen on already-contested paths. Here’s an incomplete survey of converging issues that intersect with this moment in technology:

• A rising tide of scams on every technical medium, from the legacy phone system to the web
• Scummy incumbents, willing to do anything to stay on top
• Information milling through LLMs, making text cheap to generate
• Political oppression of multiple groups:
  • Women
  • Trans people
  • Minorities
  • Immigrants
• Rising wealth inequality
• Limited competition, as incumbents stagnate and corporate concentration increases
• Repressive governments around the world testing the waters of internet regulation
• Heightened activism in response to all of these factors

There’s so much technological work to do to meet this moment. There’s so much more to invest in than just frantically stuffing AI into things.

TikTok may be banned in the US, leaving a void for a new engine of discovery to fill, at least for those users.

It will be interesting to see who figures this out first.