Check 1, 2 is the mic on?
Check on you, is the vibe off?
– Thos Moser

Most AI doom scenarios are nonsense. The idea of a rogue computer behaving in a way no humans want never made sense to me, only to Hollywood who doesn’t know all that much about computers.

Paul Christiano has one of my favorite takes, that the creation of AI will look a lot less like a visit from an alien produced by natural selection (where you are most certainly dead), and a lot more like animal breeding.


That out of the way, what is going to happen? Let’s follow out a trend.

A lot of the story of the last 50 years is the story of advertising. And it’s crazier than you think, Instagram makes $39.7B US revenue off of 169M people, which is $234 per person, or $119 per person including those who don’t use Instagram.

Google’s revenue is $305.6B, 47% from the US, so $428 per person.

For each of the 334M US people, you could make a pie chart of both their spend and their time.


Curtis has a saying, which I’ll paraphrase. Power doesn’t stop at the voter, it just flows through the voter. The 2024 election had $15.9B spent on it, so $48/person, $98/voter, and way more if you only count only the people in swing states.

How much spend would it take to change your vote? Okay, that’s too much to consider. How about, how much spend would it take to get you to change dish soap? Go to a different coffee shop?

The marketplace for your behavior is ever increasing in scope.


Now let’s introduce AI. Whenever I think about AI, I think about analogies to people.

Currently, while ads are becoming more and more targeted, this isn’t anywhere close to its limit. Imagine a room of 10 CIA agents running at 1/1000th of real time (they have 16 hours for each minute of yours). And their only job is to influence you.

They aren’t stupid. They know not to bombard you with crap. They have studied your behavior for the last 10 years. They know exactly what it takes to make you actually change your mind. They know what you respond to. There’s no infighting among them, they are all perfectly aligned with the goal of altering your behavior.

They have studied marketing. They have studied psychology. They know more about you than you know about yourself. They will make sophisticated ad campaigns in real time only to ever be consumed by you. The endgame of targeted advertising.

The only reason you don’t have 10 agents doing this right now is that you aren’t worth it. But as the cost of these influence agents falls, this will be the future. This is what Meta and Google are building, even if they aren’t fully aware of that yet.


A team of long running extremely competent people with the sole goal of manipulating you.

You don’t stand a chance.