Anatomy of a Substack Page: It Knows Where You Stopped Reading, and It Has an Opinion About Who Wrote This

Yesterday Facebook. Today the platform this publication lives on: 170 scripts, 411 switches, one visit in ten recorded, my phone number in the source, and an AI flag on 27 of my last 32 posts.

Two days ago, I pulled apart my Facebook home page and found two posts inside 4.7 megabytes, with the word Sponsored misspelled on purpose so machines couldn’t find it. Fair is fair. This morning I did the same thing to the two pages Substack shows me when I log in: the PolitiMix home page at politimix.net, and the Substack home feed at substack.com, which is the same address for every user on the platform and shows each of us something different. Same method, a text editor and a few lines of Python, nothing a reader couldn’t do with Ctrl-U. Here is what the platform you’re reading this on does while you read it.

The ratio, first. The PolitiMix home page is 678,844 bytes, about one-seventh of Facebook’s, and the words you can read on it come to 5,526 characters, which is 0.8 percent. That is twenty times Facebook’s ratio, and it is still a page that is 99.2 percent something other than what it shows you. The something is mostly one object: a block of configuration data, 396,521 bytes, more than half the file, that the page hands to its own scripts before drawing anything. There are 170 script tags, 162 of them loading external code, 159 distinct JavaScript bundles from Substack’s content network, and three outside parties: Sentry, which reports errors; Cloudflare Insights, which measures load times; and Datadog, which I’ll come back to.

The home feed is a locked room. The second page, substack.com, is 79,774 bytes and contains 208 readable characters, the names of the menu tabs and a line that says the site requires JavaScript. The feed itself, the Notes, the recommendations, the posts from the writers I follow, none of it is in the file. It is fetched after the page loads, by scripts, from endpoints I’d have to intercept to read. Facebook ships its feed in the page, obfuscated. Substack ships an empty room and furnishes it after you walk in. For the purposes of this exercise that makes the Substack feed unexaminable by the method, and I’ll say that plainly rather than pretend the smaller number means less is happening. It means I can’t see it.

The switches. Yesterday’s Facebook page carried 1,108 experiment settings. Substack’s carries 411 site configuration flags, by name rather than by number, plus 25 named experiments with the arm I’ve been placed in. Reading them is a tour of the company’s product roadmap: enable_post_summarization, pub_tts_override, bloomberg_syndication_enabled, enable_livestream_linkedin_simulcast, generate_pdf_tax_report, publication_ban_sample_rate. The experiments say which version of Substack I’m getting: I’m in the treatment group for posts_ranking_v40 and the control group for publication_ranking_v56, which means the order of posts I see is the fortieth version of one algorithm and the order of publications is the fifty-sixth version of another, and someone is measuring whether I behave differently because of it. Seven of the 25 are marked as exposures, meaning the page I loaded actually used them. The feed’s first slot, feed_post_first_slot, is an experiment with a control group. Whatever appears at the top of your Substack feed, there is a version of you in another arm seeing something else there.

It knows how far you read. This is the one I didn’t expect. Every post object embedded in the page carries read_progress and max_read_progress, two numbers between zero and one, per post, per reader. On my own page one of them reads 0.42288686605981796. That is the exact scroll depth at which I stopped reading one of my own essays, recorded to seventeen decimal places and handed back to my browser so the interface can show a progress bar. There are companion fields for audio_progress, video_progress and max_video_progress, and a field called is_viewed, true on all 32. Every Substack reader’s position in every post is a stored value. The feature it powers is the little “continue reading” bar. The data it produces is a per-reader map of exactly where each piece lost you, and writers do not get to see it, but the platform does.

It had my phone number. The configuration block includes a user object with 63 fields. Among them, in plain text: my email address, my phone number, my Stripe customer identifier, whether I have a saved payment method (I do), whether I have two-factor authentication on (I do), whether I’ve linked Facebook (I haven’t), the dates I accepted the publisher agreement and installed the reader app, and an anonymous_id, which is the identifier the analytics use when they’re pretending not to know who you are, sitting six lines below my name. All of this is visible only to me, logged in, in my own browser, so this is a design choice rather than a leak. But it means the page you’re reading on Substack right now has your phone number in it, if you’ve given them one, and anyone who sits down at your unlocked laptop and presses Ctrl-U can read it.

Ten percent of visits are recorded. Substack runs Datadog’s real-user monitoring on every page, and the initialization code is in the source: sessionSampleRate: 1, meaning 100 percent of sessions are measured, and sessionReplaySampleRate: 10, meaning one visit in ten is recorded as a session replay, a reconstruction of what the user saw and did, scroll by scroll and click by click, that an engineer can play back like a video. If you’re inside a feature called the launch assistant, the rate is 100. This is standard practice at companies that run web products, Datadog sells it to thousands of them, and the replays are used to debug. It is also the case that one in ten of your visits to this page is, somewhere in a Datadog account, a movie.

The ad slots are empty, and that’s the news. The page has eight fields for tracking pixels: Facebook, Google Analytics, Google Tag Manager, Twitter, Parsely, and three more. On PolitiMix every one of them is null. Substack builds the slots and lets the publisher fill them; I never did. There are no ad objects in the file, no SponsoredData under any spelling, no encrypted tracking tokens, no right-column unit. The commercial content of this page is three Stripe plan objects: eight dollars a month, eighty a year, two hundred founding. Yesterday’s page carried 65 ad media attachments for two posts. Today’s carries zero for 32. Whatever else is in this file, nobody paid to be in it except the people who pay me, and the page knows exactly who they are because is_founding: true is one of the 63 fields.

And it has an opinion about who wrote this. Each of the 32 post objects carries a field named is_ai_generated_text. On 27 of my posts the value is true. On four it is null and on one it is false. Beside it sits is_ai_detection_disabled: false, and in my reader settings there is aiGeneratedTextMode: "show", a word that implies the existence of a mode that isn’t.

This is Substack’s AI detection, announced July 21 in partnership with a company called Pangram, which scans text published after that date and estimates how much of it a machine wrote. Substack says readers can run the scan on posts and notes, writers can dispute a result, and writers can add a “How I make this” note. What the file shows is that the scan has already run, on the server, on my archive, and the result is stored on each post as a boolean, and a reader-side setting exists that takes that boolean as input.

I’ll say what’s true, since the flag says it for me either way. I’ve told readers before that I use AI tools to finalize my pieces, and the process is mine: I pick the subject and the register, I direct the research, I cut what’s wrong, and the pieces are checked against sources before they go out, which is more than a boolean can know. Pangram claims a false positive rate of one in ten thousand; the Atlantic’s Matteo Wong has argued it “does make mistakes, perhaps to a greater extent than is currently understood.” I have no way to know which of my 27 flags are accurate by Pangram’s definition and which are the detector reading a style. What I know is that a platform I pay to publish on has sorted my work into a bucket without telling me, and built a switch for readers to act on the bucket, and the only reason I know is that I saved the page as a file.

What it adds up to. Substack’s page is a far better deal than Facebook’s. Eight-tenths of a percent readable against four-hundredths. No ads, no misspelled disclosures, no sixty sofas, no friends’ presence logs. Where Facebook’s weight is spent measuring the reader for advertisers, Substack’s is spent on an enormous configuration object and on telemetry that mostly serves features you can see: the progress bar, the error report, the experiment that put a post in the first slot. The honest ranking is that this is a much cleaner page run by a company that also records one visit in ten and quietly grades its writers’ prose.

The thing I’ll take away is the asymmetry. The page knows where every reader stopped reading every one of my pieces, to seventeen decimal places, and it knows which of my pieces it thinks a machine wrote, and I, the person whose name is on them, can see neither number through any door the platform offers. I can see them because I looked at the plumbing. Readers, you now know what the plumbing knows. Writers, check your flags.

---

Related: