Training Data in Plain Sight

Facebook introduced the Like button in 2009. By 2010, we’d clicked it 100 billion times. Nobody asked what we were building.

Twenty years of posts, photos, likes, reactions, and narrated ordinary life accumulated quietly in the background. We thought of it as staying connected, maybe as memory. However, what people posted for friends or a small community is now treated as input for optimization and training. The traces that once functioned as memory return with a different use and a different value.

A feed refreshes without a stopping point. Autoplay keeps you watching. “Suggested” posts appear before you ask. A notification pulls you back at the right moment. All of it translates to ordinary life being rendered as countable events. Platforms trained us to produce behavior in forms that could be recorded and compared long before machines could read those patterns at scale. It emerged through incentives across capital, computation, and interface.

I’m calling this a legibility stack: the way platforms turn messy life into machine-readable layers.

I’m calling this a legibility stack: the way platforms turn messy life into machine-readable layers. Behaviour becomes measurable, then standardized through a limited set of actions, then retainable through default storage and identity stitching (systems that link your activity across devices, sessions, and contexts into a single continuous profile). Over time, it becomes learnable and repurposable. The term is not meant to rename existing critiques, but to give designers a causal vocabulary for how interface choices become reusable infrastructure over time.

Later sections track two layers of the stack: feeds that sense attention, and interfaces that recruit users to label meaning.

I’m not trying to solve that problem all at once. I want to trace the chain of cause and effect — from interface design to measurement, from measurement to supervision, from supervision to repurposing across time. The question that follows isn’t only about privacy. It’s about time. What happens when a past self remains readable and reusable, long after the moment that produced it has passed?

This is a story about how the future was quietly trained by us, with us, often without our knowing.

The Known Horizon: Cognition and Prediction as a Technical Goal

AI didn’t appear suddenly. The idea of machine cognition runs from Dartmouth’s claim that intelligence could be simulated, to Turing’s pragmatic test, to Wiener’s link between prediction and control. The ambition for thinking machines was clear, but it arrived late because the behavioural archive and the infrastructure to store and process it at scale had to be built first.

For much of the twentieth century, computational “cognition” was hard to define and test, so prediction became a practical stand-in. Prediction is measurable, and what’s measurable can be optimized. Once you can predict, you can govern: decide what gets surfaced and what gets suppressed. Prediction is not just analysis. It is a way to shape behaviour.

Early systems didn’t have continuous traces, cheap storage, or the ability to compare behaviour across millions of people in real time.

The constraint was not imagination. It was infrastructure. Prediction needs patterns. Patterns need repetition. Repetition needs data. Early systems didn’t have continuous traces, cheap storage, or the ability to compare behaviour across millions of people in real time.

The network boom of the early 2000s changed that. Always-on networks made online behaviour continuous. Online activity shifted from occasional actions to a steady stream of micro-events: clicks, watch-time, scroll depth, saves, returns, searches, and location trails. Storage became cheap, and retention became the default. Platforms started to keep everything. Mayer-Schönberger and Cukier call this ‘option value’: data can have second lives, becoming useful in ways not imaginable when first gathered. What was captured to run a service could later be reused for training, inference, and licensing. Behaviour became telemetry, and a continuous record of online behaviour took shape.

Vast datasets of online behaviour started to train prediction, and prediction started to govern attention.

By the 2010s, prediction was no longer a back-end technique; it was the interface. Vast datasets of online behaviour started to train prediction, and prediction started to govern attention. Feeds ranked before you asked. Recommendations arrived before preference was articulated. Platforms recorded behaviour and shaped it through continuous monitoring and tuning. Zuboff describes this as the rise of “behavioral futures,” where present traces are used to infer what someone might do next. Forecasting becomes a control surface.

Seen this way, generative AI is not the start of the story. It is the moment the story becomes obvious. What platforms spent two decades building was not “intelligence,” but the conditions intelligence requires: continuous traces, cheap retention, and feedback loops that make behaviour comparable at scale. The web, along with producing content, also produced a behavioural recording infrastructure. AI didn’t force this future into being. It arrived to find it already prepared.

Designing Disclosure: How Platforms Incentivised Legible Lives

If prediction needs patterns, platforms have to make patterns easy to produce. That is where interface design enters. The archive grows and gets shaped for comparability. People were never forced to share, but platforms consistently reward the kinds of sharing they could count, sort, and reuse. The result was more content in formats that could travel inside the system as data.

Standardization is the key design move. Platforms don’t need to understand you deeply in context if they can compress you into a menu of actions. Click. Watch. React. Follow. Save. Share. Comment. These are behavioural primitives. They turn social life into repeatable units.

The platform doesn’t have to interpret meaning. It can work with patterns.

Once you have primitives, you can make people comparable. Two users with different lives and reasons can still be compared if they both “save,” “skip,” “rewatch,” or “react.” The platform doesn’t have to interpret meaning. It can work with patterns.

With primitives in place, feedback loops emerge. The system ranks content based on what it can track, watches the response, updates the ranking, and repeats. The platform chooses what you see next. To do that, it captures behaviour in stable forms, far beyond what’s needed to run the service. That excess capture has been described as “behavioral surplus” and, more broadly, as datafication. Identity tightens the loop further. Stitched profiles make traces durable across time, so yesterday becomes the tuning signal for today, and today becomes the forecast for tomorrow.

Interfaces make behaviour measurable through trackable actions, comparable through standardized primitives, and retainable through identity and default storage.

This is the legibility stack in motion. Interfaces make behaviour measurable through trackable actions, comparable through standardized primitives, and retainable through identity and default storage. Once those layers exist, the system can tune itself on top of them.

So what took shape was more than a culture of sharing. It was a culture of self-legibility. People learned to describe themselves in formats the system could store and reuse. They learned what gets rewarded, what travels, and what gets ignored.

When the Machines Arrived: The Feed as Sensor, the Model as Reader

By the time generative AI arrived, platforms treated the feed like an instrument. Feeds had once been simple – chronological streams of what your network posted. But as the platforms progressed, the feed became a system that measures what happens next. To do this well, platforms needed signals and they turned the feed into a sensor array.

Today, every interaction is a measurement. A Like becomes a count. A Share traces affiliation. A Save signals intent that the system can read. Even “dwell time,” how long you hover, becomes a signal, a quiet proxy for interest that doesn’t require a click. Together, these gestures reduce unstructured social life into primitives that systems can store, compare, and optimize.

Together, these gestures reduce unstructured social life into primitives that systems can store, compare, and optimize.

Autoplay is the simplest example. It removes a decision point. Continuation becomes the default. That design choice keeps the sensor on. Now the system can measure attention as a stream, not a click. Watch-time becomes the key signal as it is continuous and comparable. Staying longer signals relevance. Skip fast, and the system reads a mismatch. Rewatch something? That’s intensity. Completion gets interpreted as satisfaction.

Once watch-time can be read this way, the system ranks content based on these proxies, watches the response, updates the ranking, and repeats. The consequences show up most clearly in short-form video, where completion rate becomes a quiet verdict. Creators don’t need to be told what the system wants. The analytics dashboard and the distribution curve teach them. If people swipe away in the first second, reach collapses. If viewers stay through the first loop, reach expands. Form follows measurement.

The feed isn’t a neutral infrastructure. It doesn’t just infer preferences. It shapes what gets surfaced, what gets rewarded, and what survives.

Measurements were being read long before generative AI. Recommendation engines were already trained on behaviour: YouTube’s systems predicted what you would watch and ranked content accordingly, and Netflix inferred preference to predict the next choice. The loop was the same: capture behaviour, learn from it, shape what comes next.

What changed in the 2020s is scale and generality. Foundation models did not invent this logic; they generalized it.

What changed in the 2020s is scale and generality. Foundation models did not invent this logic; they generalized it. What used to be platform optimization became cultural modelling. Transformers made it feasible to learn from vast corpora of human text and images by predicting what comes next. The limited use behavioral archives became priceable infrastructure to the large language models (LLMs). Reuters described an ‘underground race’ to secure training data. Reddit licensed to Google and OpenAI; the Financial Times to OpenAI. With enough captions, comments, threads, and Q&A, models learn recurring patterns of speech — how people argue, comfort, perform, and persuade — which is why the public web and corpora like Common Crawl became such a critical substrate.

The legibility stack flips here.

Telemetry alone is messy. The system also wants signals that reduce ambiguity about what behaviour meant. A recommender can run on correlation; a general model is judged on coherence, helpfulness, and plausibility, which creates demand for signals that anchor output back to intent. It quietly increases demand for labels, signals about meaning and not just behaviour.

Therefore, a second layer forms alongside measurement. The interface also invites people to explain themselves inside the platform’s categories

Therefore, a second layer forms alongside measurement. The interface also invites people to explain themselves inside the platform’s categories. Reactions, captions, tags, location pins, mood stickers, and feedback tools are not just expression. They act like labels.

These labels might not be the “truth.” People perform, joke, react ironically, hate-watch, or save and never return. The point isn’t precision per person. It’s that platforms prefer any structured signal to none, because noisy supervision still works at scale. The simpler story is that prediction became a practical requirement. Increasing legibility improved prediction. Once that infrastructure exists, it becomes useful for many future purposes, including model training.

So the incentive shifts. Not only more sensing, but more labelling. Not only clicks and watch-time, but judgements, explanations, and feedback. Platforms build interfaces that turn evaluation into participation. Participation becomes supervision. That is where annotation becomes the next layer.

Nicholas Thompson: “But if you don’t understand what’s happening, isn’t that an argument to not keep releasing new, more powerful models?”

Sam Altman: “We don’t understand what’s happening in your brain at a neuron-by-neuron level… and we can ask you to explain why you think something.”

This exchange captures the shift: systems move from sensing behaviour to extracting explanations.

Surveillance and Self-Annotation: From Being Measured to Being Recruited

Surveillance is usually imagined as something external: cameras, wiretaps, someone watching from above. Platform surveillance works differently. It begins with the feeling of being included instead of being watched, and then an invitation to explain yourself in the platform’s terms.

The feed already measures attention through countless micro-gestures. Beyond sensing only behaviour, platforms recruit users to label it. Annotation becomes a form of supervision, produced as a byproduct of participation. They neatly fold it into the interface.

Tools that feel harmless become mechanisms for self-labelling. Reactions compress emotion into a small set of classes. Tags and location pins turn context into metadata. Captions turn narration into structured description. “Not interested,” reporting, rating, and feedback prompts turn judgment into a button. None of this feels like labor. It feels like expression.

Measurement works best when it feels like care. The system runs better when you’re legible.

Surveillance feels like service. The platform “gets you,” recommends well, resurfaces memories, finds your people, completes your sentence. Measurement works best when it feels like care. The system runs better when you’re legible.

Not everyone accepts this bargain. Some learn to resist: posting ‘Stories’ that disappear from the view, using ‘Close Friends’ lists to segment audiences, and manipulating tags to avoid being read. Others practice what Brunton and Nissenbaum call obfuscation — flooding the system with noise to make their profiles less interpretable. These tactics rarely scale, and platforms continuously patch the loopholes. But they show that legibility is never total, and users are not entirely passive.

Still, most of us adapt. We learn what gets a response, what travels, what gets affirmed, and what gets buried. We learn the difference between a post that disappears and one that follows us for years. We learn the rules of a platform the way you learn the rules of a room, except the room is recording. For example: so much video now opens with a hook before context — ‘Wait for it,’ ‘You won’t believe this,’ even just a face reacting before you know what they’re reacting to. Intros shrink. Captions become mandatory because silent viewing still counts. Shots change early to prevent drop-off. Story becomes modular: a beat every few seconds to resist the scroll. Even ‘authenticity’ gets engineered. Spontaneity performs well, so creators learn to fake it. The feed edits culture into what can be measured.

The shift is from external surveillance to internalised legibility. Nobody has to force disclosure when the interface rewards it.

The shift is from external surveillance to internalised legibility. Nobody has to force disclosure when the interface rewards it. Nobody has to demand data when behaviour produces it. Surveillance becomes a style of living: narrating yourself in formats the system can store, through buttons, tags, categories, and prompts, until those formats begin to feel like default selfhood.

If self-annotation is the new form of surveillance, a temporal question follows. What happens when these labels, signals, and selves are retained and repurposed across years, long after the context that produced them has changed?

Self-annotation makes the archive interpretable. Retention makes it durable. Together they produce a new object: the self as a time-series of labeled signals, portable across contexts and reusable by default. That object outlives the moment it was meant to serve. Call it time collapse.

Time Collapse: Retroactive Extraction and the Consent Mismatch

If social media became AI’s playground, the real question is simple: who agreed to the rules of the game?

For two decades, platforms asked people to share in a social register: post for friends, comment for community, upload for memory, react for belonging. The implicit deal was to communicate with people in context. AI training breaks that frame. The recipient shifts from specific people to general models. The purpose shifts from communication to training. The power relationship inverts — from peer-to-peer to industrial extraction.

Helen Nissenbaum argues that privacy is about contextual integrity: whether information flows remain appropriate to the context they were produced in.

Helen Nissenbaum argues that privacy is about contextual integrity: whether information flows remain appropriate to the context they were produced in. AI training violates that integrity by shifting purpose, recipient, and power, usually without awareness or renewed consent. It takes expression that was situated and re-codes it into a generalizable pattern. It is not read by humans in context, but absorbed by systems that abstract, remix, and redeploy it elsewhere. This is not a small technical tweak. It is a shift in what disclosure means, intensifying the context collapse that social media already produced.

AI adds a second collapse: time. A post written years ago can become training material today. A late-night post about grief, written for a small audience in a specific voice, can end up inside a training corpus. A model does not remember the post as a story. It retains patterns of phrasing and association. Later, those patterns can resurface as tone or wording, detached from the person and moment they came from. What used to fade now stays reusable. This reusability creates a purpose drift as well.

Purpose drift is the governance problem that follows when availability gets mistaken for permission.

Purpose drift is the governance problem that follows when availability gets mistaken for permission. Something shared for one purpose gets reused for another, years later, without fresh permission. AI training is increasingly treated as an automatic next use for existing archives. But most people didn’t share their posts with that possibility in mind. The archive’s value comes from keeping future options open; its ethics depend on admitting that consent has a half-life.

Consent half-life does not mean people forget; it means permission loses validity as conditions change. Consent decays predictably when the audience shifts (friends to general systems), when the purpose shifts (communication to training), when capability shifts (new inference power), when time stretches beyond reasonable expectation, or when data is transferred (sold, licensed, merged). These are moments when renewal should be expected: points where reuse should require renegotiation rather than hiding inside boilerplate.

The U.S. FTC’s staff report on social media and video streaming highlights extensive data collection, retention practices, and weak user control. This makes it difficult for the users to understand the downstream usage of their information or to meaningfully govern it. People don’t just lack privacy at the moment. They can’t control what happens to the data their past selves created, or how it will be used to predict, influence, or limit their future selves.

That loss of control is not evenly distributed. Data collection and privacy harms don’t land evenly; low-income users and communities of color often have fewer practical ways to opt out, contest profiling, or limit downstream reuse. English-language content dominates training corpora, while public discourse from marginalized communities gets mined without compensation, even as corporate archives are negotiated and priced. Time collapse hits harder for those already over-policed: teenagers whose experimentation becomes a permanent record, activists whose organizing gets catalogued, immigrants whose documentation can be weaponized. Algorithmic systems encode existing hierarchies, making certain identities more surveilled, more categorized, more vulnerable to reuse. What gets called ‘the public web’ is already shaped by who has access, whose expression gets amplified, and whose labor gets valued. AI training inherits and compounds these asymmetries.

Time usually fades. Platforms don’t. They make the past searchable, portable, and profitable. They keep what we meant to outgrow.

Conclusion: Reading the Trace, Seeing the Horizon

What would it mean to build platforms that respect time through constraints? What if data expired by default? What if purpose changes required renegotiation? If you had a right to be unread later — not buried in settings, but enforced by design? Should the future be trained on terms no one ever agreed to?

This essay has argued that platforms did not merely capture our lives; they slowly trained our lives into legible form. Behaviour became measurable, then comparable, then retainable – until it became learnable and repurposable. The legibility stack is the curriculum we lived through without naming it.

What makes this visible is time. The long goals of these systems rarely appear at the moment of use. They arrive as convenience, connection, and service, and only later reveal themselves as infrastructure. Time collapse is that reveal: posts made for friends become patterns for models; context becomes training distribution; memory becomes material.

Not only passive capture. Alongside telemetry, platforms built an annotation layer. We didn’t just leave traces – we labelled them: likes, reactions, tags, captions, location pins, saves, “interested / not interested.” It felt like expression. It also made the archive interpretable, turning everyday life into cheap supervision.

Behind the interface sits a hidden workforce — moderators, labelers, raters — who do the interpretive labor that platforms cannot automate.

That supervision was never only self-administered. Behind the interface sits a hidden workforce — moderators, labelers, raters — who do the interpretive labor that platforms cannot automate.

They work in conditions designed for invisibility: outsourced, underpaid, often traumatized by what they must categorize. The ‘cheap’ in ‘cheap supervision’ is literal. It refers to wages, to disposability, to the deliberate erasure of labor from the story platforms tell about themselves. Legibility depends on this double extraction: users who annotate without knowing it, and workers who annotate without being seen.

Seen from here, the question is not whether sharing was naïve, or whether AI is a technical upgrade. It is whether rights can persist across shifting uses: a right to context, a right to time, and a right not to have self-annotation converted into permanent supervision. Because when the meaning of our interactions changes years later, consent cannot be treated as permanent.

Only hindsight makes the curriculum legible. The task now is to decide what should remain unread.