AI provenance will become the internet’s creator gatekeeper
AI provenance can verify a file’s history, but linking labels, ranking and identity may give platforms new power over who can reach an audience.
AI provenance is usually presented as a harmless way to tell audiences how something was made. In its mildest form, that is indeed what it is. A reader asks whether a post involved generative AI, the platform supplies an estimate, and the creator can explain the production process.
The danger begins when provenance stops providing context and starts deciding who gets recommended, trusted, paid, published or allowed to speak anonymously.
That transition is no longer hypothetical in the abstract. Substack has introduced reader-triggered AI scans through Pangram and is considering preferences that could influence what gets recommended. On August 2, 2026, the EU AI Act will make machine-readable marking and some visible disclosures legal obligations. At the same time, Western governments are building age-verification and digital-identity systems that could supply the missing identity layer.
These systems are currently separate. No law requires every creator to attach a government identity to every post. No major platform has yet announced that unsigned work will disappear from public view.
Yet the risk lies in how easily these pieces all fit together.
The central question is not whether every provenance tool is malicious. It is whether a signal designed to provide context becomes a reusable gatekeeping input. Once the same signal feeds recommendations, advertising eligibility, payments, moderation, client contracts or regulatory compliance, creators can face cumulative consequences that no single product team openly intended. Every layer can remain formally optional while the combined system makes refusal commercially expensive. That is how a disclosure mechanism can become a permission system without a single law or platform policy saying so directly.
More on AI provenance:
Substack has teased a first version of the filter
On July 21, 2026, Substack announced a Pangram-powered feature that lets readers scan eligible posts, Notes, comments and replies for estimated AI involvement. Substack’s support documentation limits the scanner to specified surfaces and content published on or after July 21, 2026.
Substack has introduced this first version with more restraint than many institutions would. Results appear only to the reader who requests them. Creators can add a “How I make this” statement, scan drafts, report suspected mistakes and disable detection on individual items. In its launch announcement, Substack acknowledges that Pangram can identify signs of AI involvement but cannot measure the amount of human care behind the work.
The concerns begin with what could come next. Substack says it is considering tools that would let readers set preferences about what gets recommended to them. It is also considering AI-content rules for individual communities. Those features do not exist yet, but the direction is clear. A detector result could eventually become an input to distribution: a decisive factor in which content gets seen.
Our recent examination of the feature found that Pangram may be one of the stronger AI detectors while still being unsuitable as proof of authorship. A probability score cannot reconstruct who conceived an argument, gathered the evidence, rejected bad suggestions, rewrote the structure or accepted responsibility for the finished work.
That distinction becomes crucial once a score affects more than one reader’s curiosity.
The control lever is distribution eligibility
The control lever is distribution eligibility.
A platform does not need to ban AI-assisted creators outright. It can create a sequence of softer disadvantages:
Add an AI estimate.
Invite readers to filter by the estimate.
Exclude flagged content from recommendations.
Make provenance a condition for monetization or advertising.
Require a process statement or appeal before restoring reach.
Treat repeated flags as an account-level trust issue.
Each individual step can be described as a user preference, quality measure or anti-spam precaution. Together, they determine which creators can reach audiences without passing through an approved production process.
This is how an informational label becomes a permission system without anyone formally announcing it as such.
Substack may never take every step on that list. Its leadership may sincerely want to protect readers from mass-produced filler, fake engagement and posts presented as personal reflection when no person did the reflecting. Those are reasonable concerns.
The problem is the precedent. Once platforms establish AI involvement as a special class of reputational evidence, every future product manager, advertiser, regulator and activist group has a ready-made signal to reuse.
AI use is being confused with inauthenticity
The central conceptual mistake is treating the use of a tool as a proxy for deception.
A reader can be deceived by content that was made entirely without generative AI:
A state-funded outlet may present government messaging as independent journalism.
An influencer may conceal a financial relationship with the product being praised.
A song may carry incomplete or incorrect performance and songwriting credits.
A video may use actors to perform a supposedly spontaneous event.
A politician may publish a speech written by an undisclosed team.
A celebrity may release a memoir produced by a ghostwriter.
A corporation may commission research designed to support a predetermined conclusion.
A publisher may attach a famous name to work produced by junior staff.
Some partial disclosure systems exist. YouTube may show a publisher-context panel for government-funded news organizations, although the panel is absent from search results and is not available universally. The US Federal Trade Commission says material relationships behind endorsements should be disclosed when they could affect how audiences weigh the recommendation. Spotify displays credits supplied by labels and distributors and directs artists back through those intermediaries when credits are missing or wrong.
What does not exist is a comparable, cross-platform reader setting that says:
“Exclude state-subsidized content, undisclosed commercial influence, staged authenticity, ghostwritten authority and unreliable creative credits from my recommendations.“
AI is easier to isolate because software can attempt to detect it and providers can attach machine-readable marks at generation time.
When detection gets this sophisticated, “slop” is no longer the target
Pangram’s technical report describes a transformer classifier trained with a method called hard negative mining with synthetic mirrors.
In simplified terms, the developers search large sets of human writing for passages their model mistakenly flags. They then create AI counterparts to those difficult examples and feed both back into training. Pangram reports that this reduces false positives and improves classification performance across multiple domains.
That is clever technical work. It also reveals what the detector is trying to do.
If an AI-assisted article is so crude, repetitive and empty that an ordinary reader can instantly identify it as filler, a sophisticated classifier adds little. Search engines, spam filters, reader behavior and basic editorial judgment can deal with obvious junk regardless of whether a human or model produced it.
Techniques like synthetic mirroring are needed, exactly because the quality of AI content can approach, equal or even exceeds that of “human“ content.
At that point, the system is no longer identifying material that readers recognize as “AI slop.” It is identifying content that may be coherent, useful and human-passable, then placing it in a separate class because an efficient tool was involved.
The practical question changes from “Is this low-quality spam?” to “Did this person make the work in the approved way?”
That goes well beyond the scope of mere quality control.
The detector cannot find the authentic creator
A detector can sometimes estimate whether a model left recognizable statistical traces. It cannot identify the authentic human contribution around those traces.
Consider two articles.
The first is generated from a one-sentence prompt and published without checking a single claim.
The second begins with an original thesis, interviews, source documents and years of subject knowledge. Its author uses AI to rearrange sections, remove repetition, test counterarguments, tighten a headline and produce metadata. The author verifies every factual claim and accepts full editorial responsibility.
A detector may place both in the same category.
Meanwhile, a fully human-written advertorial produced under pressure from a sponsor may receive a clean result. So may a ghostwritten political statement, a fabricated anecdote or an article whose central accusation has no evidence.
The machine detects a production pattern. It does not detect authenticity.
Research on AI labels also shows that the label itself can change how content is judged. One large survey experiment found that an AI label reduced the perceived accuracy of a policy article and reduced interest in its subject. A withdrawn preprint reported that AI disclosure hurt evaluations of emotionally expressive first-person poetry, although it did not produce the same effect across every kind of writing.
The label therefore does more than supply neutral information. In some contexts, it imposes a reputational discount.
Smaller publishers are being squeezed from both sides
AI assistance is already becoming difficult for serious independent publishers to avoid.
That does not mean handing a chatbot a topic and publishing whatever comes back. It means using AI for work that larger organizations divide among editors, researchers, production assistants, search specialists, copy editors and technical staff.
At Popular AI, AI is part of the editorial process. It can help examine structure, remove repetition, identify missing context, format metadata, test headlines and check whether an article meets the technical expectations placed on modern web publishing. A human editor controls the argument, verifies the evidence, rewrites the work and accepts responsibility for the result.
For a small, independent publication, hiring a full editorial and SEO team is often not realistic. The only alternative would be to leave this important work undone and publish uncompetitive content that consistently gets crushed by publishers who can absorb these extra costs.
For example, while Google does not outright require publishers to use AI, it does hint at its usefulness in getting your content picked up by search engines. Its official guidance says appropriate AI use is permitted and that useful, original, people-first content can rank regardless of how it was produced. Importantly, it also tells publishers to pay attention to accuracy, quality, relevance, title elements, meta descriptions, structured data and image alt text. It specifically describes generative AI as useful for research and adding structure to original material.
Those are reasonable expectations individually. Collectively, they create a large production burden and puts creators in a double bind.
The digital ecosystem effectively sends independent creators two messages:
Use every available efficiency tool if you want to meet the technical standards required for visibility online.
And:
Your use of the most effective efficiency tool may become a reason to reduce your visibility.
This does not resemble a system designed in good faith around output quality. It resembles a system that favors organizations wealthy enough to recreate AI’s efficiencies with salaried human labor.
The EU has made provenance a compliance category
The EU AI Act takes provenance beyond voluntary platform experimentation.
From August 2, 2026, Article 50 of the EU Artificial Intelligence Act requires providers of systems generating synthetic text, images, audio or video to make qualifying outputs machine-readable and detectable as artificially generated or manipulated, as far as technically feasible. Professional deployers must also disclose certain “deepfakes“ and some AI-generated or manipulated public-interest text. Human-reviewed text subject to editorial responsibility can qualify for an exemption. Standard editing and assistance that do not substantially alter the input or its meaning are also treated differently. The Commission’s transparency code describes the marking, detection and disclosure framework that supports compliance.
The full practical details are covered in our guide to the EU AI Act’s labeling rules for creators and publishers.
The law does not outright say that every unlabeled work is suspicious. It does not explicitly give AI detector scores legal authority. It does not impose an explicit duty on human artists and writers to prove that they avoided AI.
Yet the surrounding incentives may create such a system anyway, as explained in our article on how provenance systems may force human creators to prove their work.
The Commission’s compliance structure adds another asymmetry. Signing the voluntary Code of Practice gives providers and deployers an EU-recognized way to demonstrate compliance. Those using a different method must establish its adequacy individually before market-surveillance authorities. Large platforms can absorb that process more easily than independent developers and small publishers.
The predictable result is standardization around the methods accepted by large companies, standards bodies and regulators.
More on AI provenance:
Provenance proves a chain, not the truth
C2PA Content Credentials are often discussed as though they could solve the authenticity crisis. In reality, they can only solve a more specific problem.
A Content Credential can cryptographically bind assertions about a file’s creation and editing history to the file itself. It can show that the assertions have not been silently altered and that a credential was issued through a recognized signing system.
It cannot establish that the underlying content is accurate.
The C2PA explainer states this directly. Content Credentials do not make a value judgment about whether provenance information is true. Trust decisions depend on the signer, the included assertions and the trust lists recognized by the application. Certification authorities perform real-world checks before issuing some signing credentials.
This creates two separate questions:
Did this file come from the source named in the credential?
Is what that source says true?
Provenance can help answer the first. It cannot answer the second.
A government ministry can sign misleading propaganda perfectly. A corporation can authenticate a carefully selective press release. A news organization can prove that a photograph came from its own camera while publishing an inaccurate caption.
Conversely, a truthful recording from an anonymous whistleblower may lack every approved credential.
C2PA explicitly warns that content without credentials should not automatically be distrusted and says the standard is not intended to create a two-tier media ecosystem. That warning is admirable. It also identifies the exact misuse creators should expect platforms to be tempted by.
The larger convergence is difficult to ignore
Provenance is developing alongside a rapid Western push toward age assurance and digital identity.
These initiatives are attached to different laws, institutions and stated purposes, yet their convergence is striking because they all build compatible control mechanisms.
Australia has required age-restricted social platforms since December 10, 2025, to take reasonable steps to prevent users under 16 from maintaining accounts. The United Kingdom has required highly effective age assurance for access to pornography and certain content considered harmful to children since July 25, 2025. France passed an under-15 social-media restriction in July 2026 that depends on age verification.
The EU made its age-verification blueprint available in July 2025 and declared it feature-ready in April 2026. The system is intended to prove an age threshold without revealing other personal information. It can be adapted to thresholds such as 13 or older and is interoperable with the EU Digital Identity Wallets expected at the end of 2026. The Commission also plans lists of trusted age-verification solutions and providers.
Of course, proponents can come up with a myriad of semi-valid arguments in favor of such policies, but in reality, the operational effect they will have is difficult to get around. To know which users are minors, a service must classify users by age. To apply provenance-based trust, a service must classify content by origin. To establish a trusted origin, a system may rely on identities and credentials.
That does not automatically create a “driver’s license for the internet,” but it does create most of the components required to build one, and it does give us a glimpse of the real intent behind this push.
Provenance plus identity changes the stakes
A provenance system can remain privacy-preserving when creators control which assertions are disclosed and can publish pseudonymously. C2PA specifically allows identifying information to be omitted or redacted.
The danger appears when platforms prefer credentials from verified identities or approved certification authorities.
A future platform might accept an anonymous credential in theory while giving greater recommendation weight to credentials tied to:
A recognized publisher.
A government-issued wallet.
A professional association.
A verified employer.
An approved camera or editing application.
An advertiser-eligible business identity.
A certification authority on the platform’s trust list.
Unsigned and pseudonymous work would remain technically publishable while becoming practically invisible.
That outcome would be especially dangerous for dissidents, whistleblowers, controversial researchers and creators living under authoritarian regimes. The latter being a rapidly growing category. A credential that protects a photographer from impersonation in one country can help a hostile government attribute a politically inconvenient image in another.
Who benefits from a provenance permission layer
A broad provenance regime creates immediate advantages for actors that already possess money, credentials and institutional recognition.
▪ Large platforms gain another signal for ranking, advertising, moderation and liability management.
▪ Major publishers can maintain credential infrastructure, documented workflows, legal review and formal appeals teams.
▪ AI companies can present provider-controlled marks as proof that their systems support regulatory compliance.
▪ Compliance vendors and detector companies gain a permanent market built around certifying and investigating ordinary creative production.
▪ Governments and regulators gain a machine-readable information layer that can be inspected, audited and incorporated into enforcement.
▪ Established creative industries gain a way to characterize inexpensive production methods as less legitimate, protecting workflows based on access to capital, professional networks and paid labor.
None of these actors needs to conspire. Their incentives point in the same direction.
Who gets squeezed
The costs fall most heavily on people who use AI because they lack institutional resources.
▪ Independent publishers may rely on it for editing, research organization and technical production, while freelancers use it to compete with agencies that can divide the same work among larger teams.
▪ Artists may incorporate generative tools into a broader manual process, and disabled creators may depend on AI for transcription, rewriting, speech or interface assistance.
▪ Non-native speakers can use models to improve the presentation of their work, while anonymous creators may be unable to bind their output safely to a legal identity.
▪ Small developers face a different version of the same problem. They may lack the time, money or technical capacity to implement every approved provenance standard.
▪ Human artists can be caught in the system as well when their software, camera or distributor cannot generate credentials a platform recognizes.
That final group may provide the sharpest irony. Many artists demanding strict provenance appear to assume that the system will somehow automatically authenticate their work and expose competitors who use AI. Some will instead discover that they cannot produce a machine-readable chain that satisfies a client, platform or automated reviewer, while the AI artist, musician or writer can.
Their work will still be human. It will simply be uncertified and, in its own way, become suspicious.
The authenticity bureaucracy will not ask whether they painted the picture, performed the song or wrote the paragraph. It will ask whether the approved machine can detect the right signals.
The official case for provenance is not frivolous
There are legitimate reasons to develop provenance tools.
Mass-generated spam consumes time and attention. AI can create cheap impersonations, false evidence, automated scams and enormous volumes of disposable material. Readers may reasonably prefer a personal essay written from lived experience over one generated from a prompt. Clients may have contractual reasons to restrict AI. News organizations need methods for authenticating source material.
It’s easy to argue that readers, viewers, and listeners should be informed when a creator’s process changes the value of the work they purchase. But does that imply a right to inspect the process itself? Are consumers, platforms, and regulators really entitled to a museum tour of a creator’s private creative journey?
Even if we agree with that premise, the mistake remains that an AI label rarely answers meaningful questions.
A useful disclosure describes the human process:
What role did AI play?
Who originated the reporting or argument?
Who checked the facts?
Was the work edited and approved by a person?
Who accepts responsibility for errors?
Were any sources, quotations or experiences fabricated?
Is there a financial or institutional conflict?
“AI detected” answers none of these.
It merely tells the reader which tool may have touched the sentence.
Four realistic ways this could develop
1. AI preferences become recommendation exclusions
Substack’s first feature returns a private estimate to a reader. A future preference could hide or deprioritize content exceeding a chosen threshold.
The platform could then find that recommendation controls are useful for spam management, advertiser preferences or regulatory compliance. The personal filter quietly becomes a default ranking factor.
Creators would begin rewriting posts to satisfy a detector rather than readers. Formal, structured or highly edited prose could become a liability.
2. Credentialed media receive a trust advantage
Platforms could add a badge to content carrying recognized credentials. The badge might begin as optional context.
Later, credentialed material could gain access to news surfaces, monetization, advertising, political-content distribution or faster moderation appeals. Uncredentialed work would remain available but lose the benefits needed to reach an audience.
This is the two-tier outcome C2PA says its standard is not intended to create.
3. Creator credentials merge with identity and age systems
A platform already checking age may find it efficient to offer identity-backed creator verification through the same wallet infrastructure.
Creators could be offered better reach, payment access or impersonation protection in exchange for a verified credential. The offer would remain voluntary until operating without it became commercially unrealistic.
This sort of linkage easily becomes an attribution and approval system for political speech online.
4. Provenance becomes a business eligibility rule
Advertisers, insurers, payment companies, grant programs, employers and government procurement systems may begin requiring evidence of approved content-production processes.
A publisher could be asked to prove that qualifying images retain Content Credentials. An agency might require detector reports from freelancers. A music platform might demand standardized AI-use metadata from distributors. An insurer might offer better terms only to firms using approved creation tools.
Large organizations would treat such requirements as another compliance form. Smaller creators would lose contracts because their workflow does not produce the expected machine-readable evidence.
The petty stupidity of the authenticity obsession
Generative AI is built to imitate forms of human expression. As AI content creation systems improve, the measurable difference between competent human work and carefully supervised AI-assisted work becomes smaller.
Society is responding by building increasingly complex classifiers, watermarking systems, trust lists, credential authorities, disclosure codes and regulatory procedures to police that shrinking difference.
This would make more sense if the target were real harms.
Fraud, defamation, impersonation, forged evidence and undisclosed advertising can be defined by what the content does. Low-quality spam can be managed by quality, behavior and volume. Plagiarism can be investigated by comparing expression and sources. Attaching provenance signals to these types of content changes nothing to the harms they can cause.
Instead, the provenance obsession asks whether a creator obtained a quality result too efficiently.
When an advanced classifier is required to detect that a human-passable paragraph received AI assistance, punishing the paragraph in question does not protect audiences from “slop.“ It punishes technological proficiency and efficient content production.
That is a remarkable use of society’s time and technical talent.
What power users should expect
Power users should expect AI involvement to become a routine field in publishing systems, client contracts, platform metadata and creative-industry credits.
They should also expect inconsistent definitions.
One client may treat spelling correction as “AI assistance.“ Another may care only about generative drafting. A platform may identify AI-style language that came from a human editor. A provenance standard may record the use of a supported application while missing extensive work performed elsewhere.
The border between “human-made” and “AI-made” will not become clearer. It will, however, become more bureaucratic.
Creators should prepare for process disputes in which opaque detector scores are mistaken for proof without accepting the premise that a detector decides authorship.
More on disputing AI detection:
What creators can do now
Preserve ordinary evidence of your process
Keep outlines, source notes, original recordings, image layers, drafts, version histories and publication records.
Do this to protect yourself in disputes, not because an uncredentialed creator should be presumed guilty.
Publish a clear process statement
Explain which tools you use, what they do and where human responsibility begins.
A useful statement is more informative than a generic “AI-assisted” warning. It can say that AI helps with editing, structure or production while a named person controls the thesis, verifies sources and approves the final work.
Do not cripple good work to satisfy a detector
A detector is a classifier, not an editor. Replacing clear prose with awkward wording to obtain a preferred score makes your content worse and validates the detector’s authority.
Challenge the classification instead.
Keep distribution channels you control
Maintain an independent website, email list, subscriber export and local copies of published work.
A creator whose whole audience depends on one recommendation engine has little room to resist a new provenance rule. Or any other arbitrary changes platforms may make in the future, for that matter.
Separate identity from public attribution where possible
Use selective disclosure rather than attaching unnecessary personal information to public assets.
Inspect credentials before publication. Understand which identity assertions they include, who can read them and whether redaction survives the workflow.
Avoid a single provenance vendor
Keep original assets outside any one credential ecosystem. Use open formats and retain unmodified source files.
A creator should not lose the ability to prove a work’s history because one company closes, changes its terms or loses platform recognition.
Demand quality-based controls
Readers should be able to reduce spam, repetitive posts, automated engagement and mass-produced content regardless of whether it came from a person, a bot farm or an AI model.
Tool-based purity filters are a poor substitute for quality controls and result in feeds filled with “human slop.“
Keep using AI where it makes your work better
Creators should not abandon efficient tools to satisfy people whose preferred production methods happen to require more money, more staff or more time.
Use AI deliberately. Check its work. Preserve your own voice. Disclose material uses where readers may reasonably expect it. Accept responsibility for the result.
That is a stronger form of authenticity than a green detector badge.
More on anti-AI sentiment:
AI provenance should inform readers, not license creators
AI provenance can help establish where a file came from and how it changed. It can give audiences useful context and make some forms of impersonation harder.
It cannot determine whether an argument is honest, an artist is authentic or a publisher deserves trust.
The war over provenance is therefore becoming a war over who gets to define legitimate creation. If AI involvement becomes a reason to reduce reach while state funding, commercial influence, ghostwriting, staged authenticity and conventional manipulation remain outside the same filtering system, the policy is not defending authenticity consistently.
It is policing access to an efficient tool that has allowed independent creators to compete with institutions.
The danger is not the small label beside a post but the infrastructure behind the label. While provenance, detector scores, trusted signers, age assurance and digital identity can theoretically remain separate and voluntary, they are more likely to become a chain in which approved origin determines distribution and verified identity determines approved origin.
Merely being offered a choice is not enough when refusing a credential predictably reduces reach, income or access to appeals. A system can call itself voluntary while making the uncredentialed path unusable in practice. Creators should judge provenance infrastructure by the consequences attached to participation, not by the language used to market it.
Creators should use provenance when it serves them. They should resist any system that turns missing credentials into suspicion, AI assistance into disqualification or identity into the price of reaching an audience.
Authenticity belongs to the person who stands behind the work. It should not require a license from the machine.
Explore more from Popular AI:
Start here | Local AI | Fixes & guides | Builds & gear | Popular AI podcast
Should platforms judge content by its quality and accuracy, or by whether AI touched the creator’s workflow?