When “human-made” needs paperwork: how AI content labels may target human creators
What happens when human-made content is accused of being AI? Explore the risks of detectors, provenance systems and false authenticity claims.
The EU AI Act requires labels for some synthetic content. The next problem may be forcing human creators to prove that their work was not made by AI.
The debate over AI labels usually begins with an apparently simple question: How should people be told that an image, recording, video or article was created with artificial intelligence?
There is another question that has received much less attention. What happens when content without an AI label becomes suspicious?
Once provenance systems, invisible watermarks, AI detectors and official disclosure icons become commonplace, the absence of an AI label may no longer be treated as neutral. An unmarked photograph, illustration, song or article could instead be viewed as an unidentified object whose creator failed to provide the expected technical evidence.
An artist may insist that every brushstroke was theirs. A photographer may possess the original camera files, while a writer may have three days of drafts and a singer may have recorded every note. A platform, client, competition organizer, activist group or automated detector may still respond with the same demand:
“We think this was made with AI. Prove that it was not.”
The transparency obligations in Article 50 of the EU AI Act do not formally impose a general duty on human creators to prove that their work is human-made. Article 50 regulates providers and professional deployers of certain AI systems, covering matters such as direct interaction with AI, machine-readable marking of synthetic outputs, “deepfakes” and some AI-generated public-interest text.
That legal distinction matters, although it may offer little protection from what happens in practice. A regulatory system can avoid formally reversing the burden of proof while still creating institutions, incentives and technical tools that reverse it informally.
The law creates the question. Platforms, employers, clients, publishers, pressure groups and bureaucracies decide who must answer it.
More on the EU AI Act:
Key takeaways
The EU AI Act does not presume that content without provenance data is AI-generated. Human creators have no general Article 50 duty to certify that they avoided AI.
False accusations are inevitable. Text, image and audio detectors are probabilistic classifiers that can misidentify authentic work, especially after content has been edited, translated, compressed or removed from the detector’s test environment.
Provenance is asymmetric. Valid credentials may establish parts of a file’s history, but missing credentials establish very little. The C2PA standard itself warns against judging trustworthiness solely by the presence or absence of Content Credentials.
The practical burden may still shift. Platforms and commercial gatekeepers can demand source files, editing histories, recordings, drafts or other evidence before restoring a post, awarding a prize, paying an invoice or publishing disputed work.
False reports can be weaponized. Competitors, political opponents and online mobs can make AI accusations cheaply, while the accused creator bears the time, expense and privacy risks involved in answering them.
Human creators should begin preserving ordinary evidence of process. Original files, drafts, version histories, recordings, timestamps and clear contracts may become increasingly valuable, even though creators should never be presumed guilty because those records are unavailable.
The AI Act does not create a “prove you are human” rule
Article 50 is written around the operation and deployment of AI systems. As Popular AI’s detailed guide to the EU AI Act’s labeling requirements for creators and publishers explains, providers of systems that generate synthetic text, images, audio or video must ensure that qualifying outputs are marked in a machine-readable format and detectable as artificially generated or manipulated.
The obligation is qualified by technical feasibility and does not apply to the extent that a system performs an assistive function for standard editing or does not substantially alter the user’s input or its meaning. That distinction is important because many cameras, writing tools and editing applications contain automated features that do not turn the resulting work into synthetic content.
Professional deployers must also disclose certain “deepfakes” and certain AI-generated or manipulated text published to inform the public about matters of public interest. Evidently artistic, creative, satirical, fictional and similar works receive a more flexible disclosure regime, while public-interest text can fall within an exemption when it has undergone human review or editorial control and a person or organization accepts editorial responsibility.
More on the EU AI Act:
The full text of Article 50 does not say that every unmarked work is presumed to have used AI. It does not require creators who avoid AI to maintain certificates of non-use, give detector scores the status of legal facts or treat missing metadata as evidence that a disclosure was removed.
This concern is therefore not based on a claim that the legislation already contains a formal reverse burden. The danger is that the regulation may help construct an environment in which an informal burden develops through platforms, contracts and administrative procedures.
That environment is becoming more concrete. The European Commission published its final Code of Practice on Transparency of AI-Generated Content on June 10, 2026, with one section covering machine-readable marking and detection by providers and another covering deployer disclosures for “deepfakes” and qualifying public-interest text. The EU has also created a set of icons that deployers may use to label AI-generated content.
These measures may make disclosures clearer when labels are accurate. They may also teach audiences and institutions to expect a visible signal whenever the use of AI is suspected, turning an absence of information into a reason for further scrutiny.
The European Commission’s own announcement presents chatbot disclosure and the labeling of “deepfakes” and other AI-generated material as central parts of the new transparency framework.
A complaint can create an evidentiary burden without changing the law
The AI Act gives outsiders a route for challenging possible infringements. Under Article 85’s complaint mechanism, any natural or legal person with grounds to believe that the regulation has been infringed may submit a complaint to the relevant market-surveillance authority.
The authority must take the complaint into account when conducting market-surveillance activities and handle it under the applicable procedures. The complainant does not need to be the buyer, subject or direct victim of the disputed content.
That does not mean every allegation will lead to an investigation, and it certainly does not mean every complaint will produce a penalty. It does mean that Article 50 compliance can be challenged by outsiders whose motives may range from legitimate concern to commercial rivalry or political hostility.
Suppose an independent publication releases a controversial article without an AI disclosure. An ideological opponent complains that the article was generated by AI, that it never received meaningful human review and that the publication is avoiding a transparency obligation.
The publication may be asked to explain its workflow and identify the author or editor. It may need to produce drafts, research notes, revision records, correspondence or publishing logs, even though the initial allegation contained little more than suspicion.
The formal legal burden remains with the authority responsible for establishing an infringement. The practical evidentiary burden has still landed on the accused, who must spend time and money explaining how a piece of writing came into existence.
A larger publisher may absorb that demand through its legal and compliance teams. A small publication or independent creator may experience the same request as a serious disruption, particularly when the disputed work concerns a time-sensitive investigation, election, product launch or public controversy.
Provenance cannot prove what many people think it proves
The leading provenance standard is C2PA, the Coalition for Content Provenance and Authenticity. Its Content Credentials system can bind signed information about a file’s origin, editing history and other assertions to a digital asset.
A camera may certify that it captured an image, editing software may record that the file was cropped and a publisher may sign the final version. A generative system may also indicate that it created or altered an element.
The Content Authenticity Initiative describes Content Credentials as a way for creators to claim credit and disclose when generative tools formed part of the creative process.
These records can be useful because cryptographic signatures help establish that a credential came from a particular signer and that the credential or associated asset has not been changed since signing. They can strengthen a chain of custody and help a viewer understand what happened to a file.
Their limits are equally important. The C2PA Content Credentials explainer says that credentials do not make value judgments about whether the assertions within them are true. They help establish that provenance information is well formed, associated with the asset and free from later tampering.
A signed photograph can depict a staged scene, while a signed news video can carry a misleading caption. A signed article can contain false information, and a person with a valid signing credential can still lie.
Provenance can help verify where content came from and what recorded changes occurred. It cannot decide whether the scene, statement or interpretation represented by that content is factually accurate.
There is an even more important limitation for human creators. The absence of Content Credentials does not establish that AI was used.
C2PA explicitly says that adding provenance is optional and warns against creating a two-tier media environment in which assets without credentials are universally trusted less than those with them. Its guidance says no assumption should be made about an asset’s trustworthiness purely because it does or does not use Content Credentials.
The standard also acknowledges that provenance can be incomplete and that metadata can be removed. A file may be cropped in software that does not support Content Credentials, downloaded through a platform that strips metadata or converted into a format that no longer carries its original record.
Those warnings are technically sensible, but they may prove socially unrealistic once provenance indicators become common across newsrooms, stock-media services, government agencies and social platforms. As Popular AI has argued, provenance verification can become a gate when distribution systems begin expecting signatures, app attestations and signals from approved creative workflows.
People may quickly learn a crude three-part shortcut:
A verified origin badge means the content is probably genuine.
An AI label means the content is artificial.
No information means the content is suspicious.
The designers of a provenance standard may insist that the third conclusion is invalid. Interfaces, moderation systems and ordinary users may still reach it because a blank space is easier to interpret as missing proof than as a neutral absence of data.
More on AI provenance verification:
The absence of a label may become a label of its own
Every labeling system teaches audiences to interpret both the presence and the absence of a label. When certain advertisements carry warnings, consumers make assumptions about advertisements that do not. When some accounts display identity verification, anonymous accounts can appear less credible.
If genuine photographs increasingly carry camera-authentication credentials, unsigned photographs may begin to look exceptional. The result is an authentication premium for people who possess compatible equipment, supported software, identity credentials and recognized publishing infrastructure.
The corresponding absence penalty falls on creators who work outside those systems. That group includes many people for whom a complete digital chain of custody is impractical or dangerous:
Artists using traditional materials that produce only a final scan
Photographers with older cameras or unsupported editing tools
Writers working in plain-text editors or offline documents
Musicians recording through analog equipment
Small publishers using basic content-management systems
People who remove metadata for privacy or personal safety
Whistleblowers, anonymous sources and pseudonymous creators
Creators whose platforms strip credentials during compression
People using inexpensive or unsupported devices
Anyone who values anonymity more than institutional verification
A major newspaper can sign its photographs through a recognized corporate identity and preserve records across a controlled publishing system. A freelance witness using an inexpensive phone may have no equivalent capability.
A production studio can maintain an authenticated chain of editing records, while an independent illustrator may upload a compressed JPEG exported from an old application. Both works may be authentic, but only one arrives with institutional paperwork.
Human-rights organization WITNESS has described the risk of a digital divide between content with verifiable provenance and content without it. It warns that authentic material from journalists, witnesses and vulnerable people could be discredited because its creator lacks access to provenance tools or cannot safely use them.
An optional standard can therefore become effectively mandatory without any legislature declaring it mandatory. The transition happens through interface design, commercial expectations and the assumption that a responsible creator should be able to produce a recognized credential on demand.
This does not create a universal system of truth. It creates a system of credentialed and uncredentialed speakers, with the second group carrying a growing burden of explanation.
AI detectors classify patterns instead of reconstructing authorship
Provenance systems attempt to record history. AI detectors attempt to infer that history after the fact, which is a much less reliable undertaking.
A text detector may consider predictability, sentence structures, token distributions or stylistic regularities. An image detector may examine frequency patterns, compression artifacts, texture statistics or fingerprints associated with known generators. An audio detector may look for spectral traces associated with voice-synthesis systems.
None of these procedures reconstructs the actual creative process. They assign material to statistical categories based on patterns that correlate with the detector’s training data and decision threshold.
Every detector can therefore produce false positives and false negatives. A writer does not become an AI user because their prose falls on one side of a model’s decision boundary, and a photograph does not become synthetic because its compression pattern resembles images in a benchmark dataset.
The distinction matters whenever a detector score is treated as evidence of misconduct. The system may be identifying stylistic conformity, technical artifacts or unfamiliar data rather than the historical use of an AI tool.
A final document contains words, pixels or sound. It does not contain a perfect record of every thought, keyboard action, brush movement, edit and conversation that produced the finished work.
Text detectors can misidentify human writers
Detector vendors acknowledge that their systems can be wrong. Turnitin’s guidance for interpreting its AI Writing Report says false positives are possible and that scores below 20 percent are not displayed as exact percentages because the false-positive rate is higher in that range.
Turnitin also limits the types of material that its model is designed to assess. Its guidance says the system does not reliably detect AI-generated content in non-prose formats such as poetry, scripts, code, bullet points, tables and annotated bibliographies.
These limitations do not make the tool useless, but they do show that its result is an interpretation of qualifying text rather than a direct test of authorship. A highlighted passage is not a production log.
Turnitin’s own explanation of false positives is useful here because it demonstrates that the possibility of misclassification is acknowledged by the detector provider itself.
OpenAI reached a similar conclusion with its own detector. The company withdrew its AI-written text classifier in July 2023 because of its low accuracy.
In the published evaluation, the classifier identified 26 percent of AI-written challenge texts as likely AI-written and incorrectly labeled 9 percent of human-written texts. OpenAI also warned that the classifier was unreliable on short passages, performed worse outside English and could become confidently wrong on material unlike its training data.
Independent research has identified broader fairness concerns. A widely cited study found that several GPT detectors consistently misclassified writing by non-native English writers as AI-generated, while identifying native English writing more accurately.
Study co-author James Zou highlighted the finding when the peer-reviewed research was published.
The researchers warned that such systems may penalize writers whose linguistic expression is constrained. A writer using a restrained vocabulary, formal professional style or second language may therefore trigger greater suspicion than someone whose prose contains more unusual variation.
Another practical evaluation of AI-generated text detectors tested popular systems on unfamiliar models, subject areas, datasets and prompting methods. It found that moderate evasion efforts could significantly reduce detection and that some systems performed extremely poorly when required to maintain a low false-positive rate.
The combination is damaging. A detector may miss deliberately disguised AI writing while accusing a human writer whose style happens to resemble the detector’s target patterns. Popular AI’s investigation into Turnitin false positives and weak academic due process shows how quickly a probabilistic score can acquire the force of a verdict, even when the vendor warns that the result may be wrong.
The writer is then asked to prove a negative. Drafts and revision histories may support their explanation, but the final text alone cannot conclusively prove that no AI system touched any part of the workflow.
More on AI detector false positives:
Image detectors have the same structural weakness
Image detectors can appear more persuasive because AI-generated pictures sometimes contain visible mistakes. Strange fingers, impossible reflections and malformed text trained audiences to believe that synthetic images could be recognized through careful inspection.
That confidence becomes less justified as generators improve. It was never a dependable method for determining origin because human photographs and illustrations can also contain improbable details, optical distortions and technical artifacts.
Research has found that detector performance can depend on characteristics of benchmark datasets that have little to do with authorship. The study “Fake or JPEG?” identified biases related to JPEG compression and image size in generated-image detection datasets and showed that detectors learned from those unwanted factors.
A detector trained on poorly controlled data may learn that one file size looks “real” while another looks “synthetic.” It may identify the compression habits of the dataset rather than durable evidence of AI generation.
Those weaknesses become more serious after an image has been resized, screenshotted, recompressed, edited or passed through a social network. A system that performs well against familiar generators in a benchmark may fail when it encounters unfamiliar models or ordinary real-world transformations.
The opposite error also occurs. Genuine photographs can be labeled as synthetic or treated as suspicious because they resemble the visual language of AI-generated imagery.
Meta initially framed the challenge as making a blurry boundary between AI and human-made content easier for users to understand. Its announcement emphasized standardized labels based on technical indicators supplied by the industry.
Meta encountered a related problem when industry indicators caused photographs with minor AI-powered edits to receive a broad “Made with AI” label. The company’s updated approach to labeling AI-generated and manipulated media renamed the label “AI info” and later made it less prominent when content appeared to have been modified rather than fully generated.
Meta said that its earlier labels did not always match people’s expectations or provide enough context. A photograph retouched with an AI-powered feature could carry a technically defensible signal while communicating the much broader impression that the entire photograph was synthetic.
This is a central problem for AI labeling. A system can make a socially misleading statement from technically accurate metadata when the audience interprets “some AI-powered editing occurred” as “AI created this work.”
Human judges have also struggled to identify origin. In one reported case, an Australian photographer’s genuine image was rejected after competition organizers suspected that it had been generated by AI.
In another experiment, photographer Miles Astray entered a real photograph in an AI-image category and won a jury award before revealing that the image was authentic. One real photograph was rejected for looking synthetic, while another was rewarded for looking synthetic.
These incidents show that neither automated classifiers nor experienced viewers possess a dependable visual test for creative origin. Confidence in the judgment does not make the judgment accurate.
Audio and voice may be even harder to defend
A musician or voice actor accused of using a synthetic voice faces the same basic problem, complicated by the number of transformations applied to modern recordings.
Real-world audio passes through microphones, preamps, noise reduction, equalization, pitch correction, mastering, compression, streaming services and social-media transcoding. Each stage can alter the technical features on which a detector relies.
A study first submitted in 2025 and revised in July 2026 evaluated the robustness of ten audio-deepfake detectors under real-world corruption. It found that common audio modifications and compression could significantly reduce performance, with many models remaining particularly vulnerable to neural codecs and other forms of processing.
Clean benchmark performance does not guarantee dependable results in practical deployment. A detector that works on pristine laboratory samples may behave differently after a recording has been mastered, streamed, clipped for social media and downloaded by the reviewer.
The inverse accusation may be especially damaging for performers whose voices naturally sound processed, unusually precise or similar to common synthetic presets. A session singer could be denied payment because a detector classified the vocals as generated, even though the client has no direct evidence that a cloning system was used.
The singer may provide raw tracks, studio footage, rehearsal recordings and alternate takes. Those records strengthen the singer’s account, but a determined accuser can always retreat to a narrower allegation.
The accuser may claim that only some notes were replaced, that an AI enhancement tool was used during mastering or that the raw file was created after the dispute began. Once suspicion becomes the default, evidence does not necessarily end the argument. It merely changes the allegation.
Human review does not remove the reverse burden
Institutions often respond to detector errors by promising that a human being will make the final decision. That safeguard sounds reassuring until the reviewer’s starting information is considered.
A moderator may receive a detector score, several user reports, a missing provenance credential and an allegation that the creator previously used AI. The reviewer may also see a style that resembles popular AI imagery, an awkward hand, an unusual reflection, a familiar phrase or a suspicious vocal transition.
A contractual ban on AI may add another layer of uncertainty when the contract never defined whether spell-checking, automatic masking, denoising or other intelligent features count as prohibited assistance.
The reviewer is no longer approaching the disputed work neutrally. The accusation has already framed the question and directed attention toward details that might confirm it.
Instead of asking what evidence demonstrates that AI was used, the reviewer may ask whether the creator has provided enough evidence that AI was not used. That subtle change is the practical reverse burden.
Human review remains vulnerable to automation bias, social pressure and institutional risk aversion. A reviewer may overtrust a detector, misunderstand what provenance proves or prefer a false positive to the reputational risk of approving controversial synthetic content.
The reviewer may also dislike the creator or unconsciously apply different standards to different political, artistic or commercial viewpoints. A manual decision is not automatically an impartial decision.
Automation can scale an accusation. Human review can give it a bureaucratic stamp.
The liar’s dividend will reach ordinary creators
The term “liar’s dividend” usually describes the ability of dishonest people to dismiss authentic evidence as a “deepfake.” A politician confronted with a real recording can claim that the voice was cloned, while a public official shown genuine footage can say that it was generated.
The existence of convincing synthetic media makes denial sound more plausible. The same mechanism can be used against ordinary creators in commercial and cultural disputes.
A company can refuse to pay an illustrator by claiming that the work breached a no-AI clause. A publisher can terminate a freelancer after an article triggers a detector, and a competition organizer can remove an entry rather than defend it against an online campaign.
A political campaign can encourage supporters to report an inconvenient recording as synthetic. A rival artist can start an accusation against a successful competitor, while a platform can restrict distribution during an investigation that has no firm deadline.
In each case, the accusation is cheap and the defense is expensive. The creator may need to gather source files, restore old backups, contact collaborators, record a response, hire a lawyer or expose private details about their workflow.
The accuser may have done nothing more than paste a passage into a detector and publish a screenshot. The imbalance makes authenticity complaints an attractive tool for harassment.
The objective does not need to be a final regulatory penalty. Delay, uncertainty and reputational damage may be sufficient, particularly when the disputed work is connected to a book launch, exhibition, competition, album release or breaking investigation.
A creator who is vindicated months later may still lose the audience, revenue or opportunity attached to the original moment. A platform can restore a post, but it cannot recreate the exact period when that post mattered.
Trusted flaggers are not universal AI-authenticity judges
The Digital Services Act’s trusted-flagger system is often misunderstood. Trusted flaggers are designated organizations with expertise in identifying particular categories of illegal content, and platforms must prioritize qualifying notices from them.
The platform retains responsibility for deciding whether the reported content is illegal. Trusted flaggers are also expected to act accurately, objectively and diligently, with mechanisms available for suspending or revoking their status when their notices are repeatedly inadequate.
They are not general-purpose judges of whether a painting, photograph, article or recording is human-made. More importantly, an AI label does not independently determine whether content is illegal.
Recital 136 of the AI Act says that the requirement to label AI-generated content should not influence the assessment of whether specific content is illegal. That assessment must be made under the rules governing the legality of the content itself.
An absent AI label should therefore not transform an otherwise lawful photograph, painting or article into illegal content. The formal legal position is clearer than the practical environment surrounding platform enforcement.
The real risk arises from the overlap among several complaint and detection systems. Popular AI’s examination of the UK’s deepfake law and emerging detection infrastructure shows why safeguards such as false-positive transparency, independent oversight and fast appeals must be designed into authenticity systems before their scope expands.
AI Act complaints to market-surveillance authorities
DSA notices alleging illegal content
Platform rules covering synthetic or manipulated media
Contractual restrictions on AI-assisted work
Copyright and impersonation complaints
Advertising and consumer-protection rules
Coordinated community reports and flagging campaigns
A hostile complainant can select whichever route creates the greatest inconvenience. Even when a trusted flagger is not involved, ordinary user reports may trigger automated review, temporary restrictions or reduced distribution.
Platforms may impose rules stricter than Article 50 and may favor simple risk controls over nuanced judgments about creative origin. The law can state that an AI label does not determine illegality while a recommendation system quietly reduces the reach of anything considered suspicious.
More on AI content detection:
How likely is a real burden of proving non-AI authorship?
The answer depends on what “forced” means. A single legal requirement imposed on every creator remains unlikely, but several narrower burdens are plausible or already emerging.
A universal statutory burden appears unlikely under Article 50. The regulation does not establish a general presumption that disputed content is synthetic. It also does not require creators who avoid AI to register their work, use approved equipment or obtain certificates of human production.
Requests following a specific regulatory complaint are plausible. The right to complain under Article 85 allows natural and legal persons to report suspected infringements to market-surveillance authorities. A publisher or professional deployer may be asked for information when an authority assesses whether a transparency obligation applied.
Private requirements from clients and platforms are highly likely. “No generative AI” clauses are already appearing in competitions, commissions, publishing arrangements and creative marketplaces. Once such a clause exists, somebody must decide how compliance will be verified.
Informal demands from audiences are inevitable. Artists are accused because their work is unusually polished, anatomically strange or stylistically generic. Writers are accused because they use familiar headings or sentence structures, while photographers are accused because real scenes look improbable.
Universal suspicion of content without provenance remains avoidable. Standards bodies explicitly warn against it, but current incentives favor badges, auditable procedures and simplified moderation. A visible credential is easy to display, while a careful explanation of what missing metadata means requires more effort.
The danger is less a deliberate plan than an institutional drift toward requiring the easiest available evidence. Source files, version histories and process recordings become routine demands because they are easier to request than it is to evaluate the reliability of an accusation.
Independent and vulnerable creators will bear the heaviest burden
The costs of proving authorship will not be distributed evenly. Large organizations can buy compatible equipment, issue staff credentials, preserve asset histories and respond to complaints through legal departments.
Independent creators are less likely to possess those systems. They may move between personal and client-owned devices, work offline, delete old drafts or use software that does not support provenance records.
The burden will be particularly heavy for anonymous and pseudonymous writers, whistleblowers, dissident artists and journalists protecting vulnerable sources. It will also affect creators whose workflows produce limited digital evidence, including painters who scan a finished canvas and musicians who use analog equipment.
Remote collaboration creates additional gaps. A song may pass through several performers, engineers and studios, while an article may move among writers, editors and publishing systems that preserve different parts of its history.
Repeated compression and reposting can remove whatever metadata existed in the first version. A work may be copied from one platform to another until the version under dispute bears little technical resemblance to the creator’s original file.
Non-native writers may face an additional burden because the characteristics of their prose can overlap with features that some detectors associate with machine-generated text. A system presented as protecting creators may therefore make established creators easier to authenticate while making marginal creators easier to dismiss.
The unfairness of mandatory provenance is structural. The people most able to satisfy a demand for perfect documentation are often those already supported by institutions, while people publishing outside those institutions are treated as suspicious because their work lacks institutional traces.
Proving humanity carries a privacy price
Maintaining evidence of human authorship is not costless. A complete creative record can reveal a creator’s identity, device serial numbers, location data, working hours and private sketches.
It may also expose unpublished drafts, research sources, communications with collaborators, confidential client information and sensitive political or medical interests. For a journalist or witness, metadata could reveal the physical location of someone facing retaliation.
Provenance advocates often frame additional metadata as additional transparency. For the creator, additional metadata can also mean additional exposure.
A journalist may have good reason to strip information from a photograph before publication. An activist may need to remove device identifiers, while a domestic-abuse survivor may not want a signed identity attached to creative work.
A pseudonymous political writer may consider anonymity essential rather than deceptive. For such creators, a requirement to prove authorship through identity credentials can become a requirement to sacrifice safety.
WITNESS warns that provenance systems may be exploited to derive private information from metadata and that laws requiring personally identifiable information within provenance records could threaten freedom of expression. It argues that captured provenance should focus on how media was created or edited rather than automatically revealing who created it.
In this Content Authenticity Initiative symposium session, WITNESS examines how authenticity infrastructure can support verification while creating new risks for privacy, safety and freedom of expression.
Creators should not have to surrender anonymity in exchange for a presumption that their work is genuine. Yet that may become the unspoken bargain: identify yourself, expose your process and use approved software, or accept that your work will be treated with greater suspicion.
A false AI label can cause lasting harm
An AI label is not a neutral technical annotation. Audiences may associate it with dishonesty, manipulation, lower effort or reduced creative value, even when the label describes only a minor editing feature.
Research has found that describing material as AI-generated can reduce its perceived trustworthiness and people’s willingness to share it. The effect matters even when the underlying information is accurate or the content was actually created by a person.
Labels can also produce a second-order problem. When some content is marked as AI-generated, audiences may place excessive trust in unmarked material, including inaccurate content made entirely by humans.
Production method becomes a shortcut for evaluating truth. That shortcut fails in both directions because human-made material can be false and AI-assisted material can be accurate.
A false label can create four overlapping injuries:
Reputational injury: The creator may be accused of dishonesty, laziness or passing generated material off as human labor.
Commercial injury: Clients, readers and customers may place less value on the work, cancel commissions or demand refunds.
Distribution injury: Platforms may reduce visibility, disable monetization or exclude the work from recommendation systems.
Evidentiary injury: Future reviewers may treat the original label as evidence, even after it has been removed or corrected.
Screenshots of an accusation can circulate indefinitely. Search results may preserve the controversy, while later corrections reach only a fraction of the original audience.
The creator can win an appeal and still lose the commission, deadline, launch or audience that made the appeal necessary.
What a fair AI provenance system would require
A fair system must begin from the principle that an accusation requires evidence. Creators should not be expected to provide a perfect record of non-use simply because a detector or complainant expresses suspicion.
1. Missing provenance must remain neutral
Platforms, regulators, publishers and competition organizers should state explicitly that missing credentials do not create a presumption of AI use.
Credentials may be absent because of unsupported hardware, legacy files, privacy protection or ordinary editing. Popular AI’s analysis of AI watermarking and the limits of provenance metadata also explains how re-uploads, screenshots, platform processing and re-encoding can remove technical signals accidentally or deliberately. An absent credential is therefore weak evidence of anything.
The C2PA standard’s own guidance on optional provenance supports this approach. It says an asset should not be judged trustworthy or untrustworthy purely because it does or does not carry Content Credentials.
2. Detector scores must not constitute proof
A detector result should be treated as an investigative lead at most. Any adverse decision should require corroborating evidence connected to the actual file, workflow or tool.
Institutions should also stop presenting detector percentages as though they represented the probability that a person committed misconduct. A score describing the portion of text that resembles a target category is not the same as a 78 percent probability that the writer used AI.
3. The accuser should identify a specific violation
A complaint should identify the exact content alleged to be AI-generated or manipulated, the rule supposedly triggered and the evidence that an AI system was used.
Where the allegation concerns Article 50, the complainant should explain why the content falls within a covered category, why an exception does not apply and why the accused party qualifies as a regulated provider or deployer.
“The hands look strange” is not evidence. A screenshot showing a commercial detector score is also insufficient without information about the detector’s validated performance on that type of content.
4. The creator should receive the complete allegation
Secret scores and undisclosed reporting criteria prevent meaningful appeals. A creator cannot challenge a conclusion without knowing which system produced it, what version was used, what threshold applied and which portion of the work was flagged.
The institution should also disclose whether the initial action was automated, human or based on a combination of machine output and manual review.
5. Reviewers must distinguish creation from editing
A photograph lightly denoised by an AI-powered feature is different from a fully generated image. A human-written article checked for spelling is different from an automatically generated article, while a live recording mastered with intelligent software is different from a cloned voice.
The disclosure or enforcement decision should describe the actual intervention. Collapsing every machine-learning feature into one vague AI category misleads audiences and makes compliance unpredictable.
6. Rules must define the relevant threshold of AI use
Modern software contains many machine-learning features that users may not recognize or control. A camera may use computational photography, while an audio editor may include intelligent noise reduction and a design tool may apply automatic masking.
A fair policy must distinguish these assistive functions from systems that generate substantial expressive content. Without a clear threshold, a promise of “no AI” becomes impossible to interpret or enforce consistently.
7. Repeated abusive reports should carry consequences
Complaint systems that impose costs only on the accused will attract abuse. Platforms and authorities should detect coordinated flagging, repeated unsupported allegations and competitors using authenticity complaints as commercial weapons.
An accuser who repeatedly submits reckless reports should not retain unlimited power to disrupt other people’s work without consequence.
8. Appeals must be fast and capable of repairing harm
A correction issued after an election, news event, competition deadline or product launch may be worthless. Timeliness is therefore part of due process rather than an administrative convenience.
Successful appeals should remove the incorrect classification from internal enforcement records, restore distribution and address monetization lost because of the error. A quiet label removal does not undo a public accusation.
9. Confidential evidence must be protected
Creators should be able to submit drafts, raw files or recordings through a secure review process. Access should be limited to people who need the material for the dispute.
Evidence supplied to prove authorship should not be repurposed for model training, commercial analysis or unrelated investigations. The right to defend a work should not require surrendering trade secrets, unpublished material or journalistic sources.
10. AI labels must remain separate from legality judgments
The AI Act’s rule separating labels from illegality assessments should be reflected in platform policies. A disputed label may justify further investigation, but it should not transform lawful content into illegal content by itself.
More on AI content watermarking:
How human creators can protect their work
Human creators should not have to build a private surveillance system around their own creativity. Realistically, some ordinary recordkeeping is becoming prudent as authenticity disputes become more common.
1. Preserve original source material
Keep the earliest available form of the work rather than relying solely on the exported file that is eventually published.
Photographers can retain RAW files, original memory cards, contact sheets and unedited exports. Visual artists can keep sketches, scans, layers, project files, reference photographs and images of physical work in progress.
Writers can preserve research folders, notes, outlines, first drafts and editor comments. Musicians and voice performers can retain multitrack audio, isolated stems, rehearsal recordings, individual takes and original project sessions.
The underlying materials may contain far more useful evidence than the final JPEG, PDF, MP3 or video.
2. Maintain an ordinary version history
Cloud document histories, Git repositories, incremental saves and dated backups can show development over time. The goal is not to record every keystroke or brush movement, but to preserve enough of the creative sequence to answer a casual accusation.
A writer might retain an outline, source notes, rough draft, editor comments and final approval. An artist might preserve rough sketches, intermediate exports and layers, while a musician might keep alternate takes and separate tracks.
Evidence is more persuasive when it reflects a normal workflow rather than a package assembled only after a dispute begins.
3. Keep private evidence separate from public identity
Proof of process does not need to accompany every published work. Creators can maintain evidence privately and disclose it only when a genuine dispute arises.
Sensitive metadata should be stored securely, and location or identity information can be removed from public copies when necessary. Anonymous creators may also use a lawyer, publisher, union or trusted third party to verify records without disclosing their identities to the wider public.
4. Use provenance selectively
Content Credentials may help establish origin and editing history when a creator’s equipment and software support them. They should be treated as supporting evidence rather than a mandatory passport.
Adobe’s practical demonstration shows how a creator can apply and inspect Content Credentials within a supported workflow.
Creators should inspect what personal information a credential reveals before attaching it. They should also retain the underlying source files because no metadata system is indestructible or universally supported.
A provenance credential can strengthen a record. Its absence should never weaken the presumption that an unmarked work may be authentic.
5. Define AI use precisely in contracts
When a client prohibits generative AI, the contract should specify what the prohibition covers. Vague language invites disputes because modern creative tools contain automated features that occupy different points between conventional editing and generative production.
The agreement should clarify whether the restriction includes:
Spell-checking and grammar suggestions
Noise reduction and audio restoration
Camera autofocus and computational photography
Automatic masking and background removal
Translation and transcription
Generative fill and object replacement
AI-assisted reference search
Upscaling and frame interpolation
Features enabled by default within creative software
The contract should also state what evidence is sufficient, who pays for an investigation and whether detector scores can be considered proof.
Creators should avoid guaranteeing that no machine-learning component operated anywhere in a production chain unless the workflow can genuinely support that promise.
6. Create an authenticity challenge policy
Publishers, studios and independent creators can prepare a standard response before a dispute occurs:
We do not accept automated detector scores as conclusive evidence of AI use.
Specific allegations will be reviewed against source files, version history,
contractual definitions and other corroborating evidence.
The absence of provenance metadata does not establish AI generation.The policy can explain where complaints should be submitted, what information an accuser must provide and how confidential materials will be handled. Establishing the procedure in advance prevents each allegation from becoming an improvised public trial.
7. Preserve every platform notice
When content is labeled, restricted or removed, creators should immediately take screenshots, download the statement of reasons and record the date and time.
They should preserve the original upload, retain available metadata, request the detector or policy basis and file the internal appeal promptly. Records of lost revenue, canceled work or missed deadlines may become important if the dispute escalates.
The Digital Services Act requires explanations for certain moderation decisions and provides complaint mechanisms for eligible platform actions. Those protections are useful only when the creator has preserved a complete record of what happened.
8. Challenge the inference rather than negotiating the score
A creator should avoid becoming trapped in an argument over whether a detector should report 62 percent or 18 percent. The central question is whether the detector can establish how the work was produced.
Useful questions include:
What validated false-positive rate applies to this exact type of work?
Was the system tested on this language, genre, camera or recording process?
Which detector version produced the result?
Was the system independently audited?
Can ordinary editing, compression or translation change the score?
What corroborating evidence exists?
Does the conclusion mean generated, modified or merely statistically unusual?
These categories are different. A probabilistic output is not a production record.
9. Avoid unnecessary public disclosure
An accusation does not automatically justify publishing private drafts, client communications, raw footage or identity data. Creators should provide the minimum evidence required through an appropriate private channel.
Unrelated personal information can be redacted, and review copies can be watermarked when necessary. Originals should be preserved rather than handed over as the only available copies.
Creators should also resist social-media demands for immediate public proof when the accuser has provided no credible evidence. An online mob is not a neutral tribunal.
10. Record collaborative roles
Collaborative work creates special problems because no single participant may possess the entire production history. Writers, editors, photographers, designers, engineers, producers and performers should record their respective contributions when a contract makes AI use relevant.
A simple project log can identify who created the initial material, who edited it, which tools were used and who approved the final version. The record does not need to become an invasive monitoring system.
Its purpose is to prevent a complex human workflow from being reduced to a binary allegation that the finished work was either human or AI-generated.
11. Organize collectively
Creators’ associations, publishers, unions and professional bodies should establish shared standards for authenticity disputes. A credible framework could require disclosure of the evidence, meaningful opportunities to respond and independent review.
It could also prohibit adverse findings based solely on detectors, protect confidential source material, distinguish generation from editing and require rapid correction of false labels.
Collective standards can address compensation when reckless mislabeling causes measurable loss and can create consequences for repeated abusive reports. Without such standards, every independent creator must negotiate with large platforms and institutions from a position of weakness.
Verification should not become permission
Provenance can answer useful technical questions. It can identify who signed a file, whether the file changed after signing, what software recorded an editing action and whether a version corresponds to one issued by a known publisher.
Those questions can help readers, editors and investigators make better decisions. They should not quietly become political or commercial permission systems.
A creator should not need institutional credentials to be heard. A photograph should not require an approved camera before it can be considered genuine, and a writer should not have to expose an entire document history before publishing an opinion.
A musician should not need surveillance footage of every studio session to prove that their voice belongs to them. A witness should not be required to disclose a dangerous identity before authentic footage is taken seriously.
The internet flourished partly because people could publish without first obtaining certificates from established intermediaries. A provenance regime that divides expression into credentialed and uncredentialed classes would weaken that principle.
It would replace a demand to evaluate the work and the available evidence with a demand to produce approved production papers.
The distinction between verification and permission is therefore crucial. Verification offers information that can be weighed alongside other evidence, while permission makes a credential a condition of being believed, paid, published or heard.
The first can improve accountability. The second can become an authenticity license.
The reverse burden may arrive through private gatekeepers
The EU AI Act does not explicitly force human creators to prove that they avoided AI. That may be the least reassuring part of the problem because informal burdens are harder to challenge when nobody accepts responsibility for creating them.
The regulator points to the platform, while the platform points to industry standards. The standards body says credentials are optional, and the detector company says its result is advisory.
The reviewer says the totality of the evidence was considered. The client describes the outcome as a private commercial decision, while the online mob insists that it was merely asking questions.
At the end of that chain stands a human artist, photographer, musician or writer trying to establish that they personally created their own work.
The likely future is not one in which every creator receives a formal government order to prove non-AI authorship. It is one in which proof is repeatedly demanded by private gatekeepers, automated systems and complaint-driven procedures, with each participant claiming that the final decision belongs to somebody else.
That is how optional provenance becomes compulsory in practice. It is also how a transparency label can evolve into an authenticity license.
A fair system must preserve the correct starting point. The person making an accusation should provide credible evidence, missing metadata should remain neutral and detector scores should remain probabilistic.
Human review should test the allegation rather than demand proof of innocence. Human-made work should not need paperwork before it is allowed to count as human.
Explore more from Popular AI:
Start here | Local AI | Fixes & guides | Builds & gear | Popular AI podcast