AI-generated pull requests are dumping work on maintainers
AI-generated pull requests are flooding review queues. Learn how maintainers can require proof, limit volume, and close weak submissions.
AI-generated pull requests have changed the economics of contributing to open-source software. A coding agent can inspect a repository, edit several files, write tests, prepare a description, and open a pull request before its operator has fully read the diff.
The pull request may have taken minutes to produce. Proving that it is correct can still consume an experienced maintainer’s afternoon.
That imbalance is the real problem. Open-source projects do not need a philosophical argument about whether AI-written code is authentic. They need contribution rules that return the cost of verification to the person submitting the work.
More on AI for devs:
AI-generated pull requests: the quick verdict
Scale: Agent-authored pull requests are already appearing across GitHub at a level that makes project-wide rules necessary.
Quality: AI-generated code is not automatically bad, but merge rates do not reveal the review, correction, and maintenance work behind an accepted contribution.
Highest-cost submissions: Large, unsolicited, weakly tested, duplicated, or poorly understood changes consume the most maintainer attention.
Recommended policy: Require prior approval, material AI disclosure, reproducible test evidence, small diffs, direct human participation in review, and one open AI-assisted pull request per outside contributor.
Enforcement: Close submissions that omit required evidence. Maintainers do not need to prove that code “looks AI-generated” before protecting the review queue.
Contributor responsibility: Treat an agent’s output as a draft you own, not as work maintainers are obligated to debug.
For most open-source projects, the best policy is neither a total ban nor unrestricted acceptance.
Permit AI-assisted contributions when the contributor starts from a maintainer-approved issue or discussion, discloses material AI use, reads and understands the complete diff, provides reproducible test evidence, keeps the change small and focused, answers review questions personally, and has no other AI-assisted pull request open.
Automatically close the pull request when those conditions are missing.
This approach does not judge whether AI-written code is morally legitimate. It applies an ordinary rule of exchange: a contribution should create more value for the project than it costs maintainers to evaluate.
LLVM calls submissions that fail this test “extractive contributions”. Its policy explains that unreviewed AI output transfers design and review labor from the submitter to maintainers. LLVM also prohibits autonomous agents from acting in project spaces without human approval and reserves “good first issue” work for people learning the codebase.
AI agents have made submitting code much cheaper
Agent-authored pull requests are no longer a curiosity.
The February 2026 AIDev dataset collected 932,791 pull requests attributed to Codex, Devin, GitHub Copilot, Cursor, and Claude Code. They appeared across 116,211 repositories and involved 72,189 developers. The paper’s curated research subset contains 33,596 pull requests from 2,807 repositories with at least 100 stars.
That number should not be mistaken for 932,791 bad contributions. It does show that agent participation is large enough for maintainers to need rules before a flood reaches their own repository.
The new workflow is easy to understand:
A developer gives an agent an issue or broad instruction.
The agent reads the repository and generates a patch.
It writes a plausible explanation and may run whatever tests it discovers.
The developer submits the result, sometimes after little inspection.
The maintainer becomes the first person to examine the work with full responsibility for its consequences.
The contribution is cheap because code generation has been automated. Review is expensive because responsibility has not.
The review burden goes beyond finding obvious bugs
Maintainers are checking more than whether a function works on one example.
They need to determine whether the pull request:
Solves a problem the project considers important.
Fits the existing architecture.
Preserves compatibility.
Introduces a security problem.
Changes behavior outside the stated scope.
Weakens or manipulates tests.
Duplicates existing work.
Creates maintenance obligations that outlive the contributor.
Comes from someone capable of supporting the change.
A January 2026 study accepted at the Mining Software Repositories conference examined roughly 33,000 agent-authored pull requests. It found that unmerged submissions tended to be larger, touched more files, and failed continuous-integration checks more often. A manual analysis of 600 pull requests found recurring rejection reasons including duplicate work, unwanted features, poor reviewer engagement, and agent behavior that did not match project expectations.
That is why “the code compiles” is weak evidence. The maintainer still has to determine whether the change should exist.
This concern also applies when an agent produces a security fix. As Popular AI has previously covered, an AI coding agent can repair one function while breaking deployment, altering configuration, or touching an unrelated dependency. Faster patch generation does not eliminate the need for controlled review of AI-generated fixes.
More on AI in software development:
Concurrent agents create another layer of work
Agent submissions also compete with one another.
A July 6, 2026 preprint examined the 33,596-pull-request AIDev subset. Under exact temporal overlap, 40.2 percent of the repositories had agent-authored pull requests active at the same time. Those overlapping submissions represented 79.4 percent of the agent pull requests in the dataset.
The researchers then replayed Git merge operations on 747 overlapping pairs. Textual conflicts appeared in 19.8 percent of pairs created by the same agent type and 41.7 percent of cross-agent pairs. Most conflicted files contained source code, and roughly 42 percent of the conflicts were structural rather than simple edits to the same lines. The authors describe these results as a conservative lower bound because the test did not measure semantic conflicts between changes that merge cleanly but behave badly together.
This produces a particularly wasteful pattern:
One agent opens a fix.
Another opens a competing fix.
A third changes adjacent code.
Humans compare the implementations, resolve overlap, explain why two are being rejected, and maintain whichever version survives.
The agents generated more options. The maintainers inherited more coordination work.
Not every AI-assisted pull request is slop
A sensible policy should not pretend all agent work is worthless.
A 2025 study of 567 pull requests generated with Claude Code across 157 projects reported that 83.8 percent were eventually merged and that 54.9 percent of merged submissions required no further modification. The remainder required human revisions, particularly around project-specific requirements.
Those results show that useful agent-assisted contributions exist. They do not settle the maintainer-cost question.
The sample was limited to disclosed Claude Code pull requests in projects where such submissions were attempted and observable. A merge also does not show how long maintainers spent reviewing the work, what corrections happened through comments, or whether the accepted change created future maintenance costs.
The right conclusion is not “agent pull requests are good” or “agent pull requests are bad.”
It is this: The submitter should have to prove that a contribution is worth reviewing before scarce maintainer attention is spent on it.
What the research can and cannot prove
The available evidence is useful, but it answers different questions.
The AIDev paper measures the scale and distribution of agent-authored pull requests. It does not label those submissions as good or bad. The MSR 2026 failure study focuses on patterns associated with unmerged work and on recurring rejection reasons. It does not show that every large or failed pull request was careless.
The Claude Code pull-request study provides a valuable positive counterexample, but its disclosed, tool-specific sample should not be generalized to all coding agents or all repositories. The July 2026 merge-conflict preprint measures textual overlap among concurrent agent submissions and explicitly treats its estimates as a lower bound because clean merges can still create semantic conflicts.
A separate qualitative paper, “An Endless Stream of AI Slop”, analyzed 1,154 Reddit and Hacker News posts. It documents developer perceptions, review friction, and proposed countermeasures. It does not estimate the share of all AI-generated code that is defective.
Together, these sources support a narrow but practical conclusion. Agent-generated work is now common, useful submissions exist, and review cost can rise when contributors send poorly scoped or weakly owned changes. That is enough to justify evidence-based contribution rules without claiming that every AI-assisted pull request is bad.
The cURL case is real, but it needs a distinction
The Financial Times reported on July 11, 2026 that maintainers were facing low-value AI contributions that looked plausible but required human investigation. Its broader account of open-source maintenance labor used cURL as a central example.
cURL certainly experienced an extreme AI-review burden, but the clearest primary evidence concerns security reports, not ordinary pull requests.
In January 2026, cURL maintainer Daniel Stenberg said the project was ending its bug-bounty program after its rate of confirmed vulnerabilities fell below 5 percent amid an explosion of low-quality reports. In that same post, he explicitly said cURL had not yet experienced a comparable AI-slop problem in its GitHub issues and pull requests.
The pressure did not disappear. cURL’s vulnerability disclosure policy paused report intake for July 2026, with submissions scheduled to resume on August 3, 2026. The pause applied to vulnerability reports, while the project’s ordinary public issue and pull-request trackers remained available.
That distinction strengthens the main argument. The problem is bigger than code generation. Anywhere AI can make a submission nearly free, humans can be left paying the verification cost.
A practical AI contribution policy for maintainers
The following rules combine approaches already used by LLVM, Homebrew, Kubernetes, FastAPI creator Sebastián Ramírez, and other open-source maintainers.
1. Require approval before implementation
Do not accept unsolicited agent attempts at whatever issue the model happens to find.
Require a linked issue or discussion where a maintainer approved:
The problem.
The intended scope.
The proposed direction.
The need for an outside contribution.
This prevents maintainers from spending time reviewing technically competent solutions to problems they never wanted solved.
For small projects, approval can be as simple as a maintainer comment saying, “Yes, a pull request for this would be useful.”
2. Require material AI use to be disclosed
The contributor should identify:
The tool or model used.
Whether it generated code, tests, documentation, or the pull-request description.
Which parts were substantially generated.
How the output was checked.
Disclosure does not prove quality. It tells the reviewer what verification path may be needed.
Homebrew requires contributors to disclose the tool or model, review all generated content, and remain capable of handling every review comment. Its Responsible AI Usage guide tells contributors to treat AI output as a fallible first draft.
3. Make one human responsible for the complete diff
A pull request belongs to the account submitting it.
The contributor must be able to explain:
Why every changed file was touched.
Why the implementation fits the project.
What alternatives were considered.
What could fail.
How the tests exercise the actual behavior.
“The agent did that” is not an acceptable explanation.
Kubernetes permits disclosed AI assistance but says authors must understand every change. Its policy allows reviewers to close a pull request when the author cannot explain the code or engages through generated review replies rather than responding personally.
4. Demand evidence, not “tests passed”
Require exact commands and results.
Weak:
Tested locally.
Better:
TEST_COMMANDS:
pytest tests/test_parser.py -q
npm run lint
npm run typecheck
RESULTS:
42 tests passed.
No lint or type errors.
MANUAL_CHECK:
Imported the attached malformed configuration and confirmed that
the parser now returns ErrorCode.INVALID_SECTION.The evidence should be specific enough for a maintainer or CI worker to reproduce.
Claims about performance should include a benchmark, baseline, environment, and raw result. Claims about a bug should include a reproduction before the fix and a passing test afterward.
5. Keep the diff small
Large AI-generated changes are difficult to audit because agents often reformat code, rename symbols, rewrite comments, or “improve” adjacent areas while completing the requested task.
Set a default review threshold. A smaller project might begin with:
No more than 400 changed lines.
No more than 12 files.
No unrelated formatting.
No dependency changes without prior approval.
No generated lockfile churn unless required.
These numbers are starting points, not universal law. The principle matters more: exceeding the threshold requires maintainer approval before submission.
Kubernetes explicitly disallows large AI-generated pull requests and warns that reviewers may close any change whose review cost exceeds its expected benefit.
6. Limit contributors to one AI-assisted pull request
Do not let one person launch five agents against five issues and place the queue on maintainers.
Homebrew currently limits non-maintainers to one AI-assisted or AI-generated pull request at a time. GitHub now lets maintainers cap the number of open pull requests from contributors without write access and exempt trusted contributors through a bypass list.
A one-at-a-time rule also creates useful feedback. The contributor must learn from the first review before creating more work.
7. Protect mentoring issues
“Good first issue” work serves a different purpose from routine code production. It teaches a new contributor how the project works and starts a relationship with maintainers.
Allowing an autonomous agent to consume those issues removes the learning value while preserving the review cost. LLVM therefore prohibits using AI tools to complete issues carrying its good first issue label.
Projects do not have to ban all assistance. A contributor might use AI to explain unfamiliar code or check grammar. The implementation and conversation should still demonstrate that the person is learning.
8. Require human review conversations
Do not make maintainers negotiate with a reply generator.
Contributors may use tools privately to understand feedback. The final response should come from a person who has made a decision and can defend it.
This avoids the absurd loop where one model writes code, a reviewer explains a problem, and another model writes an agreeable paragraph without the contributor understanding either side.
9. Close unverifiable submissions without providing private tutoring
Maintainers are not obligated to reverse-engineer a contributor’s agent session, repair a broad patch, or write a free lesson explaining every failure.
The FastAPI creator’s contribution policy describes repeated automated submissions as a denial-of-service attack on human effort. It warns that accounts may be blocked when they repeatedly create large review obligations with little effort of their own.
A standard closure message is enough:
This pull request does not include the scope approval, human attestation,
or reproducible test evidence required by our contribution policy.
It has been closed without code review.
You may submit a new, smaller pull request after meeting those requirements.That is not hostility. It is queue management.
How existing projects draw the line
Projects are converging on similar principles even when their wording differs.
LLVM’s AI tool policy centers human accountability, disclosure, learning, and the rule that a contribution must be worth more than the time needed to review it. Homebrew’s contribution rules require disclosure, self-review, direct ownership of review comments, and a one-at-a-time limit for outside contributors.
Kubernetes’ pull-request guidance rejects large AI-generated changes, requires the author to understand every line, and expects review conversations to come from the human contributor. Sebastián Ramírez’s open-source contribution guidance frames repeated automated submissions as a denial-of-service attack on human effort.
These policies are not universal law. They are examples of maintainers adapting ordinary contribution standards to a workflow where generating a patch has become much cheaper than evaluating one. The shared idea is simple: tools may accelerate implementation, but they do not transfer authorship, responsibility, or the duty to make review economical.
Copy this AI contribution policy
## AI-assisted contributions
AI-assisted contributions are permitted when they reduce work for the
project rather than transferring verification work to maintainers.
Before opening a pull request:
1. Link to an issue or discussion where a maintainer approved the work.
2. Disclose any material use of AI coding tools.
3. Read and understand the complete diff.
4. Remove unrelated changes, generated filler, and unnecessary rewrites.
5. Run the project's required checks.
6. Provide the exact test commands and results.
7. Confirm that you can explain and maintain every change.
Non-maintainers may have only one AI-assisted pull request open at a time.
Large AI-generated changes require approval before implementation.
AI tools may not autonomously post issues, pull requests, review comments,
or replies under a contributor's identity.
Review discussions must be handled by the human contributor. You may use a
tool privately to understand feedback, but your response and decision must
be your own.
Maintainers may close a pull request without detailed review when:
- The work was not approved.
- AI use was not disclosed.
- The diff is larger than the approved scope.
- Reproducible test evidence is missing.
- Continuous integration fails.
- The change duplicates existing work.
- The contributor cannot explain the implementation.
- The expected review cost exceeds the likely value to the project.
Repeated noncompliant or automated submissions may result in the account
being blocked from the repository.Add these fields to the pull-request template
GitHub supports repository pull-request templates, which can be stored as .github/pull_request_template.md. Templates can prompt contributors for information and checklists before review begins.
## Problem and approval
APPROVED_ISSUE: #123
Explain the problem this solves:
Explain why this implementation was chosen:
## AI use
AI_USE: none | assisted | generated
AI_TOOL:
AI_GENERATED_AREAS:
Explain how you reviewed and verified the generated material:
## Human ownership
HUMAN_ATTESTATION: yes
I have read the complete diff, understand every changed file, and accept
responsibility for maintaining this contribution.
## Test evidence
TEST_EVIDENCE:
List the exact commands you ran and their results.
## Scope check
- [ ] This change contains no unrelated formatting or cleanup.
- [ ] This change introduces no unapproved dependency.
- [ ] Documentation has been updated where required.
- [ ] I can explain the implementation without delegating the discussion
to an AI tool.Automate the evidence gate
Do not waste time trying to judge whether prose or code “looks like AI.” Check whether the contributor supplied the required evidence.
The following GitHub Actions workflow closes pull requests whose descriptions omit the required fields:
name: Contribution evidence gate
on:
pull_request_target:
types: [opened, edited, reopened]
permissions:
contents: read
issues: write
pull-requests: write
jobs:
validate:
runs-on: ubuntu-latest
steps:
- name: Validate required contribution evidence
uses: actions/github-script@v9
with:
script: |
const body = context.payload.pull_request.body || "";
const required = {
AI_USE: /^AI_USE:\s*(none|assisted|generated)\s*$/im,
APPROVED_ISSUE:
/^APPROVED_ISSUE:\s*(#\d+|https:\/\/github\.com\/[^/]+\/[^/]+\/issues\/\d+)\s*$/im,
HUMAN_ATTESTATION:
/^HUMAN_ATTESTATION:\s*yes\s*$/im,
TEST_EVIDENCE:
/^TEST_EVIDENCE:\s*\S.+$/im
};
const missing = Object.entries(required)
.filter(([, pattern]) => !pattern.test(body))
.map(([field]) => field);
if (missing.length === 0) {
core.info("Required contribution evidence is present.");
return;
}
const owner = context.repo.owner;
const repo = context.repo.repo;
const pull_number = context.payload.pull_request.number;
await github.rest.issues.createComment({
owner,
repo,
issue_number: pull_number,
body:
"This pull request was closed automatically because the " +
`required contribution evidence is incomplete: ${missing.join(", ")}. ` +
"Update the pull request body using the repository template, " +
"then reopen it."
});
await github.rest.pulls.update({
owner,
repo,
pull_number,
state: "closed"
});This workflow reads metadata only. It does not establish that the declarations are truthful or that the code is safe. CI and human review still handle that.
It uses the privileged pull_request_target event so it can comment on and close pull requests from forks. Do not add a checkout step or execute contributor-controlled code in this job. GitHub explains that pull_request_target runs with elevated trust, and its workflow-event documentation warns that running untrusted code on that trigger can expose write privileges or secrets. GitHub’s secure-use guidance recommends pinning third-party actions to a reviewed full-length commit SHA in security-sensitive projects.
Run code tests in a separate workflow with the safer permissions appropriate for untrusted pull requests.
A workable maintainer triage process
Once the evidence gate is in place, use this order:
▪ Stage 1: Scope
Confirm that a maintainer approved the issue and that the diff matches the approved work.
Close it immediately when the project does not want the change.
▪ Stage 2: Mechanical checks
Run formatting, linting, type checking, tests, dependency checks, and security analysis.
Do not begin a detailed human review while mandatory checks are failing.
▪ Stage 3: Contributor comprehension
Ask one or two targeted questions when ownership is uncertain:
Why was this abstraction introduced?
Which test fails without this line?
What happens when the input is empty?
Why does this dependency need to change?
Which existing project pattern does this follow?
A contributor who owns the work should be able to answer directly.
▪ Stage 4: Human code review
Review only after the pull request is approved in scope, mechanically clean, and supported by a responsive contributor.
▪ Stage 5: Close or merge
Do not leave weak submissions open indefinitely.
An open pull request carries coordination cost. It attracts comments, becomes stale, conflicts with later work, and leaves other contributors uncertain about whether the task is available.
When developers should submit AI-assisted work
Submit the pull request when all of these are true:
The project has indicated that the change is wanted.
You could have completed the task without the agent, even if more slowly.
You understand the relevant code and tests.
The diff is smaller and clearer after your review.
You have reproduced the original problem.
You have tested the fix against that reproduction.
You are willing to support the change after it merges.
You can respond to maintainers without asking an agent to impersonate you.
Coding agents can be useful for repetitive refactoring, test scaffolding, documentation, and investigating unfamiliar code. Developers who prefer to keep more of that process on hardware they control can experiment with local coding agents and GGUF models. A local workflow changes where the model runs, not who owns the result.
More on local AI coding agents:
When maintainers should reject the submission
Close it when:
No one approved the work.
The description does not identify a real user or project problem.
The contributor cannot reproduce the claimed bug.
The diff performs broad cleanup around a tiny fix.
Tests are missing or fail.
The same contributor has several agent pull requests open.
A competing pull request already addresses the issue.
The author cannot explain a design decision.
Review replies appear generated and fail to answer the question.
The expected maintenance cost exceeds the feature’s value.
Projects may choose a stricter policy for security-sensitive areas, packaging infrastructure, build systems, cryptography, authentication, or code that processes untrusted input.
AI coding agents can access much more than the file being edited. Depending on permissions, they may read repository instructions, run shell commands, install dependencies, or touch secrets. Popular AI’s coverage of account, privacy, and repository risks around Claude Code explains why agent use also needs controls before the pull request is created. Those repo-access controls belong in the security model, not in an optional cleanup step after submission.
More on AI coding agents and repository risks:
What this setup costs
The minimum policy costs no money.
A small project can add:
A contribution-policy section.
A pull-request template.
A standard closure response.
A one-contribution-at-a-time rule.
A busier project can add the evidence-gate workflow and stronger CI. Initial setup should take less effort than repeatedly extracting missing information from individual contributors.
The ongoing cost is enforcement. Maintainers must be willing to close noncompliant pull requests consistently. A policy that produces five rounds of debate before every closure has failed to protect the review queue.
Frequently asked questions
Are all AI-generated pull requests low quality?
No. The Claude Code pull-request study found many accepted submissions, including merged pull requests that required no visible revision. Results still vary by task, project, agent, contributor involvement, and sample selection.
Generation method does not remove review cost. The submitter must provide enough evidence to justify that cost.
Should an open-source project ban AI-generated contributions?
A ban can make sense when a project has very little maintainer capacity, handles highly sensitive code, or has already experienced repeated abuse.
For many projects, a cost-based policy is more useful. Accept disclosed, small, tested contributions from accountable humans. Close submissions that transfer basic verification work to maintainers.
Can maintainers reliably detect undisclosed AI code?
Detection should not be the foundation of the policy.
Enforce observable requirements instead: approved scope, small diffs, test evidence, contributor comprehension, responsive review, and one open submission at a time. Those rules improve contributions whether the code came from an agent, a novice, or an overconfident senior developer.
What is the minimum viable AI contribution policy?
Require prior approval, disclosure, human ownership, reproducible tests, a small diff, one open AI-assisted pull request per outside contributor, and automatic closure when required evidence is missing.
Should AI be allowed on good-first-issue tasks?
Using AI for private explanation or learning can be reasonable. Letting an autonomous agent complete and claim the issue defeats its mentoring purpose.
Projects that use good-first-issue labels to train future contributors should reserve the implementation and project conversation for the human learner. LLVM’s policy takes this approach.
How open-source maintainers can make AI pull requests pay their own review cost
Open-source maintainers should stop treating every pull request as an automatic entitlement to bespoke human review.
Coding agents have reduced the cost of producing a plausible contribution. They have not reduced the responsibility attached to merging it. Until agents can carry that responsibility, the person pressing the submit button must do the verification work.
Require proof. Limit volume. Keep diffs small. Protect mentoring tasks. Close submissions that arrive without human ownership.
Vibe coding can be cheap for contributors. It does not have to remain expensive for everyone else.
Explore more from Popular AI:
Start here | Local AI | Fixes & guides | Builds & gear | Popular AI podcast
Should open-source projects limit AI-generated pull requests, or would stricter rules punish responsible contributors too?