Fake PoCs Are Breaking AppSec—Here’s the Fix

A CVSS 10.0 vulnerability hits a widely used web UI library. Your team does the responsible thing: search for proof-of-concept (PoC) exploits to validate exposure and prioritize patching. Within hours, GitHub fills up with “working” exploits, scanners, and blog posts.

Then the trap snaps shut: a big chunk of that “proof” is fake proof—nonworking code, edge-case demos, and outright AI-generated slop that looks credible. Defenders lose time. Risk gets mis-scored. Patching gets delayed. Meanwhile, attackers iterate past the broken public PoC and go straight for real exploitation.

This post is part of our AI in Cybersecurity series, and I’m going to take a firm stance: the industry’s obsession with PoC-driven validation is now a liability. The fix isn’t “ban AI.” It’s building security operations that treat PoCs as untrusted inputs and use AI the right way—to reduce noise, verify claims, and close the detection-to-remediation gap.

“Fake proof” is a supply chain problem for defenders

Fake PoCs create a downstream supply chain of bad decisions. Once a questionable PoC hits public repos, it doesn’t stay isolated. It gets copied into scanners, referenced in advisories, pasted into internal runbooks, and used to justify risk calls.

In the recent React2Shell saga, defenders faced a flood of public exploit code—Trend Micro reportedly observed around 145 public exploits circulating. The catch: most of the “attacks” failed to trigger the vulnerability. That’s not just annoying. It actively distorts triage.

Here’s how fake proof damages real operations:

• False negatives during validation: A PoC that requires unusual, nondefault conditions can convince teams they’re safe when they’re not.
• Misleading compensating controls: Teams may block the “thing the PoC uses” (a component, an endpoint shape, a parameter) instead of addressing the underlying flaw.
• Bad scanner propagation: People build detection logic around a broken PoC, turning one bad artifact into hundreds of internal false negatives.
• Patch deferral by implication: If the PoC doesn’t work, leadership hears “the exploit is theoretical,” and the queue reshuffles.

One line I keep coming back to when I see this pattern: if you need a public PoC to take patching seriously, your process is already behind.

The new reality: PoC volume scales faster than human review

AI didn’t invent low-quality exploit publishing. It industrialized it.

A single person can now generate dozens of plausible-looking PoCs, README files, and even “scan modules” in an afternoon. That changes the economics of defender attention. The core issue isn’t that defenders can’t evaluate one PoC. It’s that they can’t evaluate hundreds, fast, while also running production security.

And December is the worst possible time for this. Change freezes, holiday staffing gaps, and end-of-year release pressure all collide. Attackers know it. Noise is a tactic when defenders are tired.

Why “PoC-driven triage” fails in the age of AI slop

PoC-driven triage fails because it confuses exploit demonstration with risk reality. Risk is about exposure, reachability, and business impact—not whether someone on the internet posted a clean exploit script.

When teams over-index on PoCs, they create three predictable failure modes.

1) “It didn’t work, so we’re fine”

This is the most dangerous interpretation because it’s comforting. In React2Shell, some PoCs were invalid because they required developers to have explicitly exposed dangerous functionality to the client—something that wouldn’t represent the actual vulnerability conditions.

A broken PoC can still look sophisticated: HTTP requests, serialization blobs, bypass notes, WAF language. That’s enough for a rushed team to run it once, see “no shell,” and downgrade the issue.

If the vulnerability has a severe score and broad usage, treat PoC failure as non-evidence. It only proves your test didn’t reproduce.

2) “We blocked the PoC, so we mitigated the vuln”

Blocking a specific payload pattern is not the same as mitigating a class of vulnerability.

If the underlying issue is deserialization (or any structural weakness), the attacker’s job is to mutate. If your mitigation is tied to a PoC’s exact behavior, you’re playing whack-a-mole—especially when attackers can use AI to generate variants at scale.

3) “We’ll wait for a real PoC before patching”

Attackers don’t wait for your validation workflow.

In high-profile cases, exploitation begins quickly—sometimes within hours of disclosure. Public PoCs are often a lagging indicator, and in 2025 the lag can be deceptive: defenders are flooded with junk PoCs while attackers privately refine the real chain.

A nonworking public PoC doesn’t mean exploitation is hard. It often means the good exploit isn’t public.

The better approach: treat PoCs as untrusted and verify with AI

The right role for AI in cybersecurity isn’t generating more code—it’s verifying what’s real and what’s noise. You want systems that can ingest exploit chatter, score credibility, and guide action without your team reading 40 GitHub repos.

Here’s what “better AI” looks like in practice.

AI-driven credibility scoring for exploit artifacts

Start by assuming: every PoC is untrusted.

An AI-assisted pipeline can do first-pass triage on PoCs and related artifacts by scoring:

• Exploit preconditions: Does it require nondefault components, unusual config, or unrealistic client exposure?
• Behavioral alignment: Does the PoC attempt to hit the vulnerable code path, or just crash something adjacent?
• Originality and provenance: Is this a near-duplicate of other repos (a common signature of slop)?
• Claim vs. evidence: Are there logs, traces, or reproducible steps—or just assertions?
• Environmental specificity: Does it only “work” on a toy app with insecure assumptions?

This is where modern AI can shine: not by declaring “vulnerable/not vulnerable,” but by producing ranked, explainable triage that tells an analyst why an artifact is likely junk.

Snippet-worthy rule: If your PoC triage doesn’t explain assumptions, it’s not triage—it’s a demo.

Shift from “PoC works” to “reachability and exposure”

Instead of anchoring on PoCs, anchor on reachability:

• Is the vulnerable library version present?
• Is the vulnerable function reachable from an external request?
• Are dangerous inputs controllable by an attacker?
• Are there compensating controls that reduce impact (not just block one payload)?

AI can help by correlating SBOMs, dependency graphs, runtime telemetry, and API gateway logs to answer the real question: can an attacker reach the vulnerable path in our environment?

When this is done well, PoCs become optional. Useful, yes. Required, no.

Use AI to compress time-to-remediation (not just time-to-detection)

A hard truth from vulnerability management: detection is cheap; remediation is expensive. Many orgs detect thousands of issues monthly and fix only a fraction.

The gap that matters is detection → patch → validation → deploy.

AI-driven cybersecurity programs that actually reduce risk focus on:

1. Auto-grouping findings into fixable work units (one PR fixes 30 instances, not 30 tickets)
2. Patch recommendations tied to real dependency context (safe version bumps, breaking-change notes)
3. Automated regression checks (tests, builds, and deploy gates)
4. Targeted rollout (patch internet-facing and high-privilege services first)

If your AI spend is going into “more alerts” instead of “faster fixes,” you’re funding the wrong side of the equation.

A practical playbook for handling high-profile vulns (like React2Shell)

The goal is to keep decision-making stable when the internet gets noisy. Here’s a workflow I’ve found works—especially when a vulnerability is high-severity, widely deployed, and surrounded by questionable PoCs.

Step 1: Set a “PoC skepticism” policy in advance

Write it down now, not during the incident:

• PoCs are treated as untrusted
• PoC failure cannot downgrade severity by itself
• Mitigations must map to root cause, not payload strings

This prevents the classic December meltdown where every stakeholder argues from vibes.

Step 2: Decide patch priority with three numbers

You can do this quickly and consistently using:

1. Exposure: internet-facing, partner-facing, internal-only
2. Privilege: does the service handle auth, tokens, or admin paths?
3. Blast radius: how many users/systems depend on it?

If a vuln is critical and exposure is high, patching should start before PoC validation finishes.

Step 3: Validate using controlled, instrumented reproduction

When you do validate, don’t “run the PoC and hope.”

• Stand up a controlled environment that matches your runtime
• Instrument the suspected vulnerable path (logging, tracing)
• Confirm whether the vulnerable function is reached, not whether you get a shell

This makes you resilient to junk PoCs because you’re measuring code-path truth.

Step 4: Put AI where it reduces toil

Use AI to:

• summarize exploit chatter into a single analyst brief
• de-duplicate PoC repos and cluster similar payloads
• extract assumptions and preconditions from messy code
• propose fix PRs (version bump + compatibility notes)

Use humans to:

• approve changes
• reason about business impact
• decide rollout strategy

That split is realistic, and it keeps AI in a role where it’s strongest.

What security leaders should do next (especially for 2026 planning)

If your program relies on public PoCs to prioritize patching, you’re budgeting for failure. That model worked when PoCs were scarce and mostly written by careful researchers. It doesn’t work when AI can mass-produce convincing nonsense.

For 2026, I’d push leaders to invest in three capabilities that directly address the “fake proof” problem:

1. AI-assisted exploit and intel triage that ranks credibility and explains assumptions
2. Runtime reachability analysis so you can assess exposure without waiting for PoCs
3. Automation in remediation pipelines to close the detection-to-patch gap

The broader theme of AI in cybersecurity isn’t “more automation everywhere.” It’s automation where humans lose time to noise.

If you want to pressure-test your readiness, ask a blunt question: When the next CVSS 10.0 hits and the internet floods with junk PoCs, will your team patch confidently—or argue about whether the PoC is real?