AI Progress Metrics: What U.S. Tech Teams Miss

A single chart has become a kind of ritual in AI circles: every time OpenAI, Google, or Anthropic releases a new frontier model, people refresh a graph from METR (Model Evaluation & Threat Research) to see whether capabilities are “still exponential.” When Anthropic released Claude Opus 4.5 and METR reported it could complete a task comparable to about five hours of human work, the internet did what it does: it overreacted.

Here’s my stance: most companies get this wrong. Not because the chart is useless, but because they treat it like a prophecy instead of a measurement tool. If you run a U.S. SaaS product, a digital agency, a support operation, or an internal automation program, the point isn’t whether AI is “accelerating.” The point is how to track AI progress in a way that actually improves decisions, budgets, risk controls, and customer outcomes.

This post is part of our series, How AI Is Powering Technology and Digital Services in the United States. We’ll translate the METR-graph debate into practical guidance, then connect it to the other pressure point that’s showing up everywhere in 2026: power—especially as next-generation nuclear becomes part of the data center conversation.

The “exponential AI” graph is a compass, not a clock

Answer first: The METR-style capability graph is useful when you treat it as a directional signal about model competence—not as a timeline that tells you when AI will replace a role or fully automate a workflow.

The graph that keeps making the rounds attempts to quantify how model capability improves over time using standardized evaluations. It’s compelling because it turns abstract progress into a line you can point at. The problem is what people do next: they assume the line means “AI will do everything by date X.”

Capability tracking is more like performance monitoring in production software. A few realities make these graphs easy to misread:

• Benchmarks compress nuance. A model can spike on a particular task type without becoming broadly reliable.
• Task time is not business value. “Five hours of human work” doesn’t automatically equal “five hours saved” in your environment.
• Evaluations aren’t your workflow. Your systems, data, compliance needs, and edge cases often dominate outcomes.

A quotable rule I’ve found helpful: “Benchmarks measure models; customers experience systems.” Your job is system-level performance.

What to track instead of “Is it exponential?”

If you’re deploying AI in U.S. digital services—customer support, marketing ops, analytics, finance ops, IT, HR—track metrics that map to actual business constraints:

1. Reliability rate: What percent of outputs need no correction?
2. Escalation rate: How often does the AI hand off to a human, and at what point?
3. Time-to-resolution: End-to-end, including review and rework.
4. Cost per completed task: Model + tooling + human oversight.
5. Risk incidents: Privacy exposure, hallucination severity, policy violations.

That set gives you an adoption roadmap that doesn’t collapse the conversation into hype.

METR and the U.S. innovation ecosystem: why nonprofits matter to SaaS

Answer first: Independent evaluators like METR reduce the “vendor fog” that often surrounds frontier model releases, and that helps U.S. tech companies buy and deploy AI more strategically.

SaaS teams are operating in a market where model makers understandably market their strengths. Independent research organizations play a different role: they try to measure, stress-test, and communicate what models can do under controlled conditions.

For U.S. businesses, this has two practical benefits:

• Procurement clarity. When a neutral third party identifies where a model is improving fastest (or stagnating), you can align pilots and contracts accordingly.
• Risk calibration. Better evaluations help you decide what requires human approval, what can be automated, and what should never be delegated.

This matters because the AI stack is becoming more layered. Many companies aren’t “using a model” anymore—they’re using:

• a model
• an orchestration layer
• retrieval (RAG) connected to internal knowledge
• monitoring and guardrails
• human-in-the-loop workflows

Independent evaluations are one of the few common reference points across that stack.

A practical way to translate capability research into roadmaps

If you’re building AI features into a product or internal tool, take public capability signals (including METR-style trends) and convert them into a quarterly evaluation plan:

• Quarterly model bake-off: same prompts, same datasets, same scoring rubric.
• Workflow simulation tests: run “day in the life” cases, not isolated questions.
• Regression checks: verify the new model didn’t break what worked last quarter.
• Safety and privacy reviews: test for data leakage, prompt injection, and policy violations.

The win isn’t picking the “top” model. It’s knowing when switching models is worth the migration cost.

AI coding tools are shaking markets—here’s what that means for your org

Answer first: The surge in AI coding tools is pushing software teams toward smaller, faster releases—and that raises the bar for measurement, governance, and internal enablement.

Recent reporting suggests new AI coding capabilities are already “rattling the markets.” That’s not just investor drama; it reflects a real operational shift. When developers can generate scaffolding, tests, migrations, or documentation faster, two things happen inside companies:

1. Shipping speeds up. This is good—until quality controls don’t keep up.
2. The bottleneck moves. It shifts from typing code to reviewing, integrating, and maintaining it.

If you’re leading a U.S. digital service provider, the takeaway isn’t “replace engineers.” It’s “upgrade the pipeline.” Concretely:

• Add AI-aware code review guidelines (what must be checked, what can be trusted).
• Measure post-release defects tied to AI-generated changes.
• Standardize secure-by-default patterns in templates and internal libraries.
• Keep a clear policy on what code/data can be sent to external AI services.

A blunt truth: AI makes it easier to create software. It doesn’t automatically make it easier to own software. Ownership costs—security, performance, compliance, on-call—still land on you.

The power constraint is real: AI progress is colliding with the grid

Answer first: As AI workloads scale, electricity becomes a product constraint, not just an operating expense—and that’s why next-generation nuclear is back in the conversation.

If 2024–2025 was the era of “Can we build it?”, 2026 is increasingly “Can we power it?” Hyperscale AI data centers are large, power-hungry, and increasingly critical infrastructure for U.S. digital services.

That’s where next-generation nuclear power enters the story. Advanced reactor designs (and small modular reactor concepts) are being discussed as potential sources of reliable, low-carbon baseload electricity that can pair well with data center demand.

Even if you’re nowhere near building a data center, this still affects you because:

• Cloud prices are sensitive to energy costs. Higher sustained power demand can show up as pricing pressure.
• Capacity becomes strategic. Regions with constrained grids can throttle expansion timelines.
• Sustainability commitments get harder. Matching 24/7 compute with 24/7 clean power is nontrivial.

What digital service leaders should do this quarter

You don’t need to become an energy expert. You do need to plan like energy is a dependency.

• Ask your cloud provider about power sourcing and regional constraints. Not marketing claims—real capacity and timelines.
• Build a cost model that includes inference at scale. Many teams budget training and forget serving.
• Reduce waste: caching, smaller models for routine tasks, batching, and throttling.
• Adopt “right-size AI” architecture: not every feature needs a frontier model.

A simple, extractable statement: Compute strategy is now energy strategy.

Tracking AI progress without losing control (a simple operating model)

Answer first: The safest, fastest way to scale AI in U.S. tech and digital services is to run AI like any other production capability: define targets, instrument performance, and enforce gates.

If you only remember one framework from this post, use this three-layer approach.

Layer 1: Capability (what the model can do)

Track external progress with structured inputs:

• Model release notes and changelogs
• Independent evaluations (METR-style)
• Your own benchmark suite

Output: a quarterly “capability brief” that tells stakeholders what improved and what didn’t.

Layer 2: System performance (what users experience)

Instrument end-to-end workflows:

• Accuracy/rework rate by task category
• Latency by tier (p50/p95)
• Cost per workflow completion
• Customer CSAT for AI-assisted interactions

Output: dashboards that reflect outcomes, not vibes.

Layer 3: Governance (what you will and won’t allow)

This is where many orgs underinvest—until something breaks.

• Data handling rules (PII, PHI, financial data)
• Human approval thresholds (by risk class)
• Red-team tests (prompt injection, data exfiltration)
• Incident response playbooks for AI failures

Output: controls that let you scale without betting the company.

If your AI program can’t explain how it prevents privacy leakage, it’s not ready to scale.

That connects directly to a less-comfortable reality highlighted by research into large training datasets: content you put online can be scraped and end up in training corpora, including personal data if collection and filtering are weak. Your governance posture needs to assume the internet is not a private place.

A few “people also ask” answers (because your exec team will)

Answer first: These are the questions that typically decide budgets and timelines.

Is AI capability improving exponentially?

On some task families, improvements can look exponential on charts. In business practice, progress feels lumpy: sudden jumps on certain workflows, slower gains on edge cases, and frequent tradeoffs among cost, latency, and safety.

Should we wait for the next model before we ship?

No. Ship with an architecture that makes model upgrades easy. The competitive advantage is operational learning—your data, your workflows, your measurement—not waiting.

Will AI reduce headcount in digital services?

It can reduce time spent on routine tasks, but the near-term pattern I see is role reshaping: more review, more systems thinking, more customer exception handling, and more governance.

What to do next

If you’re building or buying AI for U.S. technology and digital services, the smart move is to treat capability tracking as part of normal operations—not as internet theater.

Start by picking one workflow (support email triage, contract summarization, marketing content QA, internal knowledge search), then:

• Define what “good” looks like in numbers
• Test two models against the same rubric
• Add monitoring and a human-approval gate
• Re-evaluate every quarter as capabilities shift

AI will keep improving, and the infrastructure powering it will keep changing too. The organizations that win won’t be the ones who refresh the graph the fastest—they’ll be the ones who can say, confidently, “We measured it, we controlled it, and we shipped it.”

What’s one workflow in your org where better AI progress tracking would immediately change a decision—budget, tooling, or staffing—this month?