You launch a test, wait two weeks, and variant B wins by 11 percent. You ship it. Three months later the lift never shows up in revenue. Nobody did anything wrong in the build. What probably happened is that a meaningful slice of the traffic that decided the test was never a customer at all. It was a scraper, a monitoring bot, or an AI agent checking prices, and none of it behaves like a person deciding whether to buy.
Cloudflare Radar puts bots at roughly a third of web traffic in 2026; Imperva's broader 2026 Bad Bot Report puts automated traffic above 53 percent. Either way, enough non-human traffic reaches most sites to bias A/B test conversion rates, especially on lower-traffic pages where a bot batch can be a large share of one variant's sessions. Run a sample ratio mismatch check on every test and filter known bots server-side before trusting a result.
How much of your traffic is even human right now?
The estimates differ by methodology, and that gap is itself the first thing to understand before you argue about a specific number. Cloudflare Radar, which classifies bots from the requests it can see across its network, measured bots at 34.94 percent of web traffic across the trailing 28 days to August 1, 2026, essentially level with 34.75 percent across all of July. Within that verified bot share, Cloudflare's own analysis of AI crawlers found them at 20.3 percent of verified bot traffic in May 2026, with AI-search bots adding another 6.5 percent, so AI-related activity alone accounted for roughly a quarter of the bots Cloudflare could positively identify.
Imperva's 2026 Bad Bot Report, titled "Bots in the Agentic Age," uses a wider definition that includes scrapers, monitoring tools and unverified automation, and lands on a bigger number: automated traffic above 53 percent of all web traffic in 2025, up from 51 percent the year before, with human share down to 47 percent and falling. The same report found a 12.5 times increase in detected AI-driven bot attacks year over year, pushing the daily average of blocked AI incidents from roughly 2 million to 25 million. Retail was the single most targeted industry for AI-enabled bot activity, and business logic abuse against pricing, inventory and booking flows was concentrated in retail and travel specifically.
Neither figure is directly comparable to the other, because they count different things. But both point the same direction: whatever your GA4 session count says, a growing share of it did not come from a person weighing a purchase decision.
Why does bot traffic actually break an A/B test?
A test compares two conversion rates. Bot traffic corrupts that comparison in two distinct ways, and most teams only guard against one of them.
The first is a skewed split, formally called sample ratio mismatch, or SRM. If your test is meant to run 50/50 and it settles at 55/45 instead, something is wrong with how traffic is being assigned, and a scraping bot that repeatedly hits one variant's URL, or a redirect quirk that funnels crawler traffic disproportionately into one path, is one of the more common causes. SRM shows up in an estimated 6 to 10 percent of tests industry-wide, and when it appears it is treated as an automatic disqualifier by most serious experimentation teams, because whatever distorted the split usually distorted the conversion numbers too.
The second, subtler problem is dilution without any visible SRM. A bot that loads a page, never adds to cart, and never converts still counts as a session. Enough of those sessions on one variant quietly drags its conversion rate down relative to the other, even with a clean 50/50 split, simply because the denominator grew while the numerator did not. On a high-traffic homepage test this washes out in the noise. On a lower-traffic landing page, a checkout step, or a B2B lead form, where you might only be running a few thousand sessions per variant over a test window, a single bot crawl batch can be a large enough share of one arm to move the needle on its own.
| Signal | What it means | What to do |
|---|---|---|
| Traffic split drifts from the configured ratio (e.g. 55/45 on a 50/50 test) | Sample ratio mismatch; something is assigning traffic unevenly | Stop the test, find the cause, restart. Do not read the result |
| One variant's bounce rate or session duration is oddly low or oddly uniform | Possible bot batch hitting that URL specifically | Check server logs by user agent and IP range for that variant's path |
| Conversion event fires with no prior page views or scroll activity | Scripted or headless traffic skipping normal browsing behavior | Cross-check against known crawler and datacenter IP lists |
| A spike in sessions with no matching spike in ad spend or campaign activity | Non-campaign automated traffic, possibly AI agent or price-monitoring bot | Segment by traffic source before pooling into the main test result |
| GA4's AI assistant channel or referral traffic grows faster than overall sessions | Declared AI referral traffic (ChatGPT, Perplexity) entering the funnel | Analyze separately; it behaves differently from search and direct traffic |
Are AI shopping agents bots or customers?
This is the part that makes 2026 different from the bot problem CRO teams have dealt with for a decade. A scraper checking your prices is obviously not a customer. But an AI agent acting on a real shopper's behalf, comparing options and potentially completing a purchase through the same interface a human would use, is something else. Imperva's 2026 report frames this directly as the defining challenge of what it calls the agentic era: these agents retrieve data, execute workflows and complete transactions through the same interfaces as humans, which blurs the line between legitimate and malicious traffic and makes a simple allow-list or block-list insufficient.
For experimentation purposes, the practical answer is to treat declared AI agent and assistant traffic as its own segment rather than folding it into your headline conversion rate either way. GA4 now splits some of this out automatically into its AI assistant channel, which is useful, but it only catches traffic that identifies itself. Undeclared agent traffic, and the much larger volume of plain scraping and monitoring bots, still needs to be filtered out at the server or CDN level before it ever reaches your test's denominator.
Before you trust any A/B test result on a page with under 20,000 sessions per variant per week, pull the traffic source and user agent breakdown for both arms, not just the topline conversion numbers. If one variant is carrying a disproportionate share of sessions with zero engagement events, direct entry to a deep URL, or a user agent string that does not match a real browser, exclude that segment and rerun the numbers before you ship anything based on the original result.
What should actually change in how you run tests?
Most testing platforms, including Optimizely, VWO and the merged VWO-AB Tasty stack, already run an automatic SRM check in the background. The mistake is treating that as sufficient. A clean SRM check tells you the split was even. It does not tell you whether both arms were evenly polluted by the same bot traffic, which would still distort your absolute conversion rate even if the comparison between variants stayed fair.
Three habits fix most of the exposure without adding much process. First, filter server-side wherever you can, since headless and scripted traffic often skips client-side analytics tags entirely but still hits your server logs, which is where the most reliable bot signal actually lives. Second, run the SRM check within the first 48 hours of any test launch, before enough conversion data has accumulated to tempt anyone into acting on it early. Third, treat any test on a low-traffic page as needing a longer run and a manual traffic-quality review before the result gets trusted, rather than applying the same automatic threshold you'd use on a high-traffic page. If most of your test traffic arrives through paid campaigns, add click-fraud and bot filtering at the ad platform level too, since a polluted top of funnel feeds a polluted test regardless of what you do on-site.
None of this replaces good experiment design. A well-run conversion optimization program still needs a real hypothesis, a big enough sample, and patience to reach statistical significance. But a test built on clean methodology and dirty traffic still produces a number you cannot act on, and in 2026 the traffic is dirtier than most dashboards let on.
Frequently asked questions
How much of web traffic is actually bots in 2026?
Estimates vary by methodology. Cloudflare Radar put bots at roughly 35 percent of web traffic across the trailing 28 days to August 1, 2026. Imperva's 2026 Bad Bot Report, which uses a broader definition including scrapers and AI agents, put automated traffic above 53 percent of all web traffic in 2025, with human share down to 47 percent. Either figure is high enough to matter for a funnel running live experiments.
What is sample ratio mismatch and why does it matter for A/B testing?
Sample ratio mismatch, or SRM, is when a test's actual traffic split does not match the intended split, for example 55/45 instead of 50/50. It shows up in an estimated 6 to 10 percent of tests and is a strong signal the result cannot be trusted, because whatever caused the imbalance, bot traffic hitting one variant more than the other is a common cause, likely also distorted the conversion numbers themselves.
Can bot traffic make a losing variant look like a winner?
Yes. If a crawler or scraping bot repeatedly loads one variant's URL and that bot never converts, it dilutes that variant's conversion rate and can make the other variant look artificially stronger, even if real customers responded to both equally. The reverse also happens when a bot batch happens to hit a form or add-to-cart action, inflating a metric that never reflected a real customer.
Do AI agents shopping on behalf of users count as bot traffic?
For experimentation purposes, yes, in the sense that they are non-human sessions that need to be classified and usually excluded from a standard conversion-rate test, even though some of them represent real purchase intent. Imperva's 2026 report frames this as the core challenge of the agentic era: agents use the same interfaces as humans, so simple bot lists are no longer enough to separate them out.
How do we filter bot and agent traffic out of test results?
Start with server-side bot filtering rather than relying only on client-side JavaScript, since headless and scripted traffic often skips analytics tags anyway but still hits your server and can still be logged in raw request data. Cross-check GA4's bot exclusion and known crawler lists, run an SRM check within 48 hours of any test launch, and segment declared AI referral traffic, from ChatGPT or Perplexity, separately rather than blending it into your main conversion number.
Is it worth paying for dedicated bot detection if we are a small business?
Not usually as a standalone tool at small scale. Most small and mid-size sites get sufficient protection from the bot filtering already built into GA4, their CDN or WAF, and their testing platform's own SRM alerts, combined with a habit of checking traffic source breakdowns before trusting a test result. Dedicated bot management is worth it once you are running paid traffic at volume or see repeated SRM failures with no obvious cause.
The takeaway
A statistically significant result is not the same thing as a trustworthy one. In 2026, with automated traffic somewhere between a third and over half of all web sessions depending on how you count it, the traffic quality check has to happen before you read the lift number, not after you've already shipped it. Add an SRM check and a traffic-source review to your test launch checklist, and treat any result on a lower-traffic page with extra suspicion until you've confirmed both arms were actually seeing customers.