Meta's new commercial terms let it crawl your website for AI from October 30. Here's what to check before then

Meta's updated Commercial Terms take effect on 30 October 2026, and the new wording authorises Meta to send web scrapers, bots, robots, spiders, crawlers and user agents to the websites tied to any business using its advertising or business products, Pages, Instagram professional accounts and ad accounts included. There is no separate opt-out: continuing to use Meta's commercial products after that date counts as accepting the terms. What is still unresolved, and what matters more than the clause itself, is whether "crawl" also means "train Meta's AI on your content." An early version of the terms listed training among the stated purposes, Meta reportedly dropped that word on 17 October after publisher backlash, and coverage a day later found the training language still present in at least one edition of the terms. Here is what is confirmed, what is disputed, and the check to run on your own site before the 30th.

What Meta's new Commercial Terms actually say

Meta's Commercial Terms govern anyone who uses its products for a commercial or business purpose, not only advertisers in the strict sense. The version that has stood since 6 March 2024 is being replaced by one that takes effect 30 October 2026, and it defines that scope broadly: the terms apply if you access or use Meta's products for advertising, operating an app on its platform, using its measurement services, managing a group or a Page for a business, or selling goods or services. If you run Meta or Instagram ads, manage a business Page, or sell through a shop connected to Meta's products, the new terms cover you, whether or not you've ever thought of your performance marketing account as something that touches your website's crawl settings.

The clause drawing the attention is new. Reporting on the terms describes language under which every business using Meta's products authorises Meta to send web scrapers, bots, robots, spiders, crawlers and user agents to the websites associated with its products or services, plus any URLs made available through that business's use of Meta's products or through its users. There is no separate opt-out: continuing to use Meta's commercial products after the terms take effect counts as accepting them, the same mechanism that governs every other change to Meta's terms. Meta's own spokesperson, Andy Stone, has said the intent is to let Meta pull accurate information about products for sale on third-party websites when it makes ad and content recommendations on its own platforms, calling it "not about scraping news sites."

The word "train": added, reportedly removed, reportedly still there

What that crawling clause actually authorises Meta to do with what it collects is where the story is still moving, and where you should be careful about treating any single report as settled. The timeline so far, pieced together from coverage published this month:

Meta's two named crawlers, and the gap between them and this clause

Meta already documents two crawlers by name, and neither one maps cleanly onto the new clause, which is worth knowing before you assume checking your robots.txt against the usual suspects covers you.

Both of those are Meta's existing, documented crawlers, built for jobs it has already described publicly. The new Commercial Terms clause is broader than either: it is a contractual authorisation tied to using Meta's products for business, not a description of one crawler's job. Nothing in the coverage of the new terms confirms which crawler, if any, Meta will actually point at business websites under this new authority, or whether a third one is coming. A robots.txt rule that blocks meta-externalagent blocks the crawler Meta has told you trains its models. It does not guarantee it blocks whatever the Commercial Terms clause turns out to authorise in practice. Checking and editing that file is a basic website development task, not a legal one, which is exactly why it is worth doing regardless of how the training dispute resolves.

Does this actually affect you?

• Running Meta or Instagram ads, in any amount: yes, in scope.
• Managing a Facebook Page or an Instagram professional account for a business: yes, when it is used for a commercial purpose.
• Selling through a shop or catalogue connected to Meta's products: yes.
• Building or maintaining an app on Meta's platform: yes.
• A personal profile with no advertising, Page or selling activity: no, per Meta's stated scope, though it is worth re-checking against the live terms rather than assuming.

Should you actually try to block it?

Blocking isn't free, and it isn't obviously the safe default either, which is the part most coverage of this story skips. If you sell products that Meta surfaces in its own shopping and ad-recommendation features, the stated purpose of this clause, pulling accurate product information from third-party sites, is meant to help you, not just extract from you. Blocking the crawler that reads your product pages could make Meta's own representation of your business less accurate on its own platforms. If your website's value is the content itself, guides, pricing pages, research you don't want folded into a model with no attribution, the calculus runs the other way.

How to check and block Meta's crawler, step by step

1. Open yourdomain.com/robots.txt in a browser and check whether it already names meta-externalagent or meta-externalfetcher.
2. Add a disallow rule if you want to block the training and indexing crawler: a User-agent: meta-externalagent block with Disallow: /, or specific paths if you only want to protect some pages.
3. Leave meta-externalfetcher alone unless you also want to block user-requested link previews and agentic features that rely on it, understanding Meta's own documentation says it may disregard the rule for those specific, user-triggered fetches regardless.
4. Allow up to 24 hours, then recheck that your robots.txt is live and review your server logs for requests still arriving from either user agent string.
5. Put 30 October 2026 on your calendar as the date to re-read Meta's live Commercial Terms directly, not just this post or any single report, since the training-clause dispute was still unresolved as of 18 October.

Frequently asked questions

What exactly did Meta change in its Commercial Terms?

Meta's Commercial Terms, which replace the version dated 6 March 2024 and take effect 30 October 2026, add a clause authorising Meta to send web scrapers, bots, robots, spiders, crawlers and user agents to the websites and URLs tied to any business's use of its products. Early versions of the terms also listed training Meta's AI among the stated purposes for using that crawled content.

Does this apply to me if I only run Facebook or Instagram ads?

Yes. The terms apply to anyone who uses Meta's products for a commercial or business purpose, which Meta defines to include advertising, operating an app on its platform, using its measurement services, managing a group or Page for a business, or selling goods or services. Running ads alone is enough to bring your linked website into scope.

Can Meta override my robots.txt file under these terms?

Nobody has confirmed that with certainty, and that is itself the problem. The terms do not mention robots.txt, and legal commentators covering the change have warned the contractual language could be read to authorise crawling regardless of a technical block, since continued business use of Meta's products after the effective date counts as accepting the terms. Treat robots.txt as a best-effort technical signal, not a guaranteed legal shield, until Meta states plainly how the two interact.

Is Meta still planning to use crawled content to train its AI?

That is unresolved as of this writing. Meta reportedly removed the word train from the list of purposes in its Commercial Terms on 17 October 2026 after publisher backlash, with a spokesperson saying the crawling terms only applied to its ads and business products. A day later, coverage reported that at least one regional edition of the terms still carried the training language, and that a separate licensing clause elsewhere in the terms may cover the same use regardless of that one word. Check the live terms yourself before the 30 October effective date rather than relying on any single report.

How do I block Meta's crawler if I don't want it on my site?

Add a disallow rule for the user agent token meta-externalagent in your robots.txt file, which is the crawler Meta documents as training its models and indexing content. Meta's own documentation describes a separate crawler, meta-externalfetcher, as fetching individual links at a user's request, with a stated exception that lets it disregard robots.txt for those specific fetches. Changes to robots.txt can take up to 24 hours to take effect, and neither crawler's documented behaviour has been tied explicitly to the new Commercial Terms clause, so recheck your server logs after making the change rather than assuming the rule worked.

The takeaway

Meta's new Commercial Terms do one thing clearly: they put a crawling authorisation into a contract that every advertiser and business-Page owner already has to accept to keep using Meta's products, with no separate opt-out. What Meta does with that authorisation, specifically whether "train Meta's AI" survives in the version that actually goes live on 30 October, was still moving as of this writing, with reports disagreeing by the day. Don't resolve that uncertainty by guessing. Read the live terms before the 30th, check your robots.txt against Meta's two documented crawlers, and decide deliberately whether blocking costs you product visibility on Meta's own platforms or costs you nothing, because that answer is specific to your business, not a single rule any one article, including this one, can hand you.