Google rolls out Publisher Center controls to claw back AI
Google also quietly expanded the Google-Extended user agent. Blocking it via robots.txt already keeps your content out of Bard/Gemini training, but the new docs confirm it also opts you out of AI Overview citations without removing you from classic blue-link results. For publishers who want search visibility but not the zero-click summary, that's the lever they've been asking for since May.
Implementation is straightforward. Add this to robots.txt:
User-agent: Google-Extended
Disallow: /Wait 24-48 hours for recrawl. You'll still appear in standard results; the overview card just stops quoting you. Early tests on a mid-size tech blog showed a 12% lift in CTR for affected keywords the week after flipping the switch — small sample, but the direction matters.
There's a catch. The opt-out is page-level, not site-wide. You'll need to manage it per directory or template if you run a large property. Publisher Center's new bulk-actions UI helps, but it's still manual for now. No API, no Search Console integration — yet.
Separately, Google started surfacing "AI Overview referral" as a distinct traffic source in GA4. Look under Acquisition → Traffic acquisition → Session source/medium. It shows up as google / ai-overview. If you're not seeing it, your property either hasn't been cited or the rollout hasn't hit your niche. Travel, health, and finance verticals are first.
Worth noting: the controls don't touch SGE experiments outside the U.S., and they don't affect Bing Copilot or Perplexity citations. Each engine still needs its own robots.txt directive.
Bottom line — publishers finally have visibility and a toggle. Not a full fix, but the first real agency we've had since AI Overviews went broad.