Free

Free coded apps to suit all. No sign up, no email, usable offline, updates when I can, feedback welcome. Send feedback

How-to

How to check if your site shows Googlebot something else

Turn on the Googlebot comparison in SEO by Zaweba's Settings and every page you analyse is also fetched as Googlebot and compared with the normal fetch.

How to check if your site shows Googlebot something else

Cloaking is when a site serves search engines different content from what people see. SEO by Zaweba can check for it: switch on the optional Googlebot comparison in Settings, under Crawling manners, and every page you analyse is fetched a second time as Googlebot. The two fetches are compared, so you can see whether Google would be served something different.

Most site owners never cloak on purpose. The check is mainly useful for finding things done to your site without your knowledge, or by a plugin you forgot about.

Why you might be cloaking without meaning to

Hacked sites are the classic case. Some attacks inject spam links or whole pages that only appear when the visitor says it is Googlebot, so the owner browsing normally sees nothing wrong while search results fill with junk. Other causes are innocent but still risky: a caching or optimisation plugin that serves a stripped version to crawlers, a geolocation redirect that treats crawlers differently, or a bot filter that blocks or redirects them.

Switching the check on

Open Settings and find Crawling manners. Alongside the user agent the app normally uses (an honest SEObyZaweba user agent, with a button to restore the default), the pause between requests, the time-out and the JavaScript rendering options, there is the optional Googlebot comparison fetch. Turn it on, then run an audit as usual.

The normal fetch is made by the single page crawler, which records the redirect chain, status, timings, size, compression, headers and TLS details. The Googlebot fetch is made alongside it for comparison. The app's pause between requests still applies, so the extra fetch stays polite.

Reading the comparison

If both fetches return the same page, the check passes and you can move on. If they differ, look at how. A different status code or a redirect only for Googlebot is a strong sign that something on the server treats crawlers differently. Extra links or text in the Googlebot version, especially links to unrelated sites, point towards injected spam. Less content for Googlebot can mean a plugin or firewall is cutting crawlers off.

It helps to read this beside the JavaScript rendering comparison, which compares the raw HTML with the rendered page. That one is about content that only exists after scripts run, which is a different issue from cloaking, but both tell you what a crawler is likely to understand about the page.

What to do if it fails

If you didn't set up the difference, treat it as a possible compromise. Check the server and any plugins, look at recently changed files, and talk to your host. If a plugin or a firewall rule is responsible, change its settings so crawlers get the same page as everyone else.

The Googlebot fetch only tells you what your server returns when asked by that name. Real Googlebot comes from Google's own addresses, and a server could in theory treat those differently again. If you suspect that, the log analyser on the Logs and Search Console page verifies real Googlebot visits by DNS and shows which addresses it requested, which is the other half of the picture.

Once the problem is fixed, run the audit again with the comparison still on. SEO by Zaweba keeps every scan, so the History page will show the check moving from fail to pass.