Finding duplicate titles and thin content on a site

Duplicate titles and thin pages tend to creep into any site that has grown for a few years, and SEO by Zaweba lists both after a crawl of your site. Its site checks flag pages that share a title or a meta description, pages whose content is nearly the same as another page (found with a SimHash comparison), and thin pages with little real content.

You get the lists from the same guided audit you use for a single page, with your own site crawl switched on.

Setting up the crawl

The site crawl is offered in step 4 of the guided audit. It reads robots.txt and your XML sitemaps, then crawls breadth first up to the page limit, 500 pages for your own site by default. If your site is larger, raise the limit in Settings under Limits before you start, so the comparison covers the pages that matter. The crawl is polite: it obeys robots.txt and waits between requests.

What each problem means

A duplicate title means two or more pages show the same headline in search results. Searchers can't tell them apart, and the pages compete with each other for attention. Duplicate descriptions are a smaller problem, but they waste the one line of text you control under each result.

Near duplicates are pages whose main content is almost the same, even if the titles differ. They are common on shops with product variants and on service sites that cloned one page per town. The SimHash comparison catches them without you reading every page side by side.

Thin pages are pages with very little main content. The app extracts the main content from each page, leaving out menus and footers, so a page full of navigation still counts as thin if there is not much of its own.

Working through the list

Open the Site pages tab after the scan. Start with duplicate titles, because they are quick to fix and the fix is clear: give each page a title that says what is different about it. The single page checks also estimate title length in characters and pixels, which helps when you rewrite.

For near duplicates, decide which page should be the one that ranks. Merge the others into it, rewrite them so each has its own purpose, or point a canonical at the main page. Canonical conflicts are flagged in the same site checks, so you will see if a page's canonical disagrees with how it is linked.

For thin pages, ask whether they need to exist. Some should be expanded with real detail, some should be merged, and a few can go. When pages compete for the same term, the keyword cannibalisation check will often list them too.

Keeping it fixed

The To-do list keeps the tasks the app suggests, and you can mark it done, defer it or dismiss it with a reason. At the next scan, SEO by Zaweba checks the tasks marked done and reopens any whose check still fails. File, Export table as CSV saves the table you are looking at if you would like to share the list with whoever writes the pages.