Finding and fixing thin and duplicate content on your website
How to spot thin and duplicate pages, then choose between improving, merging, canonicalising or noindexing them, including filter and tracking parameter URLs.
4 min read
Your website was rebuilt last year. Enquiries from Google have slowed, the content has not changed, and the developer says everything is fine. The cause is rarely one dramatic error. It is usually a handful of small ones that stop search engines finding, reading or trusting the pages that matter. A small business can find most of them in an afternoon with two free tools and a spreadsheet.
You need Google Search Console and a crawler. Search Console shows what Google has actually seen and indexed. A crawler shows what your site looks like to a bot that follows every link from the home page. The gap between those two views is where most problems sit.
Run the crawl from your home page with default settings. Then open the Page indexing report in Search Console and read the reasons listed for pages that are not indexed. Three patterns deserve attention.
Then use the URL Inspection tool on your five most important pages. It shows whether each page is indexed, which canonical URL Google selected and how the page rendered. If the rendered view is missing your main text, that content depends on scripts Google is not processing the way you expect.
Redesigns change URLs. When old addresses are not redirected, links from other websites and saved bookmarks land on error pages. Filter your crawl by response code and work through what it finds.
An XML sitemap should list only the URLs you want indexed: live pages that return a 200 status, carry no noindex tag and name themselves as canonical. Load the sitemap into your crawler and remove anything that redirects or errors. Resubmit it in the Sitemaps report. Google ignores the priority and changefreq fields, so skip them. An accurate lastmod date is worth keeping.
Next, open the Core Web Vitals report in Search Console. It groups similar URLs and rates each group on loading, responsiveness and visual stability, with mobile and desktop shown separately. Note each group rated poor or in need of improvement, with the template it uses, such as product pages, because a group usually fails for one shared reason. The data comes from real Chrome users, so a low-traffic site may show nothing. Record that too. Do not start fixing here. Our post on what actually slows a website down covers the fixes, cheapest first.
A crawler will report hundreds of warnings. Most will not affect your enquiries. A missing meta description on an old news item is not why your leads fell. Score each issue on two questions: does it stop a valuable page being crawled or indexed, and how many pages does it touch?
An audit is not a list of everything wrong. It is a short list of what stops your best pages being found.
After each batch of fixes, request indexing for the changed pages in URL Inspection and note the date in your sheet. Re-crawl a month later to confirm nothing has crept back. Then put the next audit in the calendar for the week after any redesign, platform migration or bulk content upload, because that is when these problems return.
Before your next redesign or plugin update, record the status code, canonical URL and index status of your ten most valuable pages, then check them again the week after.
Written by Fibvo — Bengaluru and Erode, India — working with clients in other time zones. Questions, or think we have it wrong? Tell us.
Keep reading
How to spot thin and duplicate pages, then choose between improving, merging, canonicalising or noindexing them, including filter and tracking parameter URLs.
4 min readA plan for a new site's first three months: getting indexed, choosing the first pages to publish, setting fair expectations and avoiding early mistakes.
4 min readHow to write a service page that ranks and converts: one keyword per page, a clear structure, proof from the service itself, useful FAQs and a quote request.
5 min readNext step
Send the problem in a paragraph. We will reply within one working day with an honest view of whether we can help, and what it is likely to cost.