An SEO glossary of terms every marketer should know in 2026
A practical, current glossary covering the essential SEO terminology every marketer should be comfortable with, from foundational concepts to 2026-specific terms.
6 min read
A crawlability audit answers a deceptively simple question: can Google actually reach every page you want indexed? Content quality is irrelevant if Google cannot access the page at all — crawlability is the foundational layer every other SEO effort depends on.
Step one: check for blanket blocking issues
Start with the basics — confirm robots.txt is not accidentally blocking important sections, and check for any site-wide noindex meta tags left over from a development environment. These blanket issues are rare but catastrophic when present.
- Check robots.txt directly at yourdomain.com/robots.txt for overly broad Disallow rules.
- Search for site-wide noindex meta tags, particularly after any recent migration or redesign.
- Confirm the site is not password-protected or otherwise inaccessible to crawlers.
Step two: crawl your own site like Google would
Use a crawling tool to simulate how Googlebot navigates your site, following links exactly as a crawler would. Compare the resulting page count against your CMS's actual published page count — a significant gap reveals pages that exist but are not reachable through normal internal linking.
The gap between how many pages you think you have and how many a crawler can actually reach is often where the real crawlability problem lives.
Step three: check for orphaned pages
Pages with no internal links pointing to them are effectively invisible to normal crawling, discoverable only through a sitemap submission — and even then, less likely to be prioritised. See our orphan pages guide for the full diagnostic and fix process.
Step four: check redirect chains and broken links
- Identify and fix redirect chains requiring more than one hop.
- Find and fix internal links pointing to 404 pages.
- Check for soft 404s — pages returning a 200 status but showing 'not found' content, confusing crawlers about the page's actual status.
Step five: verify JavaScript-dependent content renders correctly
If your site relies on JavaScript for navigation or content, use Search Console's URL Inspection tool to confirm the rendered version actually includes the links and content you expect — see our JavaScript SEO guide for more depth on this specific risk.
Frequently asked questions
How often should a crawlability audit be repeated?
After any major site change (redesign, migration, CMS switch) at minimum, and as a general health check every few months for actively maintained sites.
Can a page be crawlable but still not indexed?
Yes — crawlability is a prerequisite for indexing, not a guarantee of it. A crawlable page can still be excluded from the index for quality or duplication reasons.
What is the fastest way to check a single page's crawlability?
Search Console's URL Inspection tool gives a direct, authoritative answer for any individual URL, showing exactly how Google last crawled and processed it.
If you suspect crawlability issues are limiting your site's visibility, contact our team for a full audit.
The takeaway
Run a full site crawl with a tool like Screaming Frog and compare the results against your CMS's actual page count — any significant gap tells you exactly where crawlability is breaking down.
Written by Fibvo — Bengaluru and Erode, India — working with clients in other time zones. Questions, or think we have it wrong? Tell us.
Keep reading
Related articles
A hybrid SEO strategy for a world with both blue links and AI answers
A practical framework for building one content strategy that serves both traditional search ranking and AI answer engine visibility simultaneously.
6 min readGoogle's Quality Rater Guidelines explained, in plain language
What the Quality Rater Guidelines actually are, how human raters use them, and the practical lessons worth extracting from this lengthy public document.
5 min readNext step
Tell us what you are trying to grow.
Send the problem in a paragraph. We will reply within one working day with an honest view of whether we can help, and what it is likely to cost.