Getting started
What CrawlStat is, what it does on your machine, and how to go from a fresh install to your first list of issues in a few minutes.
What CrawlStat is
CrawlStat is a desktop SEO crawler. You give it a website and it works through the site the way a search engine would. It follows links, records what each URL returns, and reads each page's titles, descriptions, headings, canonicals, hreflang, structured data, images and links. When the crawl finishes, it runs 45 audits over everything it found and gives you a list of problems, each with an explanation and a fix.
It runs on macOS, Windows and Linux. Everything happens on your own computer:
- Your crawl data stays local. Each crawl is saved as a single file on your disk. Nothing about the sites you crawl is uploaded anywhere.
- It is polite by default. CrawlStat respects robots.txt, identifies itself honestly, and keeps the number of simultaneous requests modest.
- It handles big sites. Crawl data goes to disk rather than memory, so a large crawl doesn't slow your machine down. An interrupted crawl can pick up where it stopped.
Quick start
- Install CrawlStat. Download the build for your operating system from the downloads page and install it. See Installing CrawlStat if you need help for your platform.
- Start a crawl. Click New crawl, the + at the top of the sidebar (or press ⌘N on macOS). Type the site's address, for example
example.com, and click Start crawl. - Read the issues. When the crawl finishes, CrawlStat runs its audits automatically. Open the crawl's Issues tab to see what it found, ranked within errors, warnings and notices, each with a "How to fix" note.
- Share it. Click Export a client report in the crawl's header for a branded PDF you can send as it is.
That's all you need for a first look. The rest of this guide covers each step in more detail.
Finding your way around
The sidebar on the left holds the things that aren't about one crawl:
| Item | What it's for |
|---|---|
| Projects | Every crawl you've run, as cards or one row per site, with health, history and issue counts. Open, recrawl, compare, resume or delete crawls here. |
| The live crawl | Only while a crawl runs: its site, progress and a Live or Paused badge. Click it to get back to the crawl. |
| Accessibility | Check pages against WCAG. See Accessibility. |
| Competitor Intelligence | Compare your site with up to five competitors: what they cover that you don't, a content plan, and AI search readiness. See Competitor Intelligence. |
| UX Testing | Test a page against what it's for. See UX Testing. |
| Settings | Defaults for new crawls, the theme and colour, and where your files are kept. |
Everything about one crawl lives on that crawl's own page, which you open from Projects. Its header says which crawl it is and has Export a client report, Compare and Recrawl. Its tabs are:
| Tab | What it's for |
|---|---|
| Overview | The live view while the crawl runs; afterwards, its health, headline figures and top issues. |
| Issues | The audit findings, ranked, with why each one ranks where it does and the pages it affects. |
| Pages | Every URL in the crawl, in a sortable, filterable table with eleven tabs and quick lenses. |
| Plan | What to fix first, and a written report. |
| Visualise | How the site is structured and linked. |
| UX Testing | Testing the crawl's pages for the people who use them. |
| Accessibility | The crawl against WCAG 2.1 AA. |
What CrawlStat checks
The 45 audits cover the things that usually go wrong on a website:
- Responses: server errors, broken pages, fetch failures, redirect chains and loops, and slow responses.
- Links: broken internal links, links that point at redirects, orphan pages, and pages buried deep in the site.
- On the page: missing, duplicate, too long or too short titles and meta descriptions; missing or multiple H1s.
- Indexing: noindex pages, pages blocked by robots.txt, canonical problems, and hreflang errors.
- Content: exact and near duplicate pages.
- Images: missing alt text and oversized files.
- Security: mixed content and HTTP links on HTTPS sites.
- Accessibility: links and buttons without accessible names, unlabelled form fields, broken ARIA references, missing page language, and more.
Each audit has its own page under /docs/audits/, with the same explanation and fix that you see in the app. For example, see Broken internal links or Duplicate page titles.
Free and open source
CrawlStat is free and open source, under the MIT licence. Every feature is included: unlimited URLs, every audit, JavaScript rendering, client reports, CSV export, UX Testing, crawl comparison and Competitor Intelligence. There's no account and no key. See Open source and licence.
Where to go next
- Your first crawl: every option in the New crawl dialog, explained.
- Watching a crawl: what the live counters mean.
- The Pages table: finding your way through the data.
- Issues: reading and acting on audit findings.
- Client reports: sending it all to a client.
- Competitor Intelligence: seeing what your competitors cover that you don't.