Link score
PageRank over your internal links, from 0 to 100. Find the pages your site really promotes.
Features
Everything you need to grow a site.
A crawler that keeps memory flat, competitor research that shows you what to write, and 45 audits that tell you what to fix next. Free and open source.
Competitor Intelligence · New
See what your rivals cover and you don’t.
Add your site and up to five competitors. CrawlStat crawls them side by side, groups every page by meaning, and lists the topics, keywords and questions they cover that you miss or cover thinly. Each gap has a score, a funnel stage and the evidence behind it.
- Gaps by keyword, question, topic and content type
- Overlap, authority, freshness and AI readiness per rival
- Rechecks every month, so you see what changed
Content plan
From gaps to a plan you can hand over.
Opportunities are sorted into Learn, Compare and Buy, so you can see where your funnel is thin. One click turns any of them into a brief: an outline built from what competitors cover and what they leave out, ready for a writer.
- Learn, Compare and Buy columns
- Briefs built from real competitor pages
- A local AI can draft them, if you want
AI search readiness
Is your content easy for AI to quote?
Page check scores your pages and your rivals’ on what answer engines look for: a direct answer near the top, clear headings, definitions, questions and answers, authors, dates and FAQ markup. It measures what the markup shows, and says plainly that it cannot promise any AI will cite you.
- Your pages and each rival’s, side by side
- Fifteen signals, each as a share of pages
- Drafted changes to review and approve
Crawl
Watch every response land.
Live counters for crawled, queued and discovered URLs, errors, speed and time. robots.txt notices show up as they happen. Pause, resume or stop at any time, and pick up again after a crash. Built with React or Vue? Turn on Render JavaScript and each page runs in the Chrome, Edge or Brave you already have.
- Pause, resume, stop
- Optional JavaScript rendering in your own browser
- Crawl from a URL, a sitemap or a list
Pages
A table built for the long haul.
Eleven tabs (Internal, External, Response codes, Titles, Meta description, H1, Canonicals, Hreflang, Structured data, Images, Documents) over one fast table. Lenses ask the common questions for you: broken, redirected, slow, thin, not indexable, titles too long or too short. Each is counted before you click.
- Lenses with live counts: Broken, Slow, Thin content and more
- An Issues column with each page’s worst problem
- URL detail: issues, inlinks, outlinks, images, headers
Plan
Ten thousand rows is not a to do list.
Most crawlers end with a table sorted by severity and no answer to the one question you have. CrawlStat ranks the findings by how much of your site’s link value they touch and by how many pages one change fixes. Then it separates the two template edits from the three hundred single page ones.
- Impact from link value, not just severity
- Tells template fixes apart from page by page work
- A local model can write it up. Nothing is uploaded
UX testing
Does this page do its job?
Give CrawlStat one URL. It reads what the page is (a local business, a product, a SaaS signup, a lead form) and what it asks visitors to do, then tests it against that goal: clarity, layout, navigation, forms, trust, accessibility and the path to the goal. Each finding is ranked, with evidence and a fix.
- 62 checks, tailored to nine page types
- Every finding ranked, with evidence and a fix
- Retest after the change and see what now passes
Accessibility
WCAG 2.2 AA, automatic and by hand.
The rules a crawler can check run over every page: links and buttons with no name, fields with no label, broken ARIA references, missing lang, skipped heading levels. The ones that need a person get a checklist you work through, saved in the crawl. Test a staging URL before it ships.
- Automatic checks on every crawled page
- A manual WCAG 2.2 AA checklist, kept per crawl
- Check a single URL without a full crawl
Compare
Did the fix actually work?
Crawl, fix, recrawl in one click. CrawlStat matches pages across the two crawls and tells you what was fixed, what got worse and which pages started failing. A page that was simply not crawled again is reported as gone, never as fixed. Otherwise the way to a perfect score would be to delete your site.
- Pages matched across crawls, not by row number
- What got worse and newly broken pages first
- Fail a CI build when a release makes things worse
Issues
A to do list, not a wall of red.
Findings grouped by severity, then ranked by what they cost you: each one shows its impact, how much of the site it reaches and how much work it is. Every issue comes with a plain English explanation, how to fix it and the best linked pages it affects. Export them for whoever owns the fix.
- 45 SEO and accessibility audits
- Impact, reach and effort for every issue
- Template fixes flagged, so one change clears hundreds of pages
Projects
Every crawl, a file you own.
Each crawl is a single file on your disk. See recent crawls as cards, or one row per site with its health and a history of every crawl. Then open, recrawl, resume or delete. Updating CrawlStat improves what an old crawl reports, because the audits live in the app.
- One file per crawl, about 2 KB per URL
- Recrawl, or recrawl with changed settings
- Nothing uploaded, ever
Visualise
See the shape of the site.
A crawl is a map, so CrawlStat draws one. The section tree sets each folder’s share of pages against its share of link value, so a starved section stands out. The link graph puts every page in rings by clicks from home. And it tells you what to fix: here, 14% of the link value is spent on pages that can’t rank.
- Section tree: pages against link value
- Interactive link graph, in rings by click depth
- Structure problems, most important first
Client reports
A report your client will actually read.
Turn any crawl into a branded PDF or a web page in a single file: a health score, the headline figures, a ranked action plan with the effort each fix takes, accessibility barriers and every issue found. Add your name and colour, and show what improved since the last crawl.
- PDF or web page, made on your machine
- Your name and brand colour on the cover
- Progress since the last crawl
Under the hood
The details that add up.
Journeys
Walks your site from the home page like a visitor looking for pricing, support or a signup, and shows the path or where it gave up.
Topic map
Pages from every site grouped by meaning, coloured by where you stand: missing, weaker, covered, or only you.
Local AI
Models on your own machine write summaries, name topics and draft briefs. You see the exact prompt first, and nothing leaves the machine.
MCP server
AI agents can run competitor research through the same engine, using the Model Context Protocol.
Command line
Crawl, rank and compare from a terminal, and fail a CI build when a release makes things worse.
Redirect chains, hop by hop
Redirects are followed one at a time, so every hop and every loop is recorded. The chain is the finding.
Near duplicate pages
Finds pages that say the same thing in slightly different words, even across a very large site.
Hreflang checks
Missing return links, pages that do not list themselves, and invalid language codes across every alternate.
Recrawl in one click
Run a crawl again with the same settings, or change one thing first. Each run joins the site’s history.
Keyboard first
Arrows, Page Up and Down, Home and End, ⌘F to filter, ⌘E to export, ⌘N to crawl, ⌘Space to pause.
Polite by default
Follows robots.txt and Crawl delay, paces requests per host, and says who it is: CrawlStatBot.
Performance
Memory that plateaus.
The crawl queue lives on disk, not in RAM. Doubling a crawl from 30,000 to 60,000 URLs costs 1.19× the peak memory: about 717 bytes per extra URL. Crawl size becomes a question of time and disk.
Engine benchmark on an Apple Silicon laptop against a test site on the same machine. On real sites, speed is limited by the network and by how politely you crawl.
| URLs | Concurrency | URLs/sec | Peak RAM |
|---|---|---|---|
| 2,000 | 5 | 8,200 | 22 MiB |
| 30,000 | 10 | 9,600 | 111 MiB |
| 50,000 | 20 | 11,700 | 128 MiB |
| 60,000 | 10 | 10,600 | 131 MiB |
Free and open source
All of it, for everyone.
There is no paid plan and nothing held back. CrawlStat is MIT licensed, and it gets better every time someone tests it or sends a fix.
Every feature
Competitor Intelligence, UX testing, crawl comparison and client reports. Nothing is locked.
No URL limit
Crawl a site of 50 pages or 500,000. Size is a question of time and disk, not of a plan.
No account, no key
Download it and use it. There is nothing to sign up for and nothing to activate.
MIT licensed
Use it for paid client work, change it, share it. Just keep the copyright notice.
Start crawling in a minute.
Download CrawlStat, paste a URL, press Enter. Free and open source, with every feature. No account, no key, no limits.