[{"data":1,"prerenderedAt":413},["ShallowReactive",2],{"blog-\u002Fblog\u002Fcrawlstat-is-open-source":3,"blog-more-\u002Fblog\u002Fcrawlstat-is-open-source":386},{"id":4,"title":5,"author":6,"body":7,"cover":374,"date":375,"description":376,"extension":377,"meta":378,"navigation":379,"path":380,"readingTime":381,"seo":382,"stem":383,"tag":384,"__hash__":385},"blog\u002Fblog\u002Fcrawlstat-is-open-source.md","CrawlStat is now free and open source","The CrawlStat team",{"type":8,"value":9,"toc":358},"minimark",[10,22,27,30,52,55,59,66,72,78,84,88,91,111,114,121,127,133,139,144,154,159,165,170,183,187,205,212,216,219,223,230,234,242,256,259,263,274,303,311,315,318,322,325,329,351,354],[11,12,13,14,21],"p",{},"Today CrawlStat becomes free and open source, under the MIT licence. The code for the crawl engine and the desktop app is on ",[15,16,20],"a",{"href":17,"rel":18},"https:\u002F\u002Fgithub.com\u002Fclotairedamy\u002FCrawlStat",[19],"nofollow","GitHub",". You can read it, build it, change it and share it.",[23,24,26],"h2",{"id":25},"everything-is-free-now","Everything is free now",[11,28,29],{},"There is no paid plan anymore. Every feature is included for everyone:",[31,32,33,37,40,43,46,49],"ul",{},[34,35,36],"li",{},"unlimited URLs in every crawl,",[34,38,39],{},"every audit, on every page,",[34,41,42],{},"UX Testing,",[34,44,45],{},"crawl comparison,",[34,47,48],{},"progress in client reports,",[34,50,51],{},"and the new Competitor Intelligence.",[11,53,54],{},"There's no account to make, no key to paste, nothing to activate and no limit on how many URLs a crawl can fetch. You download it and use it.",[23,56,58],{"id":57},"why-we-did-it","Why we did it",[11,60,61,65],{},[62,63,64],"strong",{},"A tool that reads your sites should be one you can read too."," A crawler reads your sites, and often your clients' sites too. Now you can check exactly what it fetches, what it stores and where it keeps it. (On your disk, and nowhere else.)",[11,67,68,71],{},[62,69,70],{},"Trust is easier to give when you can check."," When you point a tool at a client's site, \"trust us\" isn't a great answer. \"Here's the code\" is a better one.",[11,73,74,77],{},[62,75,76],{},"It runs on your machine anyway."," CrawlStat never needed our servers to crawl. The only part that ever talked to us was the licence check, and the official build doesn't have one anymore.",[11,79,80,83],{},[62,81,82],{},"More people can make it better."," We're a small team. SEOs find bugs we'd never think of, on sites we'd never see. Developers can fix them. Open source makes it easy for both to help.",[23,85,87],{"id":86},"whats-new-competitor-intelligence","What's new: Competitor Intelligence",[11,89,90],{},"Competitor Intelligence is the biggest new feature since launch. It compares your site with up to five competitors and shows you what they cover that you don't.",[11,92,93,94,98,99,102,103,106,107,110],{},"You add your site and your competitors, and CrawlStat crawls them side by side. It stays polite: two requests at a time per site, and ",[95,96,97],"code",{},"robots.txt"," is respected. You choose how deep to go. ",[62,100,101],{},"Quick"," reads 200 pages per site (about a minute), ",[62,104,105],{},"Standard"," reads 500 (about three minutes) and ",[62,108,109],{},"Thorough"," reads 2,000 (about nine minutes).",[11,112,113],{},"The screenshots below show a local electrician compared with four competitors. We blurred the real site names.",[11,115,116,117,120],{},"The ",[62,118,119],{},"Competitors"," tab puts each site next to yours: its pages, topics, the topics only it covers, how much it overlaps with you, an authority estimate, freshness, AI readiness and the opportunities it shows.",[11,122,116,123,126],{},[62,124,125],{},"Gaps"," tab lists what competitors cover that you're missing, or cover only thinly. Each gap comes with the evidence, so you can see why it's there. You can filter by keywords, questions, long tail phrases, names, topics and content types, and by stage: Learn, Compare or Buy.",[128,129],"screenshot",{"alt":130,"caption":131,"src":132},"The Gaps tab of Competitor Intelligence for a local electrician, listing opportunities with a score, a stage, how many competitors cover each one and a status such as Missing","Gaps shows what competitors cover that you don't, with the evidence one click away.","\u002Fscreenshots\u002Fintel-gaps.png",[11,134,116,135,138],{},[62,136,137],{},"Topics"," tab groups pages from every site by meaning. You can see the topics as a map or as a list, coloured by where you stand: you're missing it, you're weaker, you cover it, or only you cover it.",[128,140],{"alt":141,"caption":142,"src":143},"The Topics tab as a list, with each topic coloured by where you stand and showing how many of your pages and how many pages in all it holds","Topics groups pages from every site by meaning and shows where you stand on each one.","\u002Fscreenshots\u002Fintel-topics.png",[11,145,116,146,149,150,153],{},[62,147,148],{},"Content plan"," tab sorts the opportunities into three columns: Learn, Compare and Buy. Each one has a ",[62,151,152],{},"Create brief"," button. It writes an outline that's ready to hand to a writer, built from what competitors cover and what they leave out.",[128,155],{"alt":156,"caption":157,"src":158},"The Content plan tab with opportunities sorted into Learn, Compare and Buy columns, each card with a Create brief button","The Content plan turns gaps into briefs you can hand to a writer.","\u002Fscreenshots\u002Fintel-content-plan.png",[11,160,116,161,164],{},[62,162,163],{},"Page check"," tab is a scorecard for AI search readiness, for your pages and theirs. It looks for direct answers near the top, clear headings, FAQ markup, an author, dates and more. It measures what the markup shows. It can't tell you whether an AI will actually cite your page.",[166,167,169],"h3",{"id":168},"local-ai-if-you-want-it","Local AI, if you want it",[11,171,172,173,178,179,182],{},"If you run ",[15,174,177],{"href":175,"rel":176},"https:\u002F\u002Follama.com",[19],"Ollama",", CrawlStat can use it on your machine. A small embedding model (",[95,180,181],{},"all-minilm",") matches gaps by meaning, so \"cost breakdown\" and \"pricing\" aren't counted as a gap just because the words differ. A local chat model can name the topics and draft the briefs. Both are optional.",[166,184,186],{"id":185},"a-few-more-things","A few more things",[31,188,189,192,199,202],{},[34,190,191],{},"Competitors can be rechecked every month, automatically.",[34,193,194,195,198],{},"The same tools are open to AI agents through an MCP server: ",[95,196,197],{},"crawlstat-cli intel mcp",".",[34,200,201],{},"All of it runs on your machine.",[34,203,204],{},"Scores are estimates, and the app labels them that way.",[11,206,116,207,211],{},[15,208,210],{"href":209},"\u002Fdocs\u002Fcompetitor-intelligence","Competitor Intelligence guide"," walks through every tab.",[23,213,215],{"id":214},"how-you-can-help","How you can help",[11,217,218],{},"Here's where help counts most right now.",[166,220,222],{"id":221},"test-it-on-your-own-sites","Test it on your own sites",[11,224,225,226,229],{},"Crawl the sites you know best and tell us when something looks wrong. We especially need testers on ",[62,227,228],{},"macOS and Windows",". The Linux installers are out, but the macOS and Windows installers aren't published yet. For now, you can build CrawlStat from source on those platforms. The README on GitHub explains how.",[166,231,233],{"id":232},"report-bugs","Report bugs",[11,235,236,237,241],{},"Open an issue on ",[15,238,20],{"href":239,"rel":240},"https:\u002F\u002Fgithub.com\u002Fclotairedamy\u002FCrawlStat\u002Fissues",[19],". It helps a lot if you include:",[31,243,244,247,250,253],{},[34,245,246],{},"the CrawlStat version,",[34,248,249],{},"your platform (macOS, Windows or Linux, and which version),",[34,251,252],{},"the smallest site that shows the problem,",[34,254,255],{},"what you expected, and what happened instead.",[11,257,258],{},"Please only crawl sites you own or have permission to test.",[166,260,262],{"id":261},"write-code","Write code",[11,264,265,266,269,270,273],{},"The crawl engine is written in Rust and lives in ",[95,267,268],{},"engine\u002F",". The desktop app is written in Flutter and lives in ",[95,271,272],{},"app\u002F",". The logic lives in Rust, and the app shows the results. Before you open a pull request, please run:",[275,276,281],"pre",{"className":277,"code":278,"language":279,"meta":280,"style":280},"language-sh shiki shiki-themes github-light","cargo test\ncargo clippy --all-targets\ncd app && flutter analyze && flutter test\n","sh","",[95,282,283,291,297],{"__ignoreMap":280},[284,285,288],"span",{"class":286,"line":287},"line",1,[284,289,290],{},"cargo test\n",[284,292,294],{"class":286,"line":293},2,[284,295,296],{},"cargo clippy --all-targets\n",[284,298,300],{"class":286,"line":299},3,[284,301,302],{},"cd app && flutter analyze && flutter test\n",[11,304,116,305,310],{},[15,306,309],{"href":307,"rel":308},"https:\u002F\u002Fgithub.com\u002Fclotairedamy\u002FCrawlStat\u002Fblob\u002Fmain\u002FCONTRIBUTING.md",[19],"contributing guide"," has the rest.",[166,312,314],{"id":313},"improve-the-docs-and-this-website","Improve the docs and this website",[11,316,317],{},"If a help page confused you, it will confuse someone else too. Fixes and ideas for the docs and for this website are just as welcome as code.",[166,319,321],{"id":320},"star-the-repo","Star the repo",[11,323,324],{},"A star on GitHub takes one click, and it helps more people find CrawlStat.",[23,326,328],{"id":327},"get-started","Get started",[31,330,331,337,344],{},[34,332,333,334,198],{},"Read the code, and star it, on ",[15,335,20],{"href":17,"rel":336},[19],[34,338,339,340,198],{},"See every way to help on the ",[15,341,343],{"href":342},"\u002Fcontribute","contribute page",[34,345,346,350],{},[15,347,349],{"href":348},"\u002Fdownload","Download CrawlStat"," and point it at your own site.",[11,352,353],{},"Thank you for crawling with us.",[355,356,357],"style",{},"html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}",{"title":280,"searchDepth":293,"depth":293,"links":359},[360,361,362,366,373],{"id":25,"depth":293,"text":26},{"id":57,"depth":293,"text":58},{"id":86,"depth":293,"text":87,"children":363},[364,365],{"id":168,"depth":299,"text":169},{"id":185,"depth":299,"text":186},{"id":214,"depth":293,"text":215,"children":367},[368,369,370,371,372],{"id":221,"depth":299,"text":222},{"id":232,"depth":299,"text":233},{"id":261,"depth":299,"text":262},{"id":313,"depth":299,"text":314},{"id":320,"depth":299,"text":321},{"id":327,"depth":293,"text":328},null,"2026-11-04","Every feature is now free for everyone, with no account, no key and no limits. The code is on GitHub under the MIT licence. Here's why we did it, what's new, and how you can help.","md",{},true,"\u002Fblog\u002Fcrawlstat-is-open-source",5,{"title":5,"description":376},"blog\u002Fcrawlstat-is-open-source","News","gMUrIWlqcKFDkpoRdtvZR1gQ0lBBAVSEWd_ZtIR6Cm0",[387,388,394,399,404,409],{"title":5,"path":380,"date":375,"tag":384,"readingTime":381},{"title":389,"path":390,"date":391,"tag":392,"readingTime":393},"Client reports, JavaScript rendering and a new way around a crawl","\u002Fblog\u002Fclient-reports-and-javascript-rendering","2026-10-26","Announcement",4,{"title":395,"path":396,"date":397,"tag":392,"readingTime":398},"Introducing CrawlStat","\u002Fblog\u002Fintroducing-crawlstat","2026-10-21",6,{"title":400,"path":401,"date":402,"tag":403,"readingTime":398},"The database is the frontier: how CrawlStat keeps memory flat","\u002Fblog\u002Fflat-memory-crawling","2026-10-14","Engineering",{"title":405,"path":406,"date":407,"tag":408,"readingTime":398},"Redirect chains: the chain is the finding","\u002Fblog\u002Fredirect-chains-the-chain-is-the-finding","2026-10-07","SEO guide",{"title":410,"path":411,"date":412,"tag":403,"readingTime":381},"Licensing without a database","\u002Fblog\u002Flicensing-without-a-database","2026-09-30",1793827598332]