Open-source · Free forever

The most advanced
open-source SEO crawler

FreeCrawl is a free, open-source desktop SEO crawler with JavaScript rendering, 200+ on-page checks, GSC / GA4 / Ahrefs / Moz / Semrush integrations, AI-assisted analysis, and unlimited URLs. No subscription. No telemetry. No vendor lock-in.

Download for Windows · macOS · Linux See it vs Screaming Frog
  • 200+ checks
  • 23 reports
  • 1M+ URLs
  • 16 panels
  • 0 telemetry
Comparison

Screaming Frog vs FreeCrawl, side by side.

Both are professional SEO crawlers. Screaming Frog is free up to 500 URLs, then £199 a year. FreeCrawl is free and unlimited.

Closed-source · subscription
Screaming Frog £199 / year
SEO Spider · 500 URLs free, then licensed
Open-source · free forever
FreeCrawl $0 · forever
Unlimited URLs · Windows · macOS · Linux
MIT-style license
CapabilityScreaming FrogFreeCrawl
Price for unlimited crawling£199 / year$0 — free forever
Free-tier crawl limit500 URLsUnlimited
JavaScript renderingYes (free, up to 500 URLs)Yes · no URL limit
Custom extraction (CSS + regex)Yes (free, up to 500 URLs)Yes · unlimited
GSC / GA4 / PageSpeed InsightsYes (free, up to 500 URLs)Built-in · unlimited
Scheduled crawls & crawl comparisonYesYes
AI-assisted analysis (LLM per URL)Not availableOpenAI + Anthropic + Ollama
MCP server (Claude Code)Not availableBuilt-in
Encrypted project filesNot availableAES-256-GCM .seoproject.enc
TelemetryAnonymous usage dataNone. Ever.
Source codeClosed-sourceOpen-source · MIT-style
PlatformsWindows / macOS / LinuxWindows / macOS / Linux

Comparison based on Screaming Frog’s published pricing and feature pages (screamingfrog.co.uk), verified May 2026: the free version is feature-complete up to a 500-URL crawl limit, with unlimited crawling licensed at £199/year. Features and prices may change — check screamingfrog.co.uk for current details. FreeCrawl is an independent project, not affiliated with, endorsed by, or sponsored by Screaming Frog Ltd. "Screaming Frog" and all other trademarks belong to their respective owners.

Download FreeCrawl — it reads .csv URL lists
A closer look

The whole tool, on one screen.

The same dense, professional layout SEO practitioners already know — a top bar, 30 tabs, a virtualized URL table, a 16-sub-tab detail panel, and a live status bar. Rebuilt with modern keyboard ergonomics and a native dark theme.

FreeCrawl SEO Tool
—▢×
FileViewVisualizationReportsHelp
FreeCrawl
example.com
Internal
External
Response Codes
URL
Page Titles
Meta Description
H1
H2
Content
Images
Canonicals
Directives
Redirects
Pagination
Hreflang
AMP
Structured Data
Meta Refresh
Custom Extraction
Custom Search
Security
Duplicates
Links
Broken Links
SERP
PageSpeed
Search Console
GA4
AI
SEO Authority
Search URLs / titles…
HTML ▾
Advanced
Columns
Export
Row
URL
Type
Status
Indexability
Indexability Status
Title 1
Title 1 Length
Meta Description 1
1
https://example.com/
html
200
Indexable
—
Acme — Cloud Tools for Modern Teams
35
All-in-one cloud platform for fast-growing …
2
https://example.com/pricing
html
200
Indexable
—
Pricing
7
—
3
https://example.com/products/analytics
html
200
Indexable
—
Analytics Dashboard — Acme Cloud Platform
42
Built-in analytics with custom dashboards …
4
https://example.com/solutions/enterprise
html
200
Indexable
—
Enterprise Solutions — Acme Cloud Platform
43
Custom plans and SSO for organisations …
5
https://example.com/resources/archive/2023
html
200
Non-Indexable
noindex
2023 Resource Archive
21
—
6
https://example.com/about
html
200
Indexable
—
About Acme
10
—
7
https://example.com/en
html
200
Non-Indexable
Canonicalised
Acme — Cloud Tools for Modern Teams
35
All-in-one cloud platform for fast-growing …
8
https://example.com/fr
html
200
Indexable
—
Acme — Outils Cloud pour Équipes Modernes
41
Plateforme cloud tout-en-un pour les …
9
https://example.com/blog
html
200
Indexable
—
Acme Blog — Product News and Engineering …
44
Read the latest from our product and …
10
https://example.com/products/beta
html
301
Non-Indexable
Redirected
—
—
—
11
https://example.com/labs
html
301
Non-Indexable
Redirected
—
—
—
12
https://example.com/products/api
html
200
Indexable
—
API & Webhooks — Acme Cloud Platform
37
Build integrations with our REST API and …
13
https://example.com/research/v1
html
301
Non-Indexable
Redirected
—
—
—
14
https://example.com/products/integrations
html
200
Indexable
—
Integrations Marketplace — Acme Cloud
38
Connect Acme to 300+ apps and services …
15
https://example.com/account/dashboard
html
302
Non-Indexable
Redirected
—
—
—
16
https://example.com/products/security
html
200
Indexable
—
Security & Compliance — Acme Cloud
35
SOC 2, GDPR and HIPAA compliance …
URLs: 294 (294 loaded) Selected Rows: 1 Selected Cells: 1 Selected Columns: 0
URL Details
Outline
Inlinks (16)
Outlinks (87)
Images
Resources
Extracted Data
SERP Snippet
HTTP Headers
Cookies
Structured Data
View Source
View Rendered
Screenshot
Duplicates
Analytics
Name
Value
Address
https://example.com/
Status Code
200
Status
OK
Indexability
indexable
Indexability Reason
—
Content Type
text/html; charset=UTF-8
Content Kind
html
Size (Bytes)
316,166
Response Time (ms)
133
TTFB (ms)
133
Overview URLs% of Total
▾Summary
Total Internal URLs294100.00%
Internal Indexable20670.07%
Internal Non-Indexable8829.93%
▾Crawl Data
Internal294100.00%
▾Response Codes
All294100.00%
Blocked by Robots00.00%
No Response00.00%
Success (2xx)28095.24%
Redirection (3xx)144.76%
Client Error (4xx)00.00%
Server Error (5xx)00.00%
▾Security
HTTPS URLs294100.00%
HTTP URLs00.00%
▾Indexability
Indexable20670.07%
Noindex7324.83%
Canonicalised10.34%
Blocked by Robots00.00%
▾Issues
▾Page Titles
Missing144.76%
Over 60 Characters175.78%
Below 30 Characters227.48%
Duplicate289.52%
Multiple <title> Tags28095.24%
Pixel Width Truncated (>600px)00.00%
Placeholder Title00.00%
Single-Word Title186.12%
▾Meta Descriptions
Missing8729.59%
Discovered364
Crawled295
Pending281
Failed0
URL/s7.7
Avg resp204ms
Elapsed00:00:38
FPS144
Heap14 MB
RSS1.22 GB
Sys Free7.74 GB
Per URL2 MB
Capacity4.0K
Lag2ms
Running
1
Native menu & top bar
File · View · Visualization · Reports · Help. URL · scope dropdown · Pause / Stop / +Add URL / Clear / ⚙. Blue accent Start button when idle.
2
30 top-level tabs
Internal · External · Response Codes · URL · Page Titles · Meta Description · H1 · H2 · Content · Images · Canonicals · Directives · Redirects · Pagination · Hreflang · AMP · Structured Data · Meta Refresh · Custom Extraction · Custom Search · Security · Duplicates · Links · Broken Links · SERP · PageSpeed · Search Console · GA4 · AI · SEO Authority. Active tab marked by accent underline.
3
Table toolbar
Search box (left) · quick-filter dropdown (HTML / 2xx / Noindex…) · Advanced filter · List ↔ Tree view toggle · Columns picker · 4-format Export.
4
Virtualised URL table
Coloured status chips (200 · 301 · 4xx) · Indexable / Non-Indexable badges · resizable columns · per-cell selection · row-number drag select · 500K+ rows comfortably.
5
16-sub-tab detail panel
URL Details · Outline · Inlinks · Outlinks · Images · Resources · Extracted Data · SERP Snippet · HTTP Headers · Cookies · Structured Data · View Source · View Rendered · Screenshot · Duplicates · Analytics. Name | Value table layout per row.
6
Overview sidebar (right)
Auto-grouped by Summary · Crawl Data · Response Codes · Security · Indexability · Issues (nested by topic — Page Titles · Meta Descriptions · …). URLs + % of Total on every row.
7
Live status bar
Discovered · Crawled · Pending · Failed · URL/s · Avg resp · Elapsed · FPS · Heap · RSS · Sys Free · Per URL · Capacity · Lag. Coloured threshold tiers (green / amber / red). Idle ↔ Running pulse on the right.

Tailwind surface-950 background · accent #2563eb · Inter typography · sidebar pinned right · panels are resizable. Source: apps/desktop/src/renderer/src/ in the FreeCrawl repo.

Features

Everything you need for technical SEO.

A focused toolkit, designed for clarity. No subscription, no vendor lock-in — just the data you need to make better decisions.

Industrial-grade crawl engine

  • Spider, List, SERP, and Compare modes
  • Up to 50 parallel workers
  • Token-bucket rate limiting (URL/s, RPS)
  • Scope: Subdomain / Subfolder / All / Exact URL
  • Crash recovery + queue checkpoint

Full JS rendering, included.

  • Chromium via Playwright (bundled)
  • Per-page screenshots (full / fold / mobile)
  • LCP candidate detection
  • Mobile Usability audit
  • Custom pre-render JS injection

Every signal a search engine sees.

  • Titles, meta, H1–H6, canonical, hreflang
  • Open Graph + Twitter + JSON-LD + Microdata + RDFa
  • Robots / X-Robots / Meta Refresh / Pagination
  • Custom CSS + regex extraction (10+ rules)
  • Custom search (literal + regex)

200+ checks across 14 categories.

  • Title / Description / Heading / Content
  • Indexability / Mobile / Accessibility (WCAG)
  • Performance / Security / Structured Data
  • Hreflang reciprocity / Pagination sequence
  • Near-duplicate (SimHash 64-bit + LSH)

Connect the rest of your stack.

  • Google Search Console (clicks / impressions / CTR)
  • Google Analytics 4 (sessions / users / engagement)
  • Ahrefs / Moz / Semrush / Majestic (URL metrics)
  • PageSpeed Insights (Lighthouse mobile + desktop)
  • OpenAI / Anthropic / Ollama (AI prompts per URL)

23 ready-made reports.

  • Bulk Export (XLSX / CSV / JSON / XML)
  • Pages per Directory, URL Length Histogram
  • Sitemap Orphans, External Domain Health
  • Word Count per Directory, Most/Least Linked URLs
  • Top Words, Indexability Distribution

100% local. Forever.

  • Runs entirely on your machine
  • No telemetry, no accounts
  • SQLite project files (encryption optional)
  • AES-256-GCM .seoproject.enc snapshots
  • System DNS bypass on broken networks

Scriptable, embeddable, hackable.

  • Headless CLI (.csv input, JSON output)
  • MCP server (Claude Code-ready)
  • GitHub Actions + GitLab CI templates
  • Open-source (MIT-style)
  • Hand-rolled, zero native deps
How it works

From a URL to a full audit, in four steps.

No accounts. No cloud roundtrips. Everything runs on your machine.

Step 1 · Configure
Configure

Start URL, crawl scope, depth limit, render mode, worker count, rate limits, URL rewrites, custom UA, cookies, robots, authentication.

Start URLhttps://example.com
ScopeSubdomain
Depth10
Workers12
RenderJavaScript
UAGooglebot
24 settings panels
Step 2 · Crawl
Crawl

Up to 50 parallel workers fetch, parse, and extract every signal — meta tags, links, headings, structured data, images, headers, cookies.

Crawled1,284
Pending142
URL/s42.1
Avg168 ms
crawling /blog/2026/headphone-…
1M+ URLs comfortably
Step 3 · Inspect
Inspect

30 top-level tabs surface what each search engine and audit tool would see. 200+ issue checks, near-duplicate clustering, link graph, screenshots.

Indexable 923 3xx 42 4xx 14 Title Too Long 38 Meta Missing 52 HSTS Missing 12 No Schema 18 Slow (>1s) 36 Hreflang 6
200+ checks across 14 groups
Step 4 · Export
Export

One dialog, four formats. Stream to Google Sheets or BigQuery. Drive from Claude Code via the bundled MCP server.

.xlsx.csv.json.xml
Sheetsstreaming insert
BigQueryservice account
MCP serverstdio JSON-RPC
4 formats + 2 cloud sinks

Crash recovery resumes pending URLs after a kill. Queue checkpoint every 30 s. Adaptive concurrency throttles when the renderer hits input lag > 200 ms.

Integrations

Plug into the tools you already use.

Bring your own API keys. Everything runs locally — your credentials never leave your machine.

Google Search Console Google Analytics 4 Google Sheets BigQuery
Ahrefs Moz Semrush Majestic PageSpeed Insights
OpenAI Anthropic Ollama local

Built for Claude Code

FreeCrawl ships an MCP server, so Claude Code and any MCP-compatible client can drive crawls, query results, and run reports — all against your live project. No API gateway, no cloud roundtrip.

Read the MCP guide on GitHub →
Use cases

Built for the way SEO actually works.

SEO agencies

Audit 20 client sites a month without burning subscriptions.

Run unlimited crawls in parallel across client projects. Bulk-export branded reports. Schedule monthly re-audits. Use AI prompts to surface unique opportunities per site.

E-commerce SEO

Find every broken product link before Black Friday.

JS-rendered crawls catch SPA product detail pages. Schema validation surfaces missing Product / Offer / Review markup. URL-rewriting tests redirect chains before you push.

Developers & tech SEO

Wire crawls into your CI pipeline.

Headless CLI in GitHub Actions. JSON output. Fail builds when broken-link thresholds exceed. Compare two crawls to surface regressions on every PR.

Solo consultants

Professional audits without professional bills.

Match Screaming Frog's deliverables — sitemaps, hreflang reports, redirect maps — on a freelancer budget (zero). Encrypted project files keep client data safe.

Performance & privacy

Fast where it matters. Local where it counts.

Your machine
Crawler Engine
Workers · rate limit · robots · auth · proxy. In-process Node.js.
Chromium (Playwright)
Bundled with installer. Screenshots, LCP, mobile audit.
SQLite (WAL)
.seoproject file on disk. node:sqlite, zero native deps.
React UI
Electron renderer. Virtualised tables. Multi-language (EN + TR).
Headless CLI
.csv input · JSON output. CI/CD-ready.
MCP Server
stdio JSON-RPC. Read-only DB share. Claude Code-compatible.
Optional outbound (your API keys, your call)
Direct connections to Google Search Console · GA4 · Sheets · BigQuery · PageSpeed · Ahrefs · Moz · Semrush · Majestic · OpenAI · Anthropic · Ollama. Each integration is opt-in per project. Credentials live in your OS keychain.
There is no
  • FreeCrawl cloud
    No central server holding your crawls.
  • Telemetry endpoint
    No usage stats, no error reports, no pings.
  • Sign-in
    No account, no licence server, no activation key.
  • Hidden cookie / pixel
    No tracking inside the app or this website.
AES-256-GCM encryption
Optional .seoproject.enc snapshots. Password gates the entire project file.
OS keychain for secrets
API keys stored via Electron safeStorage — never on disk in plaintext.
MIT-style open source
Audit every line on GitHub. Fork it. Submit pull requests. Run your own build.

Crawls flow from your machine directly to the sites you audit — over the same network your browser uses. FreeCrawl never sees, stores, or relays your traffic.

Benchmarks

Max URLs per crawl
1,000,000+
Parallel workers
1–50 (adaptive)
URL/s throughput
~50/s (typical broadband)
Memory monitor
Auto-pause if RSS > 4GB
Virtualized table rows
500k+
Sidebar tick latency
<5ms
Input lag (p99) during crawl
<30ms
SQLite write coalescing
1 fsync per page
DNS bypass tiers
3 (OS → public DNS → DoH)
Crash recovery
Resume queued URLs on restart
Pricing

Why is FreeCrawl free?

FreeCrawl is built and maintained by a single SEO practitioner — Kemal Acar — as an open-source side project. There is no company behind it, no investor expecting revenue, no upsell tier to push you into.

The tool is MIT-style licensed. Read the source code on GitHub. Fork it. Submit pull requests. Audit it for security. The entire thing runs on your machine; there is no server FreeCrawl could even monetise.

If you find FreeCrawl useful, the most valuable contribution is sharing it with another SEO professional, starring the repo, or reporting bugs. There is no paid tier and there will never be one.

⭐ Star on GitHub 🐛 Report an issue 💬 LinkedIn
FAQ

Frequently asked questions.

Is FreeCrawl really free? What's the catch?

There is no catch. FreeCrawl is open-source under an MIT-style license. The binary you download from GitHub is the same code in the repo. There is no premium tier, no paid features, no signup, and no telemetry.

How does FreeCrawl compare to Screaming Frog SEO Spider?

FreeCrawl covers Screaming Frog's core feature set — crawling, issue detection, custom extraction, integrations — and adds AI-powered analysis, an MCP server, encrypted project files, and unlimited crawling at no cost. See the side-by-side comparison above.

What platforms does FreeCrawl run on?

Windows (NSIS installer + portable .exe), macOS (Apple Silicon + Intel .dmg), and Linux (AppImage, .deb, .rpm). All builds are produced from the same GitHub Actions workflow on every release.

How large can a crawl be?

1,000,000+ URLs is comfortable on a modern laptop. The hard limit is your disk space and memory. The crawler auto-pauses when memory crosses a configurable threshold and resumes when it frees up.

Does FreeCrawl render JavaScript?

Yes. JS rendering is built-in via Chromium (Playwright). Installers ship with the browser bundled — no separate download on first launch.

Where is my crawl data stored?

In a SQLite file (.seoproject) on your machine. Nothing is uploaded anywhere. Optional AES-256-GCM encryption is available via "Save As Encrypted Project" in the File menu.

Can I use FreeCrawl in CI/CD?

Yes. Headless CLI mode reads URLs from .csv, writes JSON or CSV output, and ships with GitHub Actions + GitLab CI templates. Set thresholds (broken links, missing canonicals, etc.) to fail builds.

How do I integrate Google Search Console, GA4, or Ahrefs?

Open Settings → Integrations, paste your API key or OAuth token (stored in the OS-level encrypted credential store), and the data joins onto your crawled URLs automatically.

Will my Screaming Frog .csv exports work in FreeCrawl?

URL lists imported as .csv (or sitemap.xml, or GSC export) work directly in List mode. Full project import from a Screaming Frog .seospider file is not supported (proprietary format).

Where do I report bugs or request features?

GitHub Issues — github.com/kemalai/FreeCrawl-SEO-Tool/issues. For private support, email kemalacarofficial@gmail.com — replies within 48 hours.

Changelog

Shipping every week.

Recent improvements. Full history on GitHub Releases.

v0.6.5
  • JavaScript rendering via Playwright (Chromium bundled)
  • Screenshots: full / above-the-fold / mobile
  • LCP candidate detection
  • Mobile Usability audit
  • "Export Crawl Data…" dialog with XLSX / CSV / JSON / XML
v0.6.0
  • Google Search Console + GA4 integration
  • AI tab — OpenAI + Anthropic + Ollama prompts per URL
  • SEO Authority tab (Ahrefs / Moz / Semrush / Majestic)
  • Encrypted .seoproject.enc snapshots
  • Google Sheets + BigQuery export
v0.5.6
  • PageSpeed Insights integration (Lighthouse mobile + desktop)
  • Broken Links unique-URLs counter
  • macOS dock reopen fix
  • Scheduled-crawl double-fire fix
Full changelog on GitHub →
Contact

Get in touch.

For support requests, feature ideas or partnership inquiries, send an email. Replies are typically delivered within 48 hours.

Email kemalacarofficial@gmail.com Replies within 48 hours
LinkedIn kemalacarofficial Connect with the maker

Replace Screaming Frog this afternoon.

FreeCrawl is a 60-second download. Your first audit runs in under five minutes. No signup, no card, no install babysitting.

Download for Windows · macOS · Linux View source on GitHub →

MIT-style license · No telemetry, ever