Skip to the content
Know your tech before you buy it. Makers: get listed from $49/mo

WebCrawler Pro

By WebCrawler Pro · webcrawler.pro

  • Free plan
  • Windows
  • macOS
  • Linux
  • Web
  • Self-hosted
8.5 Club score Visit webcrawler.pro

What WebCrawler Pro is

WebCrawler Pro crawls websites and scores pages for technical SEO, answer-engine readiness (AEO), and generative engine optimization (GEO). Its technical checks include titles, meta descriptions, headings, canonical and Open Graph tags, response times, page size, duplicate content, orphan pages, and thin content. AEO checks cover FAQ schema and question-based headings, while GEO checks look at author attribution, freshness, factual density, authority links, and original-research signals. Reports include crawl-depth charts, score distributions, response-time histograms, page details, and site summaries, with PDF, CSV, and JSON exports. Signed-in users can generate drafts such as titles, FAQs, JSON-LD, alt text, internal links, and llms.txt using a hosted model or their own API key. HTML mode reads server responses; DOM Render executes JavaScript before extracting content. The free allowance is 50 requests per day for guests or 100 for signed-in users. Paid crawl tokens are listed at $1 a month. Self-hosting is available for the PHP and vanilla JavaScript project.

Who it suits

It suits site owners and agencies that want SEO, AEO, and GEO checks with exportable reports. Agencies can prepare client-ready PDFs with a client name and logo or hide WebCrawler branding.

Good to know, and what to watch

  • Checks technical SEO, AEO, and GEO signals.
  • Exports reports as PDF, CSV, and JSON.
  • DOM Render executes JavaScript before extracting content.
  • Client-ready PDFs can hide WebCrawler branding.
  • Can be self-hosted with PHP 7.4 or later.
  • Pages behind login walls may not be fully crawlable.
  • Robots.txt restrictions may prevent complete crawls.
  • Advanced bot detection may limit crawling.
  • Desktop installers are not Apple-notarized or Microsoft-signed.

Free guest

Free

Free signed in

Free

Paid tokens

$1/mo

Free daily allowance

Free

Free plan

Getting started

  1. Open WebCrawler Pro on the web or choose a listed desktop platform
  2. Sign in with Google or continue as a guest
  3. Select HTML mode or DOM Render for the crawl
  4. Run a crawl and review the technical, AEO, and GEO findings
  5. Export the report as PDF, CSV, or JSON
  6. Add your own OpenAI, Gemini, or Anthropic API key for draft generation

Limits to check

Guests receive 50 requests per day and signed-in users receive 100 requests per day, with no credit card required. Paid tokens are listed at $1 a month for 1,000 requests; prepaid crawl tokens are listed at $11,000, paid once and do not expire. Pages behind login walls, blocked by robots.txt, or protected by advanced bot detection may not be fully crawlable.

The verdict

WebCrawler Pro brings technical, answer-engine, and generative-engine checks together with several report formats. Keep its stated crawl limitations and desktop installer signature status in mind when choosing how to use it.

Questions people ask

What does WebCrawler Pro cost?

The free guest plan is free with 50 requests/day, and the free signed-in plan is free with 100 requests/day. Paid tokens are listed at $1 a month for 1,000 requests; prepaid crawl tokens are listed at $11,000, paid once.

Which platforms are supported?

WebCrawler Pro supports Linux, macOS, self-hosting, web, and Windows.

Can it render JavaScript?

Yes. DOM Render executes JavaScript before extracting content, while HTML mode fetches raw server responses.

Can agencies create branded reports?

Yes. Client-ready PDFs can include a client name and logo or hide WebCrawler branding, with findings available in five listed languages.

Which AI keys can signed-in users provide?

The FAQ names user-provided OpenAI, Gemini, and Anthropic API keys. The maker says a user’s own key stays on that device.

Can WebCrawler Pro be self-hosted?

Yes. The maker describes a PHP and vanilla JavaScript project requiring PHP 7.4 or later plus pdo, pdo_sqlite, and curl extensions.

What the maker publishes

AEO and GEO
AEO checks include FAQ schema and question-based headings; GEO checks include author attribution, date freshness, factual density, authority links, and original research signals. source
AEO scoring
AEO scoring covers featured-snippet readiness, FAQ schema, question headings, and entity clarity. source
Agency features
Client-ready PDFs can include a client name and logo or hide WebCrawler branding, and findings are available in English, Romanian, French, Spanish, and German. source
AI drafts
Signed-in users can generate draft titles, FAQs, JSON-LD, alt text, internal links, and llms.txt using the hosted model or their own API key. source
AI drafts
Signed-in users can generate crawl-grounded titles, FAQs, JSON-LD, image alt text, internal links, keyword briefs, cannibalization suggestions, GEO rewrites, and llms.txt. source
AI providers
Bring-your-own keys are supported for OpenAI, Gemini, and Anthropic; keys remain on the user device. source
Changelog: accessibility and indexation
Version 2.2.0 added crawl-time accessibility checks and an indexation view combining noindex, orphan, robots, canonical, redirect, and sitemap signals. source
Changelog: comparison
Version 2.2.0 added scan comparison showing added or removed URLs and score changes. source
Client PDFs
Reports can be printed as client-ready PDFs with client name, logo, and optional WebCrawler branding removal, in English, Romanian, French, Spanish, and German. source
Company classification
European Tech Map classifies WebCrawler Pro as a hybrid-deployment, proprietary-source product headquartered in Rădășeni, Romania. source
Company size
SaaSHub lists WebCrawler.pro as a Romanian startup with 1–9 employees and a 2026 founding year. source
Crawl modes
HTML mode fetches raw server responses, while DOM Render mode executes JavaScript before extracting content. source
Data handling
The maker says signed-in scans are private to the account, anonymous saved scans are accessible by unique URL, and scan data is not shared, sold, or analysed. source
Data storage
Scan data is stored as JSON files, while accounts and sessions use SQLite; signed-in scans are private and anonymous scans are reachable through their unique URLs. source
Desktop apps
Optional desktop builds fetch pages locally to avoid browser CORS restrictions and are provided for macOS, Windows, Ubuntu/Debian, and Linux AppImage. source
Desktop installer status
The maker says its macOS, Windows, and Linux installers are not Apple-notarized or Microsoft-signed and advises checking their SHA-256 sums. source
DOM Render mode
DOM Render executes page JavaScript in a hidden iframe before extraction and supports JavaScript-heavy frameworks such as React, Next.js, Vue, and Angular. source
Free plan
Yes source
GEO scoring
GEO scoring measures author attribution, freshness dates, factual density, authority links, original research, and related E-E-A-T signals. source
Headquarters
Rădășeni, Suceava, Romania source
HTML mode
HTML mode fetches raw server responses and is intended for traditional CMS sites. source
Integrations
The FAQ names Google sign-in and support for user-provided OpenAI, Gemini, and Anthropic API keys; it says the user’s own key stays on their device. source
Intended users
The maker describes the tool as an independent site crawler and says agencies can send its client-ready reports to clients. source
License
The official pages explain self-hosting but do not state an open-source license. source

Checked against the maker's pages on 14 Oct 2026.