Real-time, data-backed SEO diagnostics at a glance.

A suite of proprietary diagnostic tools to monitor keyword positions, technical page health, and SERP dynamics.

Free, no sign-up

robots.txt and Sitemap Check

Your domain's robots.txt is read and laid out in a table, the declared sitemaps are opened and their URLs counted, and the recurring mistakes that hide a site from Google are marked.

  • No sign-up needed
  • URL count for each sitemap
  • Common mistakes marked

Run the tool right here on the page

Determines which parts Googlebot may reach and whether your sitemap is intact.

  • What it costs Free
  • Data source Our own server
  • Access No sign-up
Verified

This tool executes on our own infrastructure with no purchased data behind it, so it costs nothing and stays that way. Its only limit is a daily cap that stops one bot from taking the whole capacity. The daily cap is measured per visitor. If you need more, a free account lifts your limit.

The short answer

robots.txt tells crawlers which parts of the site to skip, and the sitemap tells them which URLs exist and when each last changed. If a page of yours will not index, these two files are the first thing to look at, before anything else.

The worst event for a site fits in one line: Disallow: / . Every staging environment carries that line, and rightly, but when the site goes from staging to the live server it usually travels with it and removes the whole site from Google. No warning shows anywhere; traffic just slides to zero over a few weeks and nobody knows why. After every site migration, check this on day one.

The mistake we keep meeting on Iranian WordPress sites is blocking wp-content. That logic was once sound and is not now: to judge whether your page is mobile friendly, Google must see its CSS and JavaScript, all of which sit inside wp-content. Block that folder and Google sees an unstyled page and judges a shapeless one.

For sitemaps, the number worth reading is the gap between the URL count in the sitemap and the indexed page count in Search Console. If the sitemap holds a thousand URLs and Google has indexed two hundred, the sitemap is not at fault; Google has seen eight hundred pages and judged them not worth indexing. The right question then is the quality of those pages, not the sitemap file.

The data source and method

The file is read by our server straight from your domain; there is no external service and no fee. When robots.txt names no sitemap, the usual paths, sitemap.xml, sitemap_index.xml and wp-sitemap.xml, are attempted as well. A pattern we run into constantly: a large share of Iranian WordPress sites still keep wp-content blocked, which leads Google to render the page without CSS and score it as not mobile friendly.

The situations where it helps

The first item to inspect when a page refuses to index. Also after every site move or host swap, since a staging robots.txt usually travels with Disallow: / and removes the entire site from Google.

Where it falls short

This tool reports what robots.txt permits, not what Google actually did. A page can be fully allowed in robots.txt and still go unindexed, because Google judged it not worth it or its canonical points elsewhere. The reverse also occurs: a page blocked in robots.txt but linked from elsewhere sometimes shows in results with an empty title, because robots withholds permission to crawl, not permission to index. For do-not-index, the right instrument is the noindex tag, not robots.txt. To see what Google did with a URL, the only correct source is that site's own Search Console.

A lesson from working with it

If robots.txt declares no sitemap, this tool also tries the usual paths: sitemap.xml, sitemap_index.xml and wp-sitemap.xml. We added that on purpose, because in practice we have often seen a sitemap that exists and is healthy but is declared nowhere, with the owner assuming there is none. Declaring the sitemap in robots.txt is one line, and that line is what tells a crawler reaching your site for the first time where to begin.

What it costs

This tool executes on our own infrastructure with no purchased data behind it, so it costs nothing and stays that way. Its only limit is a daily cap that stops one bot from taking the whole capacity.

The daily cap is measured per visitor. If you need more, a free account lifts your limit.

Common questions about this tool

Is it a problem if I have no robots.txt?

Not necessarily. No file means everything is allowed, which for most small sites is exactly what they want. The file becomes necessary when you want to hold sections such as internal search pages or shop filters out of crawling.

Can Disallow remove a page from Google?

Not reliably. If the page is linked from elsewhere it can still appear in results with an empty title. For real removal use the noindex tag, and more importantly leave that page unblocked in robots.txt, because Google must be able to crawl it to see the noindex.

How often should a sitemap be updated?

Automatically, with every new piece of content. Almost every WordPress SEO plugin does this. A hand-built sitemap left un-updated for months is worse than none, because it sends the crawler to URLs that no longer exist.

What you get from Seemerce with the robots.txt and sitemap check

Your domain's robots.txt is read and shown in a table, the declared sitemaps are opened and their URLs counted, and the common mistakes that hide a site from Google are flagged.

0Free with no sign-up
2filesrobots.txt and the sitemaps
1tableCrawler rules in one view

Tap any step to read the details behind it.

Innovative solutions in The robots.txt and sitemap check

Every output has a job; see which one is yours.

Tap any solution to see exactly what it includes.

Five points on robots.txt and sitemaps

We set Google's guidance beside this tool.

1

What robots.txt does

Google says the file tells crawlers which URLs they may access.

With us The file is read and laid out in a table.

2

It does not hide pages

Google stresses robots.txt is not a way to keep a page out of Google; use noindex or a password.

With us Common mistakes that hide a site from Google are marked.

3

Sitemap limits

Google: one sitemap holds at most 50 MB uncompressed or 50,000 URLs; beyond that, split it.

With us Declared sitemaps are opened and each one's URLs counted.

4

Declaring the sitemap

Google says a sitemap can be submitted in Search Console or referenced with a Sitemap line in robots.txt.

With us Sitemaps declared in robots.txt are read.

5

Fully-qualified URLs

Google asks for absolute URLs and UTF-8 encoding.

With us If a page will not index, these two files are the first place to look.

Sources: Google Search Central — robots.txt and Sitemaps (2026)

Six numbers

Each reference's figure comes from its own source and Seemerce's from this page.

0dollars

Cost per run

Runs on our own server with no purchased data.

Seemerce
1daily cap

Per visitor

A daily cap per visitor so no bot takes all the capacity; a free account lifts it.

Seemerce
50,000URLs

Per sitemap

The cap per sitemap file.

Google
50MB

Uncompressed

The allowed uncompressed size of a sitemap.

Google
4formats

Supported

XML, RSS/Atom, text and sitemap index.

Google
2files

Read together

robots.txt and the sitemaps it declares.

Seemerce

Sources: Google Search Central — robots.txt and Sitemaps (2026)

One reference and Seemerce

Only what each platform's page says outright is marked; a blank means the page does not say.

PracticeGoogleSeemerce
robots.txt directs crawler access1 of 1Read and tabulated
Not enough to hide a page1 of 1Mistakes are marked
50,000 URLs per sitemap1 of 1A URL count per sitemap
Sitemap referenced in robots.txt1 of 1Declared sitemaps are opened

stated on their published page   not stated there (it may still be done)

Sources: Google Search Central — robots.txt and Sitemaps (2026)

Cost per run across Seemerce tools

The price of one run in dollars; tools with purchased data have a real cost deducted from a wallet, while tools on our own server are free.

Free$0.014$0.036$0.054$0.054$0.073$0.15$0.18$0.18$0.18
Free tools (4)SERP viewerRanked keywordsBacklink gapReferring domainsDomain overviewBacklinksSite keywordsKeyword suggestionsVolume and CPC (up to 100)
Prices move with the daily rate and the exact figure at the moment of a run is shown on screen. Keyword tools have one flat price for up to 100 keywords (or 20 seeds).

Sources: each tool's Seemerce page

What this tool includes

Only what this page already promises.

Checks

  • robots.txt in a table
  • Declared sitemaps and their URL counts
  • Common mistakes that hide a site

Access

  • Free with no sign-up
  • On our own server
  • An answer to “why is it not indexed?”

Limit

  • A daily cap per visitor
  • A free account lifts the cap
  • It only reads; nothing on your site changes
See+Map=SeeMap This page's name in the Seemerce family What the name means
This page's name in the Seemerce family

Reads robots.txt and sitemap.xml and says what is accidentally blocked.

The rule is simple: See joins a word to make one new word, just as See and commerce became Seemerce.

Related names