Skip to main content

SaaSyBot, our web crawler

This page is for website owners who found SaaSyBot in their logs.

SaaSyBot fetches public web pages when a SaaSy customer's AI assistant needs them. Typical cases are reading a company's website before a sales call and checking a supplier's listed price for an item the customer buys. It fetches the one page it was asked for. It does not crawl a whole site.

How to recognize it​

Every request carries this User-Agent:

SaaSyBot/1.0 (+https://docs.hellosaasy.ai/docs/saasybot)

SaaSyBot never changes or hides that header.

What it respects​

  • robots.txt. SaaSyBot follows RFC 9309. It obeys Disallow and Allow path rules. It uses a group for SaaSyBot when your file has one, and the * group when it does not. If your robots.txt returns a server error or cannot be reached, SaaSyBot treats the whole site as disallowed and tries again a few minutes later. A missing robots.txt (a 404) means everything is allowed.
  • Crawl-delay. A Crawl-delay in the group that applies to SaaSyBot sets the minimum gap between its requests to your site.
  • Its own rate limit. Without a Crawl-delay, SaaSyBot makes at most a short burst of requests to one site and then no more than about one per second.
  • No bot-wall workarounds. SaaSyBot does not solve CAPTCHAs, run JavaScript challenges, or disguise itself as a browser. If a page sits behind bot protection, SaaSyBot gets nothing back.

SaaSyBot re-reads robots.txt at least once a day, so changes take effect within 24 hours.

How to block it​

To block SaaSyBot from your whole site, add this to your robots.txt:

User-agent: SaaSyBot
Disallow: /

To block only part of your site, list those paths instead:

User-agent: SaaSyBot
Disallow: /account/
Disallow: /checkout/

Contact​

To report a problem with SaaSyBot, email support@hellosaasy.ai. Include the URL it fetched and the time of the request.