WaooEventsBot

The crawler that reads public event calendars for Road Trip's Explore board.

What it does

WaooEventsBot collects public event listings (festivals, library programs, concerts, park events) so travellers on Road Trip's Explore board can find what is on near them. It reads calendar feeds (iCalendar, RSS, JSON), event APIs and event pages that a site already publishes. Every listing we show links back to the page it came from, and we keep only what a listing needs: the title, dates, place, a short description, an image link and that link back.

How it identifies itself

Every request carries one of these User-Agent strings:

  • Mozilla/5.0 (compatible; WaooEventsBot/1.0; +https://waoonetwork.com/bot)
  • WaooNetworkEvents/1.0 (+https://waoonetwork.com/bot)

It never sends a browser's User-Agent and never tries to solve a challenge page. Some sites behind Cloudflare or Akamai turn away the TLS handshake of the .NET library we use whatever the request says; for those we repeat the request once with a browser-compatible TLS handshake, still carrying the User-Agent above. A site that refuses WaooEventsBot by name stays refused.

How often it visits

Each calendar is read once or twice a day. A visit reads the feed or the pages of the calendar (a few to a few hundred requests for a large calendar), one request after another rather than in parallel.

robots.txt

WaooEventsBot honours robots.txt as described in RFC 9309. Before it fetches anything from a site it reads that site's /robots.txt, keeps it for up to 24 hours, and fetches nothing the file disallows. It follows the group for WaooEventsBot (or WaooNetworkEvents) when there is one, and the * group otherwise. If the file cannot be read because the server errors or does not answer, it fetches nothing from that site until it can.

To block it entirely:

User-agent: WaooEventsBot
Disallow: /

To allow only your event feed:

User-agent: WaooEventsBot
Disallow: /
Allow: /events/feed/

APIs we use under their own published terms (with an API key issued to us, or an API whose terms invite programmatic use, such as OpenStreetMap's Overpass API) follow those terms rather than robots.txt.

Contact

If WaooEventsBot is causing a problem, if you would like your calendar listed, or if you want a listing corrected or removed, contact us and mention WaooEventsBot. We answer within one business day.