Bot identity & responsible crawling

MedicWatch fetches publicly published determinations from regulator websites. When doing so, it identifies itself in the requestUser-Agent header.

MedicWatchBot/1.0 (+https://medicwatch.co.uk/about/bot; contact: hello@medicwatch.co.uk)

Crawl behaviour

• One request per three seconds, per source.
• Respects robots.txt on the source sites.
• Backs off on 429 / 5xx responses.
• Only fetches publicly accessible URLs.

Contact

If our crawler is causing problems on your service, please email hello@medicwatch.co.uk and we will respond promptly.