For stores: Raqib's crawler and your data
Who we are
Raqib (rqb.app) is a free Saudi site that checks discounts against price history. We take no commission and use no affiliate links: every visit we send to your store is a direct one.
Crawler identity
Our crawler always identifies itself as: RaqibBot/1.0 (+https://rqb.app/bot). It never logs in, never bypasses a block or paywall, and never pretends to be a browser.
Rate
At most one request every 3 seconds per store, slower if your robots.txt asks for it (Crawl-delay). A single product page is read between every 6 hours and every 3 days.
robots.txt
We read robots.txt before the first request and refresh it daily, and follow both the User-agent: RaqibBot rules and the * rules. If the file can't be reached, we treat the site as disallowed until we can read it.
The second network
Some stores allow us in robots.txt while their hosting blocks data-centre addresses. Those are fetched from a second network, with the same user agent, the same rate and the same robots.txt rules.
How to block or opt out
Add to your robots.txt:
User-agent: RaqibBot
Disallow: /
We stop within a day. Or email hello@rqb.app and we remove your store by hand.
Correcting your data
If a price, name or verdict for your store is wrong, email hello@rqb.app with the link. We correct it free of charge; how verdicts are computed is on the methodology page: rqb.app/method.