KiteJobsBot

KiteJobsBot is the crawler operated by KITEJOBS PRIVATE LIMITED (CIN U62013TN2026PTC193396) for Apiva, a job-search product. It reads public job postings so that people looking for work can find them and then apply on the employer's own site. It is a reader, not a submitter. If it is causing you trouble, everything you need to slow it down, block it, or have your site removed is on this page.

How to identify it

Every request sends a stable, contactable User-Agent:

KiteJobsBot/1.0 (+https://kitejobs.ai/bot; jobs@kitejobs.ai)

Match on the KiteJobsBot/1.0 token — the parenthetical is informational and a few call sites omit the address. We do not crawl from a browser-like or disguised User-Agent, and we do not rotate identifiers: if a request does not say KiteJobsBot, it is not us.

What it fetches

• The public job-list endpoint a hiring platform publishes for a company's careers board, and where the list omits the summary, that posting's own public detail endpoint.
• Public web-index datasets, to learn which companies have a careers board at all. That step reads the index, not your site.
• Read requests only. A few platforms return their public search results from a POST endpoint, so a POST body may appear in your logs — it is a query, never a submission.

What it keeps

For each posting: the title, the location, the posted date, the identifiers the platform assigns it, a short attributed excerpt of the description capped at 300 characters, and the canonical link back to your posting. The full description is processed in memory and discarded — we do not store it and we do not republish it. Search results link out to your posting; they are not a copy of it.

What it never does

• Sign in, create an account, or fetch anything behind a login, a paywall, or a session.
• Submit a form, an application, or a candidate's details.
• Solve, bypass, or retry around a bot challenge. If you challenge it, it backs off.
• Collect personal data about your recruiters or employees.
• Crawl your site generally. It asks the specific job endpoints above and follows nothing else.

How often it asks

At most one pass over a given careers board per day, and usually less — the list of companies is long and we work through it in rotation. There is no page-by-page site crawl, no re-fetch loop, and no burst.

How to slow it down or block it

Publish a rule for the agent in your robots.txt:

User-agent: KiteJobsBot
Crawl-delay: 30
Disallow: /

Straight talk about that, because we would rather be honest than look compliant: the crawler does not yet fetch and parse robots.txt itself, and it does not yet read the Retry-After header — a 429 stops us for the rest of that run regardless of the value you send, and we come back on our normal daily rotation rather than at the time you named. A robots.txt rule you publish is authoritative for us and we will apply it, but it takes effect when we see it — not on the next request. Automated enforcement of both is being built. Until it ships, either of these works immediately and needs nothing from us:

• Throttle or block at your edge. Answer the User-Agent with 403 or 429. We read both as “stop” — not for that one URL, but for the whole platform for the remainder of that run. Effective on the very next request.
• Remove a board for good. Answer 404 or 410 and we drop it permanently. We treat only those two as permanent on purpose: a 403 or 429is a “not now”, never a “never”.
• Just ask. Email us the domain or the board and we remove it, no justification needed.

Applications are not the crawler

If your question is about a job application rather than crawling: applications are a separate, account-holder-initiated action, made with that person's own information and their own consent. Those requests are not KiteJobsBot and never carry its User-Agent. What is and is not allowed there is set out in our Acceptable Use Policy, and misuse should be reported to abuse@kitejobs.ai.

Crawl complaints

Rate-limit requests, removal requests, questions, or anything the crawler did that it should not have: jobs@kitejobs.ai — the same address the User-Agent advertises. We reply within one business day. Include the domain and, if you have them, a few log lines; that is enough for us to find the source and stop it.

Other contact channels are listed on Contact; how we handle personal data is in our Privacy Policy.