> ## Documentation Index
> Fetch the complete documentation index at: https://docs.rankraft.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Custom log integration

> Send request logs from a log drain, CDN or your own server to RankRaft AI.

From a log drain (Vercel, Netlify), Fastly or your own server, POST batches of requests to RankRaft AI. Unlike the snippet, this sees crawlers as well as referrals.

**Sees:** everything: AI crawlers, search bots and AI referrals.

## Prerequisites

* A RankRaft AI site key. Open **Integrations**, choose **Any other site** and click **Create site key**. See [Your site key](/agent-analytics/integrations/overview#your-site-key).
* A process that can read your request logs and make outbound HTTPS requests.

## Endpoint

```http theme={"system"}
POST https://ingest.rankraft.ai/v1/events
```

## Authentication

Send your site key in the `X-Site-Key` header, with `Content-Type: application/json`. Store the key as a secret, not in source control.

## Request body

```json theme={"system"}
{
  "source": "custom",
  "events": [
    {
      "ts": "2026-10-15T10:00:00Z",
      "host": "example.com",
      "method": "GET",
      "path": "/blog/post",
      "ip": "20.171.206.44",
      "user_agent": "Mozilla/5.0 AppleWebKit/537.36 (KHTML, like Gecko); compatible; GPTBot/1.2; +https://openai.com/gptbot",
      "status": 200,
      "bytes": 18234
    }
  ]
}
```

### Fields

| Field | Required | Description |
| - | - | - |
| `host` | Yes | Hostname requested. Must be one of your site's [allowed hostnames](/agent-analytics/integrations/site-data-controls#allowed-hostnames). |
| `path` | Yes | Path requested. Any query string is removed before storage. |
| `ip` | Recommended | The **client** IP address, not your load balancer's. Used for [crawler verification](/agent-analytics/crawler-verification), then stored only as a hash. |
| `user_agent` | Recommended | The User-Agent header. Needed to recognise crawlers. |
| `referer` | Recommended | The Referer header. Needed to detect AI referrals. **Leave the field out when there is none. Don't send `null`.** |
| `query` | Optional | The raw query string without `?`. Used to detect `utm_source` on AI referrals. |
| `status` | Optional | HTTP status your site returned (100–599). Powers **Crawl errors**. |
| `bytes` | Optional | Response size in bytes. |
| `ts` | Optional | ISO-8601 time of the request. Defaults to the time we receive it. |
| `method` | Optional | HTTP method. Defaults to `GET`. |
| `country` | Optional | ISO 3166-1 alpha-2 country code, such as `GB`. |

Send **every** request, not just bot requests. RankRaft AI classifies them for you. Human traffic is counted toward your daily totals so that AI share of traffic is accurate, and it isn't stored individually.

## Examples

<CodeGroup>
  ```javascript Node.js theme={"system"}
  await fetch("https://ingest.rankraft.ai/v1/events", {
    method: "POST",
    headers: {
      "X-Site-Key": process.env.RANKRAFT_SITE_KEY,
      "Content-Type": "application/json",
    },
    body: JSON.stringify({ source: "custom", events }),
  });
  ```

  ```python Python theme={"system"}
  import os, requests

  requests.post(
      "https://ingest.rankraft.ai/v1/events",
      headers={"X-Site-Key": os.environ["RANKRAFT_SITE_KEY"]},
      json={"source": "custom", "events": events},
      timeout=5,
  )
  ```
</CodeGroup>

A successful request returns `202 Accepted`:

```json theme={"system"}
{ "accepted": 1, "rejected": 0 }
```

If some events are invalid, the valid ones are still stored. The response counts the rejected ones and lists up to five `issues` with the index of each failing event.

## Best practices

* **Batch.** Up to **1,000 events** and **5 MB** per request. Send a batch every few seconds to a minute, not one request per page view.
* **Never block the response.** Send after the response has gone out, or from a background queue.
* **Retry sensibly.** On `429` or any `5xx`, keep the batch and retry with backoff. On `403` or `410` the site is paused or removed, so stop sending. On any other `4xx`, drop the batch: it won't succeed on retry.
* **Send the real client IP.** If you send your proxy's address, every genuine crawler shows as **Spoofed**.

## Check it's working

Press **Check again** at the top of **Integrations**. It turns to **Connected** within a minute of your first batch. Then open **Integrations → Advanced → Delivery log**: each batch shows how many requests were stored, counted, skipped or rejected. A high **Skipped** count means the hostnames you send aren't on your allowed list.

Need help? Email [hello@rankraft.ai](mailto:hello@rankraft.ai).


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.