Skip to content
Rime Documentation

REST API Connector

The REST API connector is a general-purpose connector for any HTTP API that returns JSON. Use it when Rime does not have a dedicated connector for your data source, or when you need to extract data from an internal API.

This connector requires more configuration than purpose-built connectors because you define the API structure, authentication, pagination, and response mapping yourself.

Base configuration

FieldDescription
NameDisplay name for this connector
Base URLRoot URL for the API (e.g., https://api.example.com/v2). All endpoint paths are relative to this URL

Authentication

The REST API connector supports four authentication methods:

API key

An API key sent as a header or query parameter.

FieldDescription
Key nameHeader or parameter name (e.g., X-API-Key, api_key)
Key valueThe API key value (encrypted at rest)
LocationWhere to send the key: header or query_parameter

Bearer token

An OAuth 2.0 bearer token sent in the Authorization header.

FieldDescription
TokenBearer token value (encrypted at rest)

The connector sends Authorization: Bearer <token> with every request.

Basic auth

HTTP Basic Authentication with a username and password.

FieldDescription
UsernameAPI username
PasswordAPI password (encrypted at rest)

The connector encodes username:password in Base64 and sends it as Authorization: Basic <encoded>.

OAuth 2.0 (Client Credentials)

For APIs that require an OAuth 2.0 client credentials flow.

FieldDescription
Token URLOAuth token endpoint (e.g., https://auth.example.com/oauth/token)
Client IDOAuth client identifier
Client secretOAuth client secret (encrypted at rest)
ScopesSpace-separated list of OAuth scopes (optional)

The connector automatically requests and refreshes access tokens before they expire.

Endpoint configuration

Each REST API connector can have multiple endpoints. Each endpoint maps to a separate table in Snowflake.

FieldDescription
PathURL path relative to the base URL (e.g., /users, /orders)
MethodHTTP method: GET (default) or POST
HeadersAdditional headers to send with requests (key-value pairs). These are merged with authentication headers
Query parametersStatic query parameters appended to every request (key-value pairs)
Request bodyFor POST requests, a JSON body template. Useful for APIs that use POST for data retrieval with complex filters
Table nameName of the destination table in Snowflake. Defaults to the path segment (e.g., /users becomes users)

You can add as many endpoints as needed. Each endpoint is extracted independently during a sync.

Pagination

Most APIs return paginated results. Configure pagination so Rime can retrieve all records across multiple pages.

Offset pagination

The API accepts offset and limit parameters.

FieldDescription
Limit parameterQuery parameter name for page size (e.g., limit, per_page)
Limit valueNumber of records per page (e.g., 100)
Offset parameterQuery parameter name for offset (e.g., offset, skip)

Rime increments the offset by the limit value after each page until the API returns fewer records than the limit.

Cursor pagination

The API returns a cursor token for the next page of results.

FieldDescription
Cursor parameterQuery parameter name to send the cursor (e.g., cursor, after)
Cursor pathJSON path to the cursor value in the response (e.g., $.meta.next_cursor, $.pagination.cursor)

Rime sends the cursor from the previous response with each subsequent request. Pagination stops when the cursor path returns null or is absent.

Page number pagination

The API accepts a page number parameter.

FieldDescription
Page parameterQuery parameter name for the page number (e.g., page)
Page size parameterQuery parameter name for page size (e.g., page_size, per_page)
Page size valueNumber of records per page
Start pageFirst page number (default: 1; some APIs start at 0)

Rime increments the page number after each request until the API returns fewer records than the page size.

The API uses RFC 5988 Link headers with rel="next".

No additional configuration is needed. Rime follows the next link in the response headers until no next link is present.

Response mapping

API responses need to be mapped from JSON to tabular data.

FieldDescription
Records pathJSON path to the array of records in the response (e.g., $.data, $.results, $.items). If the response is a top-level array, leave this blank
Column selectionAfter the first test extraction, Rime shows the inferred columns. You can include or exclude individual columns and rename them

Nested objects

Nested JSON objects are flattened into columns using underscore-separated names. For example, a response like:

{
  "data": [
    {
      "id": 1,
      "address": {
        "city": "Auckland",
        "country": "NZ"
      }
    }
  ]
}

With a records path of $.data, this produces columns: id, address_city, address_country.

Arrays within records are stored as JSON strings in a single column. They are not flattened into separate rows.

Rate limiting

FieldDescription
Requests per secondMaximum number of requests Rime sends per second (default: no limit)
Requests per minuteMaximum number of requests per minute. If both per-second and per-minute limits are set, the more restrictive limit applies
Respect Retry-AfterWhen enabled (default), Rime honors the Retry-After header in 429 Too Many Requests responses

Set rate limits conservatively to avoid disrupting the source API. If you are unsure of the API’s limits, start with 5 requests per second and adjust based on the sync results.

Error handling and retry behaviour

The connector automatically retries failed requests with exponential backoff:

  • Retryable errors: HTTP 429 (rate limited), 500 (server error), 502 (bad gateway), 503 (service unavailable), 504 (gateway timeout), and network timeouts.
  • Non-retryable errors: HTTP 400 (bad request), 401 (unauthorized), 403 (forbidden), 404 (not found). These cause the endpoint extraction to fail immediately.
  • Retry schedule: first retry after 1 second, then 2, 4, 8, 16, 32 seconds. Maximum 6 retries per request.
  • Timeout: each request has a 60-second timeout. Adjust this in the connector’s advanced settings if the API has slow endpoints.

If all retries for a request are exhausted, that endpoint’s extraction fails. Other endpoints in the same connector continue to extract. The run is marked as partially failed, and the error details are available in the run history.

Example: cursor-paginated API

To extract from https://api.example.com/v2/users with cursor pagination, set Base URL to https://api.example.com/v2, Endpoint path to /users, Records path to $.results, Pagination type to Cursor, Cursor parameter to cursor, and Cursor path to $.pagination.next_cursor. Rime requests /users, then /users?cursor=<token>, repeating until the cursor is null.

Next steps