Streams

Declare what your connector ingests. Each stream has a key and a mode (poll or webhook).

A connector advertises its data streams up front. The platform uses the catalog to schedule polls, route webhooks, manage checkpoints, and surface stream health to operators.

Declaring streams

streams: [
  { key: "events",    mode: "webhook" },
  { key: "customers", mode: "poll", schedule: "0 */6 * * *", scheduleConfigurable: true },
  { key: "invoices",  mode: "poll", schedule: "0 */6 * * *", scheduleConfigurable: true },
  { key: "payouts",   mode: "poll", schedule: "0 3 * * *" },
],

Modes

Mode	When the platform invokes you	Where the handler lives
`poll`	On the cron schedule.	`src/api/sync/<stream-key>.ts` (a route that reads `from` / `to` / cursor query params, fetches from the provider, and emits via `ingest.canonical`).
`webhook`	On every event posted to the configured webhook endpoint.	`src/api/webhooks/<provider>.ts` (a route that verifies the signature and emits via `ingest.canonical`).

The poll and backfill mapping is enforced at build/deploy time. A stream such as { key: "customers", mode: "poll" } must have a matching POST src/api/sync/customers.ts route. Webhook-mode streams do not need a sync route, but webhooks.endpoint must point at an existing POST route with the declared auth mode.

Poll streams

Each poll stream has a schedule (cron). The platform invokes the corresponding sync route with paging metadata, and the handler is responsible for:

Reading the saved cursor / page from the request.
Calling the provider’s API.
Emitting each record via ingest.canonical(...).
Returning a response that includes the new cursor and a hasMore flag for pagination.

Stripe’s customers sync is the canonical pattern — see Sync routes.

Configurable cadence

Set scheduleConfigurable: true on a poll or backfill stream when workspace admins should be able to choose that stream’s sync frequency during connector setup:

{
  key: "bank_transactions",
  mode: "poll",
  schedule: "0 */6 * * *",       // connector default
  scheduleConfigurable: true,     // admins can override per connection
}

The manifest schedule remains the default. When an admin chooses another cadence, Backfill stores the override on the connector connection and reconciles scheduled imports from that effective schedule. Use this for streams where freshness, API rate limits, or customer workflow vary by tenant. Leave it off for webhook streams and for poll streams whose cadence should be connector-controlled.

Webhook streams

A webhook-mode stream means the corresponding events arrive at the connector’s webhook endpoint (declared in webhooks: { endpoint, auth }). The webhook handler dispatches to per-stream transforms; one webhook delivery may emit records into multiple streams.

// after parsing event:
ingest.canonical(emission.canonicalType, emission.payload, {
  stream: emission.stream,                  // names the destination stream
  idempotencyKey: eventIdempotencyKey(/* ... */),
});

Stream key conventions

Lowercase + underscores: customers, payout_reconciliation, balance_transactions.
One-to-one with the provider’s logical resource name where possible.
Webhooks for events that don’t map to a single stream (e.g., a generic events topic) declare a stream named events.

Why declare them at all

Operators need to see “this connector ingests these 12 things, on these cadences, with these last-success timestamps.” The stream catalog makes that surfaceable in the dashboard without inferring from your route filenames.

← Webhooks

Overview →