Streams

Declare what your connector ingests. Each stream has a key and a mode (poll or webhook).

A connector advertises its data streams up front. The platform uses the catalog to schedule polls, route webhooks, manage checkpoints, and surface stream health to operators.

Declaring streams

streams: [
  { key: "events",    mode: "webhook" },
  { key: "customers", mode: "poll", schedule: "*/15 * * * *" },
  { key: "invoices",  mode: "poll", schedule: "*/15 * * * *" },
  { key: "payouts",   mode: "poll", schedule: "*/15 * * * *" },
],

Modes

Mode	When the platform invokes you	Where the handler lives
`poll`	On the cron schedule.	`src/api/sync/<stream-key>.ts` (a route that reads `from` / `to` / cursor query params, fetches from the provider, and emits via `ingest.canonical`).
`webhook`	On every event posted to the configured webhook endpoint.	`src/api/webhooks/<provider>.ts` (a route that verifies the signature and emits via `ingest.canonical`).

The poll and backfill mapping is enforced at build/deploy time. A stream such as { key: "customers", mode: "poll" } must have a matching POST src/api/sync/customers.ts route. Webhook-mode streams do not need a sync route, but webhooks.endpoint must point at an existing POST route with the declared auth mode.

Poll streams

Each poll stream has a schedule (cron). The platform invokes the corresponding sync route with paging metadata, and the handler is responsible for:

Reading the saved cursor / page from the request.
Calling the provider’s API.
Emitting each record via ingest.canonical(...).
Returning a response that includes the new cursor and a hasMore flag for pagination.

Stripe’s customers sync is the canonical pattern — see Sync routes.

Webhook streams

A webhook-mode stream means the corresponding events arrive at the connector’s webhook endpoint (declared in webhooks: { endpoint, auth }). The webhook handler dispatches to per-stream transforms; one webhook delivery may emit records into multiple streams.

// after parsing event:
ingest.canonical(emission.canonicalType, emission.payload, {
  stream: emission.stream,                  // names the destination stream
  idempotencyKey: eventIdempotencyKey(/* ... */),
});

Stream key conventions

Lowercase + underscores: customers, payout_reconciliation, balance_transactions.
One-to-one with the provider’s logical resource name where possible.
Webhooks for events that don’t map to a single stream (e.g., a generic events topic) declare a stream named events.

Why declare them at all

Operators need to see “this connector ingests these 12 things, on these cadences, with these last-success timestamps.” The stream catalog makes that surfaceable in the dashboard without inferring from your route filenames.

← Webhooks

Overview →