Sync API reference

The sync global — checkpoints, list-page cursors, schedules, and run status.

The sync global is the state machine behind sync routes: it persists per-stream checkpoints between runs, hands your route the cursor and page size for the current run, and exposes schedule and run status. The connector-specific list helpers used in guides are thin wrappers over these calls.

import { sync } from "@backfill-io/sdk";

Use sync in poll or backfill routes when a stream needs to resume from the last successful page, cursor, date window, or partition. It does not fetch source data or ingest canonical records by itself; it gives your route the state needed to fetch the next slice and save where to continue.

What stream do I pass?

The stream is the key from your connector’s streams[] manifest. It is the same key used in:

Choose stream keys for independently scheduled and observable units of work: usually one source collection or report per poll stream. Split streams when they need different schedules, checkpoints, retries, or operator status. Keep related pages in one stream when they share the same cursor and failure semantics.

Checkpoints

A checkpoint is a JSON object you own — the platform stores and returns it verbatim per stream:

const checkpoint = sync.getCheckpoint("customers");
// e.g. { cursor: "customer_123", syncedThrough: "2026-07-01" } or null on first run

sync.setCheckpoint("customers", {
  cursor: lastRecord.id,
  syncedThrough: today,
});

Both accept { partition } as a final options argument when one stream tracks several independent cursors (for example one per location or sub-account). Partitions are keyed by the object you pass; each partition gets its own checkpoint.

List paging

For the common “fetch a page, emit it, save the cursor” loop, listPage and commitListPage manage the checkpoint for you:

import { api, Http, ingest, sync } from "@backfill-io/sdk";

export const POST = api(async (request) => {
  const page = sync.listPage(request, "customers", {
    defaultLimit: 100,
    maxLimit: 500,
  });
  // page: { stream, limit, cursor, checkpoint }

  const response = Http.get(providerUrl({ after: page.cursor, limit: page.limit }));
  const list = sync.listResponse(response.body, { recordsKey: "data", hasMoreKey: "has_more" });
  // list: { records, hasMore }

  ingest.batch("customer", list.records.map(toCanonical), { stream: "customers" });

  const checkpoint = sync.commitListPage("customers", list.records, {
    cursorField: "id",
    hasMore: list.hasMore,
  });

  return api.json({ status: "ok", imported: list.records.length, checkpoint });
});

listPage(request, stream, opts?) resolves the limit and cursor for this run — from the incoming request when present (requestLimitKeys, requestCursorKeys), falling back to the saved checkpoint (checkpointCursorKeys), clamped to minLimit/maxLimit around defaultLimit.

commitListPage(stream, records, opts?) writes the next checkpoint from the page you just processed:

listResponse(body, opts?) normalizes a provider list body into { records, hasMore }; recordsKey and hasMoreKey name the fields when they aren’t the common defaults.

Schedule and status

sync.schedule("customers", "*/15 * * * *");
const status = sync.status("customers");

schedule re-registers the stream’s scheduled job with a new cron expression, overriding the manifest schedule default. The cron is validated and rejected with an error when the stream isn’t schedulable (webhook-mode streams, disabled streams). status(stream) reports the currently active schedule.

status(stream) returns the full run picture for dashboards and debug routes: current checkpoint, activeRun / latestRun / recentRuns (each with itemsImported, itemsFailed, progressPercent, lastError), the effective schedule, plus the platform’s concurrency, rate-limit, resume, and retry policies. Statuses are idle, scheduled, or a run state (pending, running, paused, completed, failed, cancelled).

Testing

The connector test runtime persists checkpoints in memory across calls within a test, so you can run a sync route twice and assert the second run resumes from the committed cursor.