# Syncing Accounting Data at Scale

Most accounting integrations start the same way: a customer connects QuickBooks or Xero, and you list invoices. That works until the customer has years of history, tens of thousands of transactions, and a downstream system that answers slowly. This guide is the accounting-specific playbook for that case. It covers the full sync lifecycle — an initial historical backfill, a steady-state incremental loop, and a reconciliation safety net — plus the mechanics that keep each phase from falling over: cursor pagination, high-water marks, request timeouts, rate limits, and webhooks. All paths use Apideck's [Accounting API](/apis/accounting/reference).

>
> **Apideck reads live, not from a cache.** Many integration platforms front the accounting system with a managed sync layer, so a freshly connected customer waits — sometimes hours — for an initial sync to populate before any data is available, and later reads can lag behind the source. Apideck queries the connected system directly in real time, so every call returns current data the moment a connection is authorized. The sync loops below are for *your* store (a warehouse, an analytics DB), built on top of always-current reads — you are never blocked waiting for Apideck to warm up.

## The sync lifecycle

Treat a connected accounting system as three distinct workloads, not one. Each has different volume, latency, and failure characteristics, and conflating them is the most common reason large syncs stall.

| Phase | When it runs | Shape | Primary risk |
| --- | --- | --- | --- |
| **Initial backfill** | Once, right after a connection is authorized | Bounded but large — the customer's entire history | Timeouts and rate limits on deep pages |
| **Incremental sync** | On a schedule (or on webhook) | Small — only what changed since last run | Missed or double-counted changes |
| **Reconciliation** | Periodically (daily/weekly) | Full re-list, compared against your store | Drift from deletes and closed-period edits |

The rest of this guide walks each phase and the shared mechanics they rely on.

## Pagination: always cursor, never offset

Every list endpoint in the Accounting API returns a cursor envelope. Read the next cursor from `meta.cursors.next` and pass it back as the `cursor` query parameter. `limit` accepts 1–200 and defaults to 20 — for backfills, set it to 200 to minimize round trips.

```
GET /accounting/invoices?limit=200
```

The response envelope:

```json
{
  "data": [ /* ... invoices ... */ ],
  "meta": {
    "items_on_page": 200,
    "cursors": {
      "previous": "em9oby1jcm06OnBhZ2U6OjE=",
      "current": "em9oby1jcm06OnBhZ2U6OjI=",
      "next": "em9oby1jcm06OnBhZ2U6OjM="
    }
  },
  "links": {
    "next": "https://unify.apideck.com/accounting/invoices?cursor=em9oby1jcm06OnBhZ2U6OjM%3D"
  }
}
```

Loop until `meta.cursors.next` is absent:

```
GET /accounting/invoices?limit=200
GET /accounting/invoices?limit=200&cursor=em9oby1jcm06OnBhZ2U6OjM=
GET /accounting/invoices?limit=200&cursor=<next cursor>
...
```

>
> Do not paginate by incrementing an offset or re-listing with a higher page number. Cursors encode downstream pagination state that offset math cannot reproduce, and some connectors (NetSuite, Workday-style large directories) will silently skip or repeat rows if you re-issue a list query instead of following the cursor.

Request only the fields you persist with the `fields` projection parameter. On a wide object like an invoice with line items, narrowing the payload is one of the largest wins for both latency and rate-limit headroom:

```
GET /accounting/invoices?limit=200&fields=id,number,total,updated_at,status
```

## Phase 1: the initial backfill

The backfill pulls the customer's full history once. Two rules keep it survivable.

**Bound it by date, not by "everything."** Most planning, lending, and analytics use cases don't need the full ledger back to inception. Anchor the backfill to a start date and page forward from there:

```
GET /accounting/invoices?limit=200&filter[updated_since]=2022-01-01T00:00:00Z
```

**Checkpoint every page.** Persist the last successful `cursor` per `(consumer_id, service_id)` pair before requesting the next page. If the backfill dies at page 400, resume from the stored cursor rather than restarting. Long backfills *will* be interrupted — by a redeploy, a rate-limit pause, or a downstream hiccup — so make interruption cheap.

### Slice by timeframe with filters

Filters make it trivial to pull exactly the window you want, which is the single most effective way to keep large syncs manageable. Instead of one enormous unbounded pull, request the data in bounded timeframes:

- **List endpoints** (`invoices`, `bills`, `payments`, `journal-entries`) accept `filter[updated_since]`, so you can walk history in windows — a month, a quarter, a year at a time — and checkpoint per window:

  ```
  GET /accounting/journal-entries?limit=200&filter[updated_since]=2024-01-01T00:00:00Z
  ```

- **Report endpoints** (`profit-and-loss`, `balance-sheet`, `aged-debtors`, `aged-creditors`) accept `filter[start_date]` and `filter[end_date]` (with `period_count` / `period_type`), so you can pull any reporting period directly rather than reconstructing it from raw transactions:

  ```
  GET /accounting/profit-and-loss?filter[start_date]=2024-01-01&filter[end_date]=2024-12-31&filter[period_count]=12&filter[period_type]=month
  ```

Windowing a backfill this way turns one long-running request into many short, independently retryable ones — each cheap to resume, easy to parallelize across windows, and far less likely to hit a timeout or rate limit.

For a genuinely large one-off backfill that you cannot page or window any smaller, see [Handling long-running requests](/guides/handling-long-running-requests): switch the host to `longrun.apideck.com` and raise `x-apideck-timeout`. Reach for this only after you have already applied filters, `fields`, and `limit=200` — it is headroom for known-slow connectors, not a substitute for optimizing the request.

## Phase 2: the incremental sync

Once the backfill completes, stop re-listing everything. Switch to `filter[updated_since]` and pull only what changed.

```
GET /accounting/invoices?limit=200&filter[updated_since]=2025-01-15T00:00:00Z
```

The pattern that makes this reliable is a **high-water mark** per `(consumer_id, service_id)`:

1. After each successful page, record the largest `updated_at` you saw.
2. On the next run, set `updated_since` to that high-water mark **minus a small overlap window** (15 minutes is a safe default) to absorb clock skew and out-of-order writes upstream.
3. Deduplicate on the resource `id` when you persist, so the overlap window can safely re-deliver a handful of rows without double-counting.

For ledger-level use cases, `GET /accounting/journal-entries` is the canonical change stream: any posting that touches the P&L or balance sheet appears there. For document-level flows, sync `invoices`, `bills`, `payments`, and `credit-notes` on the same `updated_since` mechanism.

>
> `updated_since` filters on the *modification* timestamp, so it captures edits to old records, not just new ones. That is exactly what you want — but it means a closed-period adjustment made today will surface with today's `updated_at`. When one lands, recompute the affected period rather than trusting a cached aggregate. Closed-period edits are the single biggest source of variance drift.

## Phase 3: reconciliation and deletes

Incremental sync has one blind spot: **deletes.** A record removed upstream stops appearing in list responses, but `updated_since` never emits a "this was deleted" row. Two mechanisms close the gap.

- **Periodic full reconciliation.** On a slower cadence (daily or weekly), re-list the resource with `updated_since` set far enough back to cover the window, compare the returned `id` set against your store, and mark anything missing as deleted. This is also your correctness backstop against any incremental run that silently dropped a page.
- **Webhooks for deletes and real-time changes** (below), which deliver delete events explicitly.

## Scaling past polling: webhooks

Polling on a schedule is the right place to start, but it scales poorly once a customer has high write volume or you're managing many consumers. When polling becomes the bottleneck, subscribe to [Apideck's unified webhooks](/guides/webhooks) and treat the periodic full sync as a reconciliation safety net rather than the primary data path.

Subscribe to the accounting lifecycle events you care about — for example `accounting.invoice.updated`, `accounting.payment.created`, `accounting.bill.created`, and the equivalents for the other resources — and drive your store from the event stream. The webhook payload tells you *what* changed; fetch the current record by `id` to get the full object. Signature verification and replay semantics are in the [Webhooks guide](/guides/webhooks) and the [Webhook signature verification guide](/guides/webhook-signature-verification).

Even with webhooks, keep a low-frequency full reconciliation running. Events can be missed (endpoint downtime, delivery failures), and reconciliation guarantees eventual consistency regardless.

## Rate limits and throttling

A backfill loop at `limit=200` is the workload most likely to hit rate limits. Respect them rather than retrying blindly:

- On a `429`, back off using the `Retry-After` / rate-limit response headers before the next request. See [Understanding unified rate limits](/guides/unified-rate-limits) for the header semantics and per-connector behavior.
- Run backfills **serially per connection** — a single connection paginating one page at a time is both simpler and less likely to trip downstream limits than parallel page fetches.
- Parallelize **across** connections (different consumers) if you need throughput, not *within* one connection.

## Connector-specific behavior

The unified model smooths over most differences, but a few connectors have quirks that matter specifically for large syncs. Verify current support in the [live coverage matrix](/apis/accounting/coverage) before relying on any single behavior in production.

| Connector | Notes for large / historical syncs |
| --- | --- |
| QuickBooks | The P&L endpoint rolls sub-accounts up by default; the journal-entries feed exposes them at the leaf level. Pick one grain and keep it consistent across the backfill and incremental loop. |
| Xero | `updated_since` is well-supported and reliable as the incremental key across invoices, bills, and journal entries. |
| NetSuite | The most segment-rich connector; deep pages are slow. Always follow the cursor, keep `limit` at 200, and consider `longrun.apideck.com` for the initial backfill. For OneWorld, use the subsidiary value to attribute rows to a legal entity. |
| Sage Intacct | User-defined dimensions may be flattened in the unified shape. If a customer reports a mismatch, check whether the slice is a UDD before assuming a sync bug, and use the [Proxy API](/apis/proxy/reference) to pull it natively. |
| Exact Online | Closed-period postings surface via `updated_since`, so they are captured by the incremental loop — recompute the period when they arrive. |

## Putting it together

A production accounting sync is these three loops running per connection:

1. **On connect** — bounded historical backfill, cursor-paginated at `limit=200`, checkpointed per page.
2. **On schedule (or webhook)** — incremental pull keyed off an `updated_since` high-water mark with a 15-minute overlap, deduped by `id`.
3. **Periodically** — full reconciliation to catch deletes and heal any dropped pages.

Get those three right and the sync stays correct as the customer's history grows from months to years.

## Next steps

- [Handling long-running requests](/guides/handling-long-running-requests) — `longrun.apideck.com` and `x-apideck-timeout` for large backfills
- [Understanding unified rate limits](/guides/unified-rate-limits) — back-off headers and per-connector limits
- [Webhooks](/guides/webhooks) and [Webhook signature verification](/guides/webhook-signature-verification) — move from polling to events
- [FP&A with the Accounting API](/guides/fpa-with-accounting-api) — a worked backfill + incremental pipeline for period actuals
- [Accounting API reference](/apis/accounting/reference)