Database reference

This reference describes how Ponder interacts with the database, which may be useful for advanced workflows around zero-downtime deployments, crash recovery, and database administration.

Instance lifecycle

When a Ponder instance starts up, it runs the following logic (simplified for clarity).

1. Start returning 200 from the /health endpoint. Return 503 from the /ready endpoint.
2. Generate a new instance_id (a random four-character string).
3. For each table defined in ponder.schema.ts, create a new table in the database named {instance_id}__{table_name}. If the user has specified a custom database schema in ponder.schema.ts, use it.
4. Begin the historical backfill (starting from scratch) and write data to the tables created in the previous step.
5. When the historical backfill is complete, for each table create a view named {table_name} as SELECT * FROM {instance_id}__{table_name}. If a view with that name already exists, drop the old one first.
6. Start returning 200 from the /ready endpoint.
7. On shutdown (e.g. due to a process exit signal), do not drop the tables or views.

The following sections describe possible deviations from this standard lifecycle.

Live views

Instances running ponder dev create the live views immediately. Instances running ponder start create the live views as described above, just before the instance begins responding as ready.

Build ID and crash recovery

During the build step, each instance generates a content hash of the entire app (config, schema, and indexing function file contents). This is called the build_id.

Then, each instance checks the _ponder_meta registry table for any instances with the same build_id that are no longer running. If it finds one, the current instance will adopt the prior instance's instance_id and resume indexing where the prior instance left off.

Crash recovery is disabled when running ponder dev.

Stale table cleanup

Shortly after startup, each instance checks the _ponder_meta registry table and drops any tables belonging to instances that are not currently running, aside from the 3 most recent instances.

Tables belonging to non-running ponder dev instances will always get dropped. They do not count towards the limit.

RPC request cache

Ponder caches RPC request data to improve reindexing performance on hot reloads and redeployments.

• The RPC request cache is located in the ponder_sync schema.
• The cache is durable, persists across instance restarts, and does not store unfinalized data.
• The cache query patterns are designed to be lock-free. It's safe for multiple instances to run in the same Postgres database at the same time (reading and writing to the same ponder_sync schema).