Upgrades

A queue is a file, and changing it is just editing that file and syncing again. The interesting question is what happens to items already on the server when the definition changes underneath them. This page covers evolving a queue without breaking work in flight.

How a change lands

Edit the queue's JSON, then sync. The local package validates every definition locally first, so a malformed queue never reaches the server. It compares your files with the server's manifest by content hash and uploads only what changed; the server re-validates and replaces the definition atomically. From the next request on, that definition is in force.

Definitions are the only thing synced. Items live on the server and are never touched by a sync: a new definition changes how future requests are validated, not the items already stored.

The local package validates before syncing

Both the local package and the server parse a definition through the same code, so a mistake is caught the moment you sync (naming the file and the problem), rather than surfacing as a failed request later:

node taskhoster-local.js --config config.json --sync
# [taskhoster-sync] task.example.com/taskhoster-queues/resize.json: taskhoster-schema-item field
#   "format": enum needs a non-empty "taskhoster-values" array

Nothing uploads until every definition parses. So a broken edit can't take a queue down; the sync just refuses and the previous definition stays in force.

Adding and changing fields

Fields are validated per submission, so the safe moves are the ones that don't invalidate requests that were already valid:

Change	Safe?	Why
Add an optional field	Yes	Old producers omit it; it is simply absent on their items. New producers can start sending it.
Add a `required` field	Careful	Producers that don't yet send it will get `400`. Roll out the producer change first, or add it optional and tighten later.
Relax a constraint (raise `max`, add an `enum` value)	Yes	Everything that validated before still validates.
Tighten a constraint (lower `max`, remove an `enum` value, make a field required)	Careful	Submissions that were fine may now be rejected. Coordinate with producers.
Remove a field	Careful	Producers may still send it; it will be dropped as an unknown field. Workers that read it get nothing. Update workers to match.
Add fields to `progress` / `result`	Yes	Optional additions don't affect existing workers; they simply won't set the new field.

Because unknown fields are dropped rather than rejected, a producer and a worker can be upgraded a little out of step without errors; the missing data is just absent until both sides speak the new shape.

Changing settings

The settings are read fresh by the sweep and by each request, so a change takes effect for the whole queue at once, including items already stored:

Setting change	Effect on existing items
`leaseSeconds`	Applies to the next claim's lease. Items already leased keep the lease they were given.
`queueSeconds`	Applied against each item's original `postedAt`. Shortening it can immediately time out items already older than the new bound; lengthening it gives in-flight items more time.
`retainSeconds` / `retainFrom`	Applied by the next sweep to done items. Shortening retention can make already-finished results eligible for deletion sooner.

Lowering queueSeconds or retainSeconds acts on items that already exist. If a queue holds live work, prefer lengthening timings, or drain it before tightening them.

Items already in flight

A stored item keeps the item data it was posted with; a schema change never rewrites it. What changes is validation of new submissions against it: a worker posting progress or result is validated against the current result schema, so if you tighten that schema, in-flight workers must post results that satisfy the new shape. Keep result-schema changes additive while items are in flight, and roll out worker changes alongside.

Renaming or deleting a queue file removes it from the server on the next sync (the local package deletes any definition the server has that you no longer keep). Any items still stored for it stop being reachable through the (now absent) endpoints. Drain a queue before you retire it.