Taskhoster

Host durable task queues for your apps. Drop a folder, get a task queue.

Taskhoster/
  task.example.com/           your queue host - one folder
    taskhoster-queues/
      resize.json              a queue: item, progress & result schemas + settings
      thumbnail.json           another queue - its own endpoint and socket
    taskhoster-users/
      alice.json               a user: a name and a 32-hex access token

How it works The family

How it works

The same idea as localhoster, pointed at work instead of pages. A folder named for a domain becomes a live queue host on your own server; declaring a queue inside it gives you an endpoint that takes tasks, hands them to workers, and reports results. All durable, each task its own file on disk.

Producers post tasks

An app POSTs a task to a queue's endpoint and gets back an id. It can fetch that id's status any time, or subscribe to it on a WebSocket and watch it move.

Workers pull & process

A worker asks the queue for a task it can handle (a query over the item schema), is handed one exclusively, reports progress, and posts a result when it's done.

Durable & reliable

Every task is a file on disk. Leases, timeouts and re-queues make sure a task that a worker drops gets picked up again. A slow worker's late result is still used if nobody beat it.

Built for workers that come and go

Distributed work is messy: workers crash, stall, and finish out of order. Taskhoster's lifecycle is designed around that so you don't have to be.

Nothing gets stuck

A claimed task is leased. If the worker's lease lapses (or its socket drops), the task returns to the queue for someone else. A whole-queue timeout ends a task that no one can finish.

Slow work still counts

A worker that was thought gone can still hand in its result, and it's accepted as long as no other worker finished it first, so effort is never wasted when everyone is slow.

Schemas on every side

Each queue defines the shape of its items, its progress records and its results, so producers and workers always agree, and errors ride back in-band in the result.

Just enough auth

Define a domain's users as small files: a name and an access token. Each queue then says who may post, read status and work: the public, or named users you list.