What a Workflow Execution Kernel Is and Why Sagas Need One
Picture an online shop’s order code. Reserve the stock. Charge the card. Print a receipt. Book the courier. Email the customer. Five calls, five different systems, and no database transaction that spans them, because the card network and the courier don’t take part in your transactions.
Most of the time all five work. The trouble is the other times. The courier’s API refuses the address after the card was charged. Or the server restarts during the charge, and nobody knows whether it went through. Each of those leaves the order half done, and half done is the worst state there is: money taken, nothing shipped, and the code that knew what to do next gone with the process.
The pattern for this has a name. A saga is a long job split into steps, where each step that changes the outside world has a matching step that changes it back. If the job can’t finish, you don’t roll back a transaction. You run the undo steps, newest first: refund the charge, then release the stock. That reversing is called compensation.
The Hard Part Isn’t the Undo
Writing a refund function is easy. Knowing when to call it is the hard part, and it gets harder exactly when things go wrong. After a crash, something has to know that this order completed reserve and charge, failed at ship, and was halfway through compensating when the lights went out. That something has to have written it down before each step, not after, and somewhere that survives the crash.
That is what a workflow execution kernel does. It is the small core that keeps a job’s place:
- It moves each run from step to step (the state transitions).
- It writes a checkpoint before and after each step, and flushes it to disk before going on.
- It retries a failed step after a wait, a set number of times.
- It runs the undo steps, newest first, when a run can’t finish.
- It reads its checkpoints back after a restart and resumes every run where it stopped.
Everything else people associate with workflow tools sits on top of that: the drag-and-drop designers, the dashboards, the hosted services with their own clusters and databases. Those are useful products. They are also optional, and for a lot of software, much bigger than the problem.
Durable Execution Without the Platform
The industry term for the kernel’s job is durable execution: code that survives the process running it. The best-known tools for it are platforms. Temporal runs as a cluster your program talks to. Others keep their checkpoints in Postgres, which is lighter but still needs a database server beside your program.
FlowChassis takes the smallest possible footprint: a Go library and one journal file. Each checkpoint is a short record appended to that file, with a checksum, flushed to disk before the step moves on. When several runs write at once, their records share one flush. After a crash, the engine reads the file from the start and knows exactly where every run stands. If the crash tore the last record in half, that half is cut off and the run carries on from the record before.
That makes it a fit for places a platform can’t go: a command-line tool, a single-binary server, an edge device, a desktop app, a product you ship to customers who won’t run your database for you.
The One Rule to Remember
A kernel like this can promise that every step runs. It can’t promise every step runs exactly once. If the process dies after the card was charged but before the checkpoint said so, the step runs again on restart, and the charge is attempted twice.
The fix is old and dependable: idempotency keys. FlowChassis gives every step a key that stays the same on every attempt and after every restart, like order-1001/charge/2. Pass it to the payment provider, and the second charge is recognized as the first. Many payment, shipping and messaging APIs accept a key like this for exactly this reason.
That’s the whole idea. Steps, checkpoints, retries, compensation, resume. The quick start shows it in one short Go program, and the testing post shows what happens when you kill it.