Safe Docker Compose rollouts to remote Linux devices
A container starting is not the same as a release succeeding. Here is a practical rollout model that contains failure, produces health evidence and keeps the previous version within reach.
Running docker compose up -d is a useful local action. It is not, by itself, a safe fleet deployment strategy. Once the same application runs on devices in shops, warehouses or customer sites, the important question changes from “did Compose accept the file?” to “can we prove the intended release is healthy on every target, and recover without visiting it?”
01 · THE FAILURE GAP
Why “the container started” is not success
Compose can create a container successfully while the application inside it is unusable. A web process may be listening but unable to reach its database. A display service may launch with an invalid configuration and show a blank screen. An architecture mismatch can fail immediately on one class of device even though the image worked on your development machine.
Remote fleets make these failures more expensive. Connectivity is intermittent, hardware revisions drift and an operator may not be available at the site. A sequential SSH script also loses context easily: which host received which image, which command timed out and whether a retry repeated a destructive step.
A release is complete only when the workload is running, its declared health evidence passes and the result is recorded against the exact device and version.
A safer model separates the rollout into four stages: prepare a deterministic artifact, prove that targets are compatible, deploy to one representative canary, then expand only after its health gate passes. Recovery is part of that design, not a command invented during an incident.
02 · DETERMINISTIC INPUT
Define a release contract before touching a device
The smallest useful release contract contains an application name, a unique version, a Compose document, target device identities and a test that represents usable service. Keep runtime secrets outside the Compose source and resolve them for each target at execution time.
Pin the thing you tested
A tag such as latest or even 2.4 is mutable. If someone pushes a different manifest under that tag between canary and fleet phases, two devices can receive different bytes while the dashboard reports the same version. Resolve private-registry tags to a manifest digest before creating the release, then persist the pinned reference.
services:
display:
image: registry.example.com/edge/display@sha256:YOUR_DIGEST
restart: unless-stopped
ports:
- "127.0.0.1:8080:8080"
environment:
DEVICE_SITE: ${DEVICE_SITE}
healthcheck:
test: ["CMD", "wget", "-qO-", "http://127.0.0.1:8080/health"]
interval: 10s
timeout: 3s
retries: 6
start_period: 20sThe digest makes the image immutable. The health check defines an application-level signal instead of treating process existence as readiness. Binding the port to loopback keeps the check local unless the workload truly needs public exposure.
Reference variables in Compose and store their values in a secret scope. A release should remain inspectable without revealing a registry password or application credential to every operator who can read it.
03 · BEFORE EXECUTION
Run preflight while failure is still cheap
Preflight should reject a release before a production device pulls anything. At minimum, validate the Compose shape, confirm every selected device still exists, compare the device-reported Docker platform with the image manifest and verify that the agent can execute the guarded release protocol.
For private images, inspect the Registry V2 manifest and record its digest and supported platforms. An ARM64 gateway should not become the place where you discover that an AMD64-only image was published. Unknown platform metadata should block the release rather than quietly downgrade the check.
- 01Inspect
Resolve each matching private image to an immutable manifest and collect its supported platforms.
- 02Compare
Match those platforms with operating system, architecture and variant reported by every target agent.
- 03Block or pin
Return explicit incompatibility reasons, or return a Compose document with mutable references replaced by digests.
Warnings still matter. If a Compose file mixes images from the selected private registry and an uninspected public registry, say so. The operator can then decide whether that uncertainty belongs in this change window.
04 · LIMIT THE BLAST RADIUS
Prove one representative canary, not one convenient device
The first target should represent the risk you care about: a common hardware revision, a realistic network, the same attached peripherals and the production configuration scope. A permanently connected lab machine may produce a comforting result while proving very little about the fleet.
During the canary phase, the agent pulls the pinned image, writes resolved variables to a root-only file, applies the Compose project and waits for health. The control plane should keep the remaining targets queued. Only a successful, reported result unlocks expansion.
Treat silence as uncertainty, not success
If the device disconnects after accepting a command, the result is unknown. Do not release the rest of the fleet merely because no failure arrived. Use a deadline, preserve the last reported phase and require a terminal success result from the canary.
Expansion should remain observable per device. Fleet progress is not one percentage: each target needs a state, timestamps and a useful failure reason. That evidence is what lets an operator distinguish a bad release from one damaged disk or one unavailable network.
05 · RECOVERY
Design rollback before the first rollout
A rollback needs more than the previous tag. Preserve the previous Compose content, exact image digest, health definition and the protected configuration snapshot required to start it. If those inputs are reconstructed during an incident, the “rollback” may actually be a new untested deployment.
Decide the failure boundary explicitly. A failed canary should stop expansion. A failure on a later target should keep the incident contained and offer a guarded rollback for the affected application. Avoid automatically deleting volumes: application data has a different lifecycle from containers and networks.
Restoring old containers is not enough. Apply the previous immutable release, wait for the same health contract and record whether recovery succeeded. If it fails, surface that failure instead of looping indefinitely.
Test this path on purpose. Deploy a known-bad health response to a non-production device and confirm that expansion stops, the previous release can be restored and the audit history explains both actions. A recovery control that has never been exercised is only a hypothesis.
06 · REPEATABLE AUTOMATION
Put the same guardrails behind the API
Automation should not bypass safety checks available in the console. In Felania, the preflight and release endpoints normalize the same input and apply the same compatibility rules. Use a dedicated Platform API key for the deployment workflow and keep it outside the repository.
{
"applicationName": "Edge Display",
"version": "2.4.0",
"strategy": "canary",
"deviceIds": ["dev_CANARY", "dev_FLEET_02"],
"healthTarget": "http://127.0.0.1:8080/health",
"registryCredentialId": "reg_PULL_ONLY",
"composeYaml": "services:\n display:\n image: registry.example.com/edge/display:2.4.0\n restart: unless-stopped"
}curl -fsS -X POST https://felania.com/api/v1/releases/preflight \
-H "Authorization: Bearer $FELANIA_API_KEY" \
-H "Content-Type: application/json" \
--data @release.json
curl -fsS -X POST https://felania.com/api/v1/releases \
-H "Authorization: Bearer $FELANIA_API_KEY" \
-H "Content-Type: application/json" \
--data @release.jsonInspect the preflight response before the second request. It reports ready and blocked targets, warnings, inspected image metadata and the pinned Compose output. The release endpoint repeats preflight server-side, so a caller cannot accidentally deploy after the target state has changed.
07 · BEFORE PRODUCTION
A compact rollout checklist
- ✓
Artifact — Every private image is resolved to a digest and supports every target platform.
- ✓
Configuration — Secrets are stored separately and resolved for the intended workspace, site or device.
- ✓
Health — The check represents usable service and has realistic startup and timeout windows.
- ✓
Canary — The first device represents production risk, not merely the easiest connection.
- ✓
Evidence — Each target keeps state, timestamps, output and an auditable actor.
- ✓
Recovery — The previous immutable release is retained and rollback has been tested end to end.
The mechanics are straightforward. The discipline is in refusing to infer success from incomplete evidence. Prepare deterministic inputs, move uncertainty into preflight, learn on one target and keep recovery close enough to use under pressure.