Containment without Containers – Sturdy Statistics

Constraining a single-node service with systemd uses only as much abstraction as the problem requires.

A Linux process inherits a surprising amount of authority. Even when it runs as a non-root user, it can:

Read large portions of the filesystem
Open outbound network connections
Bind sockets
Inspect /proc
Access devices
Create namespaces
Invoke a wide range of system calls

Because of this, most services run with far more authority than they actually require. For us, the key question is:

Assuming an critical-but-unknown bug exists in our code, what is the process permitted to do?

That question shapes how we deploy our services at Sturdy Statistics.

Security as Structural Constraint

There are two broad ways to approach security:

Monitor everything and respond quickly.
Remove authority so there is less to monitor.

We prefer the second wherever possible.

We are a small team. Security that depends on perfect human behavior does not scale for us. Therefore, we build in structural constraints wherever possible. Our post introducing bailey and malli-firewall demonstrates how we layer security within our apps; here we describe how we layer constraints outside the app, in our deployment infrastructure.

Our infrastructure design presumes a breach. With this mindset, the meaningful question is not whether a vulnerability exists, but how much damage a compromised process can cause.

Containers are a Solution, But not the Only One

Modern infrastructure culture often defaults to containers. Containers are powerful abstractions over the OS which package:

Namespaces
cgroups
Capability dropping
Seccomp filters
Filesystem isolation

But these are not strictly “container features.” They are actually Linux primitives.

For single-node services, we prefer to use the kernel directly. We do not need an orchestration layer, a control plane, or a container runtime to constrain one process. systemd exposes these primitives declaratively, and it lets us treat the kernel itself as a policy engine.

I don’t intend this to be a critique of containers, nor a prescription for others. It is simply the simplest abstraction that works for us, and it keeps our deployment model close to the operating system and easy for us to reason about.

Defense in Depth

As we described in our post about make as an orchestration tool of Ops, each of our production services is a Clojure Ring/Jetty server running as a dedicated, unprivileged user with no home directory and no shell.

The process is a signed, self-contained Uberjar. Because we assume any single security control might fail or be bypassed, we use systemd to enforce multiple, independent layers of defense.

Here is how we layer those constraints.

1. Supply Chain: Artifacts are Verified Before Execution

Before the JVM starts, we verify the signature on the deployed artifact:

ExecStartPre=/usr/bin/bash -c '\
  gpg --no-default-keyring \
      --keyring /usr/share/keyrings/sturdy-release-signing.gpg \
      --verify current-standalone.jar.asc current-standalone.jar \
'

If signature verification fails, the service does not start. No signature, no process.

There is no external deployment control plane enforcing this rule. The host itself refuses to execute unverified code. This protects us from supply-chain attacks: the Uberjar bundles all dependencies – nothing is fetched at runtime – and the entire artifact is signed with a tightly controlled GPG key.

2. Capabilities: The Process has None

CapabilityBoundingSet=
AmbientCapabilities=
NoNewPrivileges=yes

The service inherits no Linux capabilities, it cannot escalate privileges, and NoNewPrivileges=yes ensures it cannot acquire additional authority via execve.

These restrictions remove entire classes of operations rather than relying on runtime detection.

3. Isolation: The Filesystem is Read-Only by Default

ProtectSystem=strict
ProtectHome=yes
PrivateTmp=yes

ReadWritePaths=/opt/${SERVICE_NAME}/logs /opt/${SERVICE_NAME}/artifacts

The root filesystem is mounted read-only for the service. Home directories are inaccessible, and temporary files are private to the process. Only explicitly declared directories are writable.

This inverts the Linux default: nothing is writable unless we explicitly opt into it.

4. Kernel Surface: System Calls are Restricted

SystemCallFilter=@system-service
SystemCallFilter=~@privileged @resources @reboot @setuid
SystemCallArchitectures=native

We allow only a conservative set of system calls appropriate for long-running services, and we explicitly deny categories associated with privilege escalation or resource manipulation.

Again, this is structural. The process simply cannot perform certain operations, even if it is compromised.

5. Data Protection: Secrets are Injected as Credentials

As we discussed in our post about bailey, each service has its own automatically managed keychain for envelope encryption. This keychain is unique per host and encrypted using its own TPM-sealed password.

LoadCredentialEncrypted=password.bytes:/etc/${SERVICE_NAME}/password.bytes.cred

We use TPM-sealed secrets and systemd’s credential mechanism rather than plaintext environment variables or ad-hoc secret distribution.

The secret is stored encrypted, delivered in memory at runtime, tightly scoped to the service, never written in plaintext to disk, and absent from configuration files.

TPM sealing means that even if an attacker exfiltrated our database or filesystem, or even a full backup image, decryption would require the exact host with its hardware, boot state, and runtime measurements intact.

6. Exfiltration Resistance: The Network Surface is Explicit

Our service binds only to loopback and cannot reach the public internet directly.

IPAddressAllow=127.0.0.1
IPAddressDeny=any

SocketBindAllow=ipv4:tcp:3000
SocketBindDeny=any

The application listens on 127.0.0.1:3000. Inbound traffic is proxied through Nginx. Outbound requests must pass through a local proxy that blocks any domain outside of a short whitelist of the APIs we use.

Even if an attacker managed to hijack the process and execute arbitrary code, this final layer makes meaningful data exfiltration significantly more difficult.

What This Buys Us

Visibility and Control

All trust boundaries live in a single, human-readable unit file in /etc/systemd/system/. There is no hidden abstraction layer and no additional tooling beyond standard Linux. Because the sandbox is declarative, we can ask the system to evaluate it:

$ systemd-analyze security roughtime
  NAME                                                         EXPOSURE
✓ SystemCallFilter=~@swap
✓ SystemCallFilter=~@resources
...
✗ MemoryDenyWriteExecute=                                           0.1
✗ RestrictAddressFamilies=~AF_(INET|INET6)                          0.3
✗ PrivateNetwork=                                                   0.5
✗ DeviceAllow=                                                      0.1
...
✓ CapabilityBoundingSet=~CAP_MAC_*
✓ RestrictSUIDSGID=
✓ UMask=

→ Overall exposure level for roughtime.service: 1.0 OK 🙂

Defense in Depth

Because these controls are independent, a failure in one does not compromise the whole. If a code vulnerability allows RCE, the filesystem is still read-only. If the attacker escapes the filesystem, they still have no capabilities. If they find a kernel exploit, they still cannot access the TPM-sealed keys on another host. If they hijack the process and execute arbitrary code, they cannot exfiltrate data without another exploit.

We assume failure and work to constrain it.

Minimal Setup

systemd ships with every major Linux distribution. There are no agents, sidecars, or runtime daemons beyond what the OS already provides. This also means we don’t introduce other tools into our supply chain which we need to watch for vulnerabilities.

Portability

Our deployment model works on any standard Linux host. We are not tied to a specific container runtime, orchestration platform, or cloud provider.

Explicitness

Every authority the service has is declared. Anything not declared is denied.

What This is Not

This is not an orchestration strategy. It does not manage fleets, reconcile distributed state, or autoscale. If we were running hundreds of nodes across multiple regions, we would of course reach for different tools.

But for a single-node, security-sensitive service, we prefer to use only as much abstraction as the problem requires.

The Broader Philosophy

At Sturdy Statistics, we have found that clarity pays dividends. We try to make structure explicit: in our probabilistic models, in our data pipelines, and in our infrastructure. systemd lets us constrain the authority our services run with deliberately and explicitly.

For our scale and threat model, this approach has proven both sufficient and reassuringly transparent.