Secrets Management

Secrets never sit in plaintext — ever. They are injected at runtime, rotated on schedule, accesses logged.

Problem

A plaintext secret (DB password, API key, AS2 partner certificate) leaks far more often than expected: accidental Git commit, unencrypted CI/CD log, paste into a wiki, terminal screenshot, .env backup. Once leaked, rotation is manual, slow, and forgotten. The massive TJX (2007), Uber (2016), CodeCov (2021) breaches were enabled by unmanaged secrets.

Forces

• Secrets are numerous and heterogeneous (DB, API, signing keys, AS2 partner passwords).
• Manual rotation does not scale — automation required.
• A specialised vault is expensive, but a leak is more so.
• Access must be audited: who read which secret, when, from where.
• The vault becomes a critical target — HA, monitoring, tested restore required.

Solution

Centralise secrets in a dedicated vault (HashiCorp Vault, AWS Secrets Manager, GCP Secret Manager, Azure Key Vault) that: (1) encrypts at rest with a master key derived from an HSM; (2) grants access via identity (IAM role, JWT, Kubernetes ServiceAccount) — no long-lived creds; (3) generates dynamic credentials (Vault dynamic secrets) with short TTLs; (4) automates rotation; (5) logs every read/write. Applications never store the secret; they fetch on init or via sidecar (Vault Agent, CSI Secrets Store).

Lifecycle

┌──────────────┐    rotate    ┌──────────────┐
│  Generation  │ ─────────────►│  Distribution │
└──────┬───────┘               └──────┬────────┘
       │ store                        │ fetch
       ▼                              ▼
┌──────────────┐    rotate    ┌──────────────┐
│   Storage    │ ◄─── audit ──┤  Application  │
│   (Vault)    │              │ (in-memory)   │
└──────┬───────┘               └──────┬────────┘
       │                              │ logout/expire
       │ revoke                       ▼
       ▼                       ┌──────────────┐
┌──────────────┐              │  Revocation   │
│  Audit log   │ ◄────────────┤   (manual or  │
└──────────────┘              │   automatic)  │
                              └──────────────┘

EDI implementation

An EDI hub manages dozens of secrets: (a) S/MIME / AS2 certificates per partner — rotated every 12 months (PEPPOL caps at 2 years); (b) ERP API keys (SAP, Oracle, Workday); (c) outbound SFTP partner creds; (d) OAuth2 / OIDC tokens for REST CTC APIs (Italy SDI, France Chorus Pro). Implementation: Vault with a KV-v2 mount per partner, access via Kubernetes Vault Agent injector. Auto-rotation via Vault rotate_period. AS2 certificate rotation is notoriously complex (partner coordination) — handle an overlap window where both certs are valid.

Anti-patterns

• Plaintext secret in Git (even in a private repo) — almost certain eventual leak.
• Secret as pod environment variable without vault — visible via ps aux, /proc/.../environ.
• "When we have time" manual rotation — we never do.
• Vault without HA — vault down = applications down.
• No audit log — impossible to respond to an incident.
• Vault access policy with wildcard * — defeats least privilege.

Related patterns

• Least Privilege — Secrets Management is the credentials-side instrument.
• Zero Trust Architecture — demanding on secret handling.
• Mutual TLS — Secret Manager client for certs.
• Defense in Depth — identity layer.

Sources

• HashiCorp Vault — Concepts documentation. developer.hashicorp.com/vault/docs/concepts
• AWS — Secrets Manager Best Practices. docs.aws.amazon.com/secretsmanager/latest/userguide/best-practices.html
• OWASP Cheat Sheet — Secrets Management. cheatsheetseries.owasp.org/cheatsheets/Secrets_Management_Cheat_Sheet.html