Wire Tap

Plug an observer onto an existing channel without modifying it — like a snitch on a phone line, but legal and documented. Every message goes through two destinations: the original receiver and the observer. Essential to feed lake, audit and KPIs without touching the business pipeline.

Problem

Once an EDI pipeline is in production, you often want to plug "parasitic" consumers in: a data lake for analytics, a Message Store for audit, a real-time monitor for the dashboard. But you do not want to modify the existing pipeline — each change is a production risk. And you do not want an observer outage to affect the business flow. A way to graft read-only, fire-and-forget observers is needed.

Forces

• Non-intrusiveness. The source channel does not know it is tapped; adding or removing a tap changes nothing.
• Fault isolation. An observer's failure does not block the business pipeline. Tap = fire-and-forget.
• Duplication cost. Each tap = a payload copy. For large messages, combine with Claim Check.
• Leak risk. A poorly protected tap becomes an exfiltration channel. Secure it at the same level as the main flow.
• GDPR compliance. The tap moves personal data to new contexts. Anticipate a DPIA.

Solution

EIP §547 (Hohpe & Woolf, 2003) defines the Wire Tap as a component inserted in a channel which: (a) receives each message, (b) forwards it unchanged to the initial receiver, (c) sends a copy on an observation channel. In practice on a modern broker: it is an additional consumer-group on a Kafka topic, or a fan-out RabbitMQ exchange, or a Service Bus topic subscriber. The tap must be fire-and-forget on the source side — if the observer crashes, the channel keeps going.

┌──────┐                ┌──────┐
   │  A   │ ───────────▶  │  B   │
   └──────┘                └──────┘
              │
              │ wire tap = T-junction
              ▼
        ┌──────────┐
        │ observer │ ──▶ lake / store / audit
        └──────────┘
        (read-only copy, no impact on A→B)

EDI implementation

Four canonical EDI uses of the wire tap:

# Kafka wire-tap configuration
tap:
  source_topic: edi.canonical.invoic.canonical
  observer_topics:
    - lake.invoic.raw           # data warehouse ingestion
    - audit.invoic.replay       # message store
    - kpi.invoic.live           # live dashboard
  copy_mode: full               # or: headers-only, sampled-1%
  pii_redaction: enabled        # mask IBAN, email, phone
  fire_and_forget: true         # does not impact source if observer down
  consumer_group: tap-fanout

• Analytics lake feed. Every INVOIC flowing on edi.canonical.invoic.canonical is copied into lake.invoic.raw, ingested into BigQuery / Snowflake / Athena for business KPIs (top suppliers, seasonality, error rates).
• Message Store. The tap feeds the Message Store for legal archival, independent of business reprocessing.
• Compliance and audit. For CTC fiscal flows (Italy, France), a dedicated tap sends an encrypted copy to a sealing / timestamping partner producing the cryptographic proof required by the administration.
• Real-time observability. A tap sampled at 1% feeds a monitoring dashboard (latency, size, error rate) without drowning Grafana under the full volume.

GDPR and masking

Any tap directing data to a new context (lake, analytics, dashboard) must: (a) be justified by a legal basis (legitimate interest, contractual execution, legal obligation), (b) apply minimisation — mask what is not necessary for the use. A KPI dashboard does not need the IBAN; the analytics lake does not need the consumer buyer's full name. Pseudonymise or hash at tap time.

Anti-patterns

• Blocking synchronous tap. If the source channel waits for observer ACK, the latter's outage blocks the business flow. Always fire-and-forget.
• Modifying tap. A tap that alters the message breaks the invariant: two versions of the message circulate. Keep strictly read-only.
• No redaction. Copying IBANs or personal addresses into an unencrypted lake opens major GDPR risk.
• Hidden tap. An undocumented tap is a leak risk. Mandatory catalogue (who taps what, where to, for what use).
• Exhaustive tap where sample suffices. For telemetry, 1-10% sampling is enough; don't copy 100% of the payload.

Related patterns

• Message Store — fed by tap.
• Publish-Subscribe Channel — a tap is essentially an extra subscriber.
• Message History — the tap also records its passage in the history for traceability.
• Claim Check — avoid duplicating large payloads at tap time.

Sources

• Hohpe G., Woolf B. — Enterprise Integration Patterns, Wire Tap (§547). enterpriseintegrationpatterns.com — Wire Tap
• GDPR — Regulation (EU) 2016/679. Article 5 (minimisation), article 25 (privacy by design). Every tap must respect these by design.
• Apache Camel — Wire Tap EIP. The most used open-source implementation of the pattern. camel.apache.org — Wire Tap
• AWS Kinesis Data Firehose. Modern cloud service that natively implements the pattern to feed lake and analytics from a flow. aws.amazon.com/kinesis/data-firehose