ediverse

Type to search across standards, messages, tools and glossary.

ediverse Explore the platform

Spotlight PEPPOL BIS Billing 3.0 The EU e-invoicing mandate is here — France Sept 2026, Belgium Jan 2026, Germany 2025.

— 01

Foundations

— 02

Standards

— 03

Flagship messages

— 04

Tools

Normalizer

When multiple partners send messages in different formats describing the same business concept (an order), the Normalizer is the component that detects each input's format and routes to the appropriate Message Translator. It is the canonical front door of an integration hub.

Problem

A modern EDI platform receives ORDERS in 4-5 different formats depending on the partner: EDIFACT D.96A for historical European suppliers, X12 850 for North-American suppliers, cXML for buyers on Coupa, UBL for PEPPOL flows, IDoc for point-to-point SAP integrations. All these variants represent the same business information — an order — but are structurally incompatible. Without upstream normalisation, every order consumer (CRM, ERP, WMS) would have to know all 5 formats.

Forces

  • Heterogeneous sources. No partner will change their format to align with the others. Normalisation must be absorbed internally.
  • Lightweight detection. For performance and reliability, detection must rely on the first few bytes — not on a full parse.
  • Extensibility. Adding a new format (e.g. OAGIS BOD) must happen without changing the rest of the pipeline.
  • Canonical first. The downstream pipeline must see a single structure — the canonical. Everything else is upstream cleanup.

Solution

EIP §352 (Hohpe & Woolf, 2003) models the Normalizer as the composition of two components: a Content-Based Router that detects the format, followed by a format-specific Message Translator. The output of every translator is one and the same canonical model. The rest of the pipeline (validation, idempotency, business routing) only has one schema to know.

plaintext topology.txt 
EDIFACT ──┐
              │     ┌────────────────┐
   X12     ──┼───▶ │   Normalizer   │
              │     │  (detect +     │   ┌─────────────────┐
   cXML    ──┤     │   route by    │──▶│  Canonical JSON │
              │     │   format)     │   └─────────────────┘
   UBL     ──┤     └────────────────┘
              │              │
   IDoc    ──┘              ▼
                  ┌────────────────────────────────┐
                  │ Specific translator per format │
                  │  - edifact-translator           │
                  │  - x12-translator               │
                  │  - cxml-translator              │
                  │  - ubl-translator               │
                  │  - idoc-translator              │
                  └────────────────────────────────┘

Topology

Three variants:

  • Sniffer-based. Detection by reading the first bytes. Fast, but can be fooled if the format is ambiguous (JSON could describe an Order or an Invoice).
  • Header-based. Detection by transport header (HTTP Content-Type, SMTP Subject, AS2 metadata). More reliable but requires the sender to be disciplined.
  • Schema-detection. Combination: coarse detection by sniffer, then confirmation by XSD/JSON Schema validation. More robust but more expensive.

Format detection

Canonical detection sniffs the first 100 bytes and crosses with the transport header when available:

plaintext sniffer-table.txt 
Bytes 0..3   Indication                         Verdict
─────────────────────────────────────────────────────────────────
"UNB+"       EDIFACT segment                     EDIFACT
"UNA:+"      EDIFACT service advice              EDIFACT
"UNH+"       message header (unique interchange) EDIFACT
"ISA*"       X12 segment                         X12
"<cXML"      cXML root                           cXML
"<Invoice"   UBL root (with ns)                  UBL
"<Order"     UBL root                            UBL
"BEGIN_OF"   SAP IDoc text                       SAP IDoc
"{"          JSON                                JSON
"<?xml"      generic XML, needs deeper sniffing  XML undetermined

Technical note: EDIFACT allows an optional UNA service string advice segment before UNB that redefines separators (default :, +, ', ?, .). The sniffer must accept both UNA and UNB prefixes.

EDI implementation

Three architecture choices in production:

  • One input queue + Normalizer. Every message lands on inbox.raw. The Normalizer emits to inbox.canonical after translation. Simple, readable.
  • One queue per format + canonical topic. Transport adapters write to inbox.edifact, inbox.x12, etc.; translators consume their queue and publish to canonical.order. More coupling, but more scaling granularity.
  • Adapter pattern + canonical broker. Each format has its complete adapter (transport + parse + translate). All publish to a canonical broker. The canonical decomposition of modern iPaaS (MuleSoft, Boomi, SAP CPI).

Anti-patterns

  • Fragile detection on file extension. A partner naming their file .edi but putting XML inside breaks everything. The sniffer must always read the content.
  • No canonical default on error. If detection fails (unknown format), a Dead Letter Channel must receive the payload for human review — never silent drop.
  • Coupled translators. Each translator must be independent: the EDIFACT translator must share nothing with the X12 translator. A bug in one must not propagate.
  • Validation after normalisation only. Syntactic validation of the source format (CONTRL, 997) must happen before translation, so the sender is notified of the error in their own format.

Sources