Tumbling Window

How to slice an unbounded stream into disjoint time chunks to compute exact periodic aggregates.

Problem

An EDI flow is by nature unbounded: messages arrive constantly from the PEPPOL network, from AS2/AS4, from partner portals. We want to periodically answer questions like "how many invoices were received between 09:00 and 10:00?", "what is the intra-EU B2B volume for the DRR report of 30 June 2026?", "what is the top-10 partners by hourly volume?". These aggregates require slicing the stream into disjoint temporal segments on which the aggregation function applies.

Forces

• Predictable cardinality: each event belongs to exactly one window.
• Event-time vs processing-time: the correct choice depends on business function (fiscal report = strict event-time).
• Watermarks: needed to know when a window is "definitively complete".
• Late data: what to do with events arriving after closing (drop, side-output, reprocess)?
• Granularity: too coarse = excessive latency; too fine = huge state overhead.

Solution

Define a fixed window size W (e.g. 1 hour) and a temporal reference (event-time recommended: timestamp of the message at source). Each event is routed to the window [floor(ts/W)*W, floor(ts/W)*W + W). When the watermark passes the end of a window, the aggregated result is emitted and state is released. For late events, choose a policy: drop, side output to a dead-letter, or allowed lateness (Flink) that keeps the window open for additional time and emits an update. Apache Flink, Spark Structured Streaming, Kafka Streams, Google Dataflow all implement this pattern natively.

Structure

Time:    09:00────────10:00────────11:00────────12:00────────►
         ┌────────────┐┌────────────┐┌────────────┐
Window:  │  W0 (09h)  ││  W1 (10h)  ││  W2 (11h)  │
         └────────────┘└────────────┘└────────────┘

Events:  ●●  ●  ●  ●●     ●●●  ●         ●●  ●  ●●●
         │                │                │
         09:01            10:05            11:10
         │                │
         09:42 ↓
                 09:55 ◄── arrives at 10:03 (LATE for W0!)
                          → drop, side-output, or allowed lateness?

When watermark passes 10:00:
  W0 closes → emit (W0, count=5)
When watermark passes 11:00:
  W1 closes → emit (W1, count=4)

EDI implementation

Three canonical EDI cases. (1) Periodic fiscal reporting: aggregate e-invoicing invoices in 24h windows (UTC for ZATCA, Europe/Paris for PPF France) to produce the daily batch to transmit to the administration. (2) Hourly operational KPIs: count per partner, average acknowledgement latency, top syntactic errors over the past hour for SRE dashboard. (3) Accounting rate limiting: if more than N invoices per minute for the same issuer, trigger manual review (detection of suspicious invoicing robot). Typical Flink implementation: stream.keyBy(senderGLN).window(TumblingEventTimeWindows.of(Time.hours(1))).aggregate(new CountAggregator()). For DRR under ViDA (Digital Reporting Requirements, July 2030), daily event-time window with 24h allowed lateness is a canonical pattern to produce the D+1 report even with a few late messages.

Anti-patterns

• Using processing-time for a fiscal report — replays produce results inconsistent with the initial batch.
• Watermark too aggressive — many late data lost, possible fiscal under-reporting.
• Window too fine on high-cardinality flows — explodes RocksDB state to multiple TB.
• No late-data handling — a PEPPOL message arriving 6h late due to an AP outage is never accounted for.
• Mixing time zones without aligning to UTC — the 00h-01h Europe/Paris window during DST is not aligned with UTC.

Related patterns

• Sliding Window — overlapping window.
• Session Window — activity-based window.
• Watermarks — mechanism that closes windows.
• Exactly-Once Semantics — often required for exact fiscal reporting.

Sources

• Akidau T. et al. — The Dataflow Model: A Practical Approach to Balancing Correctness, Latency, and Cost in Massive-Scale, Unbounded, Out-of-Order Data Processing, VLDB 2015. research.google
• Apache Flink documentation — Windows. nightlies.apache.org/flink
• Spark Structured Streaming — Window operations on event time. spark.apache.org
• Akidau T., Chernyak S., Lax R. — Streaming Systems, O'Reilly 2018, ch. 1-3.
• Kafka Streams documentation — Windowing. kafka.apache.org