Flow ArchitecturesDS-04 · theory

Source · Flow Architectures (Bellemare, O'Reilly)

Why this matters

Flow Architectures (Bellemare), Ch. 1

Traditional integration moves data in periodic batches — nightly ETL jobs, point-to-point APIs, request/response calls. As businesses demand real-time reactions, that request-and-batch model can't keep up. Flow architectures treat data as something continuously in motion, integrated event-first across organizational boundaries.

The shift matters because it changes integration from a pull-when-needed act to a continuous stream others can tap, reshaping how systems and even companies exchange data.

The concept

Flow Architectures (Bellemare), Ch. 3–5

A flow architecture centers on data in motion: events stream continuously rather than sitting at rest to be queried. Integration becomes event-first — systems publish state changes as event streams, and other systems subscribe, instead of calling each other on demand.

Streaming pipelines transform, filter, enrich, and route these events as they flow — a continuous processing graph rather than a scheduled job. Because integration happens through shared, durable streams rather than brittle point-to-point links, adding a new consumer means subscribing to an existing flow, not building another bespoke connection. This event-first integration inverts the classic model: instead of asking a system for data when you need it, you continuously receive its changes and react. The result is loosely coupled, real-time integration that scales across teams and organizations.

Worked scenario

Flow Architectures (Bellemare), Ch. 6

A logistics company integrates warehouse, transport, and customer systems. The batch approach syncs each pair nightly, so a delayed shipment isn't reflected until the next day.

Re-architected as flow, the warehouse publishes a continuous 'ShipmentUpdated' stream. A streaming pipeline enriches each event with route data and routes it onward; the transport and customer systems subscribe and react within seconds. When a new analytics team wants the same data, they subscribe to the existing stream — no new nightly job, no point-to-point wiring. Data is integrated as it moves, event-first, in real time.

How it connects

Flow Architectures (Bellemare), Ch. 8

Flow architectures are event-driven architecture (DS-02) applied to integration at organizational scale, and they are how data-mesh (DS-03) products travel between domains. The streaming pipelines here are frequently built from serverless (DS-05) processing steps.

Where DS-01 taught scaling within a system, flow architectures scale integration between systems — data in motion as a first-class citizen.

Common traps
  • Equating flow with faster batch — flow is continuous data in motion, not a smaller scheduled job.
  • Sticking to point-to-point integration; flow favors shared, subscribable streams over bespoke links.
  • Thinking event-first just means using a message queue — it inverts integration from pull-on-demand to continuous push.
Key takeaways
  • Flow architectures treat data as continuously in motion, integrated event-first across boundaries.
  • Streaming pipelines transform and route events as a continuous graph, not scheduled batch jobs.
  • New consumers subscribe to existing streams, replacing brittle point-to-point integration.