# Batch or stream: which one fits your problem?

> Pick a real problem to see whether batch, stream or both fits it, and why. With a quick rule for choosing and the Lambda and Kappa architectures.

Canonical URL: https://learn.chabulab.com/system-design/choosing-batch-or-stream. Facts checked: 2026-11-03.

- Ask first: if the result were an hour late, would anyone lose out? If not, start with batch. If so, you need stream.
- Many systems use both: batch for exact results, stream for fast ones.
- Lambda runs two paths and keeps two copies of the logic. Kappa has one stream path and replays its log to recompute.

The technology comes later. First find out how much delay your problem can take.

## Which approach fits your problem?

Pick a problem to see whether to use batch, stream or both, and why.

*Nine real problems, each with its acceptable delay, the reasoning and suggested tools.*

## What is the quick rule for choosing?

Ask: **"If the result were an hour late, would anyone lose out?"**

- Nobody loses out: start with batch.
- Someone does: you need stream.
- Both are true in different parts of the system: combine them.

For stream, ask next what delay is acceptable. A dashboard refreshed each minute does not need the same technology as blocking fraud within 100 milliseconds.

## When you need both, how does the Lambda architecture work?

Lambda runs two paths side by side. The batch path recomputes all of history for an exact result. The speed path handles the newest data for a fast result. A serving layer merges the two.

Data source → Batch layer → Batch view → Serving

Data source → Speed layer → Realtime view → Serving

The drawback: you write and maintain two copies of the same calculation.

## How is the Kappa architecture different?

Kappa has one stream path. Every event stays in a long-lived log. To recompute history, you replay the log through a new stream job.

Event source → Long-lived log → Stream processor → Serving

The drawback: the log has to be kept long enough, and the stream engine has to be strong enough to reprocess a lot of data. Many teams now add a lakehouse so batch and stream read one table.

## Common mistakes when choosing

- **Choosing stream because it sounds modern.** If nobody needs the result now, you pay for machines and complexity and get nothing back.
- **Not asking what delay is acceptable.** "Real time" is not a requirement. A number such as "under 10 seconds" is.
- **Building Lambda from day one.** Two copies of the logic are costly to keep in step. Add the second path when you have a measured reason.
- **Forgetting that batch can run often.** A run every 5 minutes is enough for many "near real time" needs.

## Check what you learned

1. You pay 5,000 employees on the 30th of each month. What fits best?
   Show the answer: Batch, once attendance is closed. The data is only complete once the period closes, and nobody needs the result within seconds. Batch is cheaper and easier to check and re-run.
2. Why do product recommendations often use both batch and stream?
   Show the answer: The model learns from history, while what just happened needs an instant update. Training a model is throughput work, which suits batch. What the shopper just viewed is a hot signal, which needs stream.
3. What is the main drawback of the Lambda architecture?
   Show the answer: You write and maintain two copies of the same calculation. The batch and speed paths compute the same thing in two pieces of code. Kappa exists to remove one of them.

## Sources

- [Apache Kafka: introduction and use cases](https://kafka.apache.org/documentation/#uses)
- [Apache Iceberg: reading and writing one table](https://iceberg.apache.org/docs/latest/)