Apache Flink — real-time stream processing engine

Flink je nejpokročilejší engine pro stream processing. Exactly-once sémantika, event time processing a state management.

Proč Flink¶

Stream-first přístup — batch je speciální případ streamingu.

Flink SQL¶

CREATE TABLE orders (
    order_id STRING,
    amount DECIMAL(10,2),
    order_time TIMESTAMP(3),
    WATERMARK FOR order_time AS order_time - INTERVAL '5' SECOND
) WITH ('connector' = 'kafka', 'topic' = 'orders', 'format' = 'json');

SELECT
    TUMBLE_START(order_time, INTERVAL '5' MINUTE) AS window_start,
    COUNT(*) AS order_count,
    SUM(amount) AS revenue
FROM orders
GROUP BY TUMBLE(order_time, INTERVAL '5' MINUTE);

Vergleich¶

• Flink — true streaming, nejnižší latence
• Spark Streaming — micro-batch, batch+stream hybrid
• Kafka Streams — library, jednoduché transformace

Zusammenfassung¶

Flink je top volba pro nízkou latenci a exactly-once processing. Flink SQL zpřístupňuje streaming analytikům.