Builds and runs real time streaming pipelines, keeping latency low and jobs recoverable at high
volume.
- Strong hands
on Apache Flink across DataStream API and Flink SQL. - Event time processing, checkpointing, savepoints, and state management in
production. - High volume Kafka streaming.
- Data modelling for analytics and streaming into a lakehouse.
- Monitors pipeline health and handles recovery when jobs fail.
Requirements
Requirements
pache Flink
(DataStream, Flink SQL), Kafka, Java, Scala, Python, SQL, Docker, Kubernetes
- Strong hands-on experience with Apache Flink, including DataStream API and Flink SQL.
- Experience with high-volume Kafka streaming and real-time data processing.
- Strong knowledge of event time processing, checkpointing, savepoints, and state management.
- Experience with data modelling for analytics and streaming into a lakehouse.
- Experience monitoring pipeline health and handling job recovery and failures.
- Apache Iceberg or other open table formats.
- Airflow.
- On premises or self managed cluster experience.