Streaming Data Engineer

The work

Build and operate streaming pipelines with clear contracts, backpressure handling, and replay paths. Work supports real-time analytics and feature freshness for ML and Gen AI programmes — described by latency class and data quality, not customer name.

You will

• Design Kafka (or equivalent) topics, partitioning, and consumer groups for predictable throughput
• Handle late events, schema evolution, and idempotent sinks into warehouse or lakehouse layers
• Instrument lag, error rates, and poison-message quarantines with actionable alerts
• Document replay and compaction procedures for incident recovery

You likely have

• 5+ years operating production streaming systems, not proof-of-concept demos
• Hands-on with at least one stream processor (Flink, Spark Structured Streaming, or equivalent)
• Comfort with data contracts and cross-team negotiation on event schemas
• Calm incident response when downstream dashboards or models depend on your pipeline