Data Engineer – Real-Time Streaming Systems
Overview
🏢 Company | Global trading & technology group
👤 Job | Data Engineer – Real-Time Streaming Systems
🎯 Impact | Low-latency data for decision-making, analytics & automated workflows
📏 Team | Small senior engineering group
🌟 Stack | Azure, Kafka, Snowflake, Python/Scala/Java
📍 Location | London
💻 Hybrid | Office (Hybrid)
💰 Offer | Competitive salary + performance bonus
💎 Benefits | Strong learning budget, certifications, conferences, full health package
The Work
You enjoy building systems where speed and reliability actually matter.
Here, you’ll take fast-moving external feeds, normalise them, and turn them into high-quality real-time datasets used across engineering, analytics and automation. If you like low-latency engineering, repeatable pipelines and being the reason things “just work”, this will suit you.
You’ll get:
* Real ownership over a streaming platform used across the business.
* Direct collaboration with engineers, data teams, and technical end-users.
* A modern environment: Azure, streaming tech, Snowflake, Databricks, CI/CD, TDD.
* Space to influence good engineering habits: automation, observability, data contracts.
* Long-term investment in a growing technology function.
What you’ll be doing
You’ll:
* Build and operate a high-availability, low-latency streaming pipeline.
* Pull external feeds and standardise them into clean, structured datasets.
* Add reliability features: retries, validation, redundancy, graceful failover.
* Apply testing and automation across ingestion, transformation and storage.
* Define APIs, schemas and platform patterns other teams can depend on.
* Build monitoring for latency, quality and system health — and use it to drive improvements.
* Work closely with engineers, data scientists and analysts to wire your data into models and systems.
* Keep the platform documented and continually improved.
You’ll also help steer direction:
* Prioritise data sources that meaningfully improve decision-making and analytics.
* Evaluate new external providers and integrate them safely and at scale.
* Contribute to new data products: curated datasets, feature stores, and real-time decision APIs.
What you’ll bring
You probably have:
* 3+ years in data engineering or real-time systems.
* Experience with high-frequency or event-driven pipelines.
* Strong coding ability in Python, Scala or Java.
* Streaming expertise with Kafka / Confluent.
* Practical experience with Azure (ADLS Gen2, Event Hubs, Databricks/Synapse, Functions, Data Factory, Key Vault).
* Solid experience with Snowflake in production.
* Good engineering fundamentals: tests, CI/CD, automation, version control.
Nice to have:
* Airflow or similar orchestration tools.
* Data quality frameworks (e.g., Great Expectations).
* Terraform/Bicep for IaC.
* Experience in environments where milliseconds and data accuracy matter.
You’re the type who:
* Spots anomalies quickly and goes hunting for root causes.
* Communicates clearly in technical and cross-functional discussions.
* Stays calm under pressure and fixes systems properly, not temporarily.
📅 Interview Process | CV review → Intro call → Technical interview → Team conversations → Offer
If you want to build the backbone of real-time data for a global technology group, hit Apply or send your CV and we’ll set up a confidential chat.