Location: 3 days onsite in Canary Wharf, London
Overview
We are seeking a Principal Data Engineer to join a large financial services organisation undergoing a major data platform transformation. This role sits at the intersection of architecture, engineering leadership, and hands‑on development, shaping a modern, cloud‑native data ecosystem.
You will lead the design and delivery of highly scalable, real‑time and batch data platforms built on Python, AWS and Kafka, working closely with architecture, trading, risk, and analytics teams.
Key Responsibilities
Technical Leadership
* Own the design and evolution of the enterprise data platform
* Define best practices, engineering standards, and data governance patterns
* Provide technical leadership to multiple data engineering squads
* Drive adoption of event‑driven and streaming architectures
* Design and build high‑throughput, low‑latency data pipelines
* Implement robust real‑time streaming solutions using Kafka
* Develop and maintain scalable Python‑based data services
* Architect secure and resilient AWS cloud data infrastructure
* Ensure solutions meet regulatory, security and audit requirements
Stakeholder Engagement
* Partner with senior stakeholders across Risk, Trading, Compliance and Data Science
* Translate complex business requirements into scalable technical solutions
* Mentor senior and mid‑level engineers across the programme
Required Skills & Experience
Core Technical
* Deep experience building streaming platforms using Apache Kafka
* Strong hands‑on experience with AWS, including:
* S3, Lambda, ECS/EKS
* IAM, networking and security best practices
* Experience building real‑time and batch ETL/ELT pipelines
* Strong experience with data modelling and distributed systems
* Proven experience operating at Principal / Lead Data Engineer level
* Strong understanding of event‑driven and microservices architectures
* Experience working in regulated financial services environments
* Experience with data mesh / data lakehouse architectures
* Knowledge of CI/CD and DevOps tooling (Terraform, Docker, Kubernetes)
* Experience with data governance and lineage tooling
* Exposure to market, risk, or trading data environments
Working Pattern
* Hybrid: 3 days onsite in Canary Wharf, 2 days remote
* £775 per day inside IR35
* 12-month contract with strong likelihood of extension
#J-18808-Ljbffr