Lead Data Engineer
Hands‑on technical leadership building data systems that actually work
We're looking for a Lead Data Engineer to join an enterprise data engineering capability and play a critical role in designing, building, and improving real‑world data systems.
This is not a line‑management role. Instead, it's a senior, hands‑on technical leadership position for someone with deep data engineering experience who leads through delivery, judgement, and technical credibility rather than hierarchy.
You'll work in a complex environment spanning legacy platforms, modern cloud services, and evolving data needs. The challenge isn't greenfield design - it's improving what exists, fixing what's broken, and helping teams use data more effectively and responsibly. This role suits someone who enjoys rolling up their sleeves, tackling difficult problems, and seeing things through end‑to‑end.
What you'll be doing
You'll act as a senior technical anchor for data engineering work, contributing directly while guiding others through example.
You will:
1. Design, build, and enhance enterprise‑scale data pipelines, platforms, and data services
2. Lead complex data engineering work end‑to‑end - from problem definition and exploration through build, testing, and operational use
3. Work directly with technical and non‑technical stakeholders to translate real operational problems into effective data solutions
4. Analyse and synthesise complex datasets from multiple sources, shaping how data is structured, validated, and consumed
5. Provide technical leadership and direction to data engineers and adjacent roles through design decisions, code, and delivery support
6. Improve data sharing, onboarding of new data sources, and interoperability across teams and systems
7. Introduce and evaluate new tools, patterns, and technologies where they genuinely add value - balancing innovation with pragmatism
8. Build robust, production‑grade solutions, not proofs of concept
9. Treat metadata, lineage, and data quality as first‑class engineering problems
10. Diagnose and resolve complex data and platform issues, often in constrained or imperfect environments
11. Contribute to an inclusive, supportive engineering culture with high technical standards
What we're looking for
This role is for a genuine practitioner who enjoys doing the work and raising standards around them.
Essential experience:
12. Significant, hands‑on experience as a Senior or Lead Data Engineer working on complex, real‑world systems
13. Strong Python and Spark skills, with evidence of building and maintaining production data pipelines
14. Deep understanding of the full data engineering lifecycle: ingestion, transformation, storage, serving, and reuse
15. Strong experience designing data integrations, including working with diverse data sources and legacy systems
16. Proven ability to design and apply data models that support analysis, operational use, and long‑term maintainability
17. Solid understanding of data security, compliance, and governance, built into systems by design
18. Experience working within established data development practices (version control, testing, deployment, change management)
19. Ability to communicate clearly between highly technical teams and non‑technical stakeholders
20. Persistence and resilience when dealing with messy data, legacy constraints, and organisational complexity
Desirable experience
21. Experience with cloud‑based data ecosystems (including AWS services) alongside existing legacy platforms
22. Exposure to large‑scale enterprise data or analytics platforms
23. Rotating across teams or problem domains, adapting to different delivery contexts
24. Familiarity with emerging or advanced data engineering approaches (including agentic or autonomous patterns), though this is not essential
Guidant, Carbon60, Lorien & SRG - The Impellam Group Portfolio are acting as an Employment Business in relation to this vacancy.