Lead Data Engineer
Location: Hybrid (60% Office / 40% Remote) Sheffield
Contract: 6 Month Contract (Extension possible)
Rate: £400+ per day inside IR35
Role Overview We are seeking an experienced Lead Data Engineer to design, develop, and optimise enterprise-scale data platforms for large, regulated organisations, ideally in banking, financial services, or other regulated sectors. This role requires hands-on technical expertise, leadership, and a consulting mindset to deliver scalable, resilient data solutions while promoting best practices and operational excellence.
Key Responsibilities Lead the design, development, and optimisation of enterprise data engineering platforms
Build and maintain robust ETL/ELT pipelines integrating large, complex datasets
Work with structured, semi-structured, and unstructured data across SQL and NoSQL technologies
Develop solutions using Hadoop, Spark, and Splunk in large-scale environments
Write maintainable Python code, applying object-oriented and functional programming principles
Implement and maintain CI/CD pipelines, automated testing, and version control
Collaborate with BI, Analytics, and downstream teams to support reporting and insights
Pair program and mentor other engineers to promote knowledge sharing and code quality
Define and maintain technical test plans including unit and integration tests
Promote SRE principles to ensure service resilience, sustainability, and recoverability
Essential Skills & Experience Proven experience as a Lead or Senior Data Engineer in enterprise-scale environments
Hands-on expertise with Hadoop, Spark, and Splunk
Advanced Python development skills
Experience designing and optimising high-performance data pipelines
Strong understanding of CI/CD, source control, and automated testing
Analytical, problem-solving, and leadership skills
Experience working in regulated, enterprise environments (e.g., banking, fintech, government)
Agile delivery experience (Scrum/Kanban)
Consulting & Soft Skills Ability to mentor and uplift team performance
Strong communication and stakeholder engagement skills
Collaborative, delivery-focused mindset with high accountability
Knowledge of control, compliance, and regulatory requirements
Up-to-date awareness of modern tools, cybersecurity, and data privacy regulations
Champions innovation, advanced technologies, and best practices
If this role aligns with your skills and experience, we'd love to hear from you. Apply today to be considered.
TPBN1_UKTJ