Job Description
Lead Data Engineer
Location: Hybrid (60% Office / 40% Remote) SheffieldContract: 6 Month Contract (Extension possible)Rate: £400+ per day inside IR35
Role Overview
We are seeking an experienced Lead Data Engineer to design, develop, and optimise enterprise-scale data platforms for large, regulated organisations, ideally in banking, financial services, or other regulated sectors. This role requires hands-on technical expertise, leadership, and a consulting mindset to deliver scalable, resilient data solutions while promoting best practices and operational excellence.
Key Responsibilities
* Lead the design, development, and optimisation of enterprise data engineering platforms
* Build and maintain robust ETL/ELT pipelines integrating large, complex datasets
* Work with structured, semi-structured, and unstructured data across SQL and NoSQL technologies
* Develop solutions using Hadoop, Spark, and Splunk in large-scale environments
* Write maintainable Python code, applying object-oriented and functional programming principles
* Implement and maintain CI/CD pipelines, automated testing, and version control
* Collaborate with BI, Analytics, and downstream teams to support reporting and insights
* Pair program and mentor other engineers to promote knowledge sharing and code quality
* Define and maintain technical test plans including unit and integration tests
* Promote SRE principles to ensure service resilience, sustainability, and recoverability
Essential Skills & Experience
* Proven experience as a Lead or Senior Data Engineer in enterprise-scale environments
* Hands-on expertise with Hadoop, Spark, and Splunk
* Advanced Python development skills
* Experience designing and optimising high-performance data pipelines
* Strong understanding of CI/CD, source control, and automated testing
* Analytical, problem-solving, and leadership skills
* Experience working in regulated, enterprise environments (e.g., banking, fintech, government)
* Agile delivery experience (Scrum/Kanban)
Consulting & Soft Skills
* Ability to mentor and uplift team performance
* Strong communication and stakeholder engagement skills
* Collaborative, delivery-focused mindset with high accountability
* Knowledge of control, compliance, and regulatory requirements
* Up-to-date awareness of modern tools, cybersecurity, and data privacy regulations
* Champions innovation, advanced technologies, and best practices
If this role aligns with your skills and experience, we'd love to hear from you. Apply today to be considered.
JBRP1_UKTJ