Our client, a leading organisation in the technology sector, is currently seeking a Data Engineer Lead to join their team on a contract basis. This role involves leading the design and delivery of the National Energy System Operator (NESO) for the Conform data layer, providing trusted, business-ready data across the enterprise. The ideal candidate will have deep expertise in Databricks and Azure data services. Key Responsibilities: Lead the Conform team responsible for building and maintaining pipelines from bronze through silver to gold layers in Databricks. Act as the main liaison with product owners and stakeholders, managing ad hoc requests, escalations, and shifting priorities. Oversee sprint ceremonies including refinement, reviews, retrospectives, and PI planning, ensuring clear backlog prioritisation and achievable commitments. Track and report on sprint KPIs such as velocity, story points, and successful delivery of demos. Chair combined stand-ups with data product and test teams to align dependencies and resolve issues quickly. Collaborate with acquisition teams to ensure upstream data ingestion supports downstream transformations. Partner with QA vendors to validate Conform outputs, manage testing dependencies, and resolve bugs. Safeguard the team from misplaced accountability by ensuring transparent communication, issue analysis, and resolution. Mentor Databricks engineers, providing technical guidance and preparing team members for future leadership. Support cross-company collaboration by coordinating with multiple vendors and maintaining high-quality delivery standards. Job Requirements: Experience with Databricks, delivering curated silver and gold data layers. Strong expertise in Azure data services, including: Azure SQL Data Warehouse Azure Service Bus and Event Hub Azure Logic Apps and Functions Azure DevOps for agile delivery and sprint tracking Azure Data Factory (nice to have) Proficiency in Python, SQL, PySpark; Foundry nice to have Proven experience leading data engineering teams, managing sprint ceremonies, and coordinating delivery Strong stakeholder management skills, with ability to manage prioritisation and escalations across multiple business units Familiarity with PI planning and agile delivery cycles Excellent written and verbal communication, with ability to present progress and defend technical decisions If you are an experienced Data Engineer Lead looking for a new opportunity to further develop your career in the technology sector, we would love to hear from you. Apply now to join our client's dynamic and talented team.