We're supporting a major financial-systems transformation programme and are looking for a Senior Databricks Engineer to help build and enhance a scalable data-engineering framework used across critical reporting processes. You'll operate as the senior hands-on engineer, shaping reusable libraries, optimising PySpark pipelines, and guiding offshore developers to deliver high-quality, production-ready code. You'll work across ingestion, validation, transformation, and mapping layers within a Databricks-on-Azure environment, helping to establish consistent engineering patterns and ensuring all deliveries meet high standards of performance, traceability, and maintainability. What you'd be doing: Building and extending reusable Databricks/PySpark libraries (ingestion, validation, transformation, mapping). Developing scalable, optimised PySpark pipelines aligned to metadata-driven or hybrid data models. Implementing robust validation and control logic suitable for regulated financial environments. Leading engineering quality: code reviews, mentoring offshore teams, ensuring standards across readability, testing, and performance. Supporting integration paths into downstream reporting systems and managing return-feeds. Producing clear documentation, runbooks, and developer guides. My ideal candidate will have: Strong hands-on experience with Databricks and PySpark (optimisation, partitioning, orchestration). Solid Azure data-platform experience. Proven ability to build reusable frameworks/libraries with clean abstractions. Experience guiding or leading offshore engineering teams. Comfortable working with evolving requirements while maintaining strong engineering discipline. Bonus: exposure to financial or regulatory reporting, Tagetik, or metadata-driven/EAV models. This is an urgent role, will be remote with once monthly visits to client site and up to £400 per day OUTSIDE IR35.