Lead Data Engineer- £70,000
Industry:
Banking and Financial Services
Location
: Flexible- Basingstoke (3 Days a Week)
Overview of Role:
This role will
lead the design
and optimisation of our
Azure Databricks data platform
, ensuring it is secure, scalable, and well governed. You'll take ownership of change controls, lineage, and documentation while driving adoption of best practice. Alongside this, you'll support AI/ML initiatives and provide technical guidance to colleagues, driving value from data.
Key Responsibilities
* Lead the roadmap for data engineering, ensuring alignment with business needs and regulatory requirements.
* Own the design and optimisation of the Azure Databricks platform, focusing on performance, resilience, and compliance.
* Implement change controls, lineage, and observability to detect and resolve issues.
* Maintain clear documentation of data pipelines, models, and governance processes.
* Implement tools to improve data quality, efficiency, and adoption.
* Support AI/ML delivery by building/optimising, ensuring platform readiness.
* Manage SQL workloads within Databricks, balancing cost and performance
* Ensure access controls, GDPR compliance, and security standards
* Work with other stakeholders on system changes and reporting needs
* Act as the escalation point, driving root cause analysis and preventative measures.
* Mentor colleagues and promote best practice in data engineering
* Build expertise in Unity Catalogue and other governance tooling
Essential
* Strong experience designing and delivering modern data platforms in Azure and Databricks
* Advanced SQL skills with experience in large-scale environments
* Proven track record with lineage, documentation, and governance (Unity Catalogue, Purview)
* Experience implementing data quality and observability controls
* Familiarity with ITIL change and incident management
* Hands-on experience enabling AI/ML in a data platform
* Strong ETL/ELT engineering skills
Desirable
* Experience with Python and related tooling
* Understanding of MLOps practices (MLflow, Azure ML)
* Familiarity with real-time data technologies (Kafka, Delta Live Tables)
If you're passionate about transforming the banking industry and eager to leverage your expertise to drive continuous improvement and innovation for clients then click "APPLY" or email