Job Description
Data Engineer 6 Month ContractHybrid - 2 days per week onsitePlease note: Active SC Clearance is required for this role
A leading organisation within the UK financial sector is embarking on a major data transformation programme. They are looking for an experienced Data Engineer to help design and deliver a modern, cloud-first data platform that will underpin some of the organisation's most critical functions.
Responsibilities:
* Design, build and deploy scalable, secure data solutions using Azure Databricks, Data Factory and Data Lake Storage.
* Develop and optimise advanced data pipelines with Python, SQL, Spark/PySpark and Delta Lake.
* Champion strong data quality, governance and observability practices.
* Modernise legacy systems and support large-scale migrations into Azure.
* Provide technical leadership, set engineering standards and contribute to architectural decisions.
* Mentor engineers and foster a culture of continuous improvement.
* Work closely with architecture, analytics and business teams to align engineering solutions with organisational needs.
Minimum Criteria
* Extensive experience with Azure services including Azure Databricks, Azure Data Lake Storage, and Azure Data Factory.
* Advanced proficiency in SQL, Python, and Spark (PySpark), with a strong focus on performance optimization and distributed processing.
* Proven experience in CI/CD practices using industry-standard tools (e.g., GitHub Actions, Azure DevOps).
* Strong understanding of data architecture principles and cloud-native design patterns.
Essential Criteria
* Demonstrated ability to lead technical delivery, mentor engineering teams and collaborate with stakeholders to ensure alignment between data solutions and business strategy.
* Proficiency in Linux/Unix environments and shell scripting.
* Deep understanding of source control, testing strategies, and agile development practices.
* Self-motivated with a strategic mindset and a passion for driving innovation in data engineering.
Desirable Criteria
* Experience delivering data pipelines on Hortonworks/Cloudera on-prem and leading cloud migration initiatives.
* Familiarity with:
o Apache Airflow
o Data modelling and metadata management
* Experience influencing enterprise data strategy and contributing to architectural governance.
To apply for this position please submit your CV.
Guidant, Carbon60, Lorien & SRG - The Impellam Group Portfolio are acting as an Employment Business in relation to this vacancy.