Job Title: Senior Data Engineer (Azure, Spark SQL, Team Lead)
Duration: long-term contract
Location: Hybrid (3 days/week from Sheffield)
Visa: UK Citizen/ILR/Dependent visa (No Visa sponsorship)
We are looking for a Senior Data Engineer with hands-on expertise in SQL/BigQuery migration, Azure Databricks, and Spark SQL, who also brings team leadership experience and thrives in Agile/SAFe/Scrum environments.
Key Responsibilities:
* Lead and contribute to a small Agile team working on a cross-cloud data migration project.
* Migrate complex BigQuery SQL transformations to Spark SQL on Azure.
* Build & execute ETL workflows using Azure Databricks and Python.
* Drive automation of SQL workflows and artefact migration across cloud providers.
* Collaborate with developers, POs, and stakeholders on quality delivery and performance optimization.
Key Requirements:
* Strong SQL skills (BigQuery SQL & Spark SQL), Python, and ETL pipeline development.
* Experience with Azure and cloud data tools.
* Familiarity with CI/CD practices and automation.
* Proven team leadership and organisational capabilities.
* Comfortable with Agile/SAFe/PI Planning methodologies.