Data Engineer (Databricks Focus)
London, hybrid working
6 months
What you'll be doing:
As a Data Engineer, you'll be instrumental in the post-organisational change data landscape. Your key responsibilities will include:
* Designing and implementing robust data pipelines for efficient data collection, processing, and storage.
* Leading the migration of existing files and data to new systems.
* Ensuring data quality, integrity, and security throughout the migration process.
* Collaborating closely with stakeholders across the business to understand their data needs and translate them into technical solutions.
* Contributing to the development of best practices for data management and governance.
What you'll bring:
* Extensive experience as a Data Engineer, with a proven track record of delivering impactful data solutions.
* Deep and recent experience with Azure, Databricks, and Azure Data Factory. You've not just touched these technologies; you've used them extensively in recent projects.
* Strong proficiency in Python for data manipulation and automation.
* Hands-on experience with Azure DevOps for CI/CD and infrastructure as code.
* Solid understanding of Power BI for data visualisation and reporting.
* Familiarity with the Medallion architecture for data lake organisation.