Morgan Hunt are working with a housing association in London to recruit an Azure Data Engineer on a permanent basis. The purpose of the role is to lead the implementation of data flows to connect operational systems, data for analytics and BI systems. You will be responsible for designing, implementing, and maintaining the organisations data infrastructure to ensure the efficient processing, storage, and retrieval of large datasets. The role requires a deep understanding of data engineering best practices, data architecture, and the ability to work with cross-functional teams to deliver high-quality data solutions.
Key Responsibilities:
Design and Development:
* Design, develop and optimise ETL processes using Databricks
* Develop data pipelines to extract, transform, and load into Data lakes and other data storage solutions
* Implement best practices for data management, data governance and data quality assurance
Data Infrastructure:
* Build and maintain scalable data architectures, ensuring data is organised, secure and accessible
* Integrate various data platforms and tools to create a cohesive and robust data ecosystem
* Monitor and troubleshoot data pipeline issues, ensuring high availability and reliability of data services
Collaboration:
* Work closely with data scientists, analysts and other stakeholders to understand data requirements and deliver solutions
* Have a good understanding of the business problems and use cases
* Work with the IT, Digital and DevOps teams to ensure deployment and integration of data solutions
Innovation:
* Stay up to date with the latest trends and advancements in data engineering, data platforms and big data technologies
* Evaluate and improved existing data processes
Essential Experiences:
* Advanced SQL skills
* Proficient in Databricks, including Spark, Delta Lake and MLflow
* Experience migrating SSIS to Databricks
* In depth understanding of ETL/ELT processes
* Knowledge of data warehousing concepts and architectures
* Able to produce relevant data models across multiple subject areas
* Experience implementing data governance, data security and compliance standards to ensure data accuracy
£70,000 per year
Hybrid working (2 days in London office)
Permanent position
Please get in touch for further information.
Morgan Hunt is a multi-award-winning recruitment business for interim, contract and temporary recruitment and acts as an Employment Agency in relation to permanent vacancies. Morgan Hunt is an equal opportunities employer. Job suitability is assessed on merit in accordance with the individual's skills, qualifications and abilities to perform the relevant duties required in a particular role.
#J-18808-Ljbffr