Data Engineer
Redhill, Surrey – hybrid working (2-3 days pw in office)
Full-time permanent
Up to £62,500 + 5% annual performance related bonus
We have an exciting new opportunity for a Data Engineer to join Robert Walters as a Consultant. As an employed Consultant, you will benefit from permanent employment with Robert Walters and will be deployed on an interim or project basis into our clients' organisations, in return we will provide you with the opportunity to develop your skills with ongoing training and professional accreditations.
Reporting into the Head of Data & Analytics, this role sits in the Data & Analytics team alongside Data Developers and Data Engineers.
Key Responsibilities:
* Development and automation of ingestion flows, data curation and access layers
* Define technical solutions and take part in the development and deployment of applications in accordance with central standard and guidelines
* Contribute to the improvement of standards, compliance and processes to guide Data Lake evolution
* Design, develop, test and deploy data ingestion flows, data marts and core target data components
* Implement tools and end-to-end monitoring to ensure high availability of production data processing, data quality and reliability
* Maintenance of data transformation routes in the Data Lake
* Maintenance of BAU including reporting regularly on performance, risks and issues
Key Skills:
* Experience working within multi-disciplinary data teams
* Fluent with the Big Data stack in cloud environment (ADLS, Databricks, ADF, Azure DevOps etc)
* Knowledge and experience of reporting solution design including Azure platforms, Pegasus, Attunity, Wax, R, SQL, Python, Spark, Power BI, Business Objects
* Proficient in RESTful API design, implementation and consumption
* You have a deep understanding of data lake technologies, interrogation of structures and unstructured data, data warehouse and data schema design for optimised reporting performance
* You have a very good understanding of spark processes
* You are fluent in multiple development languages
* You have a good understanding of the software development process (git, ci/cd etc.)