We are seeking a Data Engineer to work closely on the data pipeline to build and support our customer reporting capabilities. The ideal candidate will have strong programming and analytical skills, expertise in Python and SQL, and a passion for debugging and improving data processes. This role requires collaborative problem-solving, a keen attention to detail, and the ability to translate complex business requirements into robust data solutions.
The Role:
Our offices are in Trafford Park, Manchester and the role consists of hybrid working, where we like to be in the office for collaboration and team building and work from home whilst doing deep work.
Your primary responsibilities will involve developing and maintaining a mix of real time and batch ETL jobs on a large and complex dataset. You will apply strong Python and SQL skills to ensure data accuracy, integrity, and scalability, while also engaging in continuous improvements to our data pipeline.
You will tackle complex data challenges in a fast-paced, exciting environment, leveraging cutting-edge Big Data open-source technologies like Apache Spark, as well as Amazon Web Services (AWS) solutions such as Elastic MapReduce (EMR), Athena, and Lambda to develop innovative and scalable data solutions.
Key Responsibilities:
· Continually review and measure the performance of the data pipeline and evaluate improvements in design, architecture and tooling.
· Become a subject matter expert for the data pipeline and supporting processes and be able to present to others to knowledge share.
· Regularly reviewing colleagues' work and providing helpful feedback.
· Working with stakeholders to fully understand requirements and be a technical reference point for product team members.
· Writing application code and tests that conform to standards.
· Adhering to Company Policies and Procedures with respect to Security, Quality and Health & Safety.
Here's what we're looking for:
· SQL and Python development experience as a Data Engineer, preferably using AWS or equivalent cloud provider.
· Analytical skills to be able to present decisions to stakeholders in a data driven way.
· The ability to problem-solve and break down complex problems, whilst working on large and complex datasets.
· Knowledge of agile software development best practices including continuous integration, automated testing and working with software engineering requirements and specifications.
· Good interpersonal skills, positive attitude and willing to help other members of the team.
· Exposure to Apache Spark (scala or python) is preferred, however training will be provided.
Mayflower is acting as an Employment Agency in relation to this vacancy.