Social network you want to login/join with:
Data Engineer
Whitehall Resources are looking for a Data Engineer. This role is hybrid working, with 2 days per week onsite in Cambridge, and the remainder remote working, for an initial 6 month contract.
***INSIDE IR35***
Job Description:
• Build strong working relationships and collaborate with analysts and data owners to design conceptual and physical data models that power HR/People reporting, dashboards, and machine learning use cases.
• Design and implement data solutions within a modern data lakehouse environment using Databricks, leveraging medallion architecture principles (Bronze, Silver, Gold & Platinum layers) to structure and refine data progressively.
• Develop scalable and efficient data pipelines using AWS Glue to ingest, transform, and curate data across various structured and semi-structured sources.
• Validate and assure data quality by collaborating with stakeholders to test and verify business-critical metrics and model outputs.
• Support CI/CD processes to ensure controlled deployment of data assets across development, staging, and production environments.
• Produce clear, user-focused documentation for implemented data models, metrics, and transformation logic.
Skills, and Experience
• Data Provisioning: Experience provisioning data from diverse source systems into cloud-based data platforms; familiarity with AWS Glue beneficial
• Data Modelling: Strong capability in conceptual, logical, and physical data modeling, particularly within lakehouse architectures, experience working with HR/People data beneficial.
• Data Engineering: Proficient in building robust and reusable pipelines using PySpark and Databricks notebooks; skilled in applying the medallion architecture for progressive data refinement and downstream usability.
• Testing & Verification: Knowledge of data validation techniques and automated testing frameworks; experience in unit, integration, and UAT testing for data pipelines.
• Stakeholder Collaboration: Demonstrated ability to work cross-functionally with technical and non-technical stakeholders including platform teams, business SMEs, and governance roles to deliver quality data products.
Technologies & Tools
• Databricks
• AWS Glue
• Amazon S3
• CI/CD Tools (e.g., Git, Terraform)
All of our opportunities require that applicants are eligible to work in the specified country/location, unless otherwise stated in the job description.
Whitehall Resources are an equal opportunities employer who value a diverse and inclusive working environment. All qualified applicants will receive consideration for employment without regard to race, religion, gender identity or expression, sexual orientation, national origin, pregnancy, disability, age, veteran status, or other characteristics.
#J-18808-Ljbffr