Start Day: ASAP
Contract Rate: £460 per day inside IR35
Duration: 6 to 12 months initially
Role Overview
Our client is seeking a Data Engineer with strong data quality and testing expertise to lead the design and implementation of QA frameworks for business‑critical ETL/ELT pipelines. This is a greenfield opportunity to define testing strategy and build scalable, automation‑first solutions in a fast‑paced, data‑driven environment. You’ll work across the full data lifecycle, ensuring accuracy and reliability of outputs including APIs, Excel reports, and XML files. Leveraging tools such as Databricks, Python (Pytest, Pandas), and GitHub Actions, you will establish best practices, improve data trust, and enable high‑quality data delivery.
Key Responsibilities
* Own and define end‑to‑end testing strategy for data pipelines
* Build scalable automation frameworks using Python and Pytest
* Develop SQL‑based validation for reconciliation and data integrity
* Automate Excel validation using Pandas and OpenPyXL
* Validate XML outputs using lxml and xmlschema
* Embed testing into CI/CD pipelines with GitHub Actions
* Create clear logging and reporting for fast issue resolution
* Collaborate with data engineers to improve pipeline quality
* Mentor team members and establish QA best practices
Skills & Experience
* Strong experience testing ETL/ELT data pipelines
* Advanced SQL for validation and debugging
* Experience with Pandas and OpenPyXL for Excel validation
* API testing with Postman/Newman
* CI/CD experience using GitHub Actions
* Familiarity with Databricks or similar platforms
* Strong understanding of data quality and validation principles
* Experience building QA processes and mentoring others
* Exposure to cloud‑based data platforms
* Experience with tools like Great Expectations
* Knowledge of Airflow or ADF
* End‑to‑end validation across pipelines, APIs, and files
* Experience scaling QA practices in data teams
#J-18808-Ljbffr