Scrumconnect Consulting is a UK-based digital transformation consultancy delivering agile, secure technology solutions for public and private sector clients. This is a fully remote role based in India, supporting our UK operations and client base. You will work closely with our UK leadership team and must be comfortable operating within UK regulatory and legal frameworks.
About the role
Test Engineer on a large‑scale cloud data engineering programme, operating across a modern AWS‑native technology stack including Apache Airflow, Amazon Athena, AWS Glue, S3, EMR, and DynamoDB. You will own testing across automated pipelines, data workflows, and cloud infrastructure – identifying risks, championing test frameworks, and coaching colleagues in quality engineering practices. This is a hands‑on, technically deep role. You will write code, build and extend test frameworks, and take accountability for test quality across releases while actively mentoring more junior team members in best practices.
Key responsibilities
* Automated testing: design, implement, and maintain automated test suites across data pipelines, cloud services, and application layers. Ensure reproducibility and reliability across all releases.
* Risk identification: identify and raise awareness of risks arising from automated test results. Translate technical test findings into clear, prioritised risk communication for the team and stakeholders.
* Framework development: research, evaluate, and implement new test frameworks suited to the AWS data stack. Continuously improve existing frameworks to increase coverage and confidence in test outcomes.
* Pipeline & data testing: validate data pipelines built on Apache Airflow, AWS Glue, Athena, and EMR – including data integrity, transformation accuracy, and performance under load.
* Mentoring & coaching: mentor and advise team members in testing practices. Share knowledge actively, contribute to a quality‑first culture, and support less experienced colleagues in developing test skills.
* CI/CD integration: integrate and manage tests within GitLab CI/CD pipelines. Ensure tests run automatically on commit, providing fast feedback loops and blocking regressions before deployment.
Technology stack
Apache Airflow, Amazon Athena, AWS Glue, AWS S3, AWS EMR, AWS EC2, AWS ECR, AWS DynamoDB, AWS CloudWatch, AWS IAM, Python, Java, SQL, Bash, GitLab CI/CD, Jupyter Notebooks.
Required skills and experience
* Demonstrable experience as a Test Engineer or QA Engineer working on data pipelines or cloud‑native environments.
* Strong Python proficiency for test scripting, data validation, and automation in a production context.
* Working knowledge of Java and SQL for test development and data querying.
* Experience scripting in Bash or an equivalent language for pipeline and environment automation.
* Hands‑on experience with the AWS ecosystem – particularly S3, Glue, Athena, EMR, CloudWatch, IAM, and DynamoDB.
* Experience configuring and running tests within GitLab CI/CD pipelines.
* Proven ability to design and implement automated test frameworks from scratch or extend existing ones.
* Experience testing data pipelines – including data quality, integrity, transformation logic, and performance.
* Ability to identify, articulate, and communicate testing risks clearly to both technical and non‑technical audiences.
* SFIA Level 4 capability – works autonomously, takes ownership of test quality, and influences others.
* Experience mentoring or coaching junior engineers in testing practices.
* Active SC clearance – mandatory at point of application, no exceptions.
Diversity and Inclusion
At Scrumconnect Consulting, we believe that diversity drives innovation. We are committed to creating an inclusive environment where every individual is respected, valued, and supported. We welcome applications from candidates of all backgrounds and experiences, and we actively encourage applications from women, people with disabilities, under‑represented communities, and those seeking flexible working arrangements.
#J-18808-Ljbffr