Permanent | Newcastle, occasional onsite attendance | Data & Testing | SC Clearance
At Peregrine, we’re always seeking Specialist Talent that have the ideal mix of skills, experience, and attitude, to place with our vast array of clients. From Business Analysts in large government organisations to Software Developers in the private sector – we are always in search of the best talent to place, now.
Role
We are seeking an SC Cleared Test Engineer to support the testing and assurance of large‑scale data platforms, big data solutions and data lake environments.
This role will focus on validating structured and unstructured data across AWS‑based environments, ensuring data pipelines, processing workflows and outputs are accurate, reliable and fit for purpose.
You will use tools such as Apache Airflow, Jupyter Notebooks, Amazon Athena and AWS EMR to execute, run, query and monitor data processes. You will also use SQL, Python and PySpark to interrogate and validate data across complex datasets.
This is an excellent opportunity for a hands‑on Test Engineer with strong data testing experience to work on modern cloud‑based data platforms within a permanent Specialist Talent role.
Responsibilities
* Test and validate big data solutions and data lake environments containing structured and unstructured data
* Interrogate and validate data using SQL, Python and PySpark
* Execute, run, query and monitor data processes using Apache Airflow, Jupyter Notebooks, Amazon Athena and AWS EMR
* Validate data ingestion, transformation, processing and output across AWS‑based data platforms
* Identify data quality issues, reconciliation errors, pipeline failures and transformation defects
* Create and execute test plans, test cases, test scripts and test evidence
* Work closely with data engineers, developers, analysts and business stakeholders to understand expected data outcomes
* Support testing across AWS services including S3, Glue, EC2, ECR, DynamoDB, IAM and CloudWatch
* Use GitLab for version control and collaboration within the delivery lifecycle
* Raise, manage and retest defects, providing clear reporting on progress, risks and quality
* Support repeatable and automated data validation approaches where appropriate
Skills & Experience
* Proven experience as a Test Engineer, ideally within data, big data, cloud or data lake environments
* Active SC Clearance required
* Experience testing big data solutions or data lakes containing structured and unstructured data
* Strong experience with SQL for querying, interrogation and data validation
* Experience with Python, ideally including PySpark, for validating and interrogating data
* Hands‑on experience with AWS EMR, Apache Airflow and Jupyter Notebooks
* Experience using tools for executing, running, querying and monitoring data processes
* Familiarity with AWS data and cloud services such as Amazon Athena, S3, Glue, ECR, EC2, EMR, DynamoDB, IAM and CloudWatch
* Experience creating test plans, test cases, test scripts and test evidence
* Strong understanding of data quality, data validation and reconciliation techniquesAbility to work at SFIA Level 4, taking ownership of assigned testing activities and working with limited supervision
* Familiarity with Terraform
* Experience with GitLab and CI/CD pipelines
* Experience testing ETL/ELT processes, batch processing or data pipeline orchestration
* Exposure to automated data validation or test automation frameworks
* Experience working in Agile delivery environments
* Experience supporting cloud‑based transformation or data migration programmes
You’ll also get access to a full range of benefits alongside your salary.
#J-18808-Ljbffr