Test Engineer – Data Platforms - SC Clearance
Permanent | Newcastle, occasional onsite attendance | Data & Testing | SC Clearance
At Peregrine, we’re always seeking Specialist Talent that have the ideal mix of skills, experience, and attitude, to place with our vast array of clients. From Business Analysts in large government organisations to Software Developers in the private sector – we are always in search of the best talent to place, now.
The role:
We are seeking an SC Cleared Test Engineer to support the testing and assurance of large-scale data platforms, big data solutions and data lake environments.
This role will focus on validating structured and unstructured data across AWS-based environments, ensuring data pipelines, processing workflows and outputs are accurate, reliable and fit for purpose.
You will use tools such as Apache Airflow, Jupyter Notebooks, Amazon Athena and AWS EMR to execute, run, query and monitor data processes. You will also use SQL, Python and PySpark to interrogate and validate data across complex datasets.
This is an excellent opportunity for a hands-on Test Engineer with strong data testing experience to work on modern cloud-based data platforms within a permanent Specialist Talent role.
Responsibilities:
* Test and validate big data solutions and data lake environments containing structured and unstructured data
* Interrogate and validate data using SQL, Python and PySpark
* Execute, run, query and monitor data processes using Apache Airflow, Jupyter Notebooks, Amazon Athena and AWS EMR
* Validate data ingestion, transformation, processing and output across AWS-based data platforms
* Identify data quality issues, reconciliation errors, pipeline failures and transformation defects
* Create and execute test plans, test cases, test scripts and test evidence
* Work closely with data engineers, developers, analysts and business stakeholders to understand expected data outcomes
* Support testing across AWS services including S3, Glue, EC2, ECR, DynamoDB, IAM and CloudWatch
* Use GitLab for version control and collaboration within the delivery lifecycle
* Raise, manage and retest defects, providing clear reporting on progress, risks and quality
* Support repeatable and automated data validation approaches where appropriate
Skills & Experience:
* Proven experience as a Test Engineer, ideally within data, big data, cloud or data lake environments
* Active SC Clearance required
* Experience testing big data solutions or data lakes containing structured and unstructured data
* Strong experience with SQL for querying, interrogation and data validation
* Experience with Python, ideally including PySpark, for validating and interrogating data
* Hands-on experience with AWS EMR, Apache Airflow and Jupyter Notebooks
* Experience using tools for executing, running, querying and monitoring data processes
* Familiarity with AWS data and cloud services such as Amazon Athena, S3, Glue, ECR, EC2, EMR, DynamoDB, IAM and CloudWatch
* Experience creating test plans, test cases, test scripts and test evidence
* Strong understanding of data quality, data validation and reconciliation techniques
* Ability to work at SFIA Level 4, taking ownership of assigned testing activities and working with limited supervision
Desirable skills:
* Familiarity with Terraform
* Experience with GitLab and CI/CD pipelines
* Experience testing ETL/ELT processes, batch processing or data pipeline orchestration
* Exposure to automated data validation or test automation frameworks
* Experience working in Agile delivery environments
* Experience supporting cloud-based transformation or data migration programmes
About Peregrine
We build workforces that deliver tech and change programmes at leading UK organisations. By combining data science from Peregrine Intelligence, our industry-accredited Peregrine Academy, and market-leading attraction and diversity initiatives, we bridge capability gaps at all levels in public and private sector organisations. We work closely with our clients to understand their challenges and deliver flexible, long-term solutions that make a real difference. When you join Peregrine, you become part of a team that’s focused on growth, both yours, our clients’, and the sectors we support. You’ll also get access to a full range of benefits alongside your salary.
How Specialist Talent Works
As a permanent employee at Peregrine, you’ll be part of our Specialist Talent team. That means you’ll work on-site or remotely with our clients, supporting them on complex, high-impact projects in Data, Digital and Business Transformation. You’ll get the variety and challenge of consultancy work, with the stability and support of a permanent role. You’re not a contractor - you’re a valued member of our team, with access to all the same benefits, learning opportunities, and community.
Our Culture
We’re curious, open, and always learning. We believe in doing great work and doing it with integrity. We support each other, share ideas freely, and celebrate progress big and small. And we care about the impact we make, whether that’s helping clients solve tough problems or creating opportunities for people from all walks of life.
Diversity and Inclusion
We’re proud to be a diverse team, with 20+ nationalities represented across Peregrine. We welcome people from all backgrounds and experiences, and we’re committed to building an inclusive workplace where everyone can thrive. Through our partnerships and internal initiatives, we’re working to improve social mobility, reduce our environmental impact, and help our clients build more inclusive teams too.
Find out more: peregrine.global or check out our LinkedIn page: peregrine-resourcing