Job Title:
Principal Test Engineer – AI Core Services
Work Location:
Cambridge, UK (Hybrid, 3 Days Onsite)
Role Overview:
We are looking for a
Senior Principal Test Engineer
to ensure the quality, performance, and security of our Core AI Services. This role focuses on validating distributed, cloud-native services and public APIs, with a strong emphasis on AI-specific testing, automation, and compliance.
You will work within a Scrum team, collaborating with developers, architects, and data scientists to design validation strategies, automated frameworks, and AI evaluation tools. This position combines deep technical expertise with a builder mindset, enabling rapid prototyping and continuous improvement.
Key Responsibilities:
Lead functional, performance, and security testing of Azure-based cloud-native services and APIs.
Design & implement automated test suites for APIs, microservices, and AI pipelines.
Automate AI system evaluations to ensure accuracy, consistency, and safety.
Define and validate service-level quality metrics and observability hooks.
Ensure compliance with AI regulatory frameworks (EU AI Act, NIST AI RMF).
Contribute to threat modelling and security validation.
Provide early lifecycle feedback to improve design and reduce defects.
Mentor junior engineers and promote a culture of innovation and ethical AI.
AI & Cloud Expertise:
LLM evaluation, output scoring, hallucination detection, and validation frameworks.
Knowledge of prompt engineering, RAG, and model orchestration.
AI performance metric definition and testing.
Awareness of AI safety, bias detection, and explainability techniques.
Strong belief in ethical AI practices and compliance.
Core Skills & Qualifications:
8-12+ years in software testing/QA for cloud-native apps using Microsoft & .NET.
Proven track record in test automation frameworks and CI/CD with Azure DevOps.
Expertise in API testing, performance profiling, and security validation (OWASP Top 10).
Strong analytical and problem-solving abilities.
Agile/Scrum experience in global R&D environments.
Demonstrated mentorship and leadership skills.