?? Job Opportunity: Principal AI Test Engineer
?? Location: Cambridge, UK
Work from Client office 2-3 days in a week - Hybrid
Role Requirement-
We are looking for detail-oriented and forward-thinking Test Engineer to ensure the
quality, performance, and security of our Core AI Services. You will help validate
distributed, cloud-native services and public APIs that form the foundation for enterprise
AI capabilities. This role demands deep technical skill and a passion for delivering
robust, secure, and ethical AI services at scale.
Youll be a part of a Scrum team and work closely with developers and architects to
design effective validation strategies, automated testing frameworks, and AI-specific
evaluation tools - with a builder mindset - rapid prototyping and continuous improvement
with agility of a start-up
Key Responsibilities:
? Perform functional, performance, and security testing on cloud-native services
deployed on Microsoft Azure .
? Design and implement automated test suites for APIs, service components, and
AI pipelines.
? Automate the evaluation of AI system outputs to ensure accuracy, consistency,
and safety of responses.
? Collaborate with developers and data scientists to establish service-level quality
metrics and observability hooks.
? Validate services against AI regulatory frameworks and ensure traceability,
fairness, and robustness in outcomes.
? Participate in threat modelling and security validation of exposed APIs and AI
services.
? Provide feedback early in the lifecycle to reduce defects and improve design.
? Mentor junior testers, encourage continuous learning, and contribute to a culture
of innovation.
? AI & Cloud Expertise
? Familiarity with LLM evaluation techniques, output scoring, and validation
frameworks.
? Understanding of key concepts such as prompt engineering, RAG, model
orchestration, and hallucination detection.
? Experience in testing for accuracy, relevance, and consistency of AI model
predictions/generations
? Defining Performance Metrics for AI services and testing for the same
? Awareness of AI safety, bias detection, and explainability techniques.
? Experience ensuring compliance with AI regulations and standards (e.g., NIST AI
RMF, EU AI Act).
? Strong belief in ethical AI practices, transparency, and end-user trust.
Core Skills and Qualifications:
? 12+ years of experience in software testing, QA, or validation roles for cloudnative
applications using Microsoft and .NET technologies.
? Proficient in designing automated testing frameworks
? Hands-on experience with Azure DevOps, CI/CD pipelines, and containerized
test environments.
? Strong understanding of API testing, performance profiling, and security testing
(including OWASP top 10).
? Excellent problem-solving skills, with the ability to analyse complex technical
challenges and propose scalable solutions.
? Experience working in Agile teams and collaborating across global R&D locations
? Demonstrated ability to mentor junior team members fostering a culture of
continuous learning and innovation
TPBN1_UKTJ