Principal Test Engineer
AI Core Services (Microsoft Azure / .Net)
We're seeking a highly skilled
Principal Test Engineer
to lead the
validation of cloud-native AI services and APIs
. This role focuses on
designing automated testing frameworks
and
AI-specific evaluation tool
s to ensure enterprise-grade reliability, security, and ethical compliance.
About the Role
As a Principal Test Engineer, you will be responsible for
leading the validation efforts for Azure-based services
, ensuring they meet the highest standards of quality and compliance.
Responsibilities
* Lead functional, performance, and security testing of
Azure-based services
* Build automated test suites for APIs, service components, and AI pipelines
* Develop tools to
assess AI outputs
for accuracy, consistency, and safety
* Collaborate with engineering and data science teams to define quality metrics and observability
* Ensure services meet
AI regulatory standards
(e.g.,
NIST AI RMF, EU AI Act
) with traceability and fairness
* Conduct
threat modelling
and validate exposed APIs for security
* Provide early feedback to improve design and reduce defects
* Mentor junior testers and foster a culture of innovation
Qualifications
* Proven experience in testing
cloud-native applications
using
Microsoft/.NET technologies
* Strong background in
automated testing frameworks
and QA leadership
* Deep understanding of
AI systems and regulatory compliance
Required Skills
* AI evaluation: LLM scoring, hallucination detection, prompt engineering, RAG, model orchestration
* AI safety: bias detection, explainability, ethical AI practices
* Cloud & DevOps: Azure DevOps, CI/CD, containerized environments
* Testing: API validation, performance profiling, OWASP security testing
* Agile collaboration across global R&D teams
* Experience with advanced AI testing methodologies