Role Requirement-
We are looking for detail-oriented and forward-thinking
Test Engineer
to ensure the
quality, performance, and security of our Core AI Services. You will help validate
distributed, cloud-native services and
public APIs
that form the foundation for enterprise
AI capabilities. This role demands deep technical skill and a passion for delivering
robust, secure, and ethical AI services at scale.
You'll be a part of a
Scrum team
and work closely with developers and architects to
design effective validation strategies, automated testing frameworks, and AI-specific
evaluation tools - with a builder mindset - rapid prototyping and continuous improvement
with agility of a start-up
Core Skills and Qualifications:
Familiarity with LLM evaluation techniques, output scoring, and validation
frameworks.
Understanding of key concepts such as prompt engineering,
RAG
, model
orchestration, and hallucination detection.
Experience in testing for accuracy, relevance, and consistency of AI model
predictions/generations
Defining Performance Metrics for AI services and testing for the same
Awareness of AI safety, bias detection, and explain ability techniques.
Experience ensuring compliance with
AI regulations and standards (e.g., NIST AI
RMF, EU AI Act).
Strong belief in ethical AI practices, transparency, and end-user trust.
12+ years of experience in software testing, QA, or validation roles for cloud native
applications using Microsoft and .NET technologies.
Proficient in designing automated testing frameworks
Hands-on experience with Azure DevOps, CI/CD pipelines, and containerized
test environments.
Strong understanding of API testing, performance profiling, and security testing
(including OWASP top 10).
Excellent problem-solving skills, with the ability to analyze complex technical
challenges and propose scalable solutions.
Experience working in Agile teams and collaborating across global R&D locations
Demonstrated ability to mentor junior team members fostering a culture of
continuous learning and innovation