About the Role
Skills and Requirements:
Proficiency in JavaScript and Python for AI testing and automation.
Expertise in adversarial testing, identifying LLM vulnerabilities, and implementing secure AI practices.
Hands-on experience with AI testing frameworks like Botium, Pyrit, or similar tools for LLM evaluation.
Familiarity with knowledge base evaluation methods and tools for assessing LLM performance (precision, recall, consistency).
Ability to design and execute adversarial testing, red-teaming, and stress-testing for LLMs.
Knowledge of LLM security vulnerabilities, ethical AI principles, and AI compliance frameworks.
Experience in prompt engineering, and analyzing LLM behavior under adversarial conditions.
Awareness of emerging trends in AI security and LLM testing techniques.