Build and scale the "Safety & Trust" engine for our agentic AI ecosystem. You will be the technical lead responsible for ensuring our AWS Bedrock-based agents meet the highest standards of FCA Operational Resilience, DORA, and EU AI Act (Art. 15) compliance.
Core Responsibilities
* Automated Red Teaming: Implement adversarial testing (Garak, Pyrit, AgentDojo) directly into CI/CD pipelines with automated release gating.
* Centralised Eval Platform: Operate a firm-wide service to measure success rates, uncertainty, hallucination, and bias across all non-deterministic systems.
* Secure Architecture: Map OWASP LLM Top 10 and agentic threats to technical controls; manage AWS Bedrock Guardrails and Knowledge Bases.
* AI Supply Chain: Own the AI-BOM, ensuring supply chain integrity, signed artifacts, and drift monitoring.
* Regulatory Evidence: Produce the technical documentation and robust testing evidence required for EU AI Act Article 15.
Technical Requirements
* AWS Bedrock Expert: Hands‑on experience with Bedrock Agents, Knowledge Bases, and model lifecycle management.
* AI/ML Depth: Strong grasp of FMs, RAG, tool‑use, and the failure modes of agentic workflows.
* Security & Compliance: Deep knowledge of NIST AI RMF, OWASP LLM Top 10, and UK/EU financial regulations (FCA/DORA).
* Testing Automation: Proven ability to build measurement frameworks for drift, memorization, and adversarial robustness.
Qualifications
* Significant experience in UK Financial Services.
* Expertise in automated adversarial testing and evaluation at scale.
* Ability to bridge the gap between complex AI engineering and rigid regulatory requirements.
#J-18808-Ljbffr