Position Overview
As a Senior Machine Learning QA Engineer in the Research Enablement team, you will work side-by-side with researchers, Machine Learning Engineers and software engineers to define and uphold quality standards for ML systems. You are a quality-focused engineer who is passionate about reliable, repeatable evaluation of ML models and data. Your skills span test strategy, automation, and a little MLOps, with a strong software engineering base. You are excited to collaborate across research and product to ship ML capabilities with clear quality gates. You are comfortable working at the intersection of research and product and are competent in using Autodesk CAD software.
Reporting Structure: You will report to an Engineering Manager in Research Enablement.
Location: United Kingdom We are a global team, located in London, San Francisco, Toronto, and remotely. Autodesk is a hybrid-first company, allowing workers to work remotely, in an office, or a mix of both.
Responsibilities
1. Define ML quality strategy and acceptance criteria across data, model, and system levels
2. Design and maintain model evaluation suites, metrics, and test datasets
3. Evaluating CAD RL model outputs for geometric validity or policy stability
4. Defining structured rubrics that translate qualitative findings into measurable evaluation gates
5. Testing ML Models from product side
6. API Testing
7. Automate ML QA workflows using Python and CI/CD (e.g., GitHub Actions, Jenkins)
8. Create and maintain test harnesses for ML services and APIs
9. Mentor teams on ML QA best practices and consistent evaluation standards
10. Build quality gates for training and deployment pipelines (e.g., regression checks, drift detection)
11. Contribute to multi-team projects and codebases, ensuring code quality and consistency
12. Participate in code reviews and provide constructive feedback to peers
13. Document and present findings and ideas across the company
Minimum Qualifications
14. Bachelor’s degree in Computer Science, Engineering, or equivalent experience
15. 7+ years of professional experience in software engineering or QA for ML/AI systems
16. Strong programming skills in Python, with experience in test automation
17. Familiarity with popular CAD environments tooling
18. Proficient in Automation and UAT test suite/framework
19. Experience designing QA frameworks or platforms used by multiple teams
20. Excellent problem-solving skills and attention to detail
21. Strong communication and collaboration skills
22. Understanding of software architecture and design patterns
23. Ability to work in an agile development environment
Preferred Qualifications
24. Experience with data validation tooling (e.g., Great Expectations) or labeling workflows
25. Familiarity with ML frameworks (e.g., PyTorch, TensorFlow)
26. Experience with CI/CD tools and processes
27. Experience with data pipelines and orchestration tools (e.g., Airflow, Metaflow)
28. Familiarity with MLOps practices (model monitoring, drift, deployment checks)
29. Experience with ML evaluation methods, metrics, and benchmarking
30. Passion for learning new technologies and improving existing systems
31. Experience with cloud providers (e.g., AWS, Azure, Google Cloud Platform)
32. Experience testing ML services in production environments
33. Knowledge of experiment tracking tools (e.g., Comet, MLflow, Weights & Biases)
The Ideal Candidate
34. You demonstrate initiative to provide solutions and to learn and develop new technologies
35. Comfortable building QA systems from scratch and writing maintainable automation
36. You enjoy learning and collaborating across global locations
37. You are comfortable working in newly forming ambiguous areas
38. You are comfortable building scalable and maintainable systems that will be relied on by others
39. You can communicate well with others
Learn More
About Autodesk
Welcome to Autodesk! Amazing things are created every day with our software – from the greenest buildings and cleanest cars to the smartest factories and biggest hit movies. We help innovators turn their ideas into reality, transforming not only how things are made, but what can be made.
We take great pride in our culture here at Autodesk – it’s at the core of everything we do. Our culture guides the way we work and treat each other, informs how we connect with customers and partners, and defines how we show up in the world.
When you’re an Autodesker, you can do meaningful work that helps build a better world designed and made for all. Ready to shape the world and your future? Join us!
Salary transparency
Salary is one part of Autodesk’s competitive compensation package. Offers are based on the candidate’s experience and geographic location. In addition to base salaries, our compensation package may include annual cash bonuses, commissions for sales roles, stock grants, and a comprehensive benefits package.
Diversity & Belonging
We take pride in cultivating a culture of belonging where everyone can thrive. Learn more here:
Are you an existing contractor or consultant with Autodesk?
Please search for open jobs and apply internally (not on this external site).