Job Title: Artificial Intelligence Engineer (Databricks)
Rate: DOE (outside IR35)
Location: Remote
Contract Length: 6 months
A consultancy client of ours have secured a project requiring a Databricks focused Artificial Intelligence Engineer. This is an exciting opportunity to work on cutting-edge machine learning projects, building scalable ML pipelines and cloud-based systems that deliver real-world impact.
Key Responsibilities:
* Lead the design, development, and optimisation of scalable machine learning workflows using Azure Databricks
* Build and deploy robust ML pipelines leveraging Delta Lake, MLflow, notebooks, and Databricks Jobs
* Apply advanced knowledge of Databricks architecture and performance tuning to support production-grade ML solutions
* Collaborate with data scientists, data engineers, and analysts to operationalise machine learning models at scale
* Champion the use of Databricks-native features (e.g., Unity Catalog, MLflow Model Registry, AutoML) to improve model lifecycle management
* Migrate legacy model training and scoring workflows into unified Databricks-based pipelines
* Ensure best practices in model reproducibility, governance, monitoring, and security within the Databricks environment
* Act as a subject matter expert on Databricks ML capabilities, advising on architecture, tools, and integrations
* Mentor peers and junior engineers on ML engineering practices, with a focus on MLOps and Databricks workflows
* Continuously improve the machine learning platform, tooling, and deployment practices to accelerate delivery
Experience and Qualifications Required:
* Deep hands-on experience with Azure Databricks, particularly in developing and deploying machine learning solutions using Delta Lake, MLflow, and Spark ML/PyTorch/TensorFlow integrations
* Strong programming skills in Python (including ML libraries like scikit-learn, pandas, PySpark) and experience using SQL for data preparation and analysis
* Experience orchestrating end-to-end ML pipelines, including data preprocessing, model training, validation, and deployment
* Solid understanding of MLOps principles, including model versioning, monitoring, and CI/CD for ML workflows
* Familiarity with Azure cloud services, including Azure Data Lake, Azure Machine Learning, and Data Factory
* Experience with feature engineering, model management, and automated retraining in production environments
* Knowledge of data governance, security, and regulatory compliance in the context of ML workflows
* Strong problem-solving skills, with the ability to debug and optimise distributed ML pipelines
* Proven track record of delivering machine learning models in production within enterprise-scale environments
* Excellent communication and collaboration skills, with experience engaging both technical and business stakeholders
* Experience mentoring others and promoting best practices in ML engineering and Databricks usage
If this sounds like an exciting opportunity please apply with your CV.