Hays are now looking for a Databricks Consultant / SME for one of our local clients:
1. Flexible rate - Outside IR35
2. 1 day a week in Swindon, the rest is WFH
3. Initial 6 month contract (extensions likely)
We’re seeking a highly experienced Databricks specialist who can confidently navigate the platform end-to-end, including leading migrations from legacy setups using DBFS-mounted tables and Hive Metastore. You’ll bring strong AWS knowledge, hands-on experience working with S3 via Python, a solid understanding of data storage formats, and practical Python development expertise.
What you’ll be doing
4. Lead the migration of Databricks assets (notebooks, jobs, clusters) from DBFS to Unity Catalog
5. Define and implement a scalable, standardised migration approach across teams
6. Support teams with ad-hoc Databricks issues related to migration and platform usage
7. Optimise the Databricks environment for structure, access, andernance
8. Automate infrastructure and deployments using Terraform and/or CloudFormation
9. Build and maintain data pipelines using Python, SQL and Databricks APIs
10. Monitor platform performance using tools such as CloudWatch, Grafana, and Kibana
11. Collaborate with engineering teams, stakeholders, and platform users
What we’re looking for
12. Strong hands-on experience with Databricks in an AWS environment
13. Proven experience with Unity Catalog and Databricks migrations
14. Strong Python skills and experience working with Databricks APIs
15. Solid understanding of data pipelines, SQL, and data platform architecture
16. Experience with Infrastructure as Code (Terraform or CloudFormation)
17. Knowledge of monitoring and observability tools ( CloudWatch, Grafana, Kibana)
18. Strong problem-solving skills and ability to work independently
19. Excellentmunication and stakeholder engagement skills
20. Exposure to Databricks Apps or AI/ML workloads (nice to have)
#4802220 - Jonathan Davies