Job Title: Databricks Architect
Location: UK/Europe - Remote
Engagement Type: Contract (Outside IR35)
Role Overview
We are seeking a Senior Databricks Architect with 15+ years of experience in Data Engineering/Analytics to design and deliver scalable, secure, and cost-efficient data solutions on AWS. This is a leadership role requiring strong hands-on expertise, proactive risk management, and excellent stakeholder communication skills.
Key Responsibilities
* Architect and implement data solutions using Databricks on AWS, PySpark, and Python
* Lead cloud migration initiatives, including on-prem to AWS migrations
* Drive GenAI-enabled automation for data asset migration
* Develop scalable data models and deliver actionable business insights
* Design and optimize ETL pipelines (Informatica or similar tools)
* Build and deploy PySpark applications for large-scale data processing
* Work on LLM pretraining, fine-tuning, augmentation, and optimization
* Ensure high data quality through cleaning, transformation, and validation (Python/SQL)
* Automate data workflows to enhance scalability and efficiency
* Maintain clear technical documentation and best practices
Required Experience
* 15+ years in Data Analyst/Data Engineer/Data Architect roles
* Strong expertise in Databricks on AWS
* Advanced Python (Pandas, Scikit-learn) and SQL skills
* Proven experience in large-scale cloud data migrations
* Hands-on experience with LLMs and GenAI-based automation
* Strong experience designing secure, high-performance data architectures
Key Competencies
* Proactive risk & issue management
* Strong requirement clarification and stakeholder engagement skills
* Excellent communication – able to translate technical insights into business value
* Critical thinker with strong problem-solving capabilities
* Self-driven, autonomous, and collaborative approach