We're seeking an experienced Databricks Engineer to help design and deliver strategic data solutions across enterprise platforms. This role will support both proof-of-concept work and the development of a Strategic Operational Data Store (ODS), with a focus on performance, scalability, and governance. This role is 3 days a week onsite in either Newcastle, London or Edinburgh and falls outside IR35.
Key Responsibilities
Lead Databricks platform evaluation, including performance benchmarking and stress testing
Collaborate with Databricks Professional Services for architectural assurance
Design and document scalable data models and integration patterns
Support enterprise integration, including Salesforce data and unified client ID strategies
Migrate Azure Data Factory flows to Databricks for improved traceability
Implement Unity Catalog for automated data lineage
Deliver backlog items through Agile sprint planning
Skills & Experience
Strong hands-on experience with Databricks, Fabric, Apache Spark, Delta Lake
Proficient in Python, SQL, and PySpark
Familiar with Azure Data Factory, Event Hub, Unity Catalog
Solid understanding of data governance and enterprise architecture
Effective communicator with experience engaging stakeholders
Desirable
Experience with Salesforce data integration
Prior involvement in platform PoCs or benchmarking
Knowledge of Strategic ODS and enterprise integration strategies