Company Description
Location: Basingstoke (hybrid working 3 office days per week)
Employment Type: Permanent, full time
Additional Benefit: Annual Bonus
About the AA
The AA is a well-loved brand with a wide range of driver services. We have an enviable set of data assets from breakdown, service, repair, insurance, telematics, digital interactions, car dealers and driving schools.
Data is a strategic priority for the group. We are modernising our data infrastructure to a cutting‑edge cloud platform and enabling machine learning and GenAI.
Job Overview
We are looking for an experienced Databricks Data Engineer who will design, build, and operate production‑grade Databricks Lakehouse solutions, including structured streaming pipelines, Unity Catalog governance, and Spark‑based data engineering at scale.
Responsibilities
* Design, build and operate production‑grade Databricks Lakehouse solutions, including structured streaming pipelines using Python and PySpark.
* Own and evolve Unity Catalog‑based governance to keep data assets secure, discoverable and well‑managed.
* Develop and maintain event‑driven data pipelines in close collaboration with backend engineering teams.
* Implement and support CI/CD pipelines in Azure DevOps for reliable and automated Databricks deployments.
* Create high‑quality, analytics‑ready datasets that deliver actionable business insight at scale.
* Proactively improve performance, reliability, automation and observability across the Databricks data platform.
Essential Experience
* Significant, hands‑on experience with Azure Databricks in a production, enterprise‑scale environment.
* Proven ability to build Spark/Databricks pipelines using Python and PySpark.
* Strong experience with structured streaming and event‑driven architectures.
* Practical experience implementing and operating Unity Catalog.
* Solid understanding of Lakehouse design principles, including dimensional and analytical modelling.
* Experience building and maintaining CI/CD pipelines, ideally using Azure DevOps.
* Confidence working with large‑scale data, performance tuning and troubleshooting complex pipelines.
* Strong debugging and problem‑solving skills, including diagnosing and resolving issues in Spark/Databricks pipelines.
What You Bring
* Hands‑on experience configuring and managing Databricks environments, including cluster setup, dependency management and package installation.
* Designed and built end‑to‑end Databricks pipelines.
* Made architectural decisions within Databricks environments.
* Worked with Spark internals, optimisation techniques and cluster configuration.
* Operated Databricks solutions in live, business‑critical contexts.
Interview Process
Candidates will be assessed on hands‑on technical ability in Azure Databricks. Expect to discuss and demonstrate how you have built, configured, optimised and troubleshooted Databricks pipelines in a production environment.
Benefits
* 25 days annual leave plus bank holidays and holiday buying scheme.
* Worksave pension scheme with up to 7% employer contribution.
* Free AA breakdown membership from Day 1 plus 50% discount for family and friends.
* Discounts on AA products including car and home insurance.
* Employee discount scheme with access to a car salary‑sacrifice scheme and discounts on healthcare, shopping, holidays, and more.
* Company‑funded life assurance.
* Diverse learning and development opportunities.
* Dedicated Employee Assistance Programme and a 24/7 remote GP service for you and your family.
Equal Opportunities
We’re an equal opportunities employer and welcome applications from everyone. The AA values diversity and the difference it brings to our culture. We actively seek people from diverse backgrounds to join an inclusive company where you can be yourself and feel empowered.
#J-18808-Ljbffr