The Role
As a Data Engineer you will be responsible for maintaining, enhancing, and evolving an existing Databricks data platform to support trusted, scalable, and high‑quality data products. Working closely with analysts, developers, and business stakeholders, you will build and extend a medallion architecture that underpins reporting, analytics, and downstream use cases. A key focus of the role will be embedding strong data quality controls and supporting clear data quality exception reporting to improve transparency, trust, and decision‑making across the organisation.
Databricks Platform & Data Engineering
* Maintain and optimise an existing Databricks environment, ensuring reliability, performance, and cost‑effectiveness
* Design, build, and enhance data pipelines using Spark and Databricks best practices
* Develop scalable, reusable data engineering patterns to support multiple data domains
* Monitor and troubleshoot pipeline failures, performance bottlenecks, and data issues
Medallion Architecture Development
* Build and evolve Bronze, Silver and Gold layers aligned to business and analytical needs
* Implement clear transformation, enrichment, and aggregation logic between layers
* Design the Gold layer data models in collaboration with analysts and BI developers to meet defined analytical and reporting consumption requirements
Data Quality & Exception Reporting
* Define and implement data quality rules, validations, and thresholds across the data lifecycle
* Develop automated data quality checks and exception reports to highlight data issues
* Proactively identify root causes of data quality incidents and implement preventative fixes
* Partner with data owners and stakeholders to resolve data quality issues and improve source system practices
Data Governance, Security & Integrity
* Support data governance standards including schema management, naming conventions, and metadata
* Ensure appropriate data access controls and security measures are applied
* Maintain data lineage and documentation to support auditability and trust
Stakeholder Collaboration
* Work closely with analysts, developers and business teams to understand data requirements
* Translate business needs into robust, maintainable data engineering solutions
* Provide guidance on data availability, structure, and appropriate usage
* Document pipelines, data models, data quality rules, and architectural decisions
* Contribute to shared standards, templates, and best practices within the Data team
* Support knowledge transfer and mentoring within the Data team
Continuous Improvement
* Continuously improve pipeline performance, reliability, and maintainability
* Identify opportunities to automate, standardise or simplify data engineering workflows
* Stay up to date with Databricks, Spark and cloud data engineering best practices
Essential Skills & Experience
* Proven experience as a Data Engineer working in a modern cloud data platform
* Strong hands‑on experience with Databricks and Apache Spark
* Experience building and maintaining medallion (Bronze/Silver/Gold) architectures
* Experience extracting and ingesting data using APIs
* Strong SQL skills for data transformation, validation, and analysis
* Proficiency in Python (or Scala) for data engineering and automation
* Experience designing and implementing data quality checks and controls
* Strong problem solving skills with a focus on data reliability and integrity
Desirable Skills & Experience
* Experience with cloud platforms such as Azure (especially services such as ADLS, Azure SQL Server, Key Vault) or similar platform services such as AWS/GCP
* Familiarity with Delta Lake and transactional data patterns
* Experience with data orchestration tools (e.g. Databricks Workflows, MS Scheduler)
* Understanding of data modelling concepts for analytics and reporting
* Experience supporting Power BI or other BI tooling through curated data layers
* Exposure to data governance, cataloguing, or lineage tools
* Familiarity with legal industry data and metrics.
Our Benefits
* Annual discretionary bonus scheme
* Company contributory pension scheme
* Hybrid working
* Life assurance
* Cash‑back healthcare scheme
* Enhanced family friendly polices
* Employee Assistance Programme
* Subsidised gym membership
* Cycle to Work scheme
* Staff referral scheme
J-18808-Ljbffr