Job Description – Lead Data Engineer Data Engineer (Azure | Databricks | PySpark) Permanent | Hybrid | UK Public Sector Project OVERViEW We are seeking a hands-on Data Engineer to join a growing Data & Analytics practice. This role focuses on building, optimising, and maintaining modern cloud data platforms. You will work across the full data lifecycle, delivering scalable, high-performance solutions using Azure and Databricks. Who We’re Looking For A strong Data Engineer with deep technical expertise in Azure and Databricks, who enjoys building robust pipelines and working in a collaborative, client-facing environment. Key Responsibilities Build and maintain scalable data pipelines (batch & real-time) Develop solutions using Azure Databricks and PySpark Implement Medallion Architecture (Bronze, Silver, Gold) Design ETL/ELT pipelines using Azure Data Factory Develop data warehouse and lakehouse solutions Support BI, analytics, and AI workloads Optimise performance, cost, and reliability Implement CI/CD pipelines using Azure DevOps Collaborate with architects, analysts, and stakeholders Mandatory Requirements Strong hands-on experience with Azure Databricks (PySpark) Experience with Azure Data Factory and Synapse Strong SQL and data warehousing expertise Experience with data modelling (star, snowflake, Kimball) Experience with Power BI and Microsoft Fabric Strong programming skills (Python, SQL, scripting) Experience with CI/CD and DevOps practices Ability to produce technical documentation (HLD, LLD) Desirable Experience in consulting / professional services Exposure to AI and advanced analytics solutions Experience with real-time / streaming data Experience in regulated industries Soft Skills Strong problem-solving skills Collaborative and team-oriented mindset Clear communication and documentation skills Ability to work in fast-paced delivery environments Contract Details Role Type: Permanent & Hybrid (2 days per week in office) Location: Guildford Clearance: Candidates must be eligible for UK Security Clearance (SC) or active SC