Required Skills & Experience
Must Have
* 8+ years of data architecture experience with at least 3 years hands-on with Azure Databricks in production environments
* Proven experience designing and implementing medallion architecture (Bronze / Silver / Gold) using Delta Lake on Azure Databricks
* Strong hands-on experience with Unity Catalog — metastore setup, catalog and schema design, access controls, and governance policies
* Experience configuring Azure Data Lake Storage Gen2 — containers, access control lists, lifecycle policies, and integration with Databricks
* Hands-on experience with Microsoft Fabric — specifically Mirrored Azure Databricks Unity Catalog and Power BI connectivity to Databricks SQL Warehouses
* Experience with enterprise data source integration — MS SQL Server (on-premises and Azure SQL) and Oracle Database ingestion into Azure landing zones
* Strong Apache Spark skills — PySpark or Scala, optimisation, partitioning, and job scheduling in Databricks
* Experience assessing and remediating partially built or vendor-abandoned data platform implementations
* Proven ability to engage senior stakeholders — Architecture Council, CIO, CDO level — presenting technical findings in clear business language
* UK-based with ability to travel to Pennon client sites (South West England) — onsite presence required particularly in Weeks 1–2
Highly Desirable
* Experience in the utilities, water, or regulated infrastructure sector — understanding of operational data, compliance requirements, and safety-critical system considerations
* Familiarity with Lakehouse Federation in Azure Databricks — federating queries across external data sources without full ingestion
* Experience with Azure Data Factory or equivalent ETL/ELT tooling for enterprise source ingestion pipelines
* Exposure to Delta Live Tables (DLT) for declarative pipeline management
* Experience with Power BI data modelling and DirectQuery / Import mode considerations for Databricks SQL Warehouse connectivity
* Familiarity with Azure Key Vault for secrets management and secure credential handling in Databricks
* Prior experience in a vendor transition or rescue engagement — inheriting and assessing another vendor's technical work
* Knowledge of Databricks Asset Bundles or Databricks Workflows for job orchestration
* TOGAF or equivalent enterprise architecture certification