The role
As a Data Engineer, you’ll play a pivotal role in delivering a proactive, high‑quality managed data service for customers using modern analytics platforms such as lakehouse, SQL, Spark and semantic models.
You’ll focus on ensuring the reliability, performance, data quality and overall optimisation of customer data environments that have been onboarded into our managed service. Working within ITIL‑aligned Incident, Change and Problem Management processes, you’ll help keep customer platforms healthy, stable and fit for purpose.
This is a hands‑on engineering role, ideal for someone who thrives on troubleshooting, performance tuning and managing controlled change. Rather than greenfield development, you’ll be enhancing and supporting live production systems while guiding customers on best practice and improvements.
What will I be doing?
Platform Monitoring & Incident Response
* Monitor pipelines, dataflows, jobs and lakehouse workloads for failures or performance issues.
* Respond to alerts, diagnose root causes and restore service quickly.
* Fix issues across pipeline steps, data refresh, connectivity and authentication.
* Safely re‑run jobs to restore normal operation.
* Support D365 governance requirements.
Performance & Capacity Management
* Monitor capacity usage, throttling and performance risks.
* Analyse performance of SQL, Spark, notebooks, Delta optimisation and semantic models.
* Implement optimisations such as query tuning, indexing, scheduling improvements and compute scaling.
Data Quality & Schema Management
* Detect schema changes, datatype shifts, anomalies and missing/late data.
* Maintain and run data quality rules (duplicates, thresholds, data completeness).
* Investigate and resolve data quality issues.
Change Delivery & Continuous Improvement
* Deliver customer‑requested changes through formal Change Management.
* Update pipelines, schemas, calculated fields, metadata and RLS roles.
* Optimise slow‑running workloads and provide impact/rollback assessments.
Connectivity, Security & Access
* Troubleshoot linked services, runtime failures and network access issues.
* Provide guidance on gateway configuration and authentication.
* Support cloud gateway remediation and manage workspace/dataset/lake permissions.
Tooling, Documentation & Knowledge Management
* Use telemetry, logs and diagnostics to investigate reliability and performance issues.
* Maintain data dictionaries, lineage documentation, runbooks and data flow diagrams.
* Ensure all changes are recorded in up‑to‑date support documentation.
What will I bring to the role?
Essential
* Experience supporting and operating production data platforms.
* Strong SQL skills, including optimisation and troubleshooting.
* Spark-based data processing and performance tuning experience.
* Hands‑on work with pipelines/orchestration, lakehouse/warehouse architectures, and semantic models.
* Familiarity with incident and change management processes.
* Strong problem‑solving and root‑cause analysis abilities.
* Clear, structured documentation skills.
Desirable
* Experience with Microsoft Fabric, Azure data services or similar cloud analytics platforms.
* DAX optimisation skills.
* Understanding of capacity‑based analytics and throttling behaviour.
* Experience in customer‑facing managed services.
* Knowledge of data governance, lineage and data quality frameworks.
Behaviours & Attributes
* Customer‑focused with a strong service mindset.
* Confident in high‑availability production environments.
* Calm, methodical incident diagnosis.
* Strong communicator and collaborative team player.
* Proactive in spotting risks and improvements.
* Able to balance reactive support with planned optimisation work