SC Cleared Senior Data Engineer - Occasional Travel - Up to £540/day Inside IR35
We are seeking an experienced Senior Data Engineer with active SC clearance to support a Central Government department on a mission-critical data and analytics programme. This role suits someone with strong Microsoft Fabric expertise and experience delivering secure, scalable data solutions in complex public sector environments.
The Role
You'll design, build, and operate production-grade data pipelines that underpin reporting, MI, and analytics. Working closely with product, data, and platform teams, you'll help modernise Legacy data estates and embed strong data engineering standards.
Key Responsibilities:
1. Build and operate scalable data pipelines using Microsoft Fabric (OneLake/Delta Lake, Data Factory, Synapse Data Engineering).
2. Develop batch processing solutions using PySpark, Spark SQL, Python, and SQL, with a focus on performance, resilience, and data quality.
3. Support reporting and MI use cases, including transformations and data models feeding Power BI.
4. Own CI/CD and version control practices; review code and uphold engineering standards.
5. Mentor engineers and provide technical input into design and architecture.
6. Wo...