FDM is a global business and technology consultancy seeking a Senior Data Engineer to work for our client within the transport sector. This is initially a 6-month contract with the potential to extend and will be a hybrid role based in Manchester.
Our client is looking for an experienced Senior Data Engineer with a strong background in Azure Data Services and Data Fabric to support the design, build, and optimisation of enterprise-grade data solutions. The role requires hands-on expertise in developing advanced data pipelines, architecting data lake structures, aligning governance frameworks, and enabling robust reporting capabilities.
The ideal candidate will be someone who can take end-to-end ownership of data engineering initiatives and deliver solutions aligned with enterprise data architecture as well as governance principles.
Responsibilities
1. Build complex ETL/ELT pipelines from scratch, integrating multiple data sources while implementing advanced transformations using mapping data flows
2. Optimise performance and manage dependencies, ensuring robust monitoring/debugging of pipelines
3. Architect and implement hierarchical data storage structures with partitioning as well as lifecycle policies
4. Define and apply advanced RBAC as well as ACL-based security models while enabling integration with Synapse, Databricks, and serverless SQL pools for analytics
5. Deliver end-to-end reporting solutions, including data modelling in Power Query and DAX using Power BI to build interactive dashboards and implement row-level security (RLS)
6. Develop reusable, scalable data transformation scripts and pipelines and optimise Spark jobs for large-scale distributed data processing
7. Implement and transfer CI/CD pipelines for data workflows using Azure DevOps while enforcing Git based version control best practices and automating the testing as well as deployment of data engineering solutions
8. Apply Data Fabric principles to ensure seamless integration, discoverability, and governance across diverse data sources while managing as well as optimising reports in the Power BI service
9. Contribute to the enterprise-wide data fabric strategy by maintaining the implementation and adoption of a unified data platform for analytics, AI, and regulatory requirements