Location: Leeds About Us We are Data Services, our mission is to unlock the value of data by delivering high-quality, reliable, and secure data services that are accessible, understandable, and actionable! We continuously evolve our offerings, using modern cloud-based technologies, and foster strong partnerships to help our colleagues in the Bank navigate the complexities of a data-driven world and achieve their strategic objectives. Job Description The use of data within Central Banking is evolving. The combination of data being collected at scale and significant advances in data technology has created the need to expand our Data Engineering team in Data Services. We are building the Bank's strategic cloud data platform to host products that solve real-world problems while putting customers at the centre. The Principal Data Engineer will play a pivotal role in designing, implementing, and supporting data pipelines on the Bank's data platform. You will lead innovative, data-centric products and predictive analysis work while developing automated solutions that continuously improve data engineering across the Bank. Your technical leadership will establish a centre of excellence, ensure regulatory compliance, and deliver smart, customer-focused designs. Role Requirements Design, develop, test, and deploy scalable, cost-effective, and secure distributed architectures using Azure services such as Azure Data Factory and Azure Databricks. Collaborate with multi-functional teams to translate business and technical requirements into innovative, automated solutions. Continuously innovate and evolve data ingestion and transformation framework design and implementation to meet modern data platform needs while providing comprehensive guidelines for customers. Manage the technical relationship with one or more Azure Data product groups, identifying common solution patterns and developing reusable frameworks. Lead the identification of dependencies and produce detailed design documents for data pipeline solutions. Leverage subject-matter expertise to drive project and release plans with key partners including SMEs and Product Owners. Participate in organisation-wide projects and steering committees while devising strategies for containerisation, orchestration, and API management. Design robust data replication architectures and implement methods to integrate AI and ML into platform and backend solutions in line with responsible AI principles. Collaborate with the data governance team to enforce a comprehensive data catalogue, maintain detailed dataset descriptions, and ensure alignment to security and privacy regulations. Work with Cloud teams and key stakeholders to optimise cloud costs using FinOps principles and enforce data platform-centric FinOps policies. Provide mentorship and coaching to data engineers on modern tools and technologies in Azure, data pipeline management, coding practices, design principles, and implementation patterns. Minimum Criteria Extensive experience in data engineering on an Azure data platform, with a strong background in modernisation and large-scale migration projects. Proven ability to coach and mentor others effectively in a senior capacity Demonstrable experience designing and managing metadata-driven frameworks using Azure Data Factory and Azure Databricks. Competence in and hands-on experience managing Databricks environments and developing lakehouse architectures with a focus on automation, performance tuning, cost optimisation, and system reliability. Proven proficiency in programming languages such as Python, T-SQL, and PySpark, with practical knowledge of test-driven development. Demonstrated capability in building secure, scalable data solutions on Azure with an in-depth understanding of data security and regulatory compliance, using tools like Microsoft Purview and Unity Catalog. Essential Critiera Proven ability to coach and mentor others effectively in a senior capacity, showcasing leadership skills. Excellent communication and collaboration skills with a track record of building and managing teams effectively. In-depth knowledge of data engineering practices and tools, including strategic decision-making within cloud environments. Strong problem-solving and adaptability to manage complex and dynamic projects. Desirable Criteria Experience working with advanced containerisation and orchestration tools such as Azure Container Instances or Kubernetes. Prior experience designing and implementing AI/ML driven solutions within data platforms. Relevant certifications such as Azure Data Engineer Associate, Azure Solutions Architect, or equivalent industry credentials. Advanced academic qualifications or industry accolades that complement practical experience in data engineering and cloud technologies. Experience in DevOps practices using Azure DevOps and GitHub Actions, designing automated CI/CD pipelines and implementing Infrastructure as Code(terraform, ARM templates, or Azure Bicep) Our Approach to Inclusion The Bank values diversity, equity and inclusion. We play a key role in maintaining monetary and financial stability, and to do that effectively, we believe we need a workforce that reflects the society we serve. At the Bank of England, we want all colleagues to feel valued and respected, so we're working hard to build an inclusive culture which supports people from all backgrounds and communities to be at their best at work. We celebrate all forms of diversity, including (but not limited to) age, disability, ethnicity, gender, gender identity, race, religion, sexual orientation and socioeconomic status. We believe that it's by drawing on different perspectives and experiences that we'll continue to make the best decisions for the public. We welcome applications from individuals who work flexibly, including job shares and part time working patterns. We've also partnered with external organisations to support us in making adjustments for candidates and employees in the recruitment process where they're needed. For most roles where work can be carried out at home, we aim for colleagues to spend half of their time in the office, with a minimum of 40% per month. Subject to that minimum requirement, individuals and managers should work together to find what works best for them, their team and stakeholders. Finally, we're proud to be a member of the Disability Confident Scheme. If you wish to apply under this scheme, you should check the box in the 'Candidate Personal Information' under the 'Disability Confident Scheme' section of the application. Salary and Benefits Information We encourage flexible working, part time working and job share arrangements. Part time salary and benefits will be on a pro-rated basis as appropriate. We offer a salary as follows: Leeds circa £72,320 - £81,360 In addition, we also offer a comprehensive benefits package as detailed below: Currently a non-contributory, career average pension giving you a guaranteed retirement benefit of 1/80th of your annual salary for every year worked. There is the option to increase your pension (to 1/65th) or decrease (to 1/105th) in exchange for salary through our flexible benefits programme each year. The Bank has the discretion to vary standard accrual rates and dial up and dial down rates at any time and to withdraw dial up and dial down options at any time. A discretionary performance award based on a current award pool. An 8% benefits allowance with the option to take as salary or purchase a wide range of flexible benefits. 26 days' annual leave with option to buy up to 12 additional days through flexible benefits. Private medical insurance and income protection. National Security Vetting Process Employment in this role will be subject to the National Security Vetting clearance process (and typically can take between 6 to 12 weeks post offer) and the passing of additional Bank security checks in accordance with the Bank policy. Further information regarding the vetting and security clearance requirements for the role will be provided to the successful applicant, and information about how the Bank processes personal data for these purposes, is set out in the Bank's Privacy Notice. The Bank of England welcomes applications from all candidates, but as a UK Visas and Immigration (UKVI) approved sponsor, we have a responsibility to comply with the Immigration Rules and guidance. As such, our ability to employ individuals who require sponsorship for immigration purposes is limited. The Bank cannot guarantee that you and / or the role you are applying for will be eligible for sponsorship and that any application made to UKVI will be successful. Eligibility will therefore be considered on a case by case basis. The Application Process Important: Please ensure that you complete the 'work history' section and answer ALL the application questions fully. All candidate applications are anonymised to ensure that our hiring managers will not be able to see your personal information, including your CV, when reviewing your application details at the screening stage. It's therefore really important that you fill out the work history and application form questions, as your answers will form a critical part of the initial selection process. The assessment process will comprise of two interview stages. This role closes on 30 May 2025. Please apply online, ensuring that you complete your work history and answer ALL the application questions fully and in detail as your application will not be considered if all mandatory questions are not fully completed.