This role sits within the Data Analytics Service Team. You will be working as our Data Engineer and will be required to review/ design, document, and streamline the ETL processes for the datasets we bring together for analysis and AI projects. We operate in Azure using ABFSS for storage and Databricks Unity catalogue for our data processing. As part of the Senior Management Team, you will ensure new datasets are being added with the correct processes, existing pipelines execute correctly, and proposing changes to these processes going forward in line with best practices.
As part of the senior management of the data analytics team, you will also have leadership responsibilities within the PSFA data team to help us improve our services.
You will work with the platform lead to identify where and how we should continue to build our Azure tenant to best support the ambitions of the PSFA data analytics service going forward. You will develop relationships with key partners across the civil service, including our key data providers and our cloud provider, to identify how we can work together to improve our tenant.
#J-18808-Ljbffr