Your newpany
One of the largest Public Sector Organisations in the UK managing trade
Your new role
Data Quality Lead – Active SC Clearance
What you'll need to succeed
Role Overview
We are seeking a strategic and technically proficient Data Quality Lead to oversee the ingestion, evaluation, and quality assurance of critical datasets supporting the Global Supply Chains Intelligence Programme (GSCIP). This role will be pivotal in ensuring data integrity, enhancing operational efficiency, and supporting high-impact analysis acrossernment andmercial domains.
This role can only be offered to candidates with Active and Existing SC or DV Clearance.
Key Responsibilities
Data Ingestion & Source Management
1. Oversee ingestion pipelines for key data sources including Sayari, S&P, and Altana.
2. Liaise with internal and external stakeholders (Microdata team, HMG equivalents and external suppliers) to ensure procurement tracking and data quality.
3. Monitor and resolve backlog issues in data quality logs.
4. Dataset Evaluation & Maintenance
5. Lead on the assessment of data quality of all GSCIP data. This includes exploratory data analysis of new datasets and pushing suppliers to rectify issues.
6. Tracking and logging existing data quality issues by coordinating with GSCIP data scientists and licence users. Prioritise issues based on feedback.
7. Raising issues with suppliers, suggesting solutions (based on DS feedback),municating updates to GSCIP users, understanding contractual agreements to keep suppliers to account.
8. Write pipelines to ensure that ingested data meets expected format/contractual criteria
9. Proactively conduct EDA and anomaly detection on data sources to identify unknown issues. Produce dashboards and reports to disseminate features of interest of the graph to GSCIP data scientists and users, where the largest outliers are, where coverage is best/worst etc.
10. Actively work on enhancements to data quality, developing reproducible pipeline functions that handle data quality features in a standardised way for gscip_utils/dashboards such as transaction deduplication, imputation etc.
11. Investigate the relative value of each data source to feed into futuremercial procurement strategy. Track the strengths and weaknesses of each data supplier.
12. Scope out new data sources to fill existing data gaps.
13. Lead annual evaluations of datasets to determine retention and relevance.
14. Conduct IRAP assessments for new datasets.
15. Maintainprehensive documentation including data dictionaries, caveats, and usage guides.
Cyber Monitoring & Pipeline Development
16. Collaborate with engineers and data scientists to support engineers to build robust, high-quality data pipelines.
17. Explore new data sources, particularly those related to risk andpany intelligence.
18. Proactively investigate data anomalies and ensure datasets meetmercial requirements.
19. Formalise ownership and tracking for datasets such as HMRC,Trade, TDM, and Shipping Instructions.
20. Coordinate with microdata teams to align procurement and sharing protocols.
Learning & Development
21. Design and deliver learning modules to support GSCIP data literacy.
22. Encourage new team members to engage in QA activities to build familiarity with datasets.
Essential Skills & Experience
23. Proven experience in quality assurance of data and working with data engineers.
24. Strong understanding ofmercial andernment data sources.
25. Ability to manage cross-functional relationships and coordinate across departments.
26. Experience with documentation standards and dataernance frameworks.
Desirable Attributes
27. Strategic thinker with a proactive approach to data quality and risk mitigation.
28. Excellentmunicator with the ability to translate technical concepts for non-technical audiences.
29. Familiarity with systems including databricks, Amazon Neptune and AWS
What you'll get in return
This is a fantastic opportunity to join a team making a difference through data here and now and with an excellent reputation in the Public Sector.