Job summary
As Lead Data Engineer you will be instrumental in the development and implementation of core borders and immigration-related management information assets within the Home Office.
The Home Office is the lead government Department for immigration and passports, crime, police, fire, and counter terrorism � significant issues which are at the heart of the agenda of HM Government. The challenges and opportunities the Department faces are significant and rapidly changing in the global environment in which we operate. Strong analytical advice is needed to navigate these challenges, and the Home Office has a reputation for putting analysis at the heart of decision-making.
Part of the Home Office Analysis & Insight directorate (HOAI), the Performance Reporting & Analysis Unit (PRAU) provides reporting and analysis of performance for the Migration & Borders Mission and associated operational Capabilities. We support Ministers, the Executive Committee, Policy and Operational leads and other analytical functions with the management information (MI) that they need, as well as producing data for publication and other forms of external release. The Unit also provides advice and support to others in the Department on how to make best use of performance information and is leading on the transformation of MI systems and capabilities.
The Data Integration team within PRAU are at the heart of multi-year transformation of the MI model and systems, introducing a cloud-based, sustainable and agilely-delivered, MI platform and service called Vantage.� As a key leader in this programme, you will be working in a challenging and exciting space that demands enthusiasm, creativity and careful management.
PRAU is geographically spread across four primary locations (London, Liverpool, and Sheffield and Croydon). Candidates will need to agree a base location of either Croydon, Liverpool or Sheffield with a requirement for periodic attendance at other Home Office locations.
Job description
As the Operational Lead Data Engineer, you will deliver the data engineering vision for Vantage as set by the Principal Data Engineer. You will focus on designing, coordinating, and implementing numerous complex data flows to connect operational systems, data for analytics and BI systems. Working in tandem with the Technical Lead Data Engineer and supporting the Principal Data Engineer with managing the unit and getting the most from the available resource.
Responsible for several data engineering sub teams, you will be providing support and management direction to their deliveries. Promoting best practice, continuous improvement, conflict resolution, constructively challenging delays, and removing blockers to project and programme deliveries.�
You will need to develop management information assets sustainably, while remaining agile and adaptive - aligning to the unit�s data strategy of building integrity, performance, scalability, security and maintainability into those designs, while being responsive to emerging business pressures and priorities.
You will identify and deliver new data sources and develop ETL (Extract, Transform, Load) processes, collaborate with the Data Acquisition, Analytics and Infrastructure teams to deliver the data strategy through the design and implementation of technology systems.
You will work closely with the Technical Lead Data Engineer to embed new ways of working and strive to build a culture of continuous personal development. �You will be open to and encourage views from all our developers and partners (a mixture of civil servants and contract suppliers), to maintain and build on our reputation for achieving great things despite the significant challenge.�
Working with our various stakeholders (operational units, MI teams, Business Analysts, Senior Managers, etc), you will exercise key stakeholder management skills to manage expectations, listen and interpret business needs, communicate progress and facilitate difficult conversations� (for example around technical or data limitations, or prioritisation), while building a 'service culture' around data.
You will lead data management, data engineering and analytical projects, working collaboratively with other teams and stakeholders to acquire and ingest new data sources, set standards, and ensure implementation and adherence to these.
You will embed and asses best practice, be responsible for data architecture decisions, provide technical direction and assistance, develop and share performance enhancements, exploit cloud computing opportunities, champion data engineering across the Department, and assist in the development of technical skills across PRAU and beyond.�
Where business needs allow, some roles may be suitable for a combination of office and home-based working. This is a non-contractual arrangement where all employees will be expected to spend a minimum of 60% of their working time in an office, subject to local estates capacity, by Spring 2024. �Applicants can request further information regarding how this may work in their team from the Vacancy Holder (see advert for contact details).
Recruitment and Retention Allowance (RRA)
This post is eligible for a DDaT RRA. Successful candidates with exceptional skills and experience may apply for an annual RRA of between �4,261 to �9,950. This allowance is subject to an initial review within six months of taking up the post and thereafter an annual review in-line with departmental priorities and could be reduced or withdrawn at any time.
Due to business requirements this vacancy is only available on Full time hours.
Person specification
Essential criteria
You will:
1. have plenty of experience of line management, including conflict resolution, staff development, team building and training delivery.
2. be a strong communicator, able to communicate ideas and concepts to a wide range of stakeholders, apply the appropriate level of technical detail to each audience, building engagement and enthusiasm.
3. have experience of large project delivery methodology (such as Agile), resource management, challenging prioritisation calls and risk management.
4. have a keen eye for detail but not let it distract you from seeing the big picture.
5. be able to balance strategic long-term delivery against the need for swift resolution to challenges.
6. be confident and familiar with database and reporting development technology, with a demonstrable passion for data and complex problem solving.
7. be confident in complex data analysis and data profiling, able to develop proof of concepts and appropriate testing to aid resolutions to data quality challenges.
8. be confident in the use of T-SQL based development and have an awareness of other data science and data engineering tooling, with an interest in staying up-to-date with everything in the data engineering space.
9. be confident in the use of T-SQL ETL tools and technologies, be familiar with various platforms (most importantlySQL Server Management Studio, Azure Synapse, PGAdmin) and be aware of language variations (such as Oracle, Microsoft SQL, Postgres) and an awareness and interest in other data science and data engineering tooling (machine learning, AI models, data mining etc), with an interest in staying up to date with everything in the data engineering space.
10. be familiar with various IT programming and automation tools, (in particular Git, Azure DevOps, Visual Basic, PySpark etc), enable pipeline delivery of regular reporting products, with an awareness of sustainable and agile delivery methodologies.
Desirable Criteria
Ideally, you will:
11. have plenty of experience of complex and compound T-SQL development and scripting.
12. be comfortable with external supplier contract management, exploiting the quick scalability of contract worker resource, whilst balancing this with permanent staff development for the longer term.
13. have an understanding of the work the Home Office�s role in protecting our borders.
Behaviours
We'll assess you against these behaviours during the selection process:
14. Leadership
15. Communicating and Influencing
16. Developing Self and Others
17. Making Effective Decisions
We only ask for evidence of these behaviours on your application form:
18. Leadership
19. Making Effective Decisions
Technical skills
We'll assess you against these technical skills during the selection process:
20. Data Engineering � Level 4
21. Data Management DATM � Level 4
22. Data Modelling and Design DTAN � Level 4
23. Database Design DBDS � Level 3
24. Innovation INOV � Level 3
25. Methods and Tools METL � Level 4
Benefits
26. Learning and development tailored to your role
27. An environment with flexible working options
28. A culture encouraging inclusion and diversity
29. A with an average employer contribution of 27%