Location: Croydon | Manchester | Sheffield | Liverpool
Salary: £62,109 plus skills allowance of up £18,921
Advert Close: Tuesday 21st April 2026
Home Office Digital designs, builds and develops services for the rest of the department and for government. Every year our systems support up to 3 million visa applications, checks on 100 million border crossings, up to 8 million passport applications and deliver 140 million police checks on people, vehicles and property.
Data Services and Analytics (DSA) is the part of DDaT which leads the build of data products to integrate data from multiple sources, and then deliver products for business intelligence, reporting and data science. DSA has the ambition to be one of the leading providers of insight services in UK Government and is currently growing significantly with over 300 colleagues, providing services to variety of functions within the Home Office.
In this role, you will identify and deliver new data sources and develop ETL processes, collaborate with the Data Acquisition, Analytics and Infrastructure teams to deliver the data strategy through the design and implementation of technology systems.
This is an opportunity to lead a team of data engineers working on complex data pipelines, ingesting and transforming data from/to a variety of sources (email, csv, ODBC/JDBC, JSON, XML, Excel, Avro, Parquet). Data pipelines are primarily developed in AWS using a variety of tools and languages including AWS S3, AWS Athena, AWS Redshift, AWS Glue, java (Docker / Spring).
The position requires leading the data engineering development for a team of data engineers working on a legacy Oracle data platform and delivering data integration and data retrieval services using PL‑SQL. A complex transformation programme is delivering the change of people, process and technology to redesign and transform the processing of the Oracle data outputs onto the strategic data platform. This role would ideally have experience of Oracle and PL‑SQL, with the aptitude to move to cloud native tooling such as AWS Athena, python, S3. Additionally, a key priority is to manage the people change to ensure the team stability and business continuity throughout the transformation.
Main Responsibilities
As a lead Data Engineer, you will showcase a true passion for data and exhibit expertise in the following areas:
* Manage data sources effectively to improve data quality and make them suitable for analysis.
* Drive and develop new data models and ETL processes, working collaboratively with the analytics team and mapping data to an appropriate matching data model.
* Utilise essential key stakeholder management skills to effectively manage expectations, actively listen and interpret business needs, communicate progress, and navigate challenging conversations, all while fostering a service orientated culture around data.
* Managing the performance of suppliers and the data team to improve capability, as well as creating the roadmap which balances immediate user needs with long-term investment.
* Developing processes to balance data availability, capacity and data archiving, while continually improving processes and efficiency in the acquisition and ingestion of data.
* Identifying data issues in data pipelines and managing their resolution.
* Proficiency in Oracle and Java for developing and maintaining data pipelines and integration solutions.
* Proven track record of leading engineering teams through transformation programs, specifically migrating from legacy platforms to modern cloud-native solutions (e.g., AWS).
* The successful candidate will be responsible for upskilling staff to transition to a new platform. They must also have leadership experience, including coaching and professional development skills.
As a Lead Data Engineer you’ll have a demonstrable passion for Data, with the following skills or some experience in:
* Effectively leading and managing teams, helping them deliver technical products and projects in data analysis, solution design and end user reporting.
* Automate IT structures by advising on best practice IT programming and automation techniques and tools such as GIThub, GITLab or other source control software, Python.
* You will lead the implementation and strategic use of Cloud Data technologies, solutions, and future Cloud Data strategies particularly in the context of data warehousing and data lake architectures. This includes expertise in AWS services such as Redshift, Athena, EC2, IAM, AWS Lambda and CloudWatch
* Working with stakeholders to gather requirements and deliver findings.
* Presenting complex ideas and recommendations that inform decisions in a clear way to technical and non-technical audiences and senior customers.
* Using tools such as industry standard ETL tools, network databases and scheduling and orchestration SQL tools.
* Strong analytical and problem-solving skills troubleshooting data issues and optimising data processes.
* Strong experience with transactional databases, including design, optimisation, and troubleshooting of high-volume data systems.
Why work for us...
Find out more information at: Benefits - Home Office Careers, but some of the primary ones are:
* A Civil Service Pension with employer contribution rates of at least 28.97%.
* In-year reward scheme for one-off or sustained exceptional personal or team achievements.
* 25 days annual leave on appointment, rising with service.
* 8 days of public holidays, plus 1 additional privilege day.
* Where business needs allow, some roles may be suitable for a combination of office and home-based working. This is a non-contractual arrangement where all employees will be expected to spend a minimum of 60% of their working time in an office.
J-18808-Ljbffr