The Role As a Data Engineer, you'll be responsible for designing, building, and maintaining robust data pipelines and architectures. You will work closely with stakeholders to understand complex data challenges, transform raw data into meaningful insights, and support analytics and reporting. This includes working with batch, streaming, real-time, and unstructured data, applying distributed compute techniques to handle large datasets efficiently. Key Responsibilities Develop and maintain data ingestion pipelines and orchestration workflows Design database schemas and data models Integrate and enrich data from multiple sources, ensuring consistency and quality Design and implement ETL/ELT processes (e.g., using Apache NiFi) Produce reusable, maintainable code with a test-driven approach Maintain and enhance existing data platforms and services Investigate and resolve operational issues in integrated datasets Implement data security measures to protect sensitive information Support Agile delivery, breaking down user requirements into actionable tasks Monitor and optimise system performance for reliability and efficiency Required Skills Apache Kafka Apache NiFi SQL and NoSQL databases (e.g., MongoDB) ETL/ELT development with Groovy, Python, or Java About the Employer With over 60 years of experience supporting government and defence programmes, this employer delivers deep technical expertise in sensors, communications, cyber, and advanced analytics. The organisation applies innovation, technology, and data to help clients make informed decisions and protect critical systems and infrastructure. Clearances Due to the nature of this role, we require you to be eligible to achieve the highest level of security clearance. Benefits & Culture Work at the cutting edge of technology in defence and national security Opportunity to spend time on innovative R&D projects and concept creation Collaborative, geeky, and creative environment that celebrates technical brilliance Competitive bonus scheme up to £3,000 / 6% of salary Generous holiday: 30 days bank holidays, 3.5 days over Christmas, option to buy/sell extra leave Supportive and engaging culture, focused on growth and innovation Hybrid working: 3 days in the office, 2 days from home, flexible to work fully on-site if needed Reasonable Adjustments: Respect and equality are core values to us. We are proud of the diverse and inclusive community we have built, and we welcome applications from people of all backgrounds and perspectives. Our success is driven by our people, united by the spirit of partnership to deliver the best resourcing solutions for our clients. If you need any help or adjustments during the recruitment process for any reason, please let us know when you apply or talk to the recruiters directly so we can support you.