Role: Data Engineer
Location: Edinburgh or Luton (hybrid/onsite as required)
Engagement: Contract - Inside IR35
Security Clearance: Active SC Clearance required
Rate: £500 - £600 per day - inside IR35
Project Description
We are seeking an experienced Data Engineer to design, build, deploy, and maintain robust data platforms and pipelines within a secure environment. You will be responsible for the end-to-end data engineering lifecycle, transforming raw data into high-quality, consumable datasets that support analytics, reporting, and advanced modelling.
You will own and optimise the data operations infrastructure, ensuring performance, reliability, scalability, and security as data volumes and processing demands grow. This role requires strong problem-solving skills, the ability to integrate data from multiple sources, and hands-on experience with modern data engineering tools and practices.
Key Responsibilities
Design, develop, deploy, and support scalable data infrastructure, pipelines, and architectures
Orchestrate ingestion and storage of raw data into structured and unstructured data solutions
Implement reliable, automated, and well-tested data ingestion and processing workflows
Build and maintain batch and real-time data processing systems
Manage and optimise performance, reliability, scalability, and security of data platforms
Support data governance, quality, and compliance requirements
Prepare data pipelines for descriptive, predictive, and prescriptive analytics
Collaborate closely with data scientists, architects, IT teams, and business stakeholders
Identify opportunities for new data acquisition and improved data utilisation
Monitor, manage, and enhance data quality and reliability through automated tooling
Skills and Experience Required
Active SC Clearance (mandatory)
Strong experience designing and maintaining data pipelines, data warehouses, and data platforms
Solid knowledge of DataOps practices, including CI/CD, containerisation, and workflow orchestration
Hands-on experience with ETL/ELT frameworks and big data tools (e.g. Spark, Airflow, Hive)
Proficiency in programming languages such as Python, Java, and SQL
Experience with SQL and NoSQL database design and optimisation
Strong understanding of batch and streaming data processing
Degree in a STEM-related field; Master's degree desirable
Data engineering certifications (e.g. IBM Certified Data Engineer) are advantageous