Hadoop Engineer Location: Birmingham or Sheffield (Hybrid, 2 days onsite p/w) Contract End Date: November 2025 Rate: Up to £500 per day Inside IR35 About the Role We’re looking for experienced Hadoop Engineers to join our team and help drive the development, optimisation, and support of our on-premises Operational Data Platform (ODP). You will work on complex, enterprise-scale data environments, ensuring performance, security, and reliability for critical operational data workloads. If you have deep expertise in the Hadoop ecosystem, strong programming skills, and a passion for building robust data pipelines, this is your chance to work on a high-impact platform supporting mission-critical analytics. Key Responsibilities Design, develop, and maintain scalable data pipelines using Hadoop technologies in an on-premises environment. Build and optimise real-time processing workflows using Apache Airflow and Spark Streaming. Develop automation and transformation solutions in Python. Collaborate with infrastructure and analytics teams to deliver operational data solutions. Monitor, troubleshoot, and fine-tune jobs to ensure platform reliability and performance. Ensure adherence to enterprise security, compliance, and governance standards. Required Skills & Experience 5 years’ experience in Hadoop and data engineering roles. Proven hands-on expertise with Python, Apache Airflow, and Spark Streaming. Strong knowledge of Hadoop components ( HDFS, Hive, HBase, YARN ) in on-prem environments. Experience with Linux systems, shell scripting, and enterprise deployment tools. Exposure to infrastructure-level or operational data analytics. Familiarity with monitoring and logging tools in on-prem environments. Preferred Qualifications Experience with enterprise ODP platforms or large-scale data systems. Knowledge of configuration management tools (e.g., Ansible, Puppet ) and on-prem CI/CD pipelines. Understanding of network and storage architecture in data centers. Awareness of data security, compliance, and audit requirements in regulated industries.