Role Title: Hadoop Engineer / ODP Platform
Location: Birmingham / Sheffield - Hybrid working with 3 days onsite per week
End Date: 28/11/2025
Role Overview:
We are seeking a highly skilled Hadoop Engineer to support and enhance our Operational Data Platform (ODP) deployed in an on-premises environment.
The ideal candidate will have extensive experience in the Hadoop ecosystem, strong programming skills, and a solid understanding of infrastructure-level data analytics. This role focuses on building and maintaining scalable, secure, and high-performance data pipelines within enterprise-grade on-prem systems.
Key Responsibilities:
* Design, develop, and maintain data pipelines using Hadoop technologies in an on-premises infrastructure.
* Build and optimise workflows using Apache Airflow and Spark Streaming for real-time data processing.
* Develop robust data engineering solutions using Python for automation and transformation.
* Collaborate with infrastructure and analytics teams to support operational data use cases.
* Monitor and troubleshoot data jobs, ensuring reliability and performance across the platform.
* Ensure compliance with enterprise security and data governance standards.
Required Skills & Experience:
* Minimum 5 years of experience in Hadoop and data engineering.
* Strong hands-on experience with Python, Apache Airflow, and Spark Streaming.
* Deep understanding of Hadoop components (HDFS, Hive, HBase, YARN) in on-prem environments.
* Exposure to data analytics, preferably involving infrastructure or operational data.
* Experience working with Linux systems, shell scripting, and enterprise-grade deployment tools.
* Familiarity with monitoring and logging tools relevant to on-prem setups.
Preferred Qualifications:
* Experience with enterprise ODP platforms or similar large-scale data systems.
* Knowledge of configuration management tools (e.g., Ansible, Puppet) and CI/CD in on-prem environments.
* Understanding of network and storage architecture in data centers.
* Familiarity with data security, compliance, and audit requirements in regulated industries.
#J-18808-Ljbffr