Salary: £46,800 - 53,004 per year Requirements: Strong hands-on experience building streaming data pipelines with Kafka (producers/consumers, schema registry, Kafka Connect, KSQL/KStreams) Experience with OpenShift / Kubernetes telemetry, including OpenTelemetry and Prometheus Proven capability integrating telemetry into Splunk (HEC, Universal Forwarders, sourcetypes, CIM, dashboards, alerting) Solid data engineering skills in Python (or similar) for ETL/ELT, enrichment and validation BPSS clearance required - candidates must be eligible Responsibilities: Design, implement and maintain scalable data pipelines to ingest and process OpenShift telemetry (metrics, logs, traces) Stream telemetry through Kafka (producers, topics, schemas) and build resilient consumer services for enrichment and transformation Engineer multi-tenant observability data models, ensuring data lineage, quality controls and SLAs across streaming layers Integrate processed telemetry into Splunk for dashboards, analytics, alerting and operational insights Implement schema management and governance using Avro/Protobuf, including versioning and compatibility strategies Build automated validation, replay and backfill mechanisms to ensure data reliability and recovery Instrument services using OpenTelemetry, standardising tracing, metrics and structured logging Apply LLMs to enhance observability, such as query assistance, anomaly summarisation and runbook generation Collaborate with Platform, SRE and Application teams to align telemetry, alerts and SLOs Ensure pipelines meet security, compliance and best-practice standards Produce clear documentation covering data flows, schemas, dashboards and operational runbooks Technologies: ETL Support Kafka Kubernetes OpenTelemetry OpenShift Prometheus Python Security Splunk More: We are looking for a Data Engineer to design and operate large-scale telemetry and observability data pipelines within a modern OpenShift and Kafka ecosystem. This 9 month contract role is based in Sheffield, offering a competitive daily rate of £395 - £442 inside IR35. As part of our team, you will play a central role in enabling proactive, Level 4 observability, delivering high-quality metrics, logs, and traces that support platform reliability, operational insight, and automation. last updated 8 week of 2026