Senior Data Engineer (Apache Flink) at N Consulting Ltd
Job Description:
The Financial Times is one of the world’s leading business news organisations. We are hiring an experienced Apache Flink Engineer (contract) to build and help operationalise the stream-processing components of a near real-time personalisation and recommender capability on AWS.
Role overview
You will bring deep hands-on expertise in production Flink on AWS, working closely with a senior FT engineer and AWS stakeholders. A key part of the role is knowledge transfer through pairing, reviews and clear documentation so the capability can be supported and extended by FT.
Responsibilities
* Design and implement stateful stream-processing jobs in Apache Flink (primarily Java), to process user interaction events and produce near real-time outputs for downstream services.
* Integrate Flink with Kafka/AWS MSK, S3 and the chosen online store (MongoDB/MongoDB Atlas).
* Own streaming non-functional requirements: correctness, fault tolerance, checkpointing/recovery, backpressure handling, and performance tuning.
* Define and implement production readiness: observability (metrics/logs/tracing), alerting, dashboards and runbooks.
* Support CI/CD and infrastructure-as-code practices to deploy and operate Flink workloads safely in AWS.
* Enable the team via knowledge transfer (pairing, design/code reviews, documentation and handover).
Skills and experience
Essential
* Strong experience building and operating production Apache Flink applications (event-time concepts, windows/watermarks, state, checkpointing and fault tolerance).
* Hands-on AWS experience operating streaming/data workloads (IAM, networking/VPC basics, deployment, observability and incident troubleshooting).
* Experience integrating Flink with Kafka (including AWS MSK) and cloud services/storage (e.g., S3).
* Highly proficient in Java and Python (production experience required).
* Strong software engineering fundamentals: clean, maintainable code; pragmatic testing; effective code review.
* Comfort mentoring/pairing with engineers to up-skill the team.
Desirable
* Experience with Amazon Managed Service for Apache Flink and/or running Flink on EKS/ECS.
* Experience with MongoDB or MongoDB Atlas for low-latency, online-serving use cases.
* Infrastructure-as-code (CloudFormation) and modern observability tooling (CloudWatch, Grafana, OpenTelemetry, Splunk).
This job has now closed
You can find more jobs over on our careers page.
#J-18808-Ljbffr