Job Title: Senior Data Engineer
Location: Remote (UK-based)
Contract: Outside IR35, Competitive Market Rate
Duration: 6 months with strong chance of extension
Why Apply?
This is a unique opportunity to play a key role in a high volume data transformation programme. You'll work with a collaborative engineering team on a large scale AWS based migration project, solving complex data partitioning and performance challenges.
Responsibilities
Design, build, and optimise large-scale ETL processes using AWS Glue.
Repartition and transform existing data pipelines.
Develop scalable logic in Glue Jobs to accommodate massive data volumes, ranging from tens to hundreds of terabytes.
Contribute to data architecture decisions by identifying non-obvious segmentation strategies and designing new formats for extraction and storage.
Ensure reliable delivery of high-volume transaction data to S3, handling millions of files across a dynamic architecture.
Requirements
Proven experience designing and delivering large-scale ETL pipelines, preferably from relational databases.
Strong understanding of relational data models and how they map into distributed data environments.
Hands-on expertise with AWS Glue, particularly in coding and embedding business logic into Glue Jobs.
Comfortable managing large data sets and optimising performance at scale in cloud environments.
Able to troubleshoot data flow and transform logic efficiently in a complex pipeline.
Experience working as a Lead/Principal Data Engineer, or Entry-Level Architect in high-volume data environments.
We are an equal opportunities employer and welcome applications from all suitably qualified persons regardless of their race, sex, disability, religion/belief, sexual orientation or age.
TPBN1_UKTJ