Overview
We are seeking a highly skilled Data Engineer with strong SQL capabilities and hands-on experience with AWS Glue or equivalent Spark-based tools (e.g., Databricks).
You will be a key contributor in our Data Modernization initiative, helping to design and build scalable data processing pipelines that support our AWS-based data lake. The role involves working with large-scale datasets, optimizing for performance through techniques like partitioning, and delivering clean, reliable data to downstream consumers.
Responsibilities
* Develop and maintain robust ETL pipelines using AWS Glue (Apache Spark) or Databricks
* Write complex SQL queries, including Common Table Expressions (CTEs), stored procedures, and views, for data transformation and analysis
* Design and implement effective partitioning strategies in Glue, Athena, and other AWS-native tools to optimize performance and cost
* Ingest, clean, and transform structured and semi-structured data from multiple sources into the AWS data lake
* Collaborate with stakeholders to understand data requirements and deliver well-structured, high-quality datasets
* Troubleshoot performance issues in data pipelines and contribute to tuning and optimization
* Support data governance, lineage, and monitoring initiatives to ensure data quality and reliability
Requirements
* Excellent SQL skills — advanced experience writing performant queries using CTEs, procedures, and views
* Hands-on experience with AWS Glue (Spark-based ETL), or similar platforms like Apache Spark or Databricks
* Strong understanding of partitioning techniques for large-scale datasets in both databases and data lake environments (e.g., Glue, Athena, Spark)
* Familiarity with cloud data lake architectures and AWS data ecosystem (S3, Athena, Glue, etc.)
* Comfortable working with large volumes of data and optimizing jobs for performance and cost
* Experience in a collaborative environment, with the ability to communicate effectively across technical and non-technical teams
* Financial services experience is a plus, especially familiarity with reference, counterparty, or instrument data
We offer
* Pension
* Employee Assistance Programme
* Enhanced Maternity policy
* Give as You Earn
* Cycle to Work Scheme
* Employee Referral Bonus Scheme
* Diversity Networks
* Access to a range of skills and certifications
#J-18808-Ljbffr