Social network you want to login/join with:
? City of London (Hybrid: Around 2-3 office days per week)
? Salary: Up to £120,000
? Industry: FinTech / RegTech
A 200-employee data and technology company specialising in Regulatory Technology (RegTech) are looking for a Lead AWS Data Engineer. They develop products that enhance data quality and help financial institutions navigate regulatory compliance.
The Role
The Lead Data Engineer will provide technical leadership to a small team of Data Engineers (onshore & offshore), driving the design, implementation, and optimisation of scalable data solutions in AWS.
Key Responsibilities
* Lead and mentor a team of onshore and offshore Data Engineers, driving technical growth and ensuring effective collaboration.
* Design, develop, and optimise scalable data pipelines and infrastructure using AWS (Glue, Athena, Redshift, Kinesis, Step Functions, Lake Formation).
* Utilise PySpark for distributed data processing, ETL, SQL querying, and real-time data streaming.
* Architect and implement robust data solutions for analytics, reporting, machine learning, and data science initiatives.
* Establish and enforce best practices in data engineering, coding standards, and architecture guidelines.
* Manage team tasks and delivery using Agile methodologies, collaborating closely with product owners and scrum masters.
Skills and Experience
* 2+ years of leadership experience in data engineering, including mentoring and team management.
* 7+ years of hands-on experience in data engineering and ETL architecture.
* Deep expertise in AWS Data Services, including Glue, Athena, Redshift, Kinesis, Step Functions, and Lake Formation.
* Strong programming skills in Python and PySpark for data processing and automation.
* Extensive SQL experience (Spark-SQL, MySQL, Presto SQL) and familiarity with NoSQL databases (DynamoDB, MongoDB, etc.).
* Proficiency in Infrastructure-as-Code (Terraform, CloudFormation) for automating AWS data workflows.
* Experience designing, implementing, and optimising ETL pipelines, including real-time data processing and streaming architectures.
? 25 days holiday + bank holidays + birthday off.
? Annual personal development allowance.
? Regular team socials in a collaborative environment.
#J-18808-Ljbffr