AWS Data Engineer (Must hold current SC clearance)
Overview:
Have you got the right qualifications and skills for this job Find out below, and hit apply to be considered.
We are seeking a highly skilled AWS Data Engineer to join a cutting-edge technology consultancy delivering solutions across the UK and Europe. This role is ideal for someone with strong experience in building scalable data pipelines and working within secure, cloud-based environments.
You will collaborate with data scientists, analysts, and software engineers to deliver efficient and reliable data solutions that support analytics, reporting, and AI/ML initiatives.
Responsibilities:
* p]:pt-0 [&>p]:mb-2 [&>p]:my-0'>
Design, develop, and maintain scalable ETL/ELT pipelines using AWS services such as Glue, Lambda, Step Functions, and Kinesis.
* p]:pt-0 [&>p]:mb-2 [&>p]:my-0'>
Work with structured and unstructured data from multiple sources, ensuring efficient ingestion, transformation, and storage.
* p]:pt-0 [&>p]:mb-2 [&>p]:my-0'>
Build and optimise data lake and warehouse solutions using S3, Redshift, Athena, and Lake Formation.
* p]:pt-0 [&>p]:mb-2 [&>p]:my-0'>
Implement data governance, security, and compliance best practices, including IAM, encryption, and access controls.
* p]:pt-0 [&>p]:mb-2 [&>p]:my-0'>
Monitor and optimise performance of data workflows using CloudWatch and other AWS-native tools.
* p]:pt-0 [&>p]:mb-2 [&>p]:my-0'>
Automate data processes using Python, PySpark, SQL, and AWS SDKs.
* p]:pt-0 [&>p]:mb-2 [&>p]:my-0'>
Collaborate with cross-functional teams to support analytics, BI, and AI/ML use cases.
* p]:pt-0 [&>p]:mb-2 [&>p]:my-0'>
Maintain and enhance CI/CD pipelines for data infrastructure using Terraform, CloudFormation, or CDK.
* p]:pt-0 [&>p]:mb-2 [&>p]:my-0'>
Troubleshoot and resolve data integration, performance, and reliability issues.
Required Skills & Experience:
* p]:pt-0 [&>p]:mb-2 [&>p]:my-0'>
5+ years of experience in data engineering, with a strong focus on AWS.
* p]:pt-0 [&>p]:mb-2 [&>p]:my-0'>
Strong proficiency in Python, PySpark, SQL, and AWS Glue.
* p]:pt-0 [&>p]:mb-2 [&>p]:my-0'>
Hands-on experience with AWS data services including Redshift, Athena, Glue, EMR, and Kinesis.
* p]:pt-0 [&>p]:mb-2 [&>p]:my-0'>
Solid understanding of data modelling, warehousing, and schema design.
* p]:pt-0 [&>p]:mb-2 [&>p]:my-0'>
Experience with streaming and event-driven architectures (e.g. Kafka or Kinesis).
* p]:pt-0 [&>p]:mb-2 [&>p]:my-0'>
Expertise in Infrastructure as Code (Terraform, CloudFormation, or CDK).
* p]:pt-0 [&>p]:mb-2 [&>p]:my-0'>
Familiarity with DevOps practices and CI/CD pipelines.
* p]:pt-0 [&>p]:mb-2 [&>p]:my-0'>
Strong problem-solving skills and experience working in Agile environments.
Desirable:
* p]:pt-0 [&>p]:mb-2 [&>p]:my-0'>
AWS certifications (Data Analytics Specialty or Solutions Architect Associate).
* p]:pt-0 [&>p]:mb-2 [&>p]:my-0'>
Experience with Airflow for orchestration.
* p]:pt-0 [&>p]:mb-2 [&>p]:my-0'>
Exposure to big data frameworks such as Spark, Hadoop, or Presto. xsngvjr
* p]:pt-0 [&>p]:mb-2 [&>p]:my-0'>
Experience supporting AI/ML data pipelines.