Senior Data Engineer (Python/AWS)
Full-time | London, UK
Hybrid working: 2-3 days per week on-site
Must be SC cleared, or eligible to obtain SC-level security clearance
Job description
We are looking for a developer with expertise in Python, AWS Glue, step function, EMR cluster and redshift, to join our AWS Development Team. You will be serving as the functional and domain expert in the project team to ensure client expectations are met.
The role involves structuring analytical solutions that address business objectives and problem solving. We are looking for hands-on experience in writing code for AWS Glue in Python, PySpark, and Spark SQL.
The successful candidate will translate stated or implied client needs into researchable hypotheses, facilitate client working sessions, and be involved in recurring project status meetings. You will develop long-lasting, trusted advisor relationships with clients, bringing an ability to work in dual shore engagement across multiple time zones and manage business uncertainty, day-to-day project operations.
An in-person interview will be required as part of the interview process.
Key responsibilities
Hands-on experience in Python programming is essential
Responsible for Build, Test and Maintain optimal AWS data pipeline architecture. Building AWS glue, step function, AWS Lambda functions and unit test
Work with fellow developers, data architects, data analysts and data scientists on data initiatives
Expected to understand the core AWS services and apply best practices regarding security and scalability
Understand the current application infrastructure, suggesting changes to it
Define and document best practices and strategies regarding application deployment and infrastructure maintenance
Define service capacity planning strategies
Implement the application's CICD pipeline using the AWS CICD stack
Project tasks deliverables and management
Deliver on project progress and ensure adherence to client expectation
Communications, including deck writing
Delivery of output
Gathering of client feedback on problem structuring
Understand and define business problems, get all background information and collect relevant data points
Create solution hypothesis and get client buy in, discuss and align on end objective, staffing need, timelines and budget
Nice to have
Hive
Pig
No-SQL database
TPBN1_UKTJ