Senior Data Engineer (Python/AWS)
Full-time | London, UK
Hybrid working: 2-3 days per week on-site
Must be SC cleared, or eligible to obtain SC-level security clearance
Job description
We are looking for a developer with expertise in Python, AWS Glue, step function, EMR cluster and redshift, to join our AWS Development Team. You will be serving as the functional and domain expert in the project team to ensure client expectations are met.
The role involves structuring analytical solutions that address business objectives and problem solving. We are looking for hands-on experience in writing code for AWS Glue in Python, PySpark, and Spark SQL.
The successful candidate will translate stated or implied client needs into researchable hypotheses, facilitate client working sessions, and be involved in recurring project status meetings. You will develop long-lasting, trusted advisor relationships with clients, bringing an ability to work in dual shore engagement across multiple time zones and manage business uncertainty, day-to-day project operations.
An in-person interview will be required as part of the interview process.
Key responsibilities
* Hands-on experience in Python programming is essential
* Responsible for Build, Test and Maintain optimal AWS data pipeline architecture. Building AWS glue, step function, AWS Lambda functions and unit test
* Work with fellow developers, data architects, data analysts and data scientists on data initiatives
* Expected to understand the core AWS services and apply best practices regarding security and scalability
* Understand the current application infrastructure, suggesting changes to it
* Define and document best practices and strategies regarding application deployment and infrastructure maintenance
* Define service capacity planning strategies
* Implement the applications CICD pipeline using the AWS CICD stack
* Project tasks deliverables and management
* Deliver on project progress and ensure adherence to client expectation
* Communications, including deck writing
* Delivery of output
* Gathering of client feedback on problem structuring
* Understand and define business problems, get all background information and collect relevant data points
* Create solution hypothesis and get client buy in, discuss and align on end objective, staffing need, timelines and budget
Nice to have
* Hive
* Pig
* No-SQL database
#J-18808-Ljbffr