You will be joining a fast growing consultancy at the forefront of secure data and cloud infrastructure, building AI-ready platforms for high-impact projects.
As a Data Engineer, you'll help build and maintain a secure batch and real-time data platform for a government client. You'll work with modern commercial and open-source tools to ingest, transform, and deliver data to support analytics and decision-making.
Duties & Responsibilities
* Build, test, and maintain data pipelines
* Design logical and physical data models
* Support the growth of the data platform to meet new use cases
* Analyse data and improve processing performance
* Ensure data aligns with agreed architecture and standards
* Troubleshoot data issues and identify improvements
Core Experience & Competencies
You'll have a degree (or equivalent experience) and meet the core requirements below.
Essential
* Experience building data pipelines
* Understanding of data modelling and data structures
* Familiarity with Agile delivery methods
* Experience with AWS (e.g. EC2, RDS, Lambda, Athena, Glue)
* Knowledge of relational and NoSQL databases
* Python or similar scripting language
* Experience with test-driven development
Preferred
1. CI tools (e.g. Jenkins, Drone)
2. Infrastructure as Code (Terraform)
3. Configuratio...