Job Description
As a Data Engineer you will join a team delivering transformative cloud hosted data platforms for a Version 1 customers.
You will have a proven track record as a Data Engineer and be adept in your ability to implement data ingestion and transformation pipelines for large scale organisations.
This career may align to individuals who like to work with a variety of technologies and data services, that will develop and deliver early proofs of concept and production implementation.
On a day to day basis you'll:
1. Design and implement highly performant data ingestion and transformation pipelines from multiple sources
2. Develop scalable and re-usable frameworks for ingestion and transformation of large data sets
3. Integrate the end-to-end data pipeline to take data from source systems to target data repositories ensuring the quality and consistency of data is maintained at all times
4. Work with other members of the project team to support delivery of additional project components (Reporting tools, API interfaces, Search)
5. Evaluate the performance and applicability of multiple tools against customer requirements.
Qualifications
6. Security Clearance eligible. You must have the right to work in the United Kingdom or Northern Ireland and have been resident in the United Kingdom or Northern Ireland for 5 or more consecutive years
7. Have practical experience designing and delivering using AWS or Azure such as: Azure SQL Data Warehouse, Azure Data Lake, AWS S3, AWS RDS, AWS Lambda or similar
8. Have experience with Open Source big data products i.e. Hadoop Hive, Pig, Impala or similar
9. Have experience with Open Source non-relational or NoSQL data repositories such as: MongoDB, Cassandra, Neo4J or similar
10. Be confident with your ability working with structured and unstructured data including imaging & geospatial data
11. Understand working in a DevOps environment with object-orientated and object function scripting languages such as Python, Java or similar
12. Have expertise in data modelling, data warehouse design and data lake concepts and practices.