Job Description
Job responsibilities
* Contribute to end-to-end data pipeline solutions on cloud infrastructure leveraging the latest technologies and the best industry practices
* Use domain modelling techniques to allow us to build best in class business products.
* Structure software so that it is easy to understand, test and evolve.
* Build solutions that avoid single points of failure, using scalable architectural patterns.
* Develop secure code so that our customers and ourselves are protected from malicious actors.
* Promptly investigate and fix issues and ensure they do not resurface in the future.
* Make sure our releases happen with zero downtime for our end-users.
* See that our data is written and read in a way that's optimized for our needs.
* Keep an eye on performance, making sure we use the right approach to identify and solve problems.
* Ensure our systems are reliable and easy to operate.
* Keep us up to date by continuously updating our technologies and patterns.
* Support the products you've built through their entire lifecycle, including in production and during incident management
Required qualifications, capabilities & skills
* Formal training or certification on data engineering concepts and applied experience
* Recent hands-on professional experience as a data engineer
* Experience in coding in a recent version of the Python programming language
* Experience in designing and implementing effective tests (unit, component, integration, end-to-end, performance, etc.)
* Excellent written and verbal communication skills in English
* Understanding of cloud technologies and distributed systems
* Understanding of data transformation frameworks and data pipeline orchestration tools
* Understanding of how to manage and optimize queries for large volumes of data within different kinds of data stores (including relational data stores)
Preferred qualifications, capabilities & skills
* Experience in working in a highly regulated environment / industry
* Understanding of AWS cloud technologies
* Understanding of data governance frameworks
* Understanding of event-base architecture, data streaming and messaging framework