Agilis are currently working exclusively with a key client who are a leading technology consultancy in their search for a Senior Data Engineer. This is a fantastic opportunity to join a fast growing, forward thinking company and helping them take their Data engineering to the next level!
Job Description:
We are seeking a highly skilled and motivated Data Engineer to join a dynamic team. The ideal candidate will have a strong background in SQL, Python, ETL processes, and data integration, ideally in Databricks. You will play a crucial role in continuing an exciting project designing, developing, and maintaining data infrastructure to ensure the seamless inflow, data sanitation/consolidation and automated report production for clients.
Key Responsibilities:
Design and Development:
* Design, develop, and maintain scalable ETL pipelines to process and integrate data from various sources.
* Implement data validation routines to ensure data quality and integrity.
* Develop and optimize SQL queries for data extraction, transformation, and loading.
Strategic Solution Design:
Data Integration:
* Integrate data from multiple sources, including APIs & relational databases.
* Collaborate with cross-functional teams to gather and understand data requirements.
Database Management:
* Design and maintain relational database schemas to support business needs.
* Ensure efficient storage, retrieval, and management of large datasets.
API Management:
* Develop and maintain APIs for data access and integration.
* Utilize tools like Postman for API testing and documentation.
* A good understanding of working with APIs: Ensure robust and efficient API integration and management.
Data bricks Management:
* Manage permissions and access controls within Databricks to ensure data security and compliance
Data Analytics and Reporting:
* Work with data analysts to provide clean and well-structured data for analysis.
* Develop and maintain documentation for data processes and workflows.
* Develop and maintain automatic report production to ensure seamless delivery of critical data
Collaboration and Communication:
* Collaborate with colleagues to gather requirements and translate them into technical specifications.
* Communicate effectively with team members to ensure alignment on data initiatives
Qualifications:
* Bachelor's degree or equivalent experience in Computer Science, Information Technology, or a related field.
* Proven experience as a Data Engineer or in a similar role.
* Strong proficiency in SQL or Python or ideally both.
* Experience with ETL processes and tools.
* Knowledge of data validation routines and data integration techniques.
* Familiarity with relational database design and management.
* Experience with API development and testing using tools like Postman.
* Experience of Databricks or similar data platforms desirable
* Excellent problem-solving skills and attention to detail.
* Strong communication and collaboration skills.
for more information please apply using the link or get in touch with Edd @ Agilis Recruitment