Job Overview:We are seeking a motivated and detail-oriented Data Engineer with a passion for designing and delivering high-quality data solutions in Databricks. You will be responsible for building and optimising data pipelines that form the foundation of our reporting and analytics ecosystem.
Using your technical expertise, you will design and maintain efficient ETL processes to integrate data from systems such as ServiceNow, JIRA, and Dynatrace, ensuring accuracy, performance, and scalability. You will thrive in a fast-paced, collaborative environment and be eager to learn, innovate, and continuously improve our data platform.If you are passionate about building reliable data foundations and enabling insights that drive smarter decisions — this is the ideal role for you!
Contract – 6 monthsLocation – Cambridge (2 days onsite)Rate – Flexible (inside IR35)
Responsibilities:
1. Design, build, and maintain scalable ETL pipelines in Databricks to integrate data from multiple business systems such as ServiceNow, JIRA, and ADO.
2. Optimise data workflows for performance, scalability, and reliability.
3. Implement data validation and quality checks to ensure trustworthy reporting in downstream tools such as Power BI.
4. Create data modelling and schema design at backend for analytics and reporting.
5. Collaborate with the Staff Data Engineer and Visualisation Developer to align technical delivery with reporting needs.
6. Manage code in Git and support CI/CD processes for Databricks deployments.
7. Contribute to data lineage documentation, standards, andernance best practices.
Required Skills and Experience:
8. Proven experience in building and maintaining ETL pipelines using Databricks.
9. End to end Data modeling experience.
10. Strong knowledge of SQL and Python for data transformation and automation.
11. Proven experience in data modelling, schema design, and data quality validation.
12. Solid understanding of data performance tuning and pipeline optimisation in Databricks.
13. Experience working with Git-based version control and collaborative development workflows.
14. Strong analytical and problem-solving skills, with an eye for efficiency and accuracy.
15. Excellentmunication and collaboration skills,fortable working with both technical and non-technical partners.
“Nice To Have” Skills and Experience:
16. Familiarity with Agile delivery methods and iterative development practices!
17. Knowledge of dataernance and data lineage documentation standards.
18. Exposure to automation and CI/CD frameworks within Databricks.