Job Description
Data Engineer
Role Summary
VE3’s Data Engineer will design, build, and optimise the data pipelines, models, and structures that support high-quality, interoperable, and AI-ready services. The role will ensure that data is consistently ingested, transformed, validated, enriched, and exposed in a way that supports reliable API performance, improved discoverability, and future analytics or AI use cases. The Data Engineer will also support metadata quality, data modelling, and operational data management.
Requirements
Key Responsibilities
* Design and develop scalable data pipelines and transformation processes for structured and semi-structured datasets.
* Build and maintain data models, schemas, mappings, and metadata structures that support API and search use cases.
* Implement data validation, profiling, reconciliation, and quality controls across ingestion and publishing workflows.
* Support the preparation of clean, consistent, and machine-readable datasets for operational and AI-enabled services.
* Work closely with API developers and AI engineers to expose data efficiently and accurately through services and search layers.
* Optimise data storage, query performance, and interoperability across source and target systems.
* Contribute to lineage, cataloguing, documentation, and governance of data assets.
* Support troubleshooting, defect resolution, and continuous improvement of data processes.
* Maintain technical documentation and support knowledge transfer into live service support models.
Skills & Experience
* Strong experience in data engineering, ETL/ELT, and data modelling.
* Experience with SQL and data processing tools/frameworks such as Python, Spark, Databricks, or equivalent.
* Experience working with relational, document, and cloud-native data platforms.
* Understanding of metadata standards, data quality management, schema evolution, and interoperability.
* Ability to design data structures that support both transactional services and downstream AI/search use cases.
* Experience with version control, automation, and engineering best practices.
Desirable
* Experience with semantic metadata, search indexing, vector-ready data preparation, or API-linked data services.
* Familiarity with data catalogues, lineage tooling, and governed data-sharing models.
* Experience supporting cloud-based and production-grade data services.
Requirements
Key Responsibilities Design and develop scalable data pipelines and transformation processes for structured and semi-structured datasets. Build and maintain data models, schemas, mappings, and metadata structures that support API and search use cases. Implement data validation, profiling, reconciliation, and quality controls across ingestion and publishing workflows. Support the preparation of clean, consistent, and machine-readable datasets for operational and AI-enabled services. Work closely with API developers and AI engineers to expose data efficiently and accurately through services and search layers. Optimise data storage, query performance, and interoperability across source and target systems. Contribute to lineage, cataloguing, documentation, and governance of data assets. Support troubleshooting, defect resolution, and continuous improvement of data processes. Maintain technical documentation and support knowledge transfer into live service support models. Skills & Experience Strong experience in data engineering, ETL/ELT, and data modelling. Experience with SQL and data processing tools/frameworks such as Python, Spark, Databricks, or equivalent. Experience working with relational, document, and cloud-native data platforms. Understanding of metadata standards, data quality management, schema evolution, and interoperability. Ability to design data structures that support both transactional services and downstream AI/search use cases. Experience with version control, automation, and engineering best practices. Desirable Experience with semantic metadata, search indexing, vector-ready data preparation, or API-linked data services. Familiarity with data catalogues, lineage tooling, and governed data-sharing models. Experience supporting cloud-based and production-grade data services.