Job Title: Senior AWS Data Engineer / Architect (AWS | AI/ML | OpenSearch)
Job Type: Permanent
Job Location: Remote, UK
Data Engineer (AWS | AI/ML | OpenSearch)
We are looking for a Principal-level Data Engineer with strong AWS Data Engineering experience to lead the design and implementation of data pipelines, metadata frameworks, AI/ML data solutions, and vector search capabilities. The ideal candidate should have hands-on experience with AWS data services, OpenSearch, vector embeddings, and large-scale data processing.
Duties and Responsibilities
• Review the existing AWS data platform architecture and provide recommendations for improvements.
• Design metadata models, taxonomy structures, and data catalog frameworks.
• Build and optimize data ingestion and orchestration pipelines using AWS services.
• Configure and manage OpenSearch vector databases and embedding pipelines.
• Develop AI/ML data export frameworks and integration patterns.
• Work closely with architects, analysts, and infrastructure teams to deliver scalable solutions.
• Handle large-scale structured and unstructured datasets.
• Support data visualization and analytics requirements.
Business and Technical Skills Required
• 7+ years of hands-on Data Engineering experience.
• Strong AWS experience with:
S3
S3 Tables
AWS Glue
Athena
OpenSearch
• Experience with Vector Search / k-NN Search.
• Experience building Vector Embedding pipelines.
• Strong understanding of AI/ML data architectures.
• Experience in Metadata Modelling and Data Taxonomy Design.
• Strong Python development skills.
• Experience with PySpark and AWS Glue job optimization.
• Experience working with Parquet, JSON-LD, and large datasets.
• Strong API integration and data pipeline development experience.
Nice to Have
• Experience with Bedrock and Generative AI solutions.
• Experience with SageMaker.
• Simulation / CAE / HPC data experience (Ansys, OpenFOAM, Siemens NX, etc.).
• Multi-cloud experience (AWS and GCP).
• Experience designing enterprise-scale data architectures.