We are seeking a highly skilled ML Ops Engineer to join our growing data team. This role is critical in designing and implementing robust, scalable, and efficient data systems that power analytics, machine learning models, and business insights. The ideal candidate will have expertise in data pipeline orchestration (e.g., Airflow), data lake and warehouse architecture and development, infrastructure as code (IaC) using Terraform, and data extraction from both structured and unstructured data sources (e.g. websites). Knowledge using the Microsoft Azure ecosystem, MLOps, Kubernetes, and other modern data engineering practices.
Core Responsibilities
Data Architecture & Development:
* Devesign and implement scalable, secure, and high-performance data lake and data warehouse solutions.
* Lerage best practices in schema design, partitioning, and optimisation for efficient storage and retrieval.
* Build and maintain data models to support analytics and machine learning workflows.
Pipeline Orchestration:
* Develop, monitor, and optimize ETL/ELT workflows using Apache Airflow.
* Ensure data pipelines are robust, error-tolerant, and scalable for real-time and batch processing.
Data Scraping & Unstructured Data Processing:
* Develop and maintain scalable web scraping solutions to collect data from diverse sources, including APIs, websites, and other unstructured data sources.
* Extract, clean, and transform unstructured data such as text, images, and log files into structured formats suitable for analysis.
* Use tools and frameworks like BeautifulSoup, Scrapy, or Selenium for web scraping, and natural language processing (NLP) techniques for text processing.
Cloud Integration:
* Design and implement cloud-native data solutions with Microsoft Azure.
* Optimize costs and performance of cloud-based data solutions.
Infrastructure as Code (IaC):
* Use Terraform to automate the provisioning and management of cloud infrastructure.
* Define reusable and modular Terraform configurations to support scalable deployment of resources.
MLOps:
* Collaborate with data scientists and machine learning engineers to operationalise machine learning models.
* Implement CI/CD pipelines for machine learning workflows, ensuring efficient model deployment and monitoring.
Containerisation and Orchestration:
* Utilize Kubernetes and containerisation technologies (e.g., Docker) to deploy scalable, fault-tolerant data processing systems.
* Manage infrastructure and resource allocation for containerised data applications.
Cross-Functional Collaboration:
* Work closely with stakeholders, including data scientists, software engineers, and business analysts, to align technical solutions with business needs.
* Mentor junior engineers and foster a culture of continuous learning within the team.
Skills / Qualifications
Education:
* Bachelor's/Master’s/PhD degree in Computer Science, Engineering, or a related field; or equivalent professional experience.
Experience:
* 5+ years of experience in data engineering or a related field.
* Strong expertise in data pipeline orchestration tools such as Apache Airflow.
* Proven track record of designing and implementing data lakes and warehouses (experience with Azure is a plus).
* Demonstrated experience with Terraform for infrastructure provisioning and management.
* Solid understanding of MLOps practices, including model training, deployment, and monitoring.
* Hands-on experience with Kubernetes and containerised environments.
Technical Skills:
* Proficiency in programming languages such as Python & SQL.
* Experience with distributed computing frameworks such as Spark.
* Familiarity with version control systems (e.g., Git) and CI/CD pipelines.
Soft Skills:
* Strong problem-solving skills and the ability to work in a fast-paced, collaborative environment.
* Excellent communication and documentation skills.
* Strong analytical mindset with attention to detail.
Company Overview
Element is one of the fastest growing testing, inspection and certification businesses in the world. Globally we have more than 9,000 brilliant minds operating from 270 sites across 30 countries. Together we share an ambitious purpose to ‘Make tomorrow safer than today’.