Overview
We are seeking a highly skilled Senior Data Engineer to design, build, and maintain scalable data pipelines and architectures. You will play a key role in enabling data-driven decision-making by ensuring data is reliable, accessible, and optimized for analytics and machine learning use cases.
Data Engineering & Architecture
* Design, develop, and maintain scalable ETL/ELT pipelines
* Build and optimize data warehouses, data lakes, and lakehouse architectures
* Ensure efficient data ingestion, transformation, and storage
* Develop reusable frameworks and standards for data engineering best practice
Data Modelling & Warehousing
* Design data models (dimensional, normalized, star/snowflake schemas)
* Collaborate with analytics teams to support BI and reporting requirements
* Optimize data structures for performance and scalability
Cloud & Platform Engineering
* Build and maintain data solutions on cloud platforms (Azure, AWS, GCP)
* Implement and manage data tools such as:
o Azure Data Factory / Synapse / Databricks
o AWS Glue / Redshift / EMR
o Google BigQuery / Dataflow
* Ensure systems are secure, scalable, and cost-efficient
Data Pipeline & Workflow Management
* Develop orchestration workflows (e.g., Airflow, Prefect, Azure Data Factory)
* Monitor and troubleshoot pipelines to ensure high availability and performance
* Implement robust error handling, logging, and alerting
Data Quality & Governance
* Ensure data quality, integrity, and consistency
* Implement validation, testing, and monitoring frameworks
* Work with governance teams on data security, compliance, and policies
Collaboration & Leadership
* Partner with Data Scientists, Analysts, and Product teams
* Mentor junior data engineers and promote best practices
* Contribute to architectural decisions and strategic planning
Technical Skills
* Strong proficiency in SQL and data modelling
* Excellent programming skills in Python, Scala, or Java
* Hands‑on experience with distributed data frameworks (Spark, Hadoop)
* Experience with data pipelines and ETL tools
* Expertise in at least one cloud platform:
o Azure (Synapse, Data Factory, Databricks)
o AWS (S3, Glue, Redshift, Lambda)
o GCP (BigQuery, Dataflow)
Data Platform Expertise
* Experience with data warehousing solutions
* Streaming technologies (Kafka, Kinesis, Pub/Sub)
Familiarity with DevOps/DataOps practices
* CI/CD pipelines
* Infrastructure as Code (Terraform, ARM, CloudFormation)
#J-18808-Ljbffr