Ocean Infinity is seeking an experienced and technically accomplished Senior Data Engineer - Integration to join our team. The successful candidate will lead the design and implementation of scalable data pipelines and automation solutions that transform complex manual workflows into reliable, production‑grade data systems.
Responsibilities
* Architect and lead the implementation of complex data automation solutions that replace manual workflows with scalable, reliable data pipelines across the organization.
* Design and build end‑to‑end Bronze Silver Gold transformation pipelines, implementing domain logic, data quality frameworks, and reproducible data products.
* Own the design of the data platform architecture, including ingestion frameworks, transformation patterns, orchestration strategies, and data governance standards.
* Establish and enforce data engineering best practices, including schema evolution, data contracts, validation frameworks, and performance optimization across all data pipelines.
* Build scalable backend data services and APIs that expose curated datasets to analytics tools, product applications, and downstream consumers.
* Lead the integration of data sources from operational systems, external applications, and sensor data, implementing both batch and streaming patterns as needed.
* Partner closely with product teams, data analysts, business stakeholders, and AI Engineers to understand complex data requirements and translate them into robust technical solutions.
* Drive technical excellence by establishing architectural standards, conducting design reviews, and documenting patterns that enable team scalability and maintainability.
* Mentor junior and mid‑level engineers through code reviews, technical guidance, architecture discussions, and hands‑on coaching to build team capability.
Qualifications
* A degree in Computer Science, Mathematics, or a related field, or equivalent significant practical experience.
* 5+ years of experience as a Data Engineer, Backend Engineer, or similar role with demonstrated technical leadership.
* Expert‑level programming skills in Python and advanced proficiency in SQL, including query optimization and performance tuning.
* Proven experience architecting and maintaining complex data pipelines, transformation workflows, and data platform components that serve multiple downstream consumers.
* Deep hands‑on experience with workflow orchestration tools (Airflow, Prefect, Flyte) including advanced patterns and best practices.
* Strong expertise with SQL and NoSQL systems (e.g., Postgres, MongoDB, Redis) including schema design and optimization.
* Advanced understanding of data structures, algorithms, distributed systems, and data modeling principles.
* Extensive experience with data lake / lakehouse architectures, implementing medallion architecture patterns, and building data products that serve structured and semi‑structured data at scale.
* Excellent communication and leadership skills, with proven ability to influence technical direction and mentor team members.
* Track record of delivering complex, high‑impact data platform solutions that automate manual workflows and enable self‑service data access in cross‑functional environments.
Desirable
* Deep experience with Delta Lake, data lake storage, and cloud data platforms (Azure preferred).
* Proven experience building event‑driven and streaming data integrations at scale.
* Expert knowledge of Azure data services (ADLS, ADF, Databricks, Event Hubs).
* Strong experience implementing and maintaining CI/CD workflows (Azure Pipelines, GitHub Actions, ArgoCD).
* Production experience deploying and managing containerized services in Kubernetes (AKS preferred).
* Experience with data quality frameworks and testing tools (Great Expectations, Soda, dbt).
* Knowledge of data governance tools and metadata management (Unity Catalog, Purview).
* Experience with infrastructure‑as‑code (Terraform, Bicep) and platform engineering.
* Understanding of feature engineering patterns and data preparation workflows for ML systems.
* Background in system design, performance optimization, and capacity planning for data platforms.
* Contributions to open‑source projects or technical community involvement.
Salary
Salary: The salary varies for this position as we are recruiting in multiple regional locations and job grades. The salary process is based on skills, abilities, and experience required.
#J-18808-Ljbffr