Key Responsibilities
* Pipeline Development: Design, develop, and maintain robust end-to-end data pipelines and orchestration workflows using Python, Apache Airflow, and Cloud Composer on GCP.
* Data Modernization: Bridge the gap between legacy/on-premise database environments and modern cloud architectures, ensuring seamless data ingestion and migration.
* Optimization & Performance: Architect and tune analytics solutions at scale within BigQuery, utilizing partitioning, clustering, and materialized views to maximize efficiency and control costs.
* CI/CD & Software Best Practices: Embed rigorous software engineering principles, including Git/GitHub version control, strict PR disciplines, automated unit/integration testing, and deployment via CI/CD pipelines.
* Data Governance & Compliance: Embed data quality controls, access management, data lineage, and privacy frameworks required to meet stringent prudential/regulatory banking compliance standards.
* Collaboration: Partner with cross-functional Agile teams, including Product Owners, DevOps Engineers, and Business SMEs, to translate complex technical requirements into business-ready assets.
Required Technical Skills & Experienc
* eGCP Core Stack: Proven production-grade experience building and managing cloud data solutions on Google Cloud Platform (specifically BigQuery, Cloud Composer/Airflow, Cloud Storage, and Dataflow)
* .Programming & Scripting: Strong hands-on coding capability in Python for data engineering, service development, and pipeline automation
* .Data Transformation & Modeling: Proficient in SQL, data modeling (Star Schema, Data Vault, normalisation), and data transformation frameworks like dbt
* .Regulated Environment Experience: Prior experience working within UK financial services, banking, or a similarly heavily regulated framework handling sensitive/critical data assets is highly preferred
* .Containerization (Desirable): Experience or familiarity with containerizing and operating services using Dockerand Kubernetes (GKE)
* .Legacy Systems (Desirable): Familiarity with enterprise legacy platforms (e.g., Informatica, Teradata, Mainframe) to assist with migration patterns is an added advantage
.