Overview
We are seeking a highly skilled and experienced Lead Software Engineer (Azure Data) to join our dynamic team at Royal London. In this role, you will provide technical leadership and deep technical expertise to design, develop, test, implement and support efficient solutions that meet business and customer requirements. The successful candidate will influence how Royal London develops and supports applications, combining leadership and technical capability. This role is focused within the Engagement Services platform with a focus on Azure Data.
Responsibilities
* Data Pipeline Development: Design, develop, and maintain ETL/ELT data pipelines using Azure Data Factory (ADF) for automating data ingestion, transformation, and orchestration from diverse data sources. Build scalable data integration processes to bring in data from on-premises, cloud, and external APIs into Azure Data Lake Storage (ADLS).
* Data Transformation & Modeling: Utilize Azure Data Factory/ Databricks (PySpark/Scala) to build scalable data processing and transformation workflows for both batch and streaming data. Develop data models and implement data partitioning, indexing, and schema optimization to improve query performance in Azure Synapse Analytics.
* Data Warehouse & Analytics Solutions: Collaborate with data architects and business stakeholders to design and implement data lake and data warehouse architectures on Azure. Create and optimize data models (star and snowflake schemas) for BI and analytics workloads, ensuring high performance and scalability.
* Data Governance & Quality: Implement data governance practices to ensure data accuracy, completeness, and integrity across the Azure environment. Develop and enforce data quality checks and validation rules to maintain high levels of data consistency using Azure Data Factory and Databricks.
* Leads the delivery of complex technical solutions, provides technical advice and guidance to other software developers and ensures that all code meets Group standards, guidelines and quality gates for given technology.
* Takes responsibility for ensuring technical solutions are safely implemented to production environments without impact to internal or external customers.
* Takes an active role in transferring knowledge and sharing experiences relating to own area of technical expertise at practice level.
* Strong experience in building and orchestrating complex data pipelines in ADF.
* Experience with Delta Lake for implementing data lakes and managing big data workloads efficiently.
* Strong knowledge of Synapse Analytics (formerly SQL Data Warehouse) for data warehousing and advanced analytics.
* Experience with T-SQL and optimizing complex queries for large datasets.
* Experience with designing and managing data lakes using ADLS, with an understanding of hierarchical namespaces, partitioning strategies, and performance optimization techniques.
* Expertise in Data Modeling (star and snowflake schemas) and writing efficient SQL queries for analytical workloads.
* Experience with Azure DevOps for CI/CD pipeline creation and deployment automation for data pipelines and infrastructure.
* Proficiency with Azure Active Directory (AAD), Azure Key Vault, and Azure RBAC for managing security and access control.
* A growth mindset with the ability to lead a team, support, coach, and mentor others.
* Able to communicate at all levels in a collaborative manner.
* Good understanding of the full software development lifecycle, and an awareness of agile development practices.
#J-18808-Ljbffr