Social network you want to login/join with:
Methods Analytics (MA) exists to improve society by helping people make better decisions with data. Combining passionate people, sector-specific insight, and technical excellence to provide our customers an end-to-end data service.
We use a collaborative, creative, and user-centric approach to data to do good and solve difficult problems. Ensuring that our outputs are transparent, robust, and foster discussion and debate as part of our approach. We will question assumptions, ambition, and process – but do so with respect and humility.
We relish difficult problems and overcome them with innovation, creativity, and technical freedom to design optimal solutions. Ethics, privacy, and quality are at the heart of our work, and we will not sacrifice these for outcomes.
We treat data with respect and use it only for the right purpose. Our people are positive, dedicated, and relentless. Data is a vast topic, but we strive for interactions that are engaging, informative, and fun in equal measure, while maintaining a steely focus on outcomes and delivering quality products for our customers.
We are passionate about our people; we want our colleagues to develop their strengths and enjoy their work.
Methods was acquired by the Alten Group in early 2022.
Requirements
On-site, Full-time.
We are seeking a seasoned Senior Data Engineer (Infrastructure) to join our team. This role is essential for designing, building, and maintaining sophisticated data infrastructure systems that operate across on-premises and Azure cloud environments. The position involves deploying and managing scalable data operations that support advanced analytics and data-driven decision-making, crucial for our organisational growth and innovation.
Responsibilities
* Develop and manage data pipelines using Python, supporting streaming and batch processing in structured, semi-structured, and unstructured data across on-premises and Azure environments.
* Implement and manage hybrid cloud and data storage solutions, ensuring seamless data integration and accessibility.
* Utilise Docker for containerisation and Kubernetes for orchestration to ensure scalable deployment.
* Automate workflows with tools like Apache Airflow.
* Manage real-time data streams using event-driven technologies such as Kafka.
* Manage security and access controls, incorporating tools like Keycloak to ensure data integrity and compliance.
* Oversee Elasticsearch setups for data search and analytics.
* Administer and optimise PostgreSQL databases for high performance and availability.
Essential Skills and Experience
* Strong Python skills for scripting and automation.
* Experience with ETL/ELT workflows in hybrid environments.
* Profound knowledge of hybrid cloud data architecture, especially Azure.
* Experience with Docker and Kubernetes.
* Proficiency with Apache Airflow.
* Experience with Kafka and real-time data streaming.
* Knowledge of security practices including Keycloak.
* Management of Elasticsearch and PostgreSQL in hybrid setups.
Your Impact
You will enable data-driven decision-making by delivering accurate, timely insights from a hybrid infrastructure, driving scalability and flexibility. Your work will foster a data-centric culture, mentoring team members and advancing engineering practices.
Desirable Skills and Experience
* Certifications in Azure and relevant technologies.
* Minimum of 5 years in data engineering, managing infrastructure in hybrid environments.
This role requires active SC clearance and willingness to undergo Developed Vetting (DV). The position demands on-site presence 5 days a week at our locations in Worcester, Gloucester, Great Malvern (England), Ebbw Vale, and Brynmawr (Wales). If passionate about using data to drive innovation in hybrid environments, we encourage you to apply.
Benefits
* Autonomy to develop skills and experience.
* Participation in impactful projects.
* Strong leadership and collaborative environment.
* Development opportunities including LinkedIn Learning and training.
* Wellness support, flexible working, social activities, generous leave, volunteering days, pension, bonuses, life insurance, medical and travel insurance, enhanced parental pay, travel schemes, and more.
#J-18808-Ljbffr