Social network you want to login/join with:
col-narrow-left
Client:
Methods
Location:
Brynmawr, United Kingdom
Job Category:
-
EU work permit required:
Yes
col-narrow-right
Job Reference:
f779f006f7b1
Job Views:
10
Posted:
26.04.2025
Expiry Date:
10.06.2025
col-wide
Job Description:
Methods Analytics
Methods Analytics (MA) exists to improve society by helping people make better decisions with data. Combining passionate people, sector-specific insight, and technical excellence to provide our customers an end-to-end data service.
We use a collaborative, creative, and user-centric approach to data to do good and solve difficult problems. Ensuring that our outputs are transparent, robust, and fostering discussion and debate as part of our methodology. We will question assumptions, ambition, and process – but do so with respect and humility.
We relish difficult problems and overcome them with innovation, creativity, and technical freedom to design optimal solutions. Ethics, privacy, and quality are at the core of our work, and we will not compromise these for outcomes.
We treat data with respect and use it only for the right purpose. Our team is positive, dedicated, and relentless. Data is a vast topic, but we aim for interactions that are engaging, informative, and fun in equal measure, while maintaining a focus on outcomes and delivering quality products for our customers.
We are passionate about our people; we want our colleagues to develop their strengths and enjoy their work.
Methods was acquired by the Alten Group in early 2022.
Requirements
On-site, Full-time.
We are seeking a seasoned Senior Data Engineer (Infrastructure) to join our team. This role is essential for designing, building, and maintaining sophisticated data infrastructure systems across on-premises and Azure cloud environments. The position involves deploying and managing scalable data operations that support advanced analytics and data-driven decision-making, crucial for our organizational growth and innovation.
Responsibilities
* Develop and Manage Data Pipelines: Design, construct, and maintain efficient and reliable data pipelines using Python, supporting streaming and batch data processing across structured, semi-structured, and unstructured data in on-premises and Azure environments.
* Hybrid Cloud and Data Storage Solutions: Implement and manage data storage solutions leveraging both on-premises infrastructure and Azure, ensuring seamless data integration and accessibility.
* Containerisation and Orchestration: Use Docker for containerisation and Kubernetes for orchestration, ensuring scalable deployment across various environments.
* Workflow Automation: Employ tools like Apache Airflow to automate data workflows within hybrid environments.
* Event Streaming Experience: Utilize event-driven technologies such as Kafka for real-time data streams.
* Security and Compliance: Manage security setups and access controls, incorporating tools like Keycloak to protect data and ensure legal compliance.
* Data Search and Analytics: Oversee Elasticsearch setups for data searching and analytics in mixed infrastructures.
* Database Management: Administer and optimize PostgreSQL databases for high performance and availability.
Essential Skills and Experience
* Strong Python Skills: Expertise in Python scripting and automation for data processes.
* Experience with ETL/ELT: Proven experience developing and optimizing workflows, especially in hybrid environments.
* Hybrid Cloud Data Architecture: Deep knowledge of integrating on-premises infrastructure with Azure cloud services.
* Containerisation and Orchestration: Experience with Docker and Kubernetes managing applications across environments.
* Workflow Automation: Practical experience with Apache Airflow in hybrid data settings.
* Event Streaming: Ability to manage platforms like Kafka.
* Data Security: Knowledge of security practices and tools like Keycloak.
* Search and Database Management: Skills in Elasticsearch and PostgreSQL in hybrid environments.
Your Impact
In this role, you will enable business leaders to make informed decisions by delivering accurate, timely data insights from a robust hybrid infrastructure. Your expertise will enhance the integration and scalability of data operations, promoting a data-driven culture and mentoring team members.
Desirable Skills and Experience
* Certifications in Azure and other relevant technologies are highly beneficial.
* At least 5 years of experience in data engineering, managing infrastructure in both on-premises and cloud settings.
This role requires active SC clearance and willingness to undergo Developed Vetting (DV). The position also involves working on-site 5 days a week at locations around Worcester, Gloucester, Great Malvern, Ebbw Vale, and Brynmawr. If you are passionate about data and technology in hybrid environments, we encourage you to apply.
#J-18808-Ljbffr