Sector: Energy, Infrastructure and storage
Our client is a dynamic powerhouse delivering grid-scale energy storage and power infrastructure across the UK. Backed by major infrastructure investors and scaling rapidly, our Client is recruiting for a Data Engineer to join them on a permanent staff basis.
The Role
The The Data Engineer will join the team building the backbone of our Client’s data platform. You'll start by getting more data into our Client’s data warehouse, building API integrations, writing Python jobs in Prefect, and helping our Client automate work currently done in Excel. Over time you'll take ownership of how our Client deploy, monitor and secure that platform.
This is a growth-shaping role in a small team, working closely with our Client to build something they’re proud of. Our Client is targeting a mid-level engineer who can hit the ground running on Python and SQL.
Key Responsibilities
Data Pipelines and Integration
Build Python jobs in Prefect (our Client’s Airflow equivalent) to pull data from APIs into our PostgreSQL data warehouse.
Work with APIs, both consuming and building integrations, to get more data into the platform.
Ingest and process operational data from our Client’s engine and BESS sites, including time-series and telemetry feeds, into the data warehouse.
Support reporting and automation for the control room by writing scripts, tidying data and building tools that save people hours.
Help make site, generation and market data available and reliable for the teams that depend on it.
Infrastructure and Platform
Maintain and improve our Client’s infrastructure, including Linode servers (main platform), Azure workloads, our Client’s MQTT broker and its backup, and replication and backups across their data warehouses.
Maintain and improve the MQTT messaging that carries data from site, ensuring feeds are reliable and resilient.
Help migrate our Client’s internal FastAPI site, which serves warehouse data as HTML to internal users, into a more mature internal web solution.
Move our Client toward infrastructure-as-code (Terraform), no-code/low-code deployment workflows, and stronger CI/CD via GitHub.
Take ownership of chunks of the platform end-to-end as you grow into the role.
Data Quality and Reliability
Monitor data pipelines and feeds, resolving issues so that data arriving from sites is timely, complete and accurate.
Build checks and alerting that catch problems before the teams relying on the data do.
Document pipelines, integrations and infrastructure so the platform is maintainable as it grows.
Security and Resilience
Help our Client level-up on cyber security by ensuring backups actually work when tested.
Adopt and promote good cyber security practice in your workflows.
Support disaster recovery testing across the data platform.
Key Requirements
Skills & Experience
Essential Skills & Qualifications:
Degree in a STEM subject, or equivalent practical experience.
Solid commercial Python experience. You've built and maintained Python in a real production environment, not just learned it.
Confident with SQL and relational databases, including joins, indexes and the basics of query tuning.
Git/GitHub in your day-to-day workflow, with evidence of branches, PRs and code review.
Experience working with APIs (consuming and ideally building too).
Knowledge Required
The postholder must have a solid working knowledge of Python in a production environment, together with strong SQL and relational database skills, including PostgreSQL. They should be comfortable working on Linux servers, using Git and GitHub as part of a normal development workflow, and building and consuming APIs. A good understanding of data pipelines and workflow orchestration is important, along with familiarity with cloud and self-hosted infrastructure such as Azure and Linode. Knowledge of messaging systems such as MQTT, infrastructure-as-code tooling such as Terraform, and CI/CD practices is valuable as the platform matures. An awareness of good cyber security practice, backups and disaster recovery, and of how to keep data reliable and well documented, will help ensure the data platform remains secure, dependable and fit for purpose as it grows.
Desirable experience and skills
PostgreSQL specifically.
Workflow orchestrators (Prefect, Airflow, Dagster).
FastAPI or similar Python web frameworks (e.g. Django).
Terraform or other infrastructure-as-code tooling.
MQTT or any pub/sub messaging.
Some PHP exposure (helpful for collaborating on our Evolve site, but not essential).
Light ML exposure. We don't need a data scientist, but curiosity here is welcome.
Any cyber-security related experience is highly desirable (VPNs, access management, backup testing)
Person Specification:
The successful candidate will be a capable and curious mid-level engineer who enjoys being a generalist. They are comfortable moving between data pipelines, web APIs, infrastructure and automation, and diving into a range of business problems rather than specialising narrowly. They will have some commercial experience and a strong set of fundamentals to build on.
They won't be looking to be handed a spec and told exactly what to build; they'll want to help shape it. They'll be happy in a small team where the work is varied, the impact is visible, and ownership grows over time. A genuine interest in infrastructure, automation and how systems fit together is essential, along with the willingness to own what they build rather than just write scripts.
#J-18808-Ljbffr