Would you be interested in a data engineering role at a fast-paced AI (LLM) start-up comprised of Meta, Google, AWS, and Microsoft alumni?
You can expect to:
* Curate and manage large-scale data ingestion and indexing pipelines, ensuring data quality and error handling.
* Design robust data processing and indexing pipelines handling large amounts of data across several modalities, ensuring robustness and timeliness.
* Ensure the secure handling of data through end-to-end encryption, data isolation, and other best practices.
* Manage cloud orchestration and data infrastructure, applying your expertise with various data processing and indexing tools to ensure scalability and reliability.
* Work closely with the team to integrate search infrastructure that meets the needs of large language models and retrieval systems.
Your first 3 months will include:
* You have a strong grasp infrastructure, architecture, performance concerns, SDLC and practices. You have successfully enhanced their data ingestion framework, demonstrating significant improvements in the efficiency and scalability of data collection.
* Your contributions have directly improved the scalability, and performance of their data infrastructure, setting new benchmarks for data quality and access.
* You are finding opportunities to learn, mentor and promote best practices in areas of your expertise. Your colleagues count on you for your technical advice.
A successful candidate will have:
* Demonstrated experience with building and optimising search infrastructure in a cloud environment.
* Prior experience working in both small and large companies
* A product orientated mindset
* Strong programming and engineering skills, with proficiency in Python and experience with cloud infrastructure (AWS, Google Cloud, Azure) and data infrastructure (SQL, NoSQL, inverted indexes, vector databases).
* Commitment to secure data practices and a keen interest in staying updated with the latest technologies and methodologies in data engineering.
* A track record of successfully scaling data pipelines to handle large volumes of data continuously.
* Independent problem-solver with a passion for data and a proactive approach to overcoming challenges.
They typically operate a 2 day per week hybrid working model with offices in London however we encourage fully remote candidates to also apply. You'll receive a highly competitive salary, equity package, comprehensive health insurance, unlimited PTO, pension plans, travel budget, office equipment allowance and VISA sponsorship if required.
Apply below to join a challenging in house LLM (large language model) project backed by significant investors!