Data Engineer - 14-Week Contract (Outside IR35) Likely to Extend Start Date: 12th January Rate: £350 per day Location: Remote (UK-based) Interview: Immediate - Offer before Christmas We are seeking an experienced Data Engineer to join a 14-week project focused on building robust data pipelines and integrating complex data sources. This is an outside IR35 engagement, offering flexibility and autonomy. Key Responsibilities Design and implement ETL/ELT pipelines with strong error handling and retry logic. Develop incremental data processing patterns for large-scale datasets. Work with AWS services including Glue, Step Functions, S3, DynamoDB, Redshift, Lambda, and EventBridge. Build and optimise vector database solutions and embedding generation pipelines for semantic search. Implement document processing workflows (PDF parsing, OCR, metadata extraction). Integrate data from REST APIs, PIM systems, and potentially SAP. Ensure data quality, governance, and lineage tracking throughout the project. Required Skills ETL/ELT pipeline design and data validation frameworks. Advanced Python (pandas, numpy, boto3) and SQL (complex queries, optimisation). Experience with AWS Glue, Step Functions, and event-driven architectures. Knowledge of vector databases, embeddings, and semantic search strategies. Familiarity with document parsing libraries (PyPDF2, pdfplumber, Textract) and OCR tools. Understanding of data governance,...