AWS Data Engineer

Data Engineer - 14-Week Contract (Outside IR35) Likely to Extend Start Date: 12th January Rate: £350 per day Location: Remote (UK-based) Interview: Immediate - Offer before Christmas We are seeking an experienced Data Engineer to join a 14-week project focused on building robust data pipelines and integrating complex data sources. This is an outside IR35 engagement, offering flexibility and autonomy. Key Responsibilities Design and implement ETL/ELT pipelines with strong error handling and retry logic. Develop incremental data processing patterns for large-scale datasets. Work with AWS services including Glue, Step Functions, S3, DynamoDB, Redshift, Lambda, and EventBridge. Build and optimise vector database solutions and embedding generation pipelines for semantic search. Implement document processing workflows (PDF parsing, OCR, metadata extraction). Integrate data from REST APIs, PIM systems, and potentially SAP. Ensure data quality, governance, and lineage tracking throughout the project. Required Skills ETL/ELT pipeline design and data validation frameworks. Advanced Python (pandas, numpy, boto3) and SQL (complex queries, optimisation). Experience with AWS Glue, Step Functions, and event-driven architectures. Knowledge of vector databases, embeddings, and semantic search strategies. Familiarity with document parsing libraries (PyPDF2, pdfplumber, Textract) and OCR tools. Understanding of data governance, schema validation, and master data management. Strong grasp of real-time vs batch processing trade-offs. Beneficial Experience CockroachDB deployment and management. PySpark or similar for large-scale processing. SAP data structures and PIM systems. E-commerce and B2B data integration patterns.Why Apply? Fully remote contract Outside IR35 Competitive day rate Immediate interviews - secure your next role before Christmas