AWS Data Engineer - SC cleared - remote - 12 months
Key Responsibilities
* Develop, maintain, and optimize ETL pipelines using AWS Glue (Informatica will be beneficial)
* Build and manage cloud-based data pipelines leveraging AWS services (eg, EMR, S3, Lambda, Glue).
* Implement scalable data processing workflows using Databricks, PySpark, Python, and SQL.
* Design and support data ingestion, transformation, and integration processes across structured and unstructured data sources.
* Collaborate with data architects, analysts, and business stakeholders to understand requirements and deliver reliable data solutions.
* Monitor pipeline performance, troubleshoot issues, and ensure data quality and reliability.
* Contribute to best practices for data engineering, including version control, CI/CD, and automation.
Required Skills & Qualifications
1. Strong hands-on experience with ETL development and orchestration (AWS).
2. Solid AWS cloud experience, including working with core data services.
3. Expertise in building distributed data pipelines using EMR, PySpark, or similar technologies.
4. Strong data processing and transformation experience across large datasets.
5. Proficiency in PySpark, Python, and SQL for data manipulation and automation.
6. Understanding of data modelling, data...