As a Data Engineer you will responsible for budding optimizing and maintaining data pipelines using distributed computing on Cloud. The ideal candidate is an experienced data wrangler who can understand and optimize data systems from the ground up. The Data Engineer will support our software developers data analysts and data scientists on data initiatives and will ensure optimal data delivery architecture that is consistent throughout ongoing projects. They must be selfdirected and comfortable supporting the data needs 01 multiple teams systems and products.
Responsibilities
* Develop Operationalize scalable and reliable data pipelines
* Assemble large complex data sets that meet functional nonfunctional business requirements
* Experience in database management techniques including logical data modeling physical database
* optimization and security
* Keep improving the integrity Of data pipelines by setting up Data Quality checks to provide a
* Work with Data and Analytics experts to strive tor greater functionality in our data systems
* Experience with On-Call support and unblock users based on severity of the issue
Qualification
* 5 Years of experience in Engineering and Operationalizing Data Pipelines with large and complex
* datasets
* 3 Years Of experience working With Unstructured data JSON
* Extensive experience in Advanced SQL
* 3 Years Of experience in Python PySpark
* 3 Years of experience with transforming data using Databricks
* 3 Years Of experience With AWS
* Experience with GIT CICD
* power Bl or similar visualization
Skills
Mandatory Skills: PySpark-Data Science
J-18808-Ljbffr