About the Job you are considering
This role involves designing, building, and supporting large-scale Big Data solutions using Hadoop and Spark technologies. You will primarily develop and debug Spark jobs in Scala, with opportunities to use Java and Python where appropriate. The position focuses on creating scalable data pipelines, optimizing distributed processing, and supporting analytics and data science teams. It also offers exposure to cloud-based Big Data platforms and modern data engineering practices in a fast-paced, data-driven environment.
Hybrid working
The places that you work from day to day will vary according to your role, your needs, and those of the business; it will be a blend of Company offices, client sites, and your home; noting that you will be unable to work at home 100% of the time.
Your Role
* Design and develop Hadoop-based applications and scalable data pipelines.
* Debug, develop, and optimize Spark jobs primarily using Scala.
* Build, operate, monitor, and troubleshoot Hadoop clusters.
* Develop robust ETL workflows using Spark, Hive, and Pig.
* Create and manage data ingestion pipelines using Sqoop, Flume, or Kafka.
* Optimize MapReduce jobs and efficiently manage HDFS storage.
* Collaborate with data scientists and analysts to meet data requirements.
* Ensure data security, compliance, and maintain technical documentation.
Your Skills
* Strong experience developing and debugging Spark applications in Scala.
* Hands‑on Java Big Data development experience.
* Good knowledge of Python for data processing and scripting (nice to have).
* Solid understanding of Hadoop ecosystem: HDFS, MapReduce, YARN.
* Experience with Hive, Pig, and HBase.
* Knowledge of data ingestion tools such as Sqoop, Flume, and Kafka.
* Ability to optimize distributed data processing jobs.
* Exposure to cloud platforms such as AWS, GCP, or Azure is a plus.
We are a Disability Confident Employer
Capgemini is proud to be a Disability Confident Employer (Level 2) under the UK Governments Disability Confident scheme. As part of our commitment to inclusive recruitment, we will offer an interview to all candidates who:
* Declare they have a disability, and
* Meet the minimum essential criteria for the role.
Please opt in during the application process.
J-18808-Ljbffr