Overview
NTT DATA is currently looking for a Data Engineer for our growing team in the UK.
Competitive salary | UK/Glasgow: hybrid working model (2-3 days on site).
Req ID: 333370.
Responsibilities
* Collaborating with cross-functional teams to understand data requirements, and design efficient, scalable, and reliable ETL processes using Python and Databricks.
* Developing and deploying ETL jobs that extract data from various sources, transforming them to meet business needs.
* Taking ownership of the end-to-end engineering lifecycle, including data extraction, cleansing, transformation, and loading, ensuring accuracy and consistency.
* Creating and managing data pipelines, ensuring proper error handling, monitoring and performance optimizations.
* Working in an agile environment, participating in sprint planning, daily stand-ups, and retrospectives.
* Conducting code reviews, providing constructive feedback, and enforcing coding standards to maintain a high quality.
* Developing and maintaining tooling and automation scripts to streamline repetitive tasks.
* Implementing unit, integration, and other testing methodologies to ensure the reliability of the ETL processes.
* Utilizing REST APIs and other integration techniques to connect various data sources.
* Maintaining documentation, including data flow diagrams, technical specifications, and processes.
* Designing and implementing tailored data solutions to meet customer needs and use cases, spanning from streaming to data lakes, analytics, and beyond within a dynamically evolving technical stack.
* Collaborate across diverse technical stacks, including Databricks, Snowflake, etc.
* Developing components in Python as part of a unified data pipeline framework.
* Contributing towards best practices for data usage across on-prem and cloud platforms.
* Assisting with testing and deployment of the data pipeline framework utilizing standard testing frameworks and CI/CD tooling.
* Monitoring the performance of queries and data loads and performing tuning as necessary.
* Providing assistance during QA & UAT to confirm issues and identify root causes and resolutions.
* Adhering to Agile practices throughout the solution development process.
* Design, build, and deploy databases and data stores to support organizational requirements.
Qualifications
* 4+ years of experience developing data pipelines and data warehousing solutions using Python and libraries such as Pandas, NumPy, PySpark, etc.
* 3+ years hands-on experience with cloud services, especially Databricks, for building and managing scalable data pipelines.
* 3+ years of proficiency in Snowflake or similar cloud-based data warehousing solutions.
* 3+ years of experience in data development and solutions in highly complex data environments with large data volumes.
* Solid understanding of ETL principles, data modelling, data warehousing concepts, and data integration best practices.
* Familiarity with agile methodologies and the ability to work collaboratively in a fast-paced, dynamic environment.
* Experience with code versioning tools (e.g., Git).
* Knowledge of Linux operating systems.
* Familiarity with REST APIs and integration techniques.
* Familiarity with data visualization tools and libraries (e.g. Power BI).
* Background in database administration or performance tuning.
* Familiarity with data orchestration tools, such as Apache Airflow.
* Exposure to big data technologies (e.g. Hadoop, Spark) for large data processing.
* Strong analytical and communication skills; ability to translate business requirements into technical designs.
* Self-starter with the ability to manage multiple projects with minimal supervision.
* Strong problem-solving skills and prioritization abilities.
Education
Bachelor’s degree in Computer Science, Engineering, or a related field (or equivalent experience).
Benefits
Our people are the most critical component of our long-term success and their health and wellbeing are our priority. You will enjoy a comprehensive, locally competitive benefits package.
About NTT DATA
NTT DATA is a $30 billion trusted global innovator of business and technology services. We serve 75% of the Fortune Global 100 and are committed to helping clients innovate, optimize and transform for long term success. As a Global Top Employer, we have diverse experts in more than 50 countries and a robust partner ecosystem of established and start-up companies. Our services include business and technology consulting, data and artificial intelligence, industry solutions, as well as the development, implementation and management of applications, infrastructure and connectivity. We are one of the leading providers of digital and AI infrastructure in the world. NTT DATA is a part of NTT Group. Visit us at us.nttdata.com
NTT DATA endeavors to make https://us.nttdata.com accessible to any and all users. If you would like to contact us regarding the accessibility of our website or need assistance completing the application process, please contact us at https://us.nttdata.com/en/contact-us. This contact information is for accommodation requests only and cannot be used to inquire about the status of applications. NTT DATA is an equal opportunity employer. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, sexual orientation, gender identity, national origin, disability or protected veteran status. For our EEO Policy Statement, please click here. If you'd like more information on your EEO rights under the law, please click here. For Pay Transparency information, please click here.
Seniority level: Mid-Senior level
Employment type: Full-time
Job function: Information Technology
Industries: IT Services and IT Consulting
#J-18808-Ljbffr