As a Data Engineer at JPMorgan Chase within the Cybersecurity and Technology Controls, you'll leverage your skills to develop and implement robust data solutions using cutting-edge technologies. You'll play a critical role in analyzing complex data structures, ensuring data accuracy, and enhancing our security analytics capabilities. Collaborate with cross-functional teams to drive innovation, implement scalable solutions, and protect our digital assets. Your contributions will be pivotal in shaping the future of cybersecurity, fostering a culture of excellence, and ensuring the integrity and security of our data infrastructure.
Job Responsibilities:
1. Design and implement complex, scalable solutions to efficiently process large volumes of data, ensuring consistent and timely delivery and availability.
2. Troubleshoot and resolve complex issues related to data architecture, including data ingestion, indexing, and search performance.
3. Create reusable frameworks with a strong emphasis on quality and long-term sustainability.
4. Collaborate with key partners to enhance understanding of data usage within the business.
5. Serve as a subject matter expert on the content and application of data in the product and related business areas.
6. Integrate data from multiple sources, including structured, semi-structured, and unstructured data.
7. Implement data quality checks and validation processes to ensure the accuracy, completeness, and consistency of the data.
8. Analyze complex data structures from various security data sources and scale data engineering pipelines.
9. Perform all Data Engineering job activities, including ELT project development, testing, and deployment activities.
10. Documents data engineering processes, workflows, and systems for reference and knowledge-sharing purposes
11. Add to team culture of diversity, opportunity, inclusion, and respect.
Required Qualifications, Capabilities, and Skills:
12. Formal training or certification on SQL concepts and proficient applied experience
13. Proficient in database management, with experience in both relational databases (SQL) and NoSQL databases.
14. Experience with Python and SQL.
15. Expertise in Python and Java development.
16. Extensive experience with Big Data technologies, including Spark, Hive, Redshift, Kafka, and others.
17. Strong understanding of ETL/ELT frameworks and tools, including DBT, Apache Airflow, Trino, Kestra, or similar technologies.
18. Hands-on experience in data pipelines and ETL/ELT processes using Python and SQL.
19. Experience with Kafka data streaming or other streaming/messaging services like Kinesis, SNS, SQS.
20. Demonstrated experience developing, debugging, and tuning complex SQL statements.
21. Experience working on real-time and streaming applications.
22. Exceptional understanding of distributed systems and cloud platforms.
23. Skilled in working with data streaming platforms such as Apache Flink and Apache Kafka.
24. Expertise in performing data transformations to enhance data quality and usability.
25. Comprehensive knowledge of the Software Development Life Cycle.
26. Solid grasp of agile methodologies, including CI/CD, application resiliency, and security best practices.
Preferred Qualifications, Capabilities, and Skills:
27. Relevant industry experience, preferably in a data engineering role focused on threat detection and security analytics.
28. Experience with advanced data streaming and transformation tools.
29. Experience with Kubernetes for container orchestration is a plus.
30. Experience onboarding datasets to Splunk, ensuring CIM compliance.
31. Be a team player and work collaboratively with team members.