Job Description
Project description
You will join the team behind an internal AI platform for processing and interacting with unstructured data. The team is currently over 30 people strong and is organized into agile teams, each of which is self-sufficient and handles the creation of features from the idea stage, through analysis, implementation, testing, production deployment, and maintenance. The team is international, and it's located in Krakow, Wroclaw and London
Responsibilities
1. Maintain and enhance the existing Python-based reporting system, ensuring stability and accuracy in regulatory reporting.
2. Monitor, troubleshoot, and resolve issues in the current reporting workflows.
3. Ensure that reports comply with regulatory requirements and meet audit standards.
4. Lead the migration of the existing reporting system to Azure Databricks, ensuring a smooth transition.
5. Design and implement new, scalable reporting solutions using PySpark, SQL, and Databricks visualization tools.
6. Optimize data processing pipelines to enhance performance, reliability, and efficiency of regulatory reports.
7. Optimize PySpark performance to handle large-scale financial data processing for regulatory reporting.
8. Improve SQL query execution to ensure fast, efficient report generation.
9. Implement best practices for structured and uns...