Techyard is supporting a growing Databricks-partnered consultancy in securing a Databricks Champion Solution Architect to lead the design and delivery of advanced data and AI solutions on the Databricks Lakehouse Platform. This strategic, high-impact role is ideal for a seasoned professional who can operate as both a hands-on architect and a trusted advisor—bridging business vision with technical excellence. You’ll play a pivotal part in driving innovation, embedding best practices, and shaping the consultancy’s data engineering capabilities as they scale their team and client base.
Key Responsibilities:
* Architect and implement end-to-end, scalable data and AI solutions using the Databricks Lakehouse (Delta Lake, Unity Catalog, MLflow).
* Design and lead the development of modular, high-performance data pipelines using Apache Spark and PySpark.
* Champion the adoption of Lakehouse architecture (bronze/silver/gold layers) to ensure scalable, governed data platforms.
* Collaborate with stakeholders, analysts, and data scientists to translate business needs into clean, production-ready datasets.
* Promote CI/CD, DevOps, and data reliability engineering (DRE) best practices across Databricks environments.
* Integrate with cloud-native services and orchestrate workflows using tools such as dbt, Airflow, and Databricks Workflows.
* Drive performance tuning, cost optimisation, and monitoring across data workloads.
* Mentor engineering teams and support architectural decisions as a recognised Databricks expert.
Essential Skills & Experience:
* Demonstrable expertise with Databricks and Apache Spark in production environments.
* Proficiency in PySpark, SQL, and working within one or more cloud platforms (Azure, AWS, or GCP).
* In-depth understanding of Lakehouse concepts, medallion architecture, and modern data warehousing.
* Experience with version control, testing frameworks, and automated deployment pipelines (e.g., GitHub Actions, Azure DevOps).
* Sound knowledge of data governance, security, and compliance, including Unity Catalog.
* Excellent communication, leadership, and problem-solving skills.
Desirable:
* Databricks certifications (e.g., Data Engineer Associate / Professional or Solutions Architect).
* Familiarity with MLflow, dbt, and BI tools such as Power BI or Tableau.
* Exposure to MLOps practices and deploying ML models within Databricks.
* Experience working within Agile and DevOps-driven delivery environments.