About the role
We are looking for a Senior Data Analyst to lead analytics and evaluation efforts for search and retrieval systems. You will own key analytical workflows, define measurement strategies, and generate insights that directly improve ranking quality, relevance, and user experience. This role is ideal for someone with deep experience in search analytics, experimentation, and data visualization, who can operate with high autonomy and influence decision‑making across cross‑functional teams.
Key Responsibilities
Search Evaluation & Analytics: Perform a leading role in analysis of search and retrieval system performance, including ranking quality and relevance. Help define and standardize search evaluation metrics (e.g., NDCG, MAP, recall, precision, CTR). Analyze query behavior, user interaction signals, and content performance to identify optimization opportunities. Conduct deep‑dive analyses on ranking performance, query intent, and retrieval gaps. Support evaluation of downstream applications (including GenAI‑powered features) where they depend on retrieval quality.
Experimentation & Reporting: Help design and lead A/B testing and experimentation frameworks for search and ranking improvements. Partner with product and data science to define success metrics and experiment strategies. Ensure statistical rigor in experiment design, analysis, and interpretation. Build reusable experimentation templates and scalable analysis workflows.
Data Visualization & Communication: Own and evolve dashboards and reporting systems tracking search performance and user engagement. Develop clear, actionable data storytelling to communicate insights to technical and business stakeholders. Enable self‑service analytics for partners through well‑designed reporting tools.
Data Management & Tooling: Work with large‑scale datasets using modern platforms (e.g., Databricks, Spark, SQL‑based systems). Ensure high standards for data quality, metric consistency, and instrumentation reliability. Collaborate with engineering to improve logging, tracking, and observability of search systems.
Cross‑functional Impact & Mentorship: Act as a key analytics partner to search data scientists, engineers, and product teams. Help elevate the team’s understanding of retrieval performance and measurement frameworks.
Required Qualifications
Master’s or PhD in Data Analytics, Statistics, Computer Science, or a related field (or equivalent practical experience). Significant experience in data analysis, business intelligence, or analytics roles. Proficiency in SQL and Python for large‑scale data analysis. Advanced experience with data visualization and BI tools (e.g., Tableau, Power BI, Looker, matplotlib, seaborn). Experience working with Databricks or similar large‑scale data platforms. Excellent understanding of experimentation design, A/B testing, and statistical analysis. Experience defining and analyzing search/retrieval metrics (e.g., NDCG, recall, precision, ranking metrics). Proven ability to translate complex data into actionable insights and influence decisions.
Preferred Qualifications
PhD in Data Analytics, Statistics, Computer Science, or a related field (or equivalent practical experience). Experience working on search, ranking, or recommendation systems. Familiarity with information retrieval concepts (e.g., indexing, ranking, query understanding). Exposure to clickstream data, user behavior analytics, and event tracking systems. Experience supporting evaluation of ML‑based ranking or retrieval systems. Familiarity with RAG systems or GenAI applications.
We are an equal opportunity employer: qualified applicants are considered for and treated during employment without regard to race, color, creed, religion, sex, national origin, citizenship status, disability status, protected veteran status, age, marital status, sexual orientation, gender identity, genetic information, or any other characteristic protected by law.
#J-18808-Ljbffr