Overview
Expert in AI models, responsible for selecting, tuning, and optimizing the large language models powering the Copilot.
Responsibilities
* Model Selection: Research and determine which large language model(s) should power the Copilot for various tasks. Evaluate options such as OpenAI's GPT-4/GPT-3.5, Microsoft's proprietary models (if available), or open-source models like LLaMA 2 or others. Consider factors like accuracy in legal language understanding, context window size (for large documents), cost per call, latency, and data privacy implications (cloud vs on-premises models). Make recommendations if a mix of models is ideal (for example, using a smaller, faster model for simple tasks and a larger one for complex queries).
* Model Tuning & Training: Oversee fine-tuning processes to adapt models to Epiq's domain. This could involve curating a training dataset of legal questions and answers or documents, and then using it to fine-tune an LLM for improved performance on those tasks. Ensure that fine-tuning avoids overfitting or unintended biases and validate that the fine-tuned model performs better on target scenarios than the base model.
* Prompt Optimization: Collaborate with the AI Instruction Architect to optimize prompts and system messages from a technical standpoint. Analyze prompting strategies or lengths, experiment with few-shot learning, and develop a library of prompt strategies to improve reliability and efficiency.
* Performance & Cost Management: Monitor model performance metrics and computational costs. Optimize parameters (e.g., temperature, max tokens) to balance output quality and length; consider response caching and cost-aware scheduling for cloud APIs when applicable.
* Hybrid AI Solutions: Design architectures where the LLM is augmented by other components (e.g., Retrieval-Augmented Generation with a vector store) to ground answers with up-to-date information. Include smaller specialized models or rule-based components for triage before selecting an LLM or prompt.
* Staying Current: Keep up with AI field developments, test new advancements, and plan upgrade paths when beneficial (e.g., newer GPT-4 versions or open-source parity for the domain).
* Guidance & Governance: Provide input on model capabilities and risks, mitigate biases, implement content filters, ensure license/compliance adherence, and document model choices for transparency and auditing.
* Consultant & Internal Roles: An external LLM Strategist may audit needs and data, set up initial best practices, and hand off to internal teams; an internal LLM Strategist will continuously tune models and stay aligned with latest AI advancements.
* Model Development & Infrastructure: Use Python for development; leverage Jupyter notebooks or VS Code; libraries like Hugging Face Transformers, PyTorch/TensorFlow; and consider OpenAI API/SDK as applicable. Include data processing with pandas and NLP libraries; for large corpora, use distributed processing with Spark or Dask. Track experiments with MLflow or Azure ML; manage code and artifacts with Git.
* Vector Databases & Collaboration: Use vector stores (e.g., Azure Cognitive Search, Pinecone, Weaviate, ElasticSearch) and document embedding for retrieval-augmented generation. Track decisions in Azure DevOps/Jira and document user guides and limitations for stakeholders.
* Guardrails & Responsible AI: Apply bias detection tools and content filtering; consider interpretability tools where appropriate; ensure safety and ethical considerations are addressed.
Qualifications & Experience
* Educational Background: Master’s or PhD in Computer Science, Data Science, Machine Learning, or related field; strong NLP focus preferred.
* AI/ML Experience: 5+ years in machine learning, with several years in NLP or language models; experience with training, evaluation, and deployment; familiarity with metrics like perplexity and BLEU, and awareness of overfitting and data leakage.
* Programming & ML Frameworks: Expert Python; experience with TensorFlow or PyTorch; ability to fine-tune transformers and use ML Ops tools (e.g., MLflow).
* Algorithmic Knowledge: Understanding of transformer architectures, embedding techniques, vector similarity search, and retrieval-based methods.
* Analytical & Evaluation Skills: Strong evaluation methodology, including quantitative and qualitative analysis; ability to design evaluation sets that reflect real-world queries.
* Communication: Ability to explain complex AI concepts to non-experts and align stakeholders on model choices and expectations.
* Preferred Domain & Certifications: Domain experience with legal, medical, or finance data; open-source contributions; publications or patents; scaling AI systems; team leadership and cross-functional collaboration. Certifications such as Microsoft Certified: Azure AI Engineer Associate, Google Professional Machine Learning Engineer, Stanford AI/Machine Learning Certificates, or other data science/NLP certifications are beneficial but not critical; advanced degree preferred.
#J-18808-Ljbffr