ALBUM

Work closely with ML engineers, data scientists, product managers, and engineering teams to improve AI capabilities across Filevine’s legal workflows

Translate legal tasks, documents, workflows, and user expectations into clear requirements for AI systems, prompts, retrieval pipelines, evaluations, and product behavior

Define what “good” looks like for legal AI outputs, including accuracy, completeness, grounding, citations, tone, usability, and appropriate handling of uncertainty

Review and evaluate model outputs across use cases such as case summarization, document analysis, deposition workflows, demand generation, medical chronology, intake, task automation, and matter-level Q&A

Create and maintain evaluation rubrics, golden answers, annotation guidelines, failure-mode taxonomies, and workflow-specific quality standards

Partner with data annotation teams to design review workflows, calibrate reviewers, resolve ambiguity, and improve the legal relevance and consistency of labeled data

Identify legal and workflow-specific edge cases, hallucination patterns, citation failures, missing context, and product behaviors that could reduce user trust

Provide structured feedback to ML and Product teams on model performance, product gaps, data quality issues, workflow friction, and opportunities for improvement

Help refine prompts, retrieval strategies, agent workflows, and human-in-the-loop review patterns for legal use cases

Strong communication and writing skills in English

JD or equivalent legal training, with practical experience in a law firm, legal department, government legal office, or legal operations environment

Strong understanding of legal workflows, legal documents, matter management, and how attorneys, paralegals, and legal staff work in practice

Ability to evaluate legal content carefully, identify missing context or inaccuracies, and explain why an AI output is or is not useful to a legal professional

Strong interest in AI, machine learning, legal technology, document intelligence, and workflow automation

Experience with AI tools, prompt engineering, evaluation workflows, document review, knowledge systems, or data annotation processes is a plus

Ability to create clear rubrics, examples, test cases, review guidelines, and structured feedback for technical and non-technical audiences

Comfort working closely with engineers, ML teams, product managers, designers, and data annotation partners in a fast-moving product development environment

Experience in litigation, personal injury, insurance defense, immigration, mass torts, corporate legal, or other high-volume legal workflows is a plus

Ability to meet periodically in our office, as we value in-person collaboration