QA Engineer (AI Forms Platform) ›
Filevine
Software Engineering, Data Science, Quality Assurance
United States
Posted on Jan 7, 2026
Responsibilities
- AI Model Validation: Design and maintain evaluation harnesses (evals) to measure the accuracy of AI auto-mapping models, ensuring they meet strict quality thresholds before release.
- Test Automation: Build robust, automated test suites (end-to-end and integration) that run within our CI/CD pipelines to catch regressions immediately.
- Golden Datasets: Maintain and expand "golden datasets" of perfectly mapped forms to serve as the ground truth for regression testing.
- Defect Prevention: Collaborate with developers to identify "model quality variance" and implement gates that prevent low-confidence predictions from reaching production.
- Migration Quality: Verify data integrity during the migration of pilot customers from the legacy platform to the new architecture.
- Efficiency: Leverage LLMs and AI coding assistants to rapidly write test scripts and generate edge-case scenarios for complex legal forms.
Qualifications
- Bachelor’s degree in Computer Science, Engineering, or related field.
- 3+ years of experience in QA Automation, with proficiency in tools like Cypress, Selenium, or Playwright.
- Coding Proficiency: Strong ability to write test scripts in JavaScript, TypeScript, or Python (Java/C++ proficiency also accepted).
- AI/Data Testing: Experience testing non-deterministic systems (AI/ML models) or complex data migration projects is highly preferred.
- Agile/Scrum: 1 to 3 years experience working in agile environments with frequent release cycles.
- Problem Solving: Demonstrated ability to use AI tools to solve coding roadblocks and optimize testing workflows.
- Communication: Excellent verbal and written communication skills to articulate quality risks to the product team.