Auto-Evaluator Opportunities

Lance Martin's team has open-sourced an auto-evaluator tool and launched a free hosted app and API to assess and improve question-answering chains built with LangChain. The tool auto-generates test sets from documents and grades QA outputs to help optimize model and retriever settings. Current limitations include slow file transfer speeds and challenges with large files containing images.