Use the vitals package with ellmer to evaluate and compare the accuracy of LLMs, including writing evals to test local models.

The generative AI technology powering ChatGPT—OpenAI's GPT-3.5 and GPT-4 LLMs—is available to R users, with a growing collection of packages and apps to choose from. ChatGPT can answer questions about ...