💨 Abstract

Meta's AI model Maverick, ranked second on LM Arena, has raised concerns as it appears to differ from the version available to developers. Meta admitted that the LM Arena version is an "experimental chat version" optimized for conversationality, suggesting it may have been customized for the benchmark test. This practice is problematic as it can make it difficult for developers to predict the model's performance in specific contexts and may be misleading.

Courtesy: techcrunch.com

Summarized by Einstein Beta 🤖