Snowflake announced Arctic, their open 17b model. The LLM perfomance chart is replete with new offerings in just a few weeks. Overall knowledge performance is asymptoting as expected. It’s hard to discern the most recent dots.

One thing stands out from the announcement - the positioning of the model.

“The Best LLM for Enterprise AI”

Snowflake focuses on the model’s enterprise performance : SQL generation, code completion, & logic. This push will be echoed by others as models start to specialize.

When Databricks/Mosaic launched DBRX, their announcement focused on openness, training efficiency, & inference efficiency.

This is a shift from the general-purpose marketing of LLMs like GPT-4, Llama3, & Orca. Arctic & DBRX are B2B models.

With the MMLU general knowledge (high school equivalency) score now asymptoting, buyers of models will begin to care about different attributes. For a data-focused customer base, SQL generation, code completion for Python, & following instructions matter more than encyclopedic knowledge of Napoleon’s doomed march to Moscow.

The push to smaller models is another form of differentiation. Smaller models are much less expensive to operate : two orders of magnitude or more.

Meta’s most recent small model, Llama3 8b parameter, demonstrates small “over-trained” models can perform equally, as well as its larger brothers.

This is all good news for founders & users : more models, more choice, better performance across domains, lower costs for training, tuning, & inference. If the recent weeks are any indication, we should expect further flurries of advances & specialization.