Welcome to my newsletter, which I call Drop-In Class because each edition is a short, fun intro class for technology concepts. Except unlike many instructors, I'm not an expert yet: I'm learning everything at the same time you are. Thanks for following along with me as I "learn in public"!
Why the open source discussion matters right now
MIT Tech Review called it a “fool’s game” to try to predict the next AI trends, since the industry is moving so fast.
I concur. I send this newsletter out biweekly, and every two weeks is always enough time for the next topic to pop up. We’ve had agents and small language models become all the rage in a short time. What’s next? Who knows!
But I can tell you what’s happening today. As you’ve probably seen, DeepSeek (the Chinese AI developer making headlines) has thrown the industry for a loop with a crazy-efficient open source model. Now the conversation is about lower compute costs, and the rise of open source LLMs.
The rate of adoption for open models is rapidly increasing. For instance, Meta’s open models have now been downloaded more than 400 million times at a rate 10 times higher than last year, with usage doubling from May through July 2024.
So let’s get into the pros and cons of open source models, and why adoption is surging after a slower start. Today’s on-topic song is “Leave the Door Open!”
What’s the difference between an open source and a closed source model?
Open source means that a technology’s code and data are available for anyone to access, use and modify. Lots of open source projects become the standard for a technology category: the Linux operating system, Git version control, the MySQL database, and so on.
An open source AI model has publicly accessible code and architecture, so you can see how it was trained and how it works.
What are the advantages of open source models? The three Cs:
Customization: With open source models, there’s more flexibility to tinker. You can go into the code and understand why a model produces certain outputs, making it easier to fine-tune the model and improve its responses.
Control: Enterprises don’t want to share their proprietary data. Especially if you’re a hospital or a bank. Open source models can be self-hosted, meaning everything stays in the enterprise’s environment.
Cost: Open source models are generally free to use. You aren’t paying for inference the way you would with a closed source model — as your use cases get bigger, you probably don’t want to keep paying someone else for all those generated responses.
And here is the biggest drawback to open models: Performance. Closed source models typically perform better than open models on benchmarking tests.
Closed source models are pretty self-explanatory: they don’t reveal the source code, you can’t change anything, and you have to pay a company. But they are currently the highest-performing models, and they offer dedicated support from the company.
So closed source models like OpenAI’s GPT-4 and Anthropic’s Claude are the most widely adopted. But change is afoot.
Closed models are in the lead, but open models are quickly gaining popularity.
In 2023, closed-source models had 80%–90% of the market share, the majority going to OpenAI, according to an a16z study. (Isn’t it funny that OpenAI is called OpenAI but it isn’t open source? This didn’t occur to me until just now.)
But 46% of the respondents said they prefer open-source models, and 60% were interested in switching to more open-source usage.
Even though closed-source models perform better, enterprises like open-source models because they’re easier to customize for their specific use cases. It’s one thing to have the highest performance for external industry benchmarks. But what about for a company’s internal benchmarks? Or developer preferences? Or benchmarks by use case?
Those all mattered more than external benchmarks in the a16z study. Which makes sense considering enterprises are worried about getting their use cases into production and showing ROI, not about having the highest-performing model.
Here’s a post that sums it up from an interesting Reddit thread on the topic (I like Reddit to get the developers’ point of view on things):
And performance doesn’t matter so much when all the models are getting pretty good. The capabilities of open models are catching up to the closed models. DeepSeek-R1, the open model making all the fuss this week, was a dramatic signal of this shift. It sends a strong message that open source models can be quite powerful, and the proprietary models need to differentiate themselves on other things.
The upshot of all this? We’ve got options. The competition is heating up.
I started this post talking about how it’s a fool’s game to make any predictions about AI, but here’s one I’ll throw out there that seems like a safe bet after doing all this reading: Like many technologies in the past, an open ecosystem will be the future for GenAI, and that will probably be a good thing for innovation.
The cooldown: Extra reading
This is a 101-level drop-in, so I keep things at a high level! For deeper reading, check out these articles.
The enterprise verdict on AI models: Why open source will win (VentureBeat)
The Biggest Winner In The DeepSeek Disruption Story Is Open Source AI (Forbes)
2023, year of open LLMs (Hugging Face)
16 Changes to the Way Enterprises Are Building and Buying Generative AI (Andreessen Horowitz)
Shifting Tides: The Competitive Edge of Open Source LLMs over Closed Source LLMs (Towards Data Science)
See you in the next class!
Cheers,
Alex