The fact that Metas models have managed to be less accessible than fully closed, API-only offerings makes me really struggle to see the value here. If you’re willing to jump through corporate hoops, OpenAI can have you up and running in a few minutes, with few relevant restrictions for most businesses. If you’re willing to trade performance and ease of use for control, then there are plenty of open source models with decent performance you can fine-tune. Metas models are like the worst of both worlds - subject to restrictive licensing and approval with limited control (at least legally), but requiring a fair bit of elbow grease to get working compared to just hitting an API.
There are two clauses in the llama license that I’m aware of that are restrictive:
1) if you have more than 700M daily active users you can’t use it.
2) you can’t use the output of this model to train other models.
Are you aware of others, cause I’m not, and those aren’t unreasonable.
The TOS also outlines a number of restricted use cases, which are broader than other models. For example, Llama prohibits any ITAR related use, while Anthropic limits specific usage related to defense and weapons. (There are many peaceful technologies that are covered by ITAR).
Actually - fair enough, I think I was overestimating how much people consider the ability to use the output to train your own models important. I guess for me these felt really restrictive since that always felt like the end game - collect a bunch of data, then train a smaller, cheaper to run model for a specific use case - but I’m clearly an outlier!
I thought llama is open weights and open to commercial with reasonable restrictions. I haven't heard practitioners who fine tune llama having trouble with it
What on earth are you talking about. Metas models are the foundations for almost all the popular open source LLMs.
In what sense are they foundations? Are they using the literal code and models? Or just using the architecture and reimplementing and retraining the new models?
Anyone know if it will be open sourced?
Of course not.
At most, it'll be like the previous one. But not open source.