Want to easily implement an open source LLM? Anyscale’s Aviary project takes off

 Want to easily implement an open source LLM?  Anyscale's Aviary project takes off

Join top executives in San Francisco July 11-12 to hear how leaders are integrating and optimizing AI investments for success. Learn more

Anyscale, the leading commercial vendor behind Ray’s open source machine learning (ML) scaling technology, is today launching the new open source Aviary project to help simplify the implementation of open source large language models (LLMs).

There are a growing number of open source LLMs, including Dolly, LLaMA, Carper AI, and Amazons LightGPT, along with dozens of others available for free on Hugging Face. However, simply having an LLM isn’t enough to make it worthwhile for an organization, the model still needs to actually be deployed across the infrastructure to enable inference and real-world usage.

Getting an open source LLM model deployed on infrastructure has often been a trial and error bespoke process as developers figure out the correct compute resources and configuration parameters. Also, it’s not easy for developers to simply compare one model to another. These are some of the challenges Anyscale is looking to solve with Aviary.

Every week, new open source models are released that people are trying out that are pushing the state of the art, Anyscale CEO Robert Nishihara told VentureBeat. Where there hasn’t been as much progress and what’s lagging behind in our view is the open source infrastructure to actually run those models.


Transform 2023

Join us in San Francisco July 11-12, where top executives will share how they integrated and optimized AI investments for success and avoided common pitfalls.

subscribe now

How Aviary works to facilitate open source LLM distributions

The Aviary project builds on the Ray open source project with a number of optimizations and configurations to facilitate LLM implementation of open source models.

Ray is already used extensively by large organizations for model training and is the technology that OpenAI uses for its models, including GPT-3 and GPT-4. The goal with Aviary is to automatically enable open source LLM users to deploy quickly with the right optimizations in place.

Nishihara explained that there are many different things that need to be configured from an infrastructure standpoint, including model parallel inference across multiple GPUs, sharding, and performance optimizations. The goal with Aviary is to have pre-configured defaults for essentially any open source LLM on Hugging Face. Users don’t have to go through a time-consuming process of figuring out infrastructure setup on their own; Aviary takes care of all of this for them.

Aviary also aims to help solve the model selection challenge. With the growing number of templates, it’s not easy for anyone to know the best template for a specific use case. Nishihara said that by making it easier to implement open source LLMs, Aviary is also making it easier for organizations to compare different LLMs. Comparisons enabled through Aviary include accuracy, latency, and cost.

As new LLMs emerge, Aviary will rapidly enable them

Aviary has been in private development at Anyscale for the past three months. It took some time initially to get the setup right for any open source LLM, but what has become clear is that there are common templates for implementation across all LLMs.

Nishihara said that when LightGPT became available, Aviary was able to add support for it in less than five minutes. You explained that there are a few different standard architectures that all open source LLMs conform to in terms of how they handle model parallelism and other critical aspects of deployment.

We don’t have to handle hundreds of special cases, Nishihara said. In fact, you only have to deal with each of the standard model architectures and then all the different LLMs fall into one of these categories.

Overall, Nishihara predicts that the number of open source models will only increase, and as a result, the problem of model selection will only become more difficult for organizations.

Our hope with Aviary is that because it’s open source, anyone in the community who wants to will be able to add new models easily, he said. This will make it easy for anyone using Aviary to simply distribute those templates without really having to do any extra work.

VentureBeat’s mission it is to be a digital city square for technical decision makers to gain insights into transformative business technology and transactions. Discover our Briefings.

#easily #implement #open #source #LLM #Anyscales #Aviary #project #takes

Previous articleCharacter.AI, a16z-powered chatbot startup, surpasses 1.7 million installs in first week
Next articlePeople with social anxiety are more likely to become overly dependent on conversational AI agents


Please enter your comment!
Please enter your name here