April 21, 2023, by Brigitte Nerlich

LLaMas, Alpacas and Dolly 2.0: Exploring an emerging AI menagerie

There was a time when llamas were llamas and small furry creatures from Alpha Centauri were small furry creatures from Alpha Centauri. Now we have LLaMas, Alpacas and AI programmes that impersonate cloned sheep. I’ll first say something about ChatGPT, the AI chatbot that launched a hundred fury creatures, then something about another chatbot that I found useful, and then I’ll inspect the animals a bit.

ChatGPT

I have written three blog posts about ChatGPT and I thought that was it for a while. But no, new AI products and speculations, for example that Elon Musk will jump on the band wagon, are everywhere.

Just to recapitulate: ChatGPT is a large, and so far, the most famous, language model or LLM (the new PPE for the post-covid generation) which is trained on billions of words sucked up from the internet and is then fine-tuned by human feedback. It identifies patterns in text and can generate output that looks as if it was written by a human and it can also produce or help with computer coding.

An LLM “is an AI model that has been trained to predict the next word or words in a text based on the preceding words”. ChatGPT was released in November 2022 and was created by OpenAI. It was followed by a new Bing chatbot, created by Microsoft and Bard, created by Google (designed on top of Google’s current AI language model, BERT).

Now, in April 2023, two new so-called AIs have come to my attention – but there are more, and with them we enter an AI zoo. But first something about an AI that I just like, but which has no direct animal connections.

Phind

ChatGPT had, in my view, two major limitations: it did not provide references and if you asked it for references, it hallucinated them. Now there is Phind (phind.com), a new AI/GPT search engine that does not only answer questions, but provides references. You can ask it to answer your questions in expert or concise or creative mode, and it makes suggestions for follow-up questions, which is really useful. It also seems to be an excellent coding tutor – not something I understand. It’s also up to date, as there is no cut-off point to the input training data. It was launched at the end of February.

Dolly 2.0

And so, we come to the next AI which launched in the middle of April. This is Dolly 2.0, “the first open-source, instruction-following LLM that’s available for commercial use & doesn’t require you to pay for API access or share data with third parties”, according to a tweet by its creator Databricks.

When I looked at the tweet, I was a bit surprised to see a logo of an abstract sheep face winking at me. Reports on the new AI feature quite a lot of ‘real’ sheep. One article is illustrated by an image created by the famous image generating AI called Midjourney which gives Dolly super-cool glasses…. We shall come across the author of the article and the glasses quite a few times in the following.

In the distant past, twenty or more years ago, I watched the creation and media spread of the real Dolly the sheep, the then so controversial first cloned mammal. What has this to do with ChatGPT-like AIs, I wondered. And what about the 2.0? Let’s start with the 2.0. This Dolly follows in the footsteps, or hoof-steps, of a previous Dolly, Dolly 1.0, that appeared on the AI scene in March 2023. And so we come to the sheep.

In a blog post published on 24 March, the Databricks team said that: “We’re calling the model Dolly — after Dolly the sheep, the first cloned mammal — because it’s an open source clone of an Alpaca, inspired by a LLaMA. We’re in the earliest days of the democratization of AI for the enterprise, and much work remains to be done, but we believe the technology underlying Dolly represents an exciting new opportunity for companies that want to cheaply build their own instruction-following models.” I have to say that this sheep which is a clone of an alpaca is a new feat in cloning!

Democratising AI sounds great and the title of the blog post, “Hello Dolly: Democratizing the magic of ChatGPT with open models”, makes that clear, alluding to a famous romcom film from 1969, “Hello, Dolly” and embedding the AI even more nicely in cultural associations. I won’t discuss the issue of democratisation here, but somebody else probably should.

Llamas, alpacas and vicunas

But what about those animals mentioned when talking about Dolly the clone: llamas and alpacas? Associations are fine, if you understand them. I did not understand the references to llamas and alpacas! Oh, and there are also koalas; about that later.

LLaMA and Alpaca  are, apparently, “well-known instruction-following models”. Well-known to some. This is a whole new language and a whole new world to me – fortunately online chat about this AI is populated with images of cute sheep and lambs, thus making this new world seem more familiar! Interestingly, an article about all this by Jesus Rodriguez is illustrated with a llama wearing super-cool almost steam punk goggles – an image generated by Midjourney. But I wanted to know a bit more about the language models.

I had found that Phind does searching for info as well as I do, if not better. So, I went straight in and asked about LLaMA. It told me that “LLaMA (Large Language Model Meta AI) was created and publicly released by Meta (previously known as Facebook) as part of their commitment to open science ai.facebook.com. This state-of-the-art foundational large language model was designed to help researchers advance their work in the subfield of AI.” (If you go and ask Phind yourself, you’ll get the sources for that info!).

As for Alpaca: “Alpaca AI was created by researchers at the Stanford Center for Research on Foundation Models (CRFM). […] The Alpaca model is based on Meta AI’s LLaMA 7B model and was fine-tuned by the researchers using a dataset generated from OpenAI’s text-davinci-003 or GPT-3.5 stanforddaily.com.” Oh and if you look here, you’ll see that the Alpaca language model has a cool logo of an alpaca wearing sunglasses. It’s even cooler in this image illustrating yet another article by Jesus Rodriguez.

And… there is even a language model called Vicuna, based on the name of the wild relative of the llama (which has the softest wool!), but I won’t go into that, other than to say that (like Dolly) the model builds on LLaMA and Alpaca. In this article, its is represented by an abstract rendition of a Vicuna head, generated by another AI image making machine, namely Stable Diffusion. Of course, it also gets its own cool shades in an image by Midjourney used in an article by you guess who… so perhaps glasses are Rodriguez’ AI writing signature, who knows – they are still cool…

Koalas

What about koalas? Koala is “a chatbot created by Berkeley AI Research(BAIR) that fine-tunes LLaMA on conversations gathered from the internet”. An article about it (by you know) is illustrated with an image generated by Midjourney of a bespectacled koala wearing a deerstalker, à la Sherlock Holmes! But I also found a YouTube video about Koala featuring, quite independently, a koala wearing sun glasses …

The AI zoo

It seems that LLaMA is a foundational language model. Alpaca is based on LLaMA and Vicuna is based on LLaMA and Alpaca. Koala is a chatbot that fine-tunes LLaMA, and Dolly is an open source clone of an Alpaca, inspired by a LLaMA. So it’s llamas all the way down, do to speak. All these language models are illustrated by their respective animal counterparts and AI image generating machines love to imagine them as wearing cool sunglasses, aviator glasses or goggles. This is no coincidence, as llamas wearing glasses is, it seems, a thing, as seen here or here. It’s a meme.

If one had time, one could do a whole iconography of large language models and the role of AI image generators in that process….. but I’ll leave that for future researchers. By they way, while I found out that Midjourney and Stable Diffusion created images for the AI menagerie of language models, OpenAI’s DALL·E 2 has also been busy on the llama wearing glasses front, depicting a llama with spectacles reading a book in a coffee shop…………………..

Image: My own personal llama from Machu Picchu, brought to me by my former student Carlos Cuevas Garcia (it has mislaid its sunglasses, I am afraid)

• Share on Facebook
• Share on Twitter
• Share on Google+
• Share on LinkedIn