June 21, 2024, by Brigitte Nerlich

How to do things with prompts: Magic words, speech acts and AI

Looking at what’s going on in AI sometimes makes me feel like the anthropologist Bronislaw Malinowski, When he first visited the Trobriand Islands of then British New Guinea about a century ago, he became fascinated by the magic words that the islanders used and the actions they were believed to perform. Visiting the land of AI, I became intrigued by the use of magical prompts and the actions they are believed to perform.

Prompt engineering

If you have ever interacted with an AI like ChatGPT, you will have found that it answers questions differently or reacts to requests differently depending on how you word the question or request. Not all words work; only some. Now a new profession, craft and art has emerged called ‘prompt engineering’ (or prompt design, or prompt tuning, or prompt optimisation).

“Prompt engineering is the process of structuring an instruction that can be interpreted and understood by a generative AI model” or a large language model or LLM. And a “prompt is natural language text describing the task that an AI should perform.” (wiki) By crafting the right instructions, you can ‘generate’ words, images, code, and so on. Or, as Emily Bender puts it: “Prompt engineering is figuring out strings to put into the LLM so that you like what comes back out”. (You can find a list of prompt examples here)

Experts in this new field also talk about zero-shot prompting, few-shot prompting, chain-of-thought prompting and much more. To get a feel for things, have a look at some of the examples of prompt engineering in this Prompt Playbook, or should it be spell book, published by Dove/Unilever. (Here the aim is to get image generators to not only generate unrealistic images of female beauty but realistic ones).

Prompt engineering has very quickly become a new domain of sometimes quite arcane expertise that allows those who are good at it to ‘manifest’ words and images seemingly out of thin air. This might appear like magic to non-experts.

That’s perhaps why some people have used the phrase ‘prompt conjurer’ instead of prompt engineer. One firm specialising in prompt engineering calls itself “Invoke AI”. And an article on prompts and control theory is entitled “What’s the magic word?” And that brings us back to Malinowski.

Magic words

In 1925 Malinowski published an article “Magic, Science and Religion”. In this essay he stressed that language can only be understood as a mode of action in a situation. Magic words and formulae uttered as part of magic rituals were of particular interest to him, especially the “use of words which invoke, state, or command the desired aim” (p. 54). That sounds a bit promptish to me.

Malinowlski also pointed out that like “the other arts and crafts, [magic] is also governed by a theory, by a system of principles which dictate the manner in which the act has to be performed in order to be effective” (p. 66). This reminds me strongly of prompt engineering. The system of principles for engineering prompts is now available to AIs themselves and AIs can now, it seems, become prompt engineers themselves – double magic.

A few years earlier, in 1922, Malinowski had published his famous book Argonauts of the Western Pacific. Here he defines ‘magic’ as “an instrument serving special purposes, intended for the exercise of man’s specific power over things” (p. 432). Prompt engineers certainly want to achieve power over LLMs and bend them to their will. (Although how that works when prompt engineering is automated, I don’t know)

In a way, what prompt engineers really want is to find magical words that are ‘performative’ that have magical ‘force’, words that “are accepted as potentially creative of acts. You utter a vow or you forge a signature and you may find yourself bound for life to a monastery, a woman or a prison” (Coral Gardens and their Magic, 1935, vol. 2, p. 53)

Some decades after Malinowski, the philosopher of language John Langshaw Austin wrote a book entitled How to do Things with Words in which he explored the use of such magical words and formulae in ordinary language.

Prompts and speech acts

Austin’s work contributed to the emergence of what became in linguistics ‘speech act theory’ where linguists study the use of utterances as actions in real life. One type of sentence fascinated Austin in particular. That was the explicit performative speech act, such as “I now pronounce you man and wife”. When a person invested with some authority says these words to two people in a particular context, these people then really are man and wife – magical! Here words really have magical power – they really do perform actions.

I am not sure whether prompts have such power yet. At the moment they are more like ordinary ‘speech acts’ or ‘directive speech acts’, such as questions or requests. It is not surprising then that linguists in the Austinean tradition have started to look at prompts, and their colleagues in computational linguistics are also getting in on the (speech) act.

As Doris Dippold said in a blog post: “In the linguistic sense, a prompt is a directive speech act. We use directive speech acts to, for example, ask a family member to open a window, or a colleague to give us a pen if we have lost ours.” She also asks a number of interesting questions, such as: Do “people who write prompts in a second language use different strategies than those who use their first language? – What output do the prompts of different kinds of writers (e.g., native vs. non-native speakers, novices vs. experienced writers) generate?” etc.

When searching a bit more, I found that people now even talk about ‘prompt linguists’! And here we are back to magic! Listen to this: “An easy metaphor to help you understand the role of a prompt linguist is to think of an AI model like a magic genie that has the ability to grant any wish you ask, but asking the wrong question can fulfill wishes that you didn’t want answered.”

So, now we have the magician or conjurer on the one hand crafting their most effective Abracadabras, and AIs, especially LLMs, on the other that act as magic genies!

Beware the lure of magic

It might appear that prompt engineers or prompt magicians use a sacred language to which the non-initiated have no access, just as they have no access to the inner workings of the sacred LLMs themselves. But that is what happens with all advanced domains of expertise, knowledge and technologies. As Arthur C. Clarke said: “Any sufficiently advanced technology is indistinguishable from magic.”

So, don’t be too bedazzled by the magic. And don’t ever think that what is invoked or conjured up or created has something to do with ‘the truth’. It has not. It’s important to remember that a bit like rabbits out of a hat, what prompts ‘invoke’ might not be what it seems; it might, indeed, be bullshit.

Prompts are the outcome of mundane but expert word craft. Their results have to be taken with a grain of salt. Prompt engineers are not magicians and LLMs are not genies.

PS: On a related topic, I just remembered an older blog post from 2016: “The ghost in the machine: Of automation, algorithms and AI”!

PS: November, 2024: What seemed magic a few months ago, is now mundane. You can now get blueprints for prompts! But I’d still love to know about how ordinary people like me keep up with such developments.

Image: Briam-Cute, Pixabay

• Share on Facebook
• Share on Twitter
• Share on Google+
• Share on LinkedIn