'Function calling', OpenAI's API new feature.
Well dear reader, I woke up this morning, checked the news and I got quite excited, OpenAI had decided to shake things up again. They've updated their LLM and one of the stand out features was the ability to add "functions" flaunting it as a 'more reliable way' to pair GPT's capabilities with external calls. Reliable, they say? I've tinkered around with it, and for now, it's playing nice. But only time and a healthy serving of complex use cases will tell if it's more than just a pretty feature.
First articles straight from the source, please read it.
https://platform.openai.com/docs/guides/gpt/chat-completions-api
https://openai.com/blog/function-calling-and-other-api-updates
The rest of the information I had about this came in two separate emails (GPT3-5, GPT4 versions) that got sent to those who are signed up to the developer API.
But it's not all sunshines and rainbows. There's manual work involved too. You see, when the model calls that function, it's up to you to manually respond—This is an API call after all. A small price to pay for the power it can provide.
So, to test this wonder of modern tech, I, being the daring visionary that I am, put together a quick "Pokemon description" example. That's right, you heard me. Out of the infinity of applications that this feature could have, I chose to have a chat about Pokemon. Shocking, isn't it? But bear with me the Pokemon API is a nice API to play and practice coding techniques on.
Before getting into the code, I want to talk about what did not go as expected.
First what I noticed while I was doing my little example was OpenAI's endpoint is overwhelmed and would occasionally get an error like the following, it's a bit scary to have the possibility for such problems to occur in a production environment, so consider this a warning.
data: {
error: {
message: 'The server had an error processing your request. Sorry about that! You can retry your request, or contact us through our help center at help.openai.com if you keep seeing this error. (Please include the request ID [REDACTED] in your email.)',
type: 'server_error',
param: null,
code: null
}
}
isAxiosError: true,Another thing I expected was that the new models would be capable of generating the code needed to implment, but this was not the case. I ended up using their example and modifying it for my use case. Even by giving the LLM a single shot example it failed, all I had was the example provided but GPT4 completely lost the plot and went off on a tangent generating unrelated code.
My demo/example code is written in Typescript, and yes I know I could add Types to the endpoint response but I will be skipping that in this example.
Into the code:
Before diving into the OpenAI code, I will first prepare the Pokemon API endpoint call. You can call this as is so there is no dependency on OpenAI's code. Simple code crudely written quickly for the sake of this example, all I care about is the returning array of descriptions.
const getPokemonDescription = async (pokemonNameOrNumber: string) => {
try {
// Since I know the pokemon's name has to be lowercase, for the sake of the example I am just cheating and converting it to lowercase.
// Get the pokemon's description, and filter out the non-english ones
const res = await fetch(
`https://pokeapi.co/api/v2/pokemon-species/${pokemonNameOrNumber.toLowerCase()}`
);
const data = await res.json();
const flavorText = data.flavor_text_entries
.filter((entry) => entry.language.name === "en")
.map((text) => text.flavor_text);
// This is an array of descriptions
return flavorText;
} catch (err) {
console.log("Pokemon endpoint error:", err);
return Promise.reject([]);
}
};Time to dive into the OpenAI code, first initialize our model. NOTE: the base/current model won't have this as of the time of writing, the model transition will only occur on 2023/06/23
import { Configuration, OpenAIApi } from "openai";
const configuration = new Configuration({
apiKey: process.env.OPENAI_API_KEY,
});
const openai = new OpenAIApi(configuration);
const model = "gpt-3.5-turbo-0613";
// const message = "Tell me about the pokemon Ditto";
const message = "Tell me about the pokemon Mew";
// const message = "Tell me about the pokemon V";
// const message = "I am chippin in";Right now let's look at instructing it about the Pokemon endpoint. Our message here is the variable's defined in the previous code snippet with various comment out states you may wish to try. For simplicity this is code only and not accepting an input from the terminal or endpoint.
Although I did not do it here, I think that the function name should be a const or an enum so it reduces the chance to make a spelling mistake, since I based this off their example I just used there object type parameter where in this case the simplest would have just been to accept a string as a parameter. To run out function the Pokemon's name or number is required otherwise the endpoint would just fail. This seems to be an instruction toward the LLM which it will use and handle for us.
const chatCompletion = await openai.createChatCompletion({
model: model,
messages: [{ role: "user", content: message }],
functions: [
{
name: "getPokemonDescription",
description: "Get the description of a pokemon",
parameters: {
type: "object",
properties: {
pokemonNameOrNumber: {
type: "string",
description: "The pokemon's name or number",
},
unit: { type: "string" },
},
required: ["pokemonNameOrNumber"],
},
},
],
function_call: "auto",
});Now if this needs to make a functional call, as I mentioned earlier we need to handle this ourselves, it won't know what to do with it, so let's crudely prepare that logic as well.
// In this example I am only supporting "getPokemonDescription" function calls
if (
chatCompletion.data.choices[0].message?.function_call &&
chatCompletion.data.choices[0].message?.function_call?.name ===
"getPokemonDescription"
) {
const args = JSON.parse(
decodeURI(
chatCompletion.data.choices[0].message?.function_call?.arguments ?? ""
)
);
try {
const answer = await getPokemonDescription(args.pokemonNameOrNumber);
const pokedexCompletion = await openai.createChatCompletion({
model: model,
messages: [
{
role: "system",
content: `You are a pokedex. Please read the following array of description for the pokemon '${args.pokemonNameOrNumber}' and respond with a summary.`,
},
{
role: "function",
name: "getPokemonDescription",
// Remember the answer is an array of descriptions, but this wants a string,
// I am cheating here by adding the "Response data:" part.
content: `Response data: ${answer}`,
},
],
});
console.log(pokedexCompletion.data.choices[0].message);
} catch (err) {
const pokedexCompletionFail = await openai.createChatCompletion({
model: model,
messages: [
{
role: "system",
content: "You are a pokedex, but you failed to find the pokemon.",
},
],
});
console.log(pokedexCompletionFail.data.choices[0].message);
}
}Since the rule written earlier stated we wanted an object as the input, we are receiving an object from the LLM. In my case I wanted a string so I am parsing the String response into JSON. For the curious the response of the step is
{
role: 'assistant',
content: null,
function_call: {
name: 'getPokemonDescription',
arguments: '{\n"pokemonNameOrNumber": "Mew"\n}'
}
}Once in JSON the Pokemon's name or number is accessible as args.pokemonNameOrNumber . This get's passed to our endpoint and the code waits for the response to complete, If it's successful it should be an array of the Pokemon's descriptions throughout the Pokemon game series.
Next comes the fun part, the code is now aware of the response, we need to inform OpenAI of this array, but that alone would just be boring, since I have control here I am also able to add the new System prompt information GPT that it is a Pokedex and should process the instruction in that manner. Quite simple, you may put this in ChatGPT and provide an array of Pokemon descriptions and it will answer with a summary. I would also like to point out the catch statement here. Here we are giving it the system prompt that it failed and it may reply as it see's fit for the role of a Pokedex to the user.
I will provide an example of the array at the end of the article, but below is the "instruction" in case you'd like to experiment with it in ChatGPT.
You are a pokedex. Please read the following array of description for the pokemon '${args.pokemonNameOrNumber}' and respond with a summary.
Right time to look at the responses:
Mew
// Tell me about the pokemon Mew:
{
role: 'assistant',
content: "Summary: Mew is an extremely rare and elusive Pokemon that is often considered a mirage by experts. Only a few people worldwide have claimed to have seen it. When viewed through a microscope, Mew's hair can be seen, which is described as short, fine, and delicate. It is believed that Mew only appears to those who are pure of heart and have a strong desire to see it. Mew is said to possess the genetic codes of all Pokemon, allowing it to learn and use any move. This has led some scientists to believe that Mew may be the ancestor of all Pokemon. Mew is also capable of making itself invisible at will, which helps it avoid being noticed even when it approaches people. Overall, Mew is a mysterious and powerful Pokemon with a connection to the genetic composition of all Pokemon."
}Ditto:
// Tell me about the pokemon Ditto
{
role: 'assistant',
content: 'Ditto is a Pokémon with the ability to transform into anything it sees. It can instantly copy the genetic code of an enemy and transform itself into a duplicate of that enemy. However, if it relies on its memory to transform into something, it may get the details wrong. When it encounters another Ditto, it will move faster than normal to duplicate that opponent exactly. Ditto can reconstitute its entire cellular structure to change into what it sees, but it returns to normal when it relaxes. Despite its transformation ability, Ditto does not get along well with its fellow Ditto.'
}When it fails or can't find information for that "Pokemon":
// Tell me about the pokemon V
{
role: 'assistant',
content: "I'm sorry, but I couldn't find any information on the Pokémon you're looking for. It's possible that this Pokémon is either very rare or it doesn't exist in the current Pokémon database. Can you please double-check the name or provide some more details about this Pokémon?"
}And finally just it reacting to normal chat - meaning it has some idea when to use the custom function or not
// I am chippin in
{ role: 'assistant', content: 'Great! How can I assist you today?' }I'll admit, for a simplistic use case and considering we're dealing with the GPT3-5 model, it's not half bad.
So there you have it. If that doesn't get your blood pumping, I don't know what will. The possibilities with this is vast have fun and it does introduce potential security issues, but remember life's not a straight line. Code dumped below
import { Configuration, OpenAIApi } from "openai";
const configuration = new Configuration({
apiKey: process.env.OPENAI_API_KEY,
});
const openai = new OpenAIApi(configuration);
const model = "gpt-3.5-turbo-0613";
// const message = "Tell me about the pokemon Ditto";
const message = "Tell me about the pokemon Mew";
// const message = "Tell me about the pokemon V";
// const message = "I am chippin in";
const getPokemonDescription = async (pokemonNameOrNumber: string) => {
try {
// Since I know the pokemon's name has to be lowercase, for the sake of the example I am just cheating and converting it to lowercase.
// Get the pokemon's description, and filter out the non-english ones
const res = await fetch(
`https://pokeapi.co/api/v2/pokemon-species/${pokemonNameOrNumber.toLowerCase()}`
);
const data = await res.json();
const flavorText = data.flavor_text_entries
.filter((entry) => entry.language.name === "en")
.map((text) => text.flavor_text);
// This is an array of descriptions
return flavorText;
} catch (err) {
console.log("Pokemon endpoint error:", err);
return Promise.reject([]);
}
};
const chatCompletion = await openai.createChatCompletion({
model: model,
messages: [{ role: "user", content: message }],
functions: [
{
name: "getPokemonDescription",
description: "Get the description of a pokemon",
parameters: {
type: "object",
properties: {
pokemonNameOrNumber: {
type: "string",
description: "The pokemon's name or number",
},
unit: { type: "string" },
},
required: ["pokemonNameOrNumber"],
},
},
],
function_call: "auto",
});
console.log(chatCompletion.data.choices[0].message);
// In this example I am only supporting "getPokemonDescription" function calls
if (
chatCompletion.data.choices[0].message?.function_call &&
chatCompletion.data.choices[0].message?.function_call?.name ===
"getPokemonDescription"
) {
const args = JSON.parse(
decodeURI(
chatCompletion.data.choices[0].message?.function_call?.arguments ?? ""
)
);
try {
const answer = await getPokemonDescription(args.pokemonNameOrNumber);
const pokedexCompletion = await openai.createChatCompletion({
model: model,
messages: [
{
role: "system",
content: `You are a pokedex. Please read the following array of description for the pokemon '${args.pokemonNameOrNumber}' and respond with a summary.`,
},
{
role: "function",
name: "getPokemonDescription",
// Remember the answer is an array of descriptions, but this wants a string,
// I am cheating here by adding the "Response data:" part.
content: `Response data: ${answer}`,
},
],
});
console.log(pokedexCompletion.data.choices[0].message);
} catch (err) {
const pokedexCompletionFail = await openai.createChatCompletion({
model: model,
messages: [
{
role: "system",
content: "You are a pokedex, but you failed to find the pokemon.",
},
],
});
console.log(pokedexCompletionFail.data.choices[0].message);
}
}
Pokemon description in array form
Ditto:
[
'Capable of copying\n' +
"an enemy's genetic\n" +
'code to instantly\ftransform itself\n' +
'into a duplicate\n' +
'of the enemy.',
'Capable of copying\n' +
"an enemy's genetic\n" +
'code to instantly\ftransform itself\n' +
'into a duplicate\n' +
'of the enemy.',
'When it spots an\n' +
'enemy, its body\n' +
'transfigures into\fan almost perfect\n' +
'copy of its oppo\n' +
'nent.',
'It can transform\n' +
'into anything.\n' +
'When it sleeps, it\fchanges into a\n' +
'stone to avoid\n' +
'being attacked.',
'Its transformation\n' +
'ability is per\n' +
'fect. However, if\fmade to laugh, it\n' +
"can't maintain its\n" +
'disguise.',
'When it encount\n' +
'ers another DITTO,\n' +
'it will move\ffaster than normal\n' +
'to duplicate that\n' +
'opponent exactly.',
'DITTO rearranges its cell structure to\n' +
'transform itself into other shapes.\n' +
'However, if it tries to transform itself\finto something by relying on its memory,\n' +
'this POKéMON manages to get details\n' +
'wrong.',
'DITTO rearranges its cell structure to\n' +
'transform itself into other shapes.\n' +
'However, if it tries to transform itself\finto something by relying on its memory,\n' +
'this POKéMON manages to get details\n' +
'wrong.',
'A DITTO rearranges its cell structure to\n' +
'transform itself. However, if it tries to\n' +
'change based on its memory, it will get\n' +
'details wrong.',
'It can freely recombine its own cellular\n' +
'structure to transform into other life-\n' +
'forms.',
'Capable of copying an opponent’s genetic\n' +
'code to instantly transform itself into a\n' +
'duplicate of the enemy.',
'It has the ability to reconstitute\n' +
'its entire cellular structure to\n' +
'transform into whatever it sees.',
'It has the ability to reconstitute\n' +
'its entire cellular structure to\n' +
'transform into whatever it sees.',
'It has the ability to reconstitute\n' +
'its entire cellular structure to\n' +
'transform into whatever it sees.',
'It can transform into anything.\n' +
'When it sleeps, it changes into a\n' +
'stone to avoid being attacked.',
'Its transformation ability is perfect.\n' +
'However, if made to laugh, it\n' +
'can’t maintain its disguise.',
'It has the ability to reconstitute\n' +
'its entire cellular structure to\n' +
'transform into whatever it sees.',
'It has the ability to reconstitute\n' +
'its entire cellular structure to\n' +
'transform into whatever it sees.',
'It can reconstitute its entire cellular\n' +
'structure to change into what it sees,\n' +
'but it returns to normal when it relaxes.',
'It can reconstitute its entire cellular\n' +
'structure to change into what it sees,\n' +
'but it returns to normal when it relaxes.',
'It has the ability to reconstitute its entire cellular\n' +
'structure to transform into whatever it sees.',
'It can freely recombine its own cellular structure to\n' +
'transform into other life-forms.',
'Ditto rearranges its cell structure to transform itself into other\n' +
'shapes. However, if it tries to transform itself into something\n' +
'by relying on its memory, this Pokémon manages to get\n' +
'details wrong.',
'Ditto rearranges its cell structure to transform itself into other\n' +
'shapes. However, if it tries to transform itself into something\n' +
'by relying on its memory, this Pokémon manages to get\n' +
'details wrong.',
'It can reorganize its cells to make itself into a\n' +
'duplicate of anything it sees. The quality of the\n' +
'duplicate depends on the individual.',
'With its astonishing capacity for\n' +
'metamorphosis, it can get along with anything.\n' +
'It does not get along well with its fellow Ditto.',
'While it can transform into anything, each Ditto\n' +
'apparently has its own strengths and\n' +
'weaknesses when it comes to transformations.',
'It transforms into whatever it sees. If the thing\n' +
'it’s transforming into isn’t right in front of it,\n' +
'Ditto relies on its memory—so sometimes it fails.',
'When it spots an enemy, its body transfigures\n' +
'into an almost-perfect copy of its opponent.',
'When it spots an enemy, its body transfigures\n' +
'into an almost-perfect copy of its opponent.',
'It can reconstitute its entire cellular\n' +
'structure to change into what it sees,\n' +
'but it returns to normal when it relaxes.',
'When it encounters another Ditto, it will move\n' +
'faster than normal to duplicate that opponent exactly.'
]Mew:
[
'So rare that it\n' +
'is still said to\n' +
'be a mirage by\fmany experts. Only\n' +
'a few people have\n' +
'seen it worldwide.',
'So rare that it\n' +
'is still said to\n' +
'be a mirage by\fmany experts. Only\n' +
'a few people have\n' +
'seen it worldwide.',
'When viewed\n' +
'through a micro\n' +
"scope, this\fPOKéMON's short,\n" +
'fine, delicate\n' +
'hair can be seen.',
'Apparently, it\n' +
'appears only to\n' +
'those people who\fare pure of heart\n' +
'and have a strong\n' +
'desire to see it.',
'Its DNA is said to\n' +
'contain the genet\n' +
'ic codes of all\fPOKéMON, so it can\n' +
'use all kinds of\n' +
'techniques.',
'Because it can\n' +
'learn any move,\n' +
'some people began\fresearch to see if\n' +
'it is the ancestor\n' +
'of all POKéMON.',
'MEW is said to possess the genetic\n' +
'composition of all POKéMON.\n' +
'It is capable of making itself invisible\fat will, so it entirely avoids notice even\n' +
'if it approaches people.',
'MEW is said to possess the genetic\n' +
'composition of all POKéMON.\n' +
'It is capable of making itself invisible\fat will, so it entirely avoids notice even\n' +
'if it approaches people.',
'A MEW is said to possess the genes of all\n' +
'POKéMON. It is capable of making itself\n' +
'invisible at will, so it entirely avoids\n' +
'notice even if it approaches people.',
'A POKéMON of South America that was\n' +
'thought to have been extinct. It is very\n' +
'intelligent and learns any move.',
'So rare that it is still said to be a\n' +
'mirage by many experts. Only a few people\n' +
'have seen it worldwide.',
'Because it can use all kinds of\n' +
'moves, many scientists believe MEW\n' +
'to be the ancestor of Pokémon.',
'Because it can use all kinds of\n' +
'moves, many scientists believe MEW\n' +
'to be the ancestor of Pokémon.',
'Because it can use all kinds of\n' +
'moves, many scientists believe MEW\n' +
'to be the ancestor of Pokémon.',
'Apparently, it appears only to\n' +
'those people who are pure of heart\n' +
'and have a strong desire to see it.',
'Its DNA is said to contain the genetic\n' +
'codes of all Pokémon, so it can\n' +
'use all kinds of techniques.',
'Because it can use all kinds of\n' +
'moves, many scientists believe Mew\n' +
'to be the ancestor of Pokémon.',
'Because it can use all kinds of\n' +
'moves, many scientists believe Mew\n' +
'to be the ancestor of Pokémon.',
'Because it can use all kinds of\n' +
'moves, many scientists believe Mew\n' +
'to be the ancestor of Pokémon.',
'Because it can use all kinds of\n' +
'moves, many scientists believe Mew\n' +
'to be the ancestor of Pokémon.',
'Because it can use all kinds of moves, many\n' +
'scientists believe Mew to be the ancestor\n' +
'of Pokémon.',
'Its DNA is said to contain the genetic codes of all\n' +
'Pokémon, so it can use all kinds of techniques.',
'Mew is said to possess the genetic composition of all\n' +
'Pokémon. It is capable of making itself invisible at will,\n' +
'so it entirely avoids notice even if it approaches people.',
'Mew is said to possess the genetic composition of all\n' +
'Pokémon. It is capable of making itself invisible at will,\n' +
'so it entirely avoids notice even if it approaches people.',
'When viewed through a microscope, this\n' +
'Pokémon’s short, fine, delicate hair can be seen.',
'When viewed through a microscope, this\n' +
'Pokémon’s short, fine, delicate hair can be seen.'
]