(Update April 2024 - when this article was written OpenAI didn't have function calling within tools, read to the end for the updated code)

In this article you'll find out how to fallback from gpt4 turbo preview to gpt 4 when using function calling (Relevant November 2023 as gpt4-turbo is in preview and heavily rate limited), and with that response validate and type your function calling responses

The function calling feature is very powerful (we use it all the time to turn unstructured data into structured data) as it allows us to get JSON data for whatever function we want to call. But it's still possible that GPT could hallucinate bad JSON data, or the wrong type, or a wrong field name...I could go on (just last week the JSON coming back couldn't be parsed because it was malformed when telling GPT4 to use emojis).

Besides those outliers, we want our response from GPT to be typed for use elsewhere in our application rather than just having it as "any".

So with that context let's get into the code. We're going to use an example where we're taking in a question from a user asking questions on an e-commerce site as a way to show multiple functions we might call based on the user's question.

Code

Real quick install OpenAI:

and instantiate the OpenAI client:

GPT4 Turbo to GPT4 Fallback

Now we want to set up the ability to fallback from one GPT model to another. As I mentioned before, it's currently November 2023 and GPT4 Turbo is still in preview and rate limited. So for going to prod, we need to have a fallback to plain old GPT4 😔

So just a few try catches and we've got our fallback setup. You could use this for falling back from any model to another.
You might be wondering why I didn't use Omit<..., "model"> rather than passing each of the parameters individually. I tried this, spreading omitted objects into the OpenAI call, but it broke the return type as it uses generics internally based on whether you pass functions in... so try to get that working at your own peril.

Now, time to make our type-safe call to OpenAI.

So... a lot of code there, but let's break it down.

1. At the top of the file we're defining both the data we want to come back from open ai for each function and a mirror schema in Zod for each type to be passed into our functions.

2. We create a map for all the functions and the parser to sanitise the data coming back from OpenAI.

3. We use the function we defined above to make the call with a fallback from GPT4 turbo to regular GPT4. If you're reading this in the future you will most likely not need this, but who knows maybe you'll swap out the models from GPT4 turbo and GPT4 to GPT5 turbo and GPT5

4. We use the map we defined to call the function with the data that's sanitised with the appropriate parser. We're using Promise.all here as Open AI just announced that function calling can return multiple function calls, but adjust to your use case

Remember this is just an example to show how to make an Open AI calls type-safe and the whole e-comm customer support thing is erroneous. This has been very useful for us when we forgot to mark fields as required in the open AI schema but they came back as undefined, or catching random bad responses from Open AI.

From there, you're good to integrate OpenAI into your typescript project and be able to sleep at night knowing that the data is at least in the right format as it flows through your program.

Update April 2024

Since writing the article OpenAI has changed their APIs slightly to move function calling within tools, and we've also figured out how to reduce the duplication between Zod and the JSON schemas.

Updating Functions To Tools in OpenAI

Updating our fallback function to using tools looks like this:

Not much of a change, just moving away from the function calling API to use the new Tools API.

Now we can change our call to OpenAI:

We're still using the example from above with a few minor changes.

Tools API

We're now passing an array of objects that specify the type of tool we're calling. For now, this is always "function" but in the future, there may be options to use data analytics and more.

 Inferring JSON Schemas from Zod

In the original example, we were writing out the JSON schemas to match our zod parsers, but with the help of zod-to-json-schema we can now infer the descriptions and rules directly from Zod.

Thanks for reading!