Fine-Tuning Vercel AI SDK
Clifton Beale
Posted on September 27, 2023
Creating custom responses based on user data:
Jump to section:
Overview
I recently unveiled a sneak peak into my work over the last few months on 'Next-Meal'. One of the major features of the web app is the Sous-chef which leverages Vercel's AI SDK to incorporate a live chat bot to assist users in their search for their next meal.
At first, the functionality remained pretty basic, as I simply followed Vercel's documentation to implement it which was pretty straightforward. (See my blog post on adding Vercel's AI SDK here).
After adding some cool features to the web app like user preferences (allowing users to filter through meals based on their account's preferences), I figured it was time to take the AI a step further. With a desire to sharpen my skills involving AI, I embarked on the task of implementing the user preferences filtering into the chat bot as well.
This proved to be difficult, as once again I was navigating unfamiliar territory that has not been charted by many (not much documentation outside of Vercel for issues/implementation).
What is the purpose?
While it is true that the AI on its own is a fascinating tool, for the sake of my use-case it could have proven dangerous. Once users set their preference on Next-Meal, most user's would assume that all of the app's functionality would automatically take their preferences into consideration. With this not being the case (out of the box), users could potentially consume food from the Sous-chef's suggestions that they may be allergic too or have an intolerance for unknowingly.
My goal for Next-Meal, from the beginning, has been to give the users the most seamless experience when searching for a meal or recipe. With this extra step involved with the Sous-chef, users can be at ease knowing that their preferences are being accounted for in all areas of the web app.
Implementation and Troubleshooting
Without a doubt, I ran into many different challenges during my attempts to implement this functionality. While I did persevere and eventually find a viable solution, I figured I would share my steps and some of my hardships that I faced so that maybe someone else won't run into the same issues.
First off, I needed to figure out how to get the chatbot to understand each user's preferences. In order to achieve this, I gathered the preference information on the client, and sent it with the message to the chat API route on the server.
Above is the function that retrieves the user preference values for my web app.
For reference, the user profile is an array that contains two objects, preferences and favorites. For this case, I am iterating through the key value pairs on the userPreferences object and only returning the key for the values that are truthy booleans (meaning that these preferences are set by the user already).
This part definitely took some trial and error, and I am ashamed to admit I spent a great deal of time treating the preferences object as an array and trying to leverage built in JS array methods before realizing my error.
Well, now I have the preferences on the client, but how do I get them over to the server when I am using a built in custom hook to send my information over to the server?
After doing some digging on the Vercel docs for the best method to handle this issue, I came across this documentation for their useChat hook (shoutout Vercel for amazing docs!). After realizing you can pass an extra body object to the hook to be accessed by the server, I was back in the ball game. I quickly altered my code:
Now I was checking to make sure that there was an active session (because the Sous-chef can be used by users that have not created an account) and also checking to make sure that there were preferences to send from the user, and if not then sending nothing (obviously) - incase the users had not yet set preferences.
I will also admit from here, I was a bit unsure how to leverage this body property as I was getting some weird errors trying to access the req.body on the server side.
Once I realized my error, I felt stupid... The body off the request was the body of the useChat hook, which also encompassed other values, such as the message from the hook. Once I realized this much, I was able to simple destructure the preferences from the req.json, right alongside the messages.
Finally, things are starting to come together. Now I just need to somehow tell Chat-GPT about these preferences that I now have on the server. Vercel's AI SDK offers a very easy solution to customize your prompts, which I had already tinkered with previously. Once I had the preferences on the server, I simply adjusted my code to let the system know about the preferences, and voila!
Conclusion
I now, after several days of troubleshooting and reading through documentation (and also consulting with Chat-GPT), had finished.
The functionality is all in tact, and users can now leverage their preferences through the Sous-chef.
Above is a screenshot of the Sous-chef in action. I personally don't care for dairy or pork, so I have marked those two ingredients in my preferences to avoid. As you can see, the chatbot provides accurate suggestions without needing to be reminded of the preferences manually. There was some tinkering with the prompts, of course, before the Sous-chef was able to achieve this behavior. Prompt engineering for AI is very important, as it only knows information it is given. When working towards a solution such as this, the model is not trained on the user preferences that I was seeking it to have, so I had to get the prompt just right.
Overall, this challenge was very rewarding to overcome. Now, I have a better understanding of the fundamentals of the AI model, as well as how to better manipulate the data given and received from Vercel's AI SDK.
Resources
Posted on September 27, 2023
Join Our Newsletter. No Spam, Only the good stuff.
Sign up to receive the latest update from our blog.
Related
