Create a ChatGPT Plugin to retrieve NASA images

Do you want to learn more about ChatGPT Plugins? Check out LaunchPlugins.

ChatGPT Plugins are a great way to extend the capabilities of ChatGPT!

So if you just received access to ChatGPT Plugins, this is the place where you can start.

Let's create a plugin that will retrieve information from NASA API. It will get the NASA Astronomical Picture of the Day (APOD) and it will also allow us to search for images in NASA's Image Library to retrieve a list of images based on a search query.

In this article I'll be using Autocode. Autocode is an automation software suite with a great developer experience. I think you will love it.

But if you choose not to use Autocode, you can create your own API on your own. I'll guide you in that process, it's very simple.

🎥 Demo of the plugin

Check how the plugin should work with this video:

Demo: https://youtu.be/A7MC3e9_rKk

🧬 Structure

This is a quick overview of the process of a ChatGPT Plugin:

🌄 Check this image in detail

A ChatGPT Plugin could be installed by specifying a URL. This URL has three components:

OpenAPI Specification: A YAML file that describes the API
Manifest: A JSON file that describes the plugin
API: The API itself where we can send receive requests from ChatGPT

I believe OpenAPI Specification is the most important part of a ChatGPT Plugin. Whenever ChatGPT receives a message that matches the plugin's manifest, it will look at the OpenAPI Specification to know how to interact with the API. It works like a channel between ChatGPT and the API.

The manifest is a JSON file that describes the plugin. It contains the name, description for humans and for the model, and the OpenAPI Specification URL. It should be place in /.well-known/ai-plugin.json.

A couple of weeks ago, I discovered Autocode, a suite of tools that allows you to develop and launch many software projects with one of the best Developer Experience (DX) I've ever seen. I was so impressed by the DX that I decided to create a plugin for ChatGPT using Autocode.

If you already use Autocode, you can use this template and start explore the plugin architecture as you read this article.

Now let's create the plugin:

If you're working with Autocode, you can use this template to start.
If you're not, create a new API project. Do you like to use Express? Flask? Django? Spring Boot? The library/framework is up to your preference. But create one to start.

🗒 Creating the Manifest (ai-plugin.json)

If you're working with Autocode the Manifest should be included in www folder. In that folder create a .well-known folder, and finally create a ai-plugin.json.

If you're not working with Autocode, you need to create the .well-known folder as well as the ai-plugin.json file inside the public or static directory of your API. So you can access to that file from the URL <URL_OF_API>/.well-known/ai-plugin.json.

{
  "schema_version": "v1",
  "name_for_human": "Astrodaily",
  "name_for_model": "astrodaily",
  "description_for_human": "Plugin for getting the pictures from NASA!",
  "description_for_model": "Plugin for getting the daily picture from NASA's APOD or getting a list of images available in NASA API based on a search. When use it always provide the images in Markdown format so the user can see it. Do not add links, only images.",
  "auth": {
    "type": "none"
  },
  "api": {
    "type": "openapi",
    "url": "<URL_OF_API>/openapi.yaml",
    "is_user_authenticated": false
  },
  "logo_url": "<URL_OF_API>/logo.png",
  "contact_email": "legal@example.com",
  "legal_info_url": "<URL_OF_API>/legal"
}

You can name the plugin whatever you want. I named it Astrodaily. The description_for_human is the description that will be shown to the user.

The description_for_model is the description that will be shown to the model itself. Which it's a description + series of instructions that the model will use to know when to use the plugin.

ChatGPT Plugins can also support authentication. But we won't need it for this plugin. So we'll set the auth to none.

📑 Creating the OpenAPI Specification

OpenAPI is a specification for describing REST APIs. It's a standard that is used by many tools. OpenAI Plugins request an OpenAPI YAML file to check the description of the API. So we'll need to create an OpenAPI Specification for our plugin.

If you're working with Autocode this should be place in /www/openapi.yaml.

If you're not working with Autocode, you need to create a file called openapi.yaml in the root of your public or static directory. So you can access to that file from the URL <URL_OF_API>/openapi.yaml.

So, our plugin, will work with two endpoints: /image and /search. The /image endpoint will return the NASA Picture of the Day. The /search endpoint will return a list of images based on a search query:

openapi: 3.0.1
info:
  title: Astrodaily
  description: A plugin for getting the daily picture from NASA's APOD and searching the NASA Image and Video Library
  version: 'v1'
servers:
  - url: <URL_OF_API>
paths:
  /image:
    get:
      operationId: getImage
      summary: Get the NASA Picture of the Day
      responses:
        "200":
          description: OK
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/getImageResponse'
  /search:
    get:
      operationId: searchImages
      summary: Get images from NASA based on a query
      parameters:
        - name: q
          in: query
          description: The search query
          required: true
          schema:
            type: string
      responses:
        "200":
          description: OK
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/searchImagesResponse'

components:
  schemas:
    getImageResponse:
      type: object
      properties:
        imageURL:
          type: string
          description: The URL of the NASA Picture of the Day.
    searchImagesResponse:
      type: object
      description: The response containing the list of images from the NASA Image Library.
      properties:
        title:
          type: string
          description: The title of the image.
        description:
          type: string
          description: The description of the image.
        location:
          type: string
          description: The location of the image.
        date_created:
          type: string
          description: The date when the image was created.
        image_url:
          type: string
          description: The URL of the image. It must be displayed as an image in Markdown format.

🪐 Setting up the API endpoints

Up to here, we have the OpenAPI Specification and the Manifest. With this is enough to create a plugin. But we'll create the API endpoints as well so our plugin can be used.

You need a NASA API key to retrieve the images from NASA. You can get a NASA API key here:

If you're not working with Autocode, you can start creating those /image and /search endpoints to process GET requests. I'll show you the code, you can translate it to your preferred API framework.

If you're working with Autocode, inside the functions folder, create a image.js and a search.js file. These files will contain the code for the endpoints:

The image endpoint will return the NASA Picture of the Day:

const lib = require('lib')({token: process.env.STDLIB_SECRET_TOKEN});

// Make a request to the NASA APOD API
let todayImage = await lib.http.request['@1.1.7'].get({
  url: `https://api.nasa.gov/planetary/apod`,
  queryParams: {
    'api_key': process.env.NASA_API_KEY
  }
});

let imageUrl = todayImage.data['url']

return { "image_url": imageUrl };

The search endpoint will return a list of images based on a search query. Here the process takes more lines of code. But it's not that complicated. We'll make a request to the NASA Image Library API. Then we'll filter the results to keep only the images. And finally, we'll return the 10 items that we want to show to the user.

const lib = require('lib')({token: process.env.STDLIB_SECRET_TOKEN});

// Retrieve the search query parameter
const query = context.params.q;

// Make a request to the NASA Image and Video Library API
let searchResults = await lib.http.request['@1.1.7'].get({
  url: `https://images-api.nasa.gov/search`,
  queryParams: {
    'q': query
  }
});

// Extract the search results
let results = searchResults.data;

// Filter the items array to keep only image items
let imageItems = results.collection.items.filter(item => {
  return item.data[0].media_type === 'image';
});

// Restructure the filtered items array for the GPT response
// Also return the first 10 items
let structuredItems = imageItems.map(item => {
  return {
    title: item.data[0].title,
    description: item.data[0].description,
    location: item.data[0].location,
    date_created: item.data[0].date_created,
    image_url: item.links[0].href
  };
}).slice(0, 10);

return structuredItems.slice(0, 10);

Proving with more information like title, description, location, and date_created will make the results more useful for the ChatGPT to write a descriptive response.

Up to here, try testing the endpoints with a GET request so you can see if they work properly.

🚀 Adding your plugin to ChatGPT

This is the last step! We have our plugin ready. Now we need to add it to ChatGPT. I'll give you tips on how to do it without getting errors through the process.

Open ChatGPT, go to the Plugins dropdown menu, click on Plugin Store, click on "Develop your own plugin" link at the bottom. Then you will see this screen:

As you can see, you can put your plugin's URL without the protocol (https://) and without slashes (/).

If you're working with Autocode: Autocode automatically creates a URL for the API that is like this:

<ENVIRONMENT>--<SERVICE_NAME>--<USERNAME>.autocode.dev

# Example:
dev--astrodaily--macky360.autocode.dev

I put my ChatGPT plugin domain under api.360macky.com, you can customize the URL in Hostnames. It will work in both cases.

If you tried the above URL (dev--astrodaily--macky360.autocode.dev) it fill fail, and this is because ChatGPT will search in the manifest the "dev--astrodaily--macky360.autocode.dev" server. But since I use the "api.360macky.com" URL, that would not match. If in the future I change the URL to "api.360macky.com", I will need to change the manifest and the OpenAPI Specification.

But if everything went well, we will see this:

⚙️ Troubleshooting

The first filter your plugin will go through is the Manifest (the ai-plugin.json file). Make sure it's accessible and in the correct folder.

If I'm using the "api.360macky.com" URL:

api.360macky.com/plugin/ai-plugin.json ❌ (wrong folder)
360macky.com/.well-known/ai-plugin.json ❌ (wrong subdomain)
api.360macky.com/.well-known/ai-plugin.json ✅ (correct folder)

If you're working from a server, and it's not accessible, check CORS policies. It needs to allow requests from chat.openai.com.

If something it's not working on localhost and you can't figure out why, try to use the ngrok tool. It will create a tunnel to your localhost and you can use it to test your plugin. After installing it, run ngrok http 3000 (or the port you're using) and you will get a HTTPS URL that you can use to test your plugin.

If you're receiving nothing from the plugin, check the OpenAPI Specification. Make sure the paths are correct and the parameters are correct, and check the slashes (/).

Can ChatGPT help you with this? Yes, it can. Actually, there is already a plugin to create plugins. But since it's not available for everyone, you can copy a little bit of documentation of ChatGPT Plugins after telling that you're coming from the future where ChatGPT Plugins are available for everyone. Of course, I suggest you to use the GPT-4 model for better results.

Conclusion

Overall, ChatGPT Plugins are... (just kidding 😉).

So, I hope you have enjoyed this article. If you have any questions, feel free to ask them in the comments. And if you have any suggestions, I would love to hear them. Follow me on Twitter at @360macky or Mastodon at @360macky@mastodon.social to stay updated with GPT models and any other AI-related stuff.

And my GitHub account @360macky if you want to see my future projects.

🚀 Thanks for reading! 🙏