As AI becomes more accessible to every developer, we will continue to see the use cases for it expand. For example, when designing a new landing page or product, I use Tailwind, but what if you could apply Tailwind to your bedroom?

Well, you can now.

@nutlope gifted us another AI project example, RoomGPT.

header image was generated using midjourney

What is Nutlope/roomGPT?

This project dreams up designs of your room using AI.

It uses an ML model called ControlNet to generate variations of rooms. This application allows you to upload a photo of any room, which will be sent through this ML Model using a Next.js API route, and return your generated room.

How does it work?

The ML Model is hosted on Replicate and Upload is used for image storage. During my 9 days of OpenAI series, I have focused on OpenAI, but in this example, the developer uses Replicate.

The project uses Next.js, and the AI power is in the api/generate.ts file. Similar to OpenAI, there is an API endpoint from Replicate.

Replicate lets you run machine learning models with a cloud API without having to understand machine learning or manage infrastructure. You can run open-source models others have published or package and publish your own. Those models can be public or private. The nice part is a lot of them are open-source on GitHub, which makes it helpful for learning.

Explore some hosted replicate cogs

The predictions endpoints take id, which in this example is pointed to the hosted version of ControlNet model.

You can get an idea of how the prompt is being passed from the UI by taking a look at the default Game Room prompt.

// pages/api/generate.ts

  const {
  imageUrl,
  theme,
  room
} = req.body;
// POST request to Replicate to start the image restoration generation process
let startResponse = await fetch("https://api.replicate.com/v1/predictions", { // predictions
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    Authorization: "Token " + process.env.REPLICATE_API_KEY,
  },
  body: JSON.stringify({
    version: "854e8727697a057c525cdb45ab037f64ecca770a1769cc52287c2e56472a247b", // Controlnet version
    input: {
      image: imageUrl,
      prompt: room === "Gaming Room" ?
        "a room for gaming with gaming computers, gaming consoles, and gaming chairs" :
        `a ${theme.toLowerCase()} ${room.toLowerCase()}`,
      a_prompt: "best quality, extremely detailed, photo from Pinterest, interior, cinematic photo, ultra-detailed, ultra-realistic, award-winning",
      n_prompt: "longbody, lowres, bad anatomy, bad hands, missing fingers, extra digit, fewer digits, cropped, worst quality, low quality",
    },
  }),
});

OpenAI vs Replicate

I am making this comparison because I had the question instantly after looking at the code.

After a brief look at projects using both APIs, my takeaway is that Replicate gives a more straightforward developer experience through the API. OpenAI provides an experience you can fine-tune if you know want you to. Replicate allows you to run models without having to understand the intricacies of machine learning or manage your infrastructure. OpenAI doesn't require expert knowledge of machine learning, but it allows those with the ability to fine-tune models.

If I need to correct something, or if you have some insight into the code, please comment. I enjoyed walking through the code and learning how this works. Thanks to Nutlope for sharing this with us, and be sure to contribute back upstream. Open Source FTW!

Also, if you have a project leveraging OpenAI or similar, leave a link in the comments. I'd love to take a look and include it in my 9 days of OpenAI series.

Find more AI projects using OpenSauced

Also, check out my bedroom redesign.

Stay saucy.