Game Engine with AI✨ Part 2 - DALL E Sprites
Abhishek
Posted on July 7, 2023
Hey, devs! 🖐🎮
Hope you're all having a nice day! I have some cool updates on helicity.ai
For reference, this was part 1 (nobody saw it🙃)
I'm making an AI JS Browser Game Engine
Inbuilt DALL E Support
I've integrated DALL-E support! 🎨🖌️ Yes, so from now on, you can define your sprite URL as "lookup: some description", and Helicity.ai will generate a unique sprite for you on the fly using DALL-E. Create unique, custom images without having to leave the IDE.
How I made it
- So it all happens in the Gameobject.js module. Whenever you make a new instance of the GameObject, this code runs
- When you put "lookup:" string in your image source, it takes this as a request and calls a firebase function made in Express
- The firebase function calls the DALL E API looking up the string after "lookup:" and asynchronously returns the image-url, this is sent back to the game object.
- The game object image is drawn every frame using *render() *(automatically in the game loop) or else, its just a black box
- The savedImages makes sure that if 2 game objects are created with the same lookup description, multiple called aren't made to dall e. For example pipes in flappy bird.
Gameobject.js
var savedImages = {};
export class GameObject {
constructor(x, y, width, height, type, imageSrc) {
this.x = x;
this.y = y;
this.width = width;
this.height = height;
this.type = type; // can be Used for collision detection
this.image = new Image();
this.image.loaded = false;
//check if imagesrc contains the string lookup:
if (imageSrc.includes("lookup:")) {
//check if the image is already saved
if (savedImages[imageSrc]) {
this.image = savedImages[imageSrc];
this.image.loaded = true;
} else {
//make a post request. this is in html to use XMLHttpRequest
var xhr = new XMLHttpRequest();
xhr.open(
"POST",
"https://us-central1-myaigameengine.cloudfunctions.net/api/Image",
true
);
xhr.setRequestHeader("Content-Type", "application/json");
xhr.send(
JSON.stringify({
userPrompt: imageSrc.split("lookup:")[1],
})
);
xhr.onreadystatechange = () => {
if (xhr.readyState == 4 && xhr.status == 200) {
//console.log(xhr.responseText);
//get the imageURL from the response
var imageURL = JSON.parse(xhr.responseText).image_url;
this.image = new Image();
this.image.src = imageURL;
this.image.onload = () => {
this.image.loaded = true;
//save the image
savedImages[imageSrc] = this.image;
};
this.image.onerror = () => {
console.error(`Error loading image ${this.imageSrc}`);
};
}
};
}
} else {
this.imageSrc = imageSrc;
this.image = new Image();
this.image.src = imageSrc;
this.image.onerror = () => {
console.error(`Error loading image ${this.imageSrc}`);
};
this.image.onload = () => {
this.image.loaded = true;
};
}
Game.addGameObject(this);
}
update(deltaTime) {
// Overridden by child classes
}
render() {
//draw a black rectangle if there isnt an image else draw it
if (this.image.loaded) {
Renderer.drawImage(this.image, this.x, this.y, this.width, this.height);
} else {
Renderer.context.fillStyle = "black";
Renderer.context.fillRect(this.x, this.y, this.width, this.height);
}
}
}
Firebase function's Express Image Endpoint
app.post("/Image", async (req, res) => {
//var uuid = req.body.uuid;
var userPrompt = req.body.userPrompt;
//console.log("uuid is " + uuid);
console.log("user prompt is ");
openai
.createImage({
prompt: userPrompt,
n: 1,
size: "256x256",
})
.then((result) => {
image_url = result.data.data[0].url;
console.log(image_url);
res.send({
success: true,
image_url: image_url,
});
});
});
Thats it!
Think about it - need a sprite of a "purple dragon with a top hat?" 🐉🎩 Just type lookup: purple dragon with a top hat and voila, your personalized sprite is ready to be used in your game! It's that simple! 🎲🚀
Multiplayer support is coming soon!
If you'd like to talk about and participate in the development, hit me up on discord!
Stay tuned for more updates! Happy coding 🎮💖
Posted on July 7, 2023
Join Our Newsletter. No Spam, Only the good stuff.
Sign up to receive the latest update from our blog.
Related
