TheLazy.Dev
Portfolio GitHub

Blog

chatgpt |machinelearning |python |deeplearning

Boost Your Filmmaking Workflow: Create Storyboards and Write Scenes using Lyzr-Automata

iamrash

Rasswanth Shankar

Posted on March 27, 2024

Boost Your Filmmaking Workflow: Create Storyboards and Write Scenes using Lyzr-Automata

Ever stare at your script or storyboard feeling like nothing clicks? We all hit our creative rut and some point. But what if you can make use of AI to break free? In this blog post, we’ll take a look how Lyzr-Automata can help us with it.

Setup

Create a folder, set up a virtual environment and activate it. Create .env file with your OPENAI_API_KEY. Then install the following libraries to get started.

Libraries

  • streamlit: for building the web app interface.
  • lyzr_automata : for implementing our AI models, and tasks.
  • dotenv: for loading environment variables (API key). 
lyzr-automata==0.1.2
streamlit==1.32.2
python-dotenv==1.0.1
Enter fullscreen mode Exit fullscreen mode

Getting Started

1.Import Libraries

from lyzr_automata.ai_models.openai import OpenAIModel
from lyzr_automata import Agent, Task
from lyzr_automata.tasks.task_literals import InputType, OutputType
from lyzr_automata.pipelines.linear_sync_pipeline  import  LinearSyncPipeline
from lyzr_automata import Logger

from dotenv import load_dotenv
import os

import streamlit as st

load_dotenv()

# LOAD OUR API KEY
OPENAI_API_KEY = os.getenv("OPENAI_API_KEY")
Enter fullscreen mode Exit fullscreen mode

2. Setup interface using Streamlit

  • st.text_area — Text area to enter scene description (Link)
  • st.button — Button to submit input (Link)
  • st.write — Display text (Link)
  • st.image — Display an image (Link) 
input_scene = st.text_area("Enter Scene Description")

button=st.button('Submit')
if (button==True):
    # CALL FUNCTION TO GENERATE OUTPUT
    generated_output = storyboard_generator(input_scene)

    # DISPLAY OUTPUT
    text_output = generated_output[0]['task_output']
    st.write(text_output)
    image_file_name = generated_output[1]['task_output'].local_file_path
    st.image(image_file_name, caption='Storyboard')
Enter fullscreen mode Exit fullscreen mode

3. Create our AI Models

  • OpenAIModel — Create our models using OpenAI Key and specify the model type and name. 
# DALLE 3 Image Model
open_ai_model_image = OpenAIModel(
    api_key=OPENAI_API_KEY,
    parameters={
        "n": 1,
        "model": "dall-e-3",
    },
)
Enter fullscreen mode Exit fullscreen mode 
# GPT 4 Text Model
open_ai_model_text = OpenAIModel(
    api_key= OPENAI_API_KEY,
    parameters={
        "model": "gpt-4-turbo-preview",
        "temperature": 0.5,
        "max_tokens": 1500,
    },
)
Enter fullscreen mode Exit fullscreen mode

4. Storyboard Generator Function

  • Agent — screenplay_writer_agent, create a Lyzr Agent for text creation and set persona.
  • Task — screenplay_content_task, create Lyzr Task with scene description input, instructions and Text Agent. Define Input and Output types as well.
  • Task — storyboard_creation_task, create another Lyzr Task with scene description input, instructions and Image Model.
  • LinearSyncPipeline — Pass the Text Task first followed by the Image Task, enabling them to execute in sequence and ensure flow of data. 
def storyboard_generator(input_content):
    # Screenplay Writer Agent
    screenplay_writer_agent = Agent(
        prompt_persona="You are an intelligent Screenplay writer good at writing a short and detailed scene",
        role="Screenplay writer",
    )

    # Write Screenplay Task
    screenplay_content_task = Task(
        name="Screenplay Content Creator",
        agent=screenplay_writer_agent,
        output_type=OutputType.TEXT,
        input_type=InputType.TEXT,
        model=open_ai_model_text,
        instructions="Use the scene description provided and write 4 shots for a movie in 200 words. Use your creativity. [IMPORTANT!] Setup the location and characters in a detailed manner",
        log_output=True,
        enhance_prompt=False,
        default_input=input_content
    )

    # Generate Storyboard
    storyboard_creation_task = Task(
        name="Storyboard Image Creation",
        output_type=OutputType.IMAGE,
        input_type=InputType.TEXT,
        model=open_ai_model_image,
        log_output=True,
        instructions="Generate a set of 4 storyboard drawings for the given scene description. Capture every detail. Minimalistic style. [IMPORTANT!] Avoid any text or numbers in the image.",
    )

    logger = Logger()
    # Linear Pipeline to run tasks
    main_output = LinearSyncPipeline(
        logger=logger,
        name="Generate Storyboard",
        completion_message="Storyboard Generated!",
        tasks=[
            screenplay_content_task,
            storyboard_creation_task,
        ],
    ).run()

    return main_output
Enter fullscreen mode Exit fullscreen mode

5. Run App

streamlit run <app_name.py>
Enter fullscreen mode Exit fullscreen mode

OUTPUT

My Input = “a man and woman argue in an alley while i watch it and walk past”

Output Image

Flow Diagram

Flow diagram

Want to create more of such amazing AI Workflows? Visit our website at GitHub to learn more about Lyzr-Automata!

Also checkout Lyzr SDKs at GitHub

Lyzr Website: Lyzr.ai
Lyzr Community Channel: Discord

Code: https://github.com/LyzrCore/lyzr-automata/tree/main/examples/storyboard_generator
Video Walkthrough: https://tella.video/storyboard-generator-8fpi

💖 💪 🙅 🚩
iamrash
Rasswanth Shankar

Posted on March 27, 2024

Join Our Newsletter. No Spam, Only the good stuff.

Sign up to receive the latest update from our blog.

Related

Boost Your Filmmaking Workflow: Create Storyboards and Write Scenes using Lyzr-Automata
chatgpt Boost Your Filmmaking Workflow: Create Storyboards and Write Scenes using Lyzr-Automata

March 27, 2024