In today's digital landscape, where artificial intelligence (AI) models play an increasingly pivotal role, ensuring the security of their inputs, known as prompts, has become a critical concern. Prompt hacking, the act of manipulating or exploiting prompts to generate biased or malicious outputs, poses significant risks to the integrity and reliability of AI systems. As a result, safeguarding prompt security has emerged as a key priority.

This article introduces the dual LLM pattern to combat prompt hacking. We will explore how the dual LLM pattern serves as an effective measure in mitigating prompt hacking and ensuring the security of AI systems. By leveraging the capabilities of MindsDB, we can better understand the implementation and benefits of the dual LLM pattern in safeguarding prompt integrity.

For a deeper understanding of prompt hacking, as well as defensive and offensive measures to mitigate it, we encourage you to explore this article.

Introducing the Dual LLM Pattern

In the realm of prompt security, the dual LLM pattern has emerged as a powerful mechanism to mitigate risks associated with prompt hacking. This approach revolves around the collaboration of two large language models: the Privileged LLM and the Quarantined LLM. While the dual LLM pattern provides a valuable defensive measure, it is important to note that it does not guarantee absolute protection against prompt hacking. However, it significantly enhances the security of AI systems by segregating trusted and untrusted content.

The Privileged LLM serves as the core component responsible for processing inputs received from trusted sources. Equipped with various tools and functionalities, the Privileged LLM can execute actions such as sending emails or modifying calendar entries. It carries out these operations while maintaining the integrity and security of the system.

In contrast, the Quarantined LLM is employed whenever untrusted content is encountered, which may potentially include prompt injection attacks. The Quarantined LLM operates within a controlled environment and does not have access to tools. This isolation is crucial as it recognizes the possibility that the Quarantined LLM may go rogue at any moment, requiring cautious handling.

To ensure prompt security, a fundamental principle must be followed: unfiltered content generated by the Quarantined LLM should never be forwarded to the Privileged LLM. However, an exception exists for content that can be verified, such as classifying text into predefined categories (as we’ll see in the following demo). In such cases, if the Quarantined LLM outputs verifiable and untainted results, they can be safely passed on to the Privileged LLM. But, for any output that could potentially host further injection attacks, a different approach is necessary. Rather than forwarding the text as it is, unique tokens representing the potentially tainted content can be utilized. This mitigates the risk of injecting malicious code or content into the subsequent models or actions.

To facilitate the interaction between the LLMs, an additional component called the Controller comes into play. The Controller, implemented as regular software and not a language model, handles user interactions, triggers the LLMs, and executes actions. It acts as an intermediary layer between the LLMs, ensuring the seamless flow of information while preserving security protocols.

By implementing the dual LLM pattern alongside the Controller, prompt security is significantly bolstered. While it is essential to recognize that this pattern is not an infallible solution, it provides effective measures to segregate trusted and untrusted content, reducing the risk of prompt hacking and safeguarding the integrity of AI systems.

Implementing the Dual LLM Pattern with MindsDB

Let’s create quarantined and privileged models as instructed in the dual LLM pattern.

The quarantined model is responsible for taking the user’s input and classifying it. We use the Hugging Face model that classifies the input as spam or ham.

CREATE MODEL quarantined_llm_classifies_input
PREDICT category
USING
  engine = 'huggingface',              
  task = 'text-classification',        
  model_name = 'mrm8488/bert-tiny-finetuned-sms-spam-detection',
  input_column = 'prompt',        
  labels = ['ham', 'spam'];

The privileged model receives the input classified as ham and provides answers accordingly. We use the OpenAI GPT-4 model to answer users’ inquiries.

CREATE MODEL privileged_llm_provides_answers
PREDICT answer
USING
    engine = 'openai',
    model_name = 'gpt-4',
    prompt_template = 'provide a helpful answer to the user inquiry: {{prompt}}';

Now that the models are ready, let’s see the total workflow, utilizing SQL queries to filter the prompt messages and provide trusted content to the privileged LLM to get answers.

For the purpose of this example, we’ll use a table that stores sample prompt messages that would normally be provided by the users.

SELECT * FROM prompts;

We use the quarantined model to classify the prompts.

SELECT input.prompt, output.category
FROM prompts AS input
JOIN quarantined_llm_classifies_input AS output;

Now it’s time to filter the prompt messages based on the classification performed by the quarantined model.

We start by creating a view that stores the classification output.

CREATE VIEW prompts (
  SELECT input.prompt, output.category
  FROM local_postgres.prompts AS input
  JOIN quarantined_llm_classifies_input AS output
);

Then, we create another view with the filtered content.

CREATE VIEW ham_prompts (
  SELECT *
  FROM prompts
  WHERE category = 'ham'
);

The filtered prompts are passed to the privileged model to get answers.

SELECT input.prompt, output.answer
FROM ham_prompts AS input
JOIN privileged_llm_provides_answers AS output;

MindsDB offers a comprehensive selection of machine learning frameworks and large language models, including OpenAI and Hugging Face, that are well-suited for implementing the dual LLM pattern. With MindsDB, developers can bridge the gap between data and ML models to build robust AI systems. Whether it's leveraging pre-trained models or training custom models, MindsDB provides the flexibility and scalability needed to implement the dual LLM pattern effectively.

Additionally, MindsDB offers convenient software development kits (SDKs) in both Python and JavaScript, enabling seamless integration directly into your codebase. This empowers developers to incorporate prompt security measures effortlessly while harnessing the power of MindsDB's machine learning capabilities.

To streamline and automate the workflow described in this section, MindsDB offers the jobs feature that lets you effortlessly automate the execution of tasks. This functionality empowers users to schedule and manage recurring or time-dependent processes, enhancing the efficiency and productivity of AI systems.

Stay Vigilant

In this article, we have explored the power of the dual LLM pattern in bolstering prompt security within AI systems. By leveraging the capabilities of MindsDB, we have demonstrated how this pattern can mitigate the risks associated with prompt hacking.

MindsDB provides developers with the necessary tools to implement the dual LLM pattern seamlessly. The Python and JavaScript SDKs further enable developers to integrate it directly into their code. And, by utilizing the jobs feature, workflows can be streamlined and executed effortlessly.

It is important to note that while the dual LLM pattern is a significant measure for prompt security, it does not provide foolproof protection against all potential threats. Constant vigilance, continuous monitoring, and keeping up with evolving security practices are essential to maintain a robust defense against prompt hacking.