Generative AI has driven the popularity of vector databases, but the technological landscape seems to be changing rapidly. As one of the world's most renowned AI projects, AutoGPT has announced its decision to no longer utilize vector databases, a move that may come as a surprise to many. After all, from the outset, vector databases have consistently supported the long-term memory of AI agents.

Why has this fundamental design approach changed? What new solution is set to replace it? Are vector databases essential for large-scale model applications?

AutoGPT gives up vector database

AutoGPT, released on March 30th this year as an 'AI agent', similar to LlamaIndex and LangChain, made a significant impact immediately after its launch. Within just 7 days of going live, it garnered 44,000 stars on GitHub. In contrast to the conventional method of repeatedly feeding prompt words into models, AutoGPT can autonomously work, plan tasks, break down problems into smaller components, and execute them individually. Undoubtedly, it's an ambitious initiative.

The design concept of AutoGPT involved a method of managing an AI agent's memory in an embedded format, along with a set of vector databases for storing and retrieving memories when necessary. From that perspective, the vector databases were considered a crucial part of the entire solution. Moreover, other Artificial General Intelligence (AGI) projects have also adopted similar methods, such as BabyAGI.

Initially, AutoGPT supported five storage modes by default:

• LocalCache (renamed to JSONFileMemory)
• Redis
• Milvus
• Pinecone
• Weaviate

However, reviewing AutoGPT's documentation now reveals a surprising warning:

AutoGPT has recently undergone a 'Vector Memory Refactor,' removing all vector database implementations, including Milvus, Pinecone, Weaviate, retaining only a few classes responsible for memory management. Presently, JSON files have become the default method for storing memories/embeddings.

Why?

Maintainer Reinier van der Leer in May this year raised a query on GitHub regarding opinions on the 'value of adding different storage modes.' They were contemplating a refactor and intended to discard everything except the 'local' memory provider (now known as json_file) while striving to implement Redis VectorMemoryProvider.

Some developers expressed agreement, suggesting that if the backend is good enough, there's no reason to retain these vector databases. 'But I suggest integrating Pinecone (or Redis if it's advantageous) into a customized JSONFileMemory.'

As of now, AutoGPT's choice to "abandon" vector databases likely stems from the realization that the operational and usage costs of employing these databases outweigh their benefits. Under these circumstances, building a solution from scratch aligns better with the long-term gains of the project. After all, in software development, premature optimization can lead to high development costs and risks, resulting in uncontrollable software complexity.

Do we still need vector database?

For scenarios requiring storage of vast amounts of vectors, such as extensive image or audiovisual retrieval, it's evident that using a vector database can offer more powerful and specialized functionalities. However, for scenarios with less substantial data volumes, employing libraries like Numpy in Python for computations might be more efficient and convenient. Within the realm of vector databases, there are various types, including lightweight and heavyweight options. Choosing between utilizing plugins like pgvector on PostgreSQL or opting for a dedicated distributed vector database necessitates specific application analysis before making a decision.

As far as our current knowledge goes, not only AutoGPT but also other models like GPT Engineer, GPT Pilot, and even GitHub Copilot, refrain from using vector databases. Instead, they derive contextual relevance from recent files, proximity within the file system, or locating references to specific classes/functions.

The decision to use vector databases depends on the specific context, and AutoGPT's abandonment of vector databases marks an important step in the right direction. This move reflects a focus on delivering value rather than getting bogged down in the technical intricacies.