Guillaume St-Pierre
Posted on August 26, 2022
About three years ago, I posted an article on the (now defunct) Manifold blog on creating an offline-first app using React and CouchDB. Besides the fact the post is not available anymore, it was also very outdated given how it was built on a very old version of React. Yet, I think the subject matter of the article is still very much a concern today.
A lot of applications require their users to have a constant network connection to avoid losing their work. There are various strategies, some better than others, to make sure users can keep working, even when offline, by syncing their work once they come back online. The technology has improved a lot in three years and I still think CouchDB is a tool worth considering when building an offline-first application.
Join me again as well explore CouchDB and its features as we build a to-read list, which definitely isn't a to-do list in disguise.
What is CouchDB?
CouchDB is a NoSQL database built to sync. The CouchDB engine can support multiple replicas (Think of a database server) for the same database and can sync them in real-time with a process not dissimilar to git. That allows us to distribute our applications all over the world without the database being the limiting factor. These replicas are also not limited to servers. CouchDB compatible databases like PouchDB allow you to have synced databases on the browser or on mobile devices. That enables truly offline-first applications, users work on their own local database that happens to sync with a server when possible and required. The sync depends on the exact replication protocol chosen, and it can be manually triggered. With PouchDB, that happens when any changes trigger a sync. Of course, a server has to be up for the sync to happen! The replication will pause if the replica is offline, which enables the eventual consistency we'll talk about below.
When you create a document in CouchDB, it creates revision for easy merging and conflict detection with its copies. When the database syncs, CouchDB compares the revisions and changes history, tries to merge the documents, and triggers a merge conflict if it can’t.
{
"_id":"SpaghettiWithMeatballs",
"_rev":"1–917fa2381192822767f010b95b45325b",
"_revisions":{
"ids":[
"917fa2381192822767f010b95b45325b"
],
"start":1
},
"description":"An Italian-American delicious dish",
"ingredients":[
"spaghetti",
"tomato sauce",
"meatballs"
],
"name":"Spaghetti with meatballs"
}
All this is handled through a built-in REST API and a web interface. The web interface can be used to manage all your databases and their documents, as well as user accounts, authentication, and even document attachments. If a merge conflict occurs when a database syncs, this interface gives you the ability to handle those merge conflicts manually. It also has a JavaScript engine for powering views and data validation.
Back in 2019, CouchDB was used to power CouchApps. In short, you could build your entire backend using CouchDB and its JavaScript engine. I was a big fan of CouchApps, but the limitation of CouchDB -- and also of database-only backends -- made CouchApps far less powerful than a more traditional database+application server. As we walk the road to v4 (at the time of writing this article), CouchDB has become closer to an alternative to Firebase or Hasura than an alternative to your backend.
So, should I switch everything to CouchDB then?
As with everything in software engineering, it depends.
CouchDB works wonders for applications where data consistency doesn't matter as much as eventual consistency. CouchDB cannot promise all your instances will be consistently synced. What it can promise is that data will eventually be consistent, and that at least one instance will always be available. It’s used or was used by huge companies like IBM, United Airlines, NPM, the BBC, and the LHC scientists at CERN (Yes, that CERN). All places that care about availability and resilience.
CouchDB can also work against you in many other cases. It does not care about making sure the data is consistent between instances outside of syncing, so different users may see different data. It is also a NoSQL database, with all the pros and cons that come with it. On top of that, third-party hosting is somewhat inconsistent; you have Cloudant and Couchbase, but outside of those, you are on your own.
There are a lot of things to consider before choosing a database system. If you feel like CouchDB is perfect for you, then it’s time to fasten your seat belt because you’re in for an awesome ride.
What about PouchDB?
PouchDB is a JavaScript database usable on both the browser and server, heavily inspired by CouchDB. It's a powerful database already thanks to a great API, but its ability to sync with one or more databases makes it a no-brainer for offline capable apps. By enabling PouchDB to sync with CouchDB, we can focus on writing data directly in PouchDB and it will take care of syncing that data with CouchDB, eventually. Our users will keep access to their data, whether the database is online or not.
Building an offline-first app
Now that we know what CouchDB is, let's build an offline-first app with CouchDB, PouchDB, and React. When searching CouchDB + React for the initial article, I found a lot of to-do apps. I thought I was very funny by making the joke that I was creating a to-read app, all while claiming that a list of books to read is totally different to a list of tasks to do. For consistency, let's keep the joke alive. Also, to-read apps are totally different from to-do apps.
All the code for this application is available on GitHub here: https://github.com/SavoirBot/definitely-not-a-todo-list. Feel free to follow along with the code.
The first thing we need is a JavaScript project for our app. We'll use Snowpack as our bundler. Open a terminal located in a directory for the project and type npx create-snowpack-app react-couchdb --template @snowpack/app-template-minimal. Snowpack will create a skeleton for our React application and install all dependencies. Once it's done doing its job, type cd react-couchdb to get into the newly created project directory. create-snowpack-app is very similar to create-react-app in how it sets-up your project, but it's a lot less intrusive (You don't even need to use eject at any point).
To finish setting up the project, install all the dependencies with the following command:
npm install react react-dom pouchdb-browser
With our project in hand, we now need a CouchDB database. To keep things simple, let's start it in a docker container using docker-compose, which will allow us to start and stop it very easily. Create a docker-compose.yaml file and copy this content into it:
# docker-compose.yaml
version: '3'
services:
couchserver:
image: couchdb
ports:
- "5984:5984"
environment:
- COUCHDB_USER=admin
- COUCHDB_PASSWORD=secret
volumes:
- ./dbdata:/opt/couchdb/data
This file defines a CouchDB server with a few variables to set the admin username and password. We also define a volume that will sync the CouchDB data from inside of the container to a local folder called dbdata. This will help keep our data when we close the container.
Type docker compose up -d in a terminal opened in the same folder where you started this project. Once pulled, the container will start, and make your CouchDB database available under http://localhost:5984. Accessing this URL in your browser or with curl should return a JSON welcome message. To make our local application work, we have to configure CORS on our database. Access the CouchDB dashboard under http://localhost:5984/_utils in your browser. Use the configured admin username and password, then click on the Settings tab, followed by the CORS tab, then click on Enable CORS and select All domains ( * ).
Configuring PouchDB for our app
For this project, we'll be using a few hooks to configure PouchDB and fetch our to-read items. Let's start by configuring PouchDB itself. Create a directory called hooks and then create a file called usePouchDB.js in this directory, with this code.
// hooks/usePouchDB.js
import { useMemo } from 'react';
import PouchDB from 'pouchdb-browser';
const remoteUrl = 'http://localhost:5984/reading_lists';
export const usePouchDB = () => {
// Create the local and remote databases for syncing
const [localDb, remoteDb] = useMemo(
() => [new PouchDB('reading_lists'), new PouchDB(remoteUrl)],
[]
);
return {
db: localDb,
};
};
This hook uses the useMemo hook from React to create two new instances of PouchDB. The first instance is a local database, installed in the browser, called reading_lists. The second instance is a remote instance, which instead connects to our CouchDB container. Since we only need the local instance in our application, we return an object with that local database only.
Let's now configure the synchronization for those two databases. Go back to usePouchDB.js and update the code with these changes.
// hooks/usePouchDB.js
import { useMemo, useEffect } from 'react';
import PouchDB from 'pouchdb-browser';
const remoteUrl = 'http://localhost:5984/reading_lists';
export const usePouchDB = () => {
// Previous code omitted for brevity
const [localDb, remoteDb] = useMemo(...);
// Start the sync in a separate effect, cancel on unmount
useEffect(() => {
const canceller = localDb
.sync(remoteDb, {
live: true,
retry: true,
});
return () => {
canceller.cancel();
};
}, [localDb, remoteDb]);
return {
db: localDb,
};
};
We added a useEffect hook to start the two-way synchronization between the local and remote databases. The sync uses the live and retry option, which causes PouchDB to stay connected with the remote database rather than only sync once, and retry if the sync could not happen. This effect returns a function which will cancel the sync if the component happens to unmount while syncing.
It would be nice to show a small message to our users whenever the CouchDB database is disconnected or unavailable. PouchDB's sync provides events we can listen to like paused and active, which the doc mentions may trigger when the database is unavailable. However, these hooks are only related to the act of syncing the data. If nothing needs to be synced, the sync will trigger the paused event regardless of the state of the remote database and then ignore the state of the remote database. Instead, we need to use the info method on the database on a regular interval to check the status of the remote database.
// hooks/usePouchDB.js
import { useMemo, useEffect, useState } from 'react';
import PouchDB from 'pouchdb-browser';
const remoteUrl = 'http://localhost:5984/reading_lists';
export const usePouchDB = () => {
const [alive, setAlive] = useState(false);
// Previous code omitted for brevity
const [localDb, remoteDb] = useMemo(...);
useEffect(...);
// Create an interval after checking the status of the database for the
// first time
useEffect(() => {
const cancelInterval = setInterval(() => {
remoteDb
.info()
.then(() => {
setAlive(true);
})
.catch(() => {
setAlive(false);
});
}, 1000)
});
return () => {
clearTimeout(cancelInterval);
};
}, [remoteDb]);
return {
db: localDb,
ready,
alive,
};
};
We added the state hook for the variable alive, which will track if the remote database is available. Next, we added another useEffect hook to set up an interval that will call the info method every second to check if the database is still alive. Like the previous useEffect, we need to make sure to cancel the interval when the component unmounts to avoid memory leaks.
Fetching all the documents
With our PouchDB hook, we are ready to create our next hook for fetching all the to-read documents from the local database. Let's create another file in the hooks directory called useReadingList.js for the documents fetching logic.
// hooks/useReadingList.js
import { useEffect, useState } from 'react';
export const useReadingList = (db, isReady) => {
const [loading, setLoading] = useState(true);
const [documents, setDocuments] = useState([]);
// Function to fetch the data from pouchDB with loading state
const fetchData = () => {
setLoading(true);
db.allDocs({
include_docs: true,
}).then(result => {
setLoading(false);
setDocuments(result.rows.map(row => row.doc));
});
};
// Fetch the data on the first mount, then listen for changes (Also listens to sync changes)
useEffect(() => {
fetchData();
const canceler = db
.changes({
since: 'now',
live: true,
})
.on('change', () => {
fetchData();
});
return () => {
canceler.cancel();
};
}, [db]);
return [loading, documents];
};
This hook does a few things. First, we create some state variables for keeping the loading state and our fetched documents. Next, we define a function to fetch the documents from the database using allDocs, then adding the documents to our state variables once loaded. We use the include_docs option for the allDocs function to make sure we fetch the entire document. By default, allDocs will only return the ID and revision. include_docs makes sure we get all the data.
We then create a useEffect hook which starts the data fetching process, then listen to changes from the database. Whenever we change something through the app, or the synchronization changes data in the local database, the change event will be triggered and we'll fetch the data again. The live option makes sure this keeps happening for the entire lifecycle of the application, or until the listener is cancelled when the component unmounts.
Putting it all together
With our hooks ready, we now need to build the React application. First, open the index.html file created by snowpack and replace <h1>Welcome to Snowpack!</h1> with <div id="root"></div>. Next, rename the index.js file created by snowpack to index.jsx and replace the content of that file with this code:
// index.jsx
import React from 'react';
import { createRoot } from 'react-dom/client';
const App = () => null;
createRoot(document.getElementById('root')).render(<App />);
You can now start the snowpack app with npm run start, this should start the application, give you a URL to open in your browser, and show you a blank screen (normal since we return null from our app!). Let's start building our App component.
// index.jsx
// rest of the code remove for brevity
import { usePouchDB } from '../hooks/usePouchDB';
import { useReadingList } from '../hooks/useReadingList';
const App = () => {
const { db, ready, alive } = usePouchDB();
const [loading, documents] = useReadingList(db);
return (
<div>
<h1>Definitely not a todo list</h1>
{!alive && (
<div>
<h2>Warning</h2>
The connection with the database has been lost, you can
still work on your documents, we will sync everything once
the connection is re-established.
</div>
)}
{loading && <div>loading...</div>}
{documents.length ? (
<ul>
{documents.map(doc => (
<li key={doc._id}>
{doc.name}
</li>
))}
</ul>
) : (
<div>No books to read added, yet</div>
)}
</div>
);
};
The application loads our PouchDB hook, followed by our hook loading all our to-read items. We'll then return a basic HTML structure that can show a warning message if the database happens to disconnect, a loading message when we're fetching the documents, and finally the to-read items from the database. The _id property is the internal unique ID property in CouchDB/PouchDB, which makes a perfect key for our list items.
Showing all the items is pretty nice, but to be able to show any items, we need a way to add new to-read items to our database. Let's go back to our index.jsx file and add this code in these.
// index.jsx
import React, { useState } from 'react';
// rest of the code remove for brevity
import { usePouchDB } from '../hooks/usePouchDB';
import { useReadingList } from '../hooks/useReadingList';
// Component to add new books with a controlled input
const AddReadingElement = ({ handleAddElement }) => {
const [currentName, setCurrentName] = useState('');
const addBook = () => {
if (currentName) {
// If the currentName has data, clear it and add a new element.
handleAddElement(currentName);
setCurrentName('');
}
};
return (
<div>
<h2>Add a new book to read</h2>
<label htmlFor="new_book">Book name</label>
<input
type="text"
id="new_book"
value={currentName}
onChange={event => setCurrentName(event.target.value)}
/>
<button onClick={addBook}>Add</button>
</div>
);
};
const App = () => {
const { db, ready, alive } = usePouchDB();
const [loading, documents] = useReadingList(db);
const handleAddElement = name => {
// post sends a document to the database and generates the unique ID for us
db.post({
name,
read: false,
});
};
return (
<div>
{/* rest of the code remove for brevity */}
<AddReadingElement handleAddElement={handleAddElement} />
</div>
);
};
We added a new component to this file for adding new books to read. A separate component helps make the structure a bit clearer, feel free to extract it in another file. This component uses a state hook to control an input, and then triggers the post method on the local database when the Add button is clicked.
Go back to your browser and try adding a few books to read, they should show up in the list when the button is clicked.
Finally, it would be great to be able to set books as read or delete some books we don't want in our list anymore. Open the index.jsx file again and add this code in there.
// index.jsx
// rest of the code remove for brevity
const App = () => {
const { db, ready, alive } = usePouchDB();
const [loading, documents] = useReadingList(db);
// rest of the code remove for brevity
const handleAddElement = name => ...;
// The remove method removes a document by _id and rev. The best way to send
// both is to send the document to the remove method
const handleRemoveElement = element => {
db.remove(element);
};
// The remove method updates a document, replacing all fields from that document.
// like _id and rev, it needs both to find the document.
const handleToggleRead = element => {
db.put({
...element,
read: !element.read,
});
};
return (
<div>
{/* rest of the code remove for brevity */}
{documents.length ? (
<ul>
{documents.map(doc => (
<li key={doc._id}>
<input
type="checkbox"
checked={doc.read}
onChange={() => handleToggleRead(doc)}
id={doc._id}
/>
<label htmlFor={doc._id}>{doc.name}</label>
<button
onClick={() => handleRemoveElement(doc)}
>
Delete
</button>
</li>
))}
</ul>
) : (
<div>No books to read added, yet</div>
)}
{/* rest of the code remove for brevity */}
</div>
);
};
We added two functions in our App. The update method uses the put method to update a document. The post method on the local database creates a document without a unique ID and generates it once the element is inserted. put can both update and insert, but it requires an ID and revision to select the document to put. In our case, we use it using the existing document, toggling the read property. The second function uses the remove method with the document, which makes sure PouchDB can find the document and delete it.
Finally, we replaced the list of documents to add a checkbox and a button. When the checkbox is toggled, the update method will fire and toggle the read property. The button will fire the remove method to delete the element when clicked.
Go back to your browser and try toggling the checkboxes or deleting elements. It should work without any issues.
Testing the offline-first capabilities
Now, it's time to test the app while the database is offline. Open a new terminal where your project is located (so as not to kill the npm run start command) and type docker compose stop couchserver. You should immediately see the warning message appear in the React app. Yet, you should still be able to interact with the app and add/change/delete documents. Type docker compose start couchserver to restart the database and reload the page once the warning message disappears. Every change you made should still be in the app, and you should be able to see the change in the CouchDB dashboard.
Conclusion
We now have a functional app with an offline-first focus. Regardless of the state of the database, our users can keep adding books to read and set their read state. The message is an added bonus which helps our users know not to clear their cache until we have properly synced the app.
Of course, acting on the database directly from the client may not be the best solution for most apps. Especially if we sync that data without any validation from the database. Please let me know in the comments below if you'd like a second post in this series implementing a backend for validating and syncing data in an offline-first application.
I'd love to hear your thoughts - please comment or share the post
We are building up Savoir, so keep an eye out for features and updates on our website at savoir.dev. If you'd like to subscribe for updates or beta testing, send me a message at info@savoir.dev!
Savoir is the french word for Knowledge, pronounced sɑvwɑɹ.
Posted on August 26, 2022
Join Our Newsletter. No Spam, Only the good stuff.
Sign up to receive the latest update from our blog.
Related