Firestore Counters - Everything There is to Know!

UPDATE: 2/17/24

This article out-of-date, as there have been many changes and updates to Firestore. Please see my latest blog post:

https://code.build/p/the-four-ways-to-count-in-firestore-nNnqKF

As I stated in this post, Firestore does not handle counters automatically like nearly every other single database.

I personally think it is ridiculous. Please send a 10,000th feature request to Firebase here, they literally won't read it, and may give you excuses like scalability issues, costs, and sharding. This is ridiculous. This post (which is disabled because they don't want to hear complaints) is ignorant.

Yes, I am talking to you three as well:

You guys may actually have no say in this matter, but I see you all active in the community so I need someone to complain to. I also conversely apologize if there is someone more active than Frank I did not mention. :0

Indexes

Add the ability to create an index for counters in Firestore. You can do this for search, why not do this for counters? That way it is on the USER if they want to pay more etc. I believe the indexes could use Order Static Trees, as I have mentioned before.

This should be a given (as well as search, but I already complained about that). The sharding should be done automatically under the hood, NOT in a Firebase Function.

There is not just one type of counter. Here are the ones I could think of:

Collection Counters - Total Number of Documents in a Collection
Query Counters - Total Number of Documents in a Query of a Collection, including where filters, etc.
Custom Counters - Page Views, Likes, Bookmarks etc.

Collection Counters

Method 1: - use my adv-firestore-functions package... one line of code in your firebase function... done.

Backend Version Notes

If you decide to implement my collection counter function instead of Firestore Rules, create a separate function from the rest of your code. You will quickly learn that if one thing messes up in your Firebase Functions, the rest of the code is moot, and your count is off.

Method 2: - Firestore Rules - See Below...

Query Counters

Method 1: - Condition Counters

Method 2: - Query Counters

Custom Counters - Page Views

Method 1: - Callable Function / Simply add a counter manually. Page views could also be done to track the user's IP address or UID to count unique IDS etc...

Method 2: - Google Analytics - You could use Firebase Google Analytics to display your page views. I didn't google too deep on this, but it is possible.

Method 3: - Distributed Counters - Whenever you deal with any real scaling, you need something like the sharding to accurately keep track of document counts. If you have more than 1 page view a sec, or more than one person clicks like a post a second, the server will slow down, and potentially be inaccurate. You can fix this with sharding. Here is the Firebase Extension and the Source Code. The code is really well written, but it should never have had to be written! Indexes, indexes, indexes!

It scales from 0 updates per second to a maximum of 10,000 per second. Twitter only writes 6000 new tweets a second, so you're good; although that is just an average.
The Distributed Counter is not good for Counting Documents from an onWrite() trigger. You would need to add idempotent protection, as the functions could run more than once. I already wrote this for you in my adv-firestore-functions as an event. Simply see if an event exists (if the function has been run), if not it creates it!
It is ironic that the classic example is likes, when you can't do a follower feed in Firestore.

Firestore Rules

These are pretty much good for many situations.

Pros: You can't screw up the count from the frontend, as your rules will prevent it.
Cons: You need more front end code, and you can still screw it up from the backend (with Functions or directly from the console)

However, here is my method. Replace your set and delete functions with these batch functions. The set will automatically create the counter document in _counters, so make sure it is writable. If you already have a few documents, it will auto-count it, but don't use if for a mature app, as it cannot count past 500.

Frontend

set

async setWithCounter(
  ref: DocumentReference<DocumentData>,
  data: {
    [x: string]: any;
  },
  options: SetOptions): Promise<void> {

  // counter collection
  const counterCol = '_counters';

  const col = ref.path.split('/').slice(0, -1).join('/');
  const countRef = doc(this.afs, counterCol, col);
  const countSnap = await getDoc(countRef);
  const refSnap = await getDoc(ref);

  // don't increase count if edit
  if (refSnap.exists()) {
    await setDoc(ref, data, options);

    // increase count
  } else {
    const batch = writeBatch(this.afs);
    batch.set(ref, data, options);

    // if count exists
    if (countSnap.exists()) {
      batch.update(countRef, {
        count: increment(1),
        docId: ref.id
      });
      // create count
    } else {
      // will only run once, should not use
      // for mature apps
      const colRef = collection(this.afs, col);
      const colSnap = await getDocs(colRef);
      batch.set(countRef, {
        count: colSnap.size + 1,
        docId: ref.id
      });
    }
    batch.commit();
  }
}

delete

async deleteWithCounter(
  ref: DocumentReference<DocumentData>
): Promise<void> {

  // counter collection
  const counterCol = '_counters';

  const col = ref.path.split('/').slice(0, -1).join('/');
  const countRef = doc(this.afs, counterCol, col);
  const countSnap = await getDoc(countRef);
  const batch = writeBatch(this.afs);

  // if count exists
  batch.delete(ref);
  if (countSnap.exists()) {
    batch.update(countRef, {
      count: increment(-1),
      docId: ref.id
    });
  }
  /*
  if ((countSnap.data() as any).count == 1) {
    batch.delete(countRef);
  }*/
  batch.commit();
}

Note: I use Angular Firestore 9, but any framework should be easily translated. You can also uncomment out the bottom lines if you want to delete an empty counter document (count=0).

Backend Security Rules

function counter() {
  let docPath = 
/databases/$(database)/documents/_counters/$(request.path[3]);
  let afterCount = getAfter(docPath).data.count;
  let beforeCount = get(docPath).data.count;
  let addCount = afterCount == beforeCount + 1;
  let subCount = afterCount == beforeCount - 1;
  let newId = getAfter(docPath).data.docId == request.path[4];
  let deleteDoc = request.method == 'delete';
  let createDoc = request.method == 'create';
  return (newId && subCount && deleteDoc) 
  || (newId && addCount && createDoc);
}

function counterDoc() {
  let doc = request.path[4];
  let docId = request.resource.data.docId;
  let afterCount = request.resource.data.count;
  let beforeCount = resource.data.count;
  let docPath = /databases/$(database)/documents/$(doc)/$(docId);
  let createIdDoc = existsAfter(docPath) && !exists(docPath);
  let deleteIdDoc = !existsAfter(docPath) && exists(docPath);
  let addCount = afterCount == beforeCount + 1;
  let subCount = afterCount == beforeCount - 1;
  return (createIdDoc && addCount) || (deleteIdDoc && subCount);
}

and use them:

match /posts/{document} {
  allow read;
  allow update;
  allow create: if counter();
  allow delete: if counter();
}
match /_counters/{document} {
  allow read;
  allow write: if counterDoc();
}

You could greatly simplify the functions, but I wanted them to be easy to read, and universal so you don't have to think about it, and they work for all root collections. You're welcome.

The _counters/posts document will have a count=X on it. This is the same format as my colCounter() backend version.

More Reading - I really enjoyed this posts. I have already written the backend version, and better IMHO, but he has a lot of good points and knowledge here.

I also want to give a shout-out to the core idea of my Firestore Rules Counters to this post.

If you made it this far, like my post. I believe I have unique info here and I am just using these posts to procrastinate writing a real project with a real database.

Update: 10/2/21

I thought I would share the code if you want to also update the counter for a user. Ex: The number of posts a user has:

function userCount() {
  let colId = request.path[3];
  let docPath = 
/databases/$(database)/documents/users/$(request.auth.uid);
  let beforeCount = get(docPath).data[colId + 'Count'];
  let afterCount = getAfter(docPath).data[colId + 'Count'];
  let addCount = afterCount == beforeCount + 1;
  let subCount = beforeCount == beforeCount - 1;
  return (addCount && request.method == 'create')
  || (subCount && request.method == 'delete');    
}
match /posts/{document} {
  allow read;
  allow update;
  allow create: if counter() && userCount(); 
  allow delete: if counter() && userCount();
}

Which will check the update for users -> uid -> postsCount

async setWithCounter(
  ref: DocumentReference<DocumentData>,
  data: {
    [x: string]: any;
  },
  options?: SetOptions,
  uid = ''
): Promise<void> {

  options = options ? options : {};

  // counter collection
  const counterCol = '_counters';

  const col = ref.path.split('/').slice(0, -1).join('/');
  const countRef = doc(this.afs, counterCol, col);
  const countSnap = await getDoc(countRef);
  const refSnap = await getDoc(ref);

  // don't increase count if edit
  if (refSnap.exists()) {
    data.updatedAt = serverTimestamp();
    await setDoc(ref, data, options);

    // increase count
  } else {
    const batch = writeBatch(this.afs);
    data.createdAt = serverTimestamp();
    batch.set(ref, data, options);

    // if userCount
    if (uid) {
      batch.update(
        doc(this.afs, `users/${uid}`),
        {
          [col + 'Count']: increment(1)
        }
      );
    }

    // if count exists
    if (countSnap.exists()) {
      batch.update(countRef, {
        count: increment(1),
        docId: ref.id
      });
      // create count
    } else {
      // will only run once, should not use
      // for mature apps
      const colRef = collection(this.afs, col);
      const colSnap = await getDocs(colRef);
      batch.set(countRef, {
        count: colSnap.size + 1,
        docId: ref.id
      });
    }
    batch.commit();
  }
}

async deleteWithCounter(
  ref: DocumentReference<DocumentData>,
  uid = ''
): Promise<void> {

  // counter collection
  const counterCol = '_counters';

  const col = ref.path.split('/').slice(0, -1).join('/');
  const countRef = doc(this.afs, counterCol, col);
  const countSnap = await getDoc(countRef);
  const batch = writeBatch(this.afs);

  // if userCount
  if (uid) {
    batch.update(
      doc(this.afs, `users/${uid}`),
      {
        [col + 'Count']: increment(-1)
      }
    );
  }

  // if count exists
  batch.delete(ref);
  if (countSnap.exists()) {
    batch.update(countRef, {
      count: increment(-1),
      docId: ref.id
    });
  }
  if ((countSnap.data() as any).count == 1) {
    batch.delete(countRef);
  }
  batch.commit();
}

Obviously you should simplify these functions if you need more of them... (make a before-after function check etc)

Blog