Consider a scenario, you launched a social media app and it's doing great. Within a week or so you managed to get around 30k users worldwide. Excited right! but things started going wrong. The feature where one user can search other users is taking more time to respond than expected, which may lead to a bad UX. And you know what users hate to wait.


That's where indexing can save your life. Indexing is a technique used in databases which help to optimize search or disk reads in a database while processing a query.
Enough of theory to understand now let's jump into code.
We will work with data of 10k pets. Each document has fields name, type, breed, age and index. Download the sample dataset from here.

To import this json file into a collection fire the command in your terminal.

mongoimport --db test --collection pets --authenticationDatabase admin --username <user> --password <password> --drop --file ~\downloads\data.json

This imports data.json file into test database with pets as collection name.
Now try out a search query, let's find pets with name Fido. You can run these commands in terminal or use a GUI tool like MongoDB compass or robo3T.

db.pets.count({ name: 'Fido' })

This query returns the number of pets having a name Fido. Now let's see how much time is taken to execute and fetch results for this query. This can be done using mongoDB's cursor.explain.

db.pets.find({ name: 'Fido' }).explain("executionStats")

In this screenshot, you can see executionTimeMillis: 13 which shows this query takes 13ms. In winning plan, it shows stage: 'COLLSCAN' which means it would sequentially scan all 10k documents in the database. Imagine running such a query with millions of documents in the database. Mind blown right! it would take forever to respond to such query.


Let's optimize this by creating an index. An index is essentially a search optimized data structure ( B/B+ Tree ).

db.pets.createIndex({ name: 1 })

This command would create an index over the key name, however, you can make indexes over as many keys as you want.

.

Now again fire the same search query and examine the execution stats.

db.pets.find({ name: 'Fido' }).explain("executionStats")

The executionTimeMillis shows 3 which means this search now takes 3ms. From 13ms to 3ms, damn fast right!


If you closely look at the results, you can observe a lot more changes.


Here, in winning plan input stage, you can see stage:'IXSCAN' which means now this query scans keys of the index, not the entire collection. The result has a lot more information to observe. That's it for this post, in part 2 of this post I would show how you can implement full-text search using pure MongoDB without any library or something.
Till then you can connect to me over Linkedin or Github..

cover by Gregory Taxerman on dribble