Turning an idea to side project to finished app

Every developer should try to have some time to work on a side project, either to improve his development skills, or to feed his creativity hunger. That can be tough having a full time job but, the beauty of the side project is that you can work on it on your own pace. The first step is to find an idea that you would want to work on. It can be anything but it will help a lot with the procrastination that will eventually come, if it is something close to your own interests.
Also, keep in mind to try and experiment with new technologies during the development as it will help you advance your career but also keep it interesting till the end.

When you got the initial idea, the next hard step is to actually start working on it. Even if you actually start the process (which for the most ideas you will not) there is a big chance you will abandon the project relatively soon. But, there will be ideas worth pursuing and having fun trying to finish them. One of those ideas is the CookList app.

Idea from daily use

I was searching for a side project to work on but anything i started was quickly abandoned either by lose of interest or by pure laziness. Some ideas that i had were either too complicated or required a lot of work (not easily done alone). I tried to find something relative to my daily routine. One of my biggest interest is in music so, I tried to find something relative to that. I am also a regular Spotify user so, when i searched online and found out that they have an API to work with their data, I instantly started studying it to derive a plan for an application.

Dont be afraid to actually write down your thoughts, it helps

After some research on Spotify's great and really well documented API i came up with the root of an idea, for an app that will help users to create playlists and discover new music, based on their liked artists. What i did right after that is to look online for similar projects, to draw inspiration and find new things to add. There were some applications that really close to what i had first envisioned and others that provided some quality ideas in the end. Notably, the ones most important are these :

1. Music-Map is a web app that finds bands that a user will probably like depending on his favorite band. It is part of a bigger discovery system called Gnod or Global Network of Discovery.
2. everynoise is a great project that tracks and analyzes data from Spotify to generate a scatter-plot of all the music genres.
3. playlists is a very well known app that generates a playlist on Spotify based on a single artist/group that the user will provide. You can also search and listen other people's playlists.
4. Musicroamer is a web app that visualizes the relations of the artists you like, generating a graph starting by a single group/artist that the user will provide, using data from last.fm.

All the above projects gave me inspiration and helped me to fine-tune my idea and create something similar but also unique. The key difference of this idea is that the user will create a list of artists, starting with one that he will provide and following with related to that one and to his subsequent selections. This list will serve as the "recipe" for playlists as he, and other users later on, will be able to create playlists with random tracks from those artists. Furthermore, informations about the artists picks and also the tracks of the playlists will be visualized accordingly. And so, the journey to build CookList began (got it? because it creates "recipes"...ha).

App Development

Next up you have to pick the stack you want to work with. On the Front-End part, i was arguing between picking React (with i have only played around a little with some tutorials/small projects) or Angular (which is my main framework at work) and i ended up picking Angular because i wanted to checkout Teradata's Covalent UI Platform. On the Back-End part I played around with the idea of a Spring-Boot application but i ended up with a NodeJS along with Express Framework instead as it would have been much more suitable for the application needs. As for database I went with the safe choice of MongoDB.

I started implementing my backend service creating simple endpoints that i knew later on will transform into the actual application features. Another useful point that can be made here is that you don't have to make everything from scratch. Search and reuse what others have created and try to build upon them. What came as a great help at that point is a very nice NodeJS wrapper for the Spotify's API from thelinmichael. Using that I started creating endpoints that were searching for artists, then getting related artists, then getting tracks for a specific artist etc. Once I had the very basic logic written in endpoints, i knew i could start implementing a simple accompanying UI.

The UI part of the applications is a regular Angular 6 app but as i said i wanted to use Covalent UI Platform with it. The first iterations of the applications were build solely on Covalent's UI style but the outcome wasn't quite to my liking. Although it was handy that i could use all the powerful components of Covalent's Platform, the aesthetics of it were too much like a dashboard instead of an app. Although i kept to my initial decision for some time, in the end i removed most of Covalent and went on to transform a beautiful HTML5+CSS3 theme from html5up into Angular code. Moreover, i wanted to be able to create a couple of visualizations that will enrich the data i could get from Spotify's endpoints. For that purpose i chose the powerful Echarts library with which i could build beautiful charts and diagrams for the purposes of the application.

Coffee, the real power of the developer

The development went on to finish up the core functionality of the idea. The required steps could be summed up into the below :

1. A user should be able to type and search for an artist
2. Related artists of the searched one would be available for selection
3. Every subsequent selection would trigger a new request for related artists (populating the initial list with the new/unique ones)
4. Creating a pool of popular songs from the selected artists and randomly picking a subset of those.
5. If the user wants to save the resulting playlist to his Spotify, authorize him by loging in to Spotify and getting a required access token.
6. Create the playlist on user's profile and add the required tracks.
7. Save on MongoDB the created recipe and the resulting playlist.

After the initial rounds of development other ideas came up as well, like :

1. response caching ( node-cache is a very powerful library for that ).
2. storing track's features on MongoDB for created playlists so users could see recipe's graphs without logging in.
3. storing and re-using user's refresh token instead of requiring logging in every time
4. Tag filtering based on genres.
5. Utilize Unsplash's API ( along with their own unsplash-js library ) to download a random image to set as playlist's cover.

App Deployment

When all the above were finished and i was quite satisfied with the outcome, i decided to freeze the new features and go on with the deployment of the application. I searched online to check what my options were for serving a NodeJS application and, although during development i was experimenting with Heroku, eventually i ended up picking Google Cloud which offers a great free trial were you get to spend up to 300$ on your first year, more than enough for at least the tests i wanted to run. Most importantly, as i had previous experience with AWS infrastructure, i wanted to check Google Cloud as well, were i had limited experience mostly with Google AppEngine. From then on, i started studying about Docker and Kubernetes where i had very little knowledge and after a while i could build a docker image of my application, upload it to Google's Container Registry and through Kubernetes deploy it into a cluster, using gcloud cli tool.

Not that kind of containers

Last step was to make the application truly public. So, i went over to GoDaddy and bought a brand new .app domain and after some struggles ( .app domains enforce https, something i didn't know at first and resulted in some weird situations ) CookList was up and running for anyone to see and use.

Future Improvements

There are many more ideas and features that can be built into this project. The most obvious and missing one for a web app of the 21st century is the social integration like facebook sharing etc. Other than that, the next big feature on the core mechanism is to be able (upon user request with specific option flag) to mix along tracks from artists/groups that you didn't pick but you will most probably like. Another one is to be able to find related recipes (based on the artists that you picked) and also be able to filter the recipes on artists just like you can do now based on genres. Some other form of visualization or other way of using the track's feature data might be a good idea as well. Last but not least, this app can become a mobile application as well, either using a framework like NativeScript or by building it up from scratch for Android and Iphone.

Sum Up

The only real Playlist

All this work took me almost 2 months, based on Github's commit history although, most of that time i had limited job done during week days as most of my commits are during weekends. During that time I learned to use a new UI Platform, further develop my skills on NodeJS and MongoDB and, familiarize myself and learn to use Docker and basic functions of Kubernetes along with Google Cloud Infrastructure. On top of that, in the end my work was materialized into a working app with which i am quite happy and i see myself coming back to it and implement more things either from my ideas or from user's requests.
So, I strongly suggest everyone to do the same. Pick your next side project, start it up either by yourself or in a team of friends, develop features, learn new things and most importantly, have fun doing it. If all the above are there then you will benefit a lot from the process and if you get to the finish line, then you will also have something working and available for anyone to see.