Picture - Outside the cathedral at Trinity College at the University of Cambridge, Cambridge, UK. Taken by myself in Summer 2018. I just really like it.

As it has been a while since posting here on Dev, I thought I would give a bit of an update.

The Job Offer

I had alluded to getting a job offer in my last post , and I have since accepted it. After graduating from my PhD program in Summer 2020, I will be starting as a Software Development Engineer (SDE I) at Amazon Web Services. Needless to say, I am incredibly excited and proud of myself for getting the position, having not come from a traditional Computer Science background and being largely self-taught.

The final decision was not an easy one for me, as it was a final confirmation to myself that I really wanted a change in my life. I realized that I wanted something else in life other than what I initially envisioned for myself 12 years ago, and finally admitting that to myself and to my academic peers was not easy. A lot of my identity is still wrapped up in being an academic, and I'm sure that the transition will be not be overnight. However, despite this, I don't regret deciding to study Astrophysics. I think it is a wonderful field that both inspires the imagination and teaches you how to solve complex problems, and it will always be part of who I am in some way.

Containers With Docker And Kubernetes

As I am going to be working with a lot of Cloud concepts, I have started to flesh out my understanding of containers: what they are, what they are used for, and how I go about making use of them myself.

I actually started the process a bit backwards due to a lack of a firm grasp on where to start. I started with the official the Kubernetes tutorials , which by themselves are nice introductions to distributed systems and how they help better ensure service reliability.

After going through the tutorials, working through them on the Kurbernetes sandbox, the documentation then suggested looking at a few courses on third-party platforms. This led me to Pluralsight and the work of Nigel Poulton .

After a brief course overview on his Kubernetes course, I realized that I needed to start with Docker if I really wanted to understand containers. For those that are new to these technologies, think of Docker as a service that allows you to host isolated copies of a service with all the various dependencies (known as a container) on a single or multiple machines. Kubernetes is more of a way to manage these containers, balancing requests and automatically handling failure events. They are in the same technology regime, but looking at the problems from different angles. Note that it is possible to handle management across multiple machines with Docker via the use of Docker Swarm, but it seems like Kubernetes is the preferred choice in this space.

I have gone through:

1. Docker and Kubernetes - The Big Picture
2. Getting Started With Docker

The latter of the two was especially interesting, as it allowed me to use the free-tier AWS EC2 service while I was following along in the tutorials in building the container swarm. I am currently going through his Docker Deep Dive .

Assuming you can afford a Pluralsight subscription and you are interested in learning about these technologies, I would start there. Nigel is a fantastic teacher, explaining things in an entertaining and clear manner. While I have not read it, Nigel also wrote his own book on the subject that may provide a more economic way to learn it.

MongoDB

One thing that I never have understood about the culture of Astrophysics is our choice to not utilize more modern database technologies. Most of us have only seen one or two SQL queries in our entire career, and anything more complex than that is relegated to being stored as often-convoluted directories of .txt, .hdf5, .npy, and .fits files. We hope that our naming scheme is good enough for our future selves to be able to traverse the file maze 6 months down the line (and so often...it isn't).

Sure, some of this is due to the often very specific use cases, but I honestly think much of this is due to a lack of willingness to change. This has led to mountains of technical debt and the looming crisis of Big Data Astronomy.

Because of this lack of domain experience, I have taken an interest in learning about databases and query languages. I started this process a few months ago when I chose to learn SQL through Codecademy as part of their Web Development path.

Recently, I decided to learn more about NoSQL, and in particular MongoDB. I chose to focus on this technology due to its popularity and it having incredible learning resources on the official website. In particular, for those who are interested, I would highly recommend checking out the official MongoDB University , which provides Coursera-quality courses for free.

On MongoDB University, I started the Developers path and have finished:

1. M001 - MongoDB Basics
2. M103 - Basic Database Administration

These classes provide an excellent introduction to the query syntax and the concepts of this document-based query language. The second one is especially useful in learning about database concepts, like replication and sharding.

I think learning MongoDB while also working on learning more about Docker and Kubernetes actually makes a lot of sense. They both rely on the concept of a distributed system, and how one manages problems of service scaling and reliability, and as such the concepts learned in one will reinforce what is learned in the other.

After I have a bit more time, I will be continuing with M220JS - MongoDB for Javascript Developers .

Side Projects

As I am an incredibly busy graduate student, much of my dev learning is done during lunch or, if I have the mental capacity left, after work. My work on my side projects has been a bit slower than I would have liked it to be, but I suppose I should give myself some slack given that I just went through the existential crisis of finding (and deciding to accept) a dev job.

I am currently working on a URL shortener project (MERN stack), and have completed the skeleton of the Express middleware. I am currently using a simple Javascript object for now to represent the database (just for testing before hooking it up to a MongoDB instance), and the React front-end is still in its early stages.

Conclusion

Thanks for those that read my posts and have followed me here on Dev (over 100!). I am still trying to work out my niche on what technical things I would like to talk about, so forgive me if I mostly just do these simple personal blog posts. I try to include resources that I have found useful, as I hope that my readers will be able to take something from what I've written.

Hope you all have a great day, and happy coding!