Four Weeks of Experiments

At the beginning of 2022, I set the goal to experiment with different project ideas and technologies. I want to learn new things and build something, and after four weeks it is time to pause and reflect on the progress.

A Kind of New Year's Resolution

Around the beginning of the new year, I reflected on what I had done in the past year and what I wanted to do next. I felt like my growth as a software engineer had stagnated a bit. I was experimenting with Rust, either building small open source tools or trying to learn game development. At work, I was using a mix of Java, TypeScript, and mostly Bash scripts. But neither in my personal time nor at work did I feel like I was making much progress.

As a result, a goal for 2022 became to dedicate more time and focus to software development. Instead of only messing around with ideas, I want to push myself to actually "complete" one (assuming I can control the inevitable scope creep). I expect to learn from the process. Both in terms of "boring" work that I would typically try to avoid in my experiments, as well as in having something "in production" that I am solely responsible for.

But I started the year without a good idea what to do. There are a handful of projects that I have started, stopped, and restarted more often than I can count. My idea for a video game for programmers is one of those. But there are reasons why I eventually stopped working on them. Scope, feasibility, time and money. And I wasn't sure what kind of tech stack I wanted to explore. Stay with Rust, go back to backend development, or finally dive deep into React?

So I decided to just jump in and see where the road would lead me.

Building a GitHub App with Rust

Developer tools have always excited me. Maybe that is because I like solving my own problems. One idea that I have been coming back to many times is figuring out a better way to keep template files in sync on GitHub. A lot of configuration files in my projects are identical, and I tend to improve them as I learn more. Maybe most notably, my GitHub Actions are constantly evolving as the platform matures. When I make changes in repository A, I want those changes to also be applied to repository B that uses the same tech stack. This is a problem that can be solved with a GitHub App.

So I started the year working on this idea. I was set on using Rust for the project. It only requires a simple web server to interface with GitHub, and then a bunch of business logic to run through the actual workflow of syncing the files. I had just bought the book Zero to Production, and this was a great opportunity to implement its lessons. It was easy to get started, and the first few days were a lot of fun. I learned a ton by following the book, especially about areas like tracing and instrumentation that I wasn't yet familiar with.

But towards the end of the week, doubts started to creep up. For one, I was implementing a very narrow workflow. But it still required a lot of code just to get started. Interfacing with GitHub's webhook, GitHub's API, and the Git object model all introduced unforeseen complexities. Because I had to do all this work anyways, I started thinking of broadening the scope. In the end, I had a grandious idea of an automation engine that provides a flow-based, no-code environment to automate arbitrary workflows. Think Zapier but for developers. I really like the idea, but it is too ambitious for the time I have. And I didn't like the idea of investing that much time to build something that fully depends on someone else's platform, in this case GitHub.

You can read more about the first week in these devlogs:

Reflecting on this first week, I really liked the routine of journaling about the progress on the blog. Getting swept away by ever bigger dreams about an automation engine was fun as well. As was working with Rust. But the nagging feeling of building something that could become obsolete any day by a new feature of GitHub was frustrating.

Building a Desktop App with Rust

The second week is a logical consequence of the first, as it took what I liked about week 1 and does the opposite for the rest. Instead of aiming to build an overly complex and ambitious enterprise, it started with the goal to create a desktop app. A well-defined scope, no operational responsibility, and privacy-focused by keeping data on the user's device. Since it was still the beginning of the year, my goal was to build an app that helps users set goals. I wanted to solve my own problem again.

What I liked about week 1 was using Rust. And building a desktop app with Rust was a great excuse to check out two projects that I had been interested in for a long time:

Tauri is an alternative to Electron and creates desktop apps from web frontends
Yew is a framework to build web apps with Rust

The first two days were great and a lot of run again. I started with Yew, and really enjoyed playing with the framework. It takes a lot of inspiration from React, which feels quite intuitive. And it is still Rust, so you get all the benefits of the language. A strong type system, helpful compiler errors, and great developer tooling.

use yew::prelude::*;

use crate::components::page::{PageContent, PageHeader};

pub struct Dashboard {}

impl Component for Dashboard {
    type Message = ();
    type Properties = ();

    fn create(_ctx: &Context<Self>) -> Self {
        Self {}
    }

    fn view(&self, _ctx: &Context<Self>) -> Html {
        html! {
            <div id="dashboard">
                <PageHeader text="Dashboard" />
                <PageContent>
                    { "This is the dashboard's content." }
                </PageContent>
            </div>
        }
    }
}

But figuring out how to adept Yew for a desktop app and make it work well with Tauri was a bit too much for me. Tauri does not have a Rust SDK, and officially only supports Node apps. And Yew is designed for the browser, not a Node runtime. So I got stuck trying to figure out a way to access the local file system so that I could save the user's data.

I tried to work around this by embedding a web server in the Tauri app. The idea was to run a local web server that the web frontend could interact with, thus avoiding the need to access the file system from Yew. This approach also allowed me to go back to Zero to Production and build a web server. But it made things way too complicated. It also didn't help that I wanted to use GraphQL. In the end, I had thousands of lines of boilerplate code just to get a simple form saved.

Building a desktop app with Tauri and Yew just didn't feel feasible at this point in time. At least not with the knowledge that I had about both projects. And going back to my goals for 2022, I didn't feel that the experimentation I was doing was aligned with the goal of shipping something to production.

Ironically, a week later a blog post about building desktop apps with Tauri and Yew got published. It answered the questions that I had, and showed how to get past the problems that I was experiencing. But at that point I had already moved on.

Building a (Deskop|Web) App with Next.js

The next iteration was an attempt to replace Yew with Next.js. Next.js is a framework that I always wanted to try out, and it seemed logical at first to switch from Rust to Node. I would still be using a component-based framework, but with better support for Tauri.

I quickly remembered why I didn't like working with the Node ecosystem. Getting started with Next.js was easy, but soon enough I ran into the first problems. The testing framework didn't play nicely with the way that assets got imported. Tauri's NPM package wasn't compatible with the default Webpack setup (due to the use of ES modules). I probably spent more time debugging these issues than writing code.

More fundamentally, however, I realized that Next.js is not the right framework for desktop apps. It is designed for web applications, offering features such as server-side rendering and a way to build APIs. I am truly impressed by the framework, but it simply wasn't the right tool for the job.

I switched to React next, and spend a day or two experimenting with that approach. But when it came to saving data, I still ran into the build issues with Tauri and the question how to interact with the file system. I was ready to abandon the idea of building a desktop app. If I were to build a web app, I could use Next.js again. But how would I store data? Privacy is important to me, and I didn't want to just throw everything into Firebase or any other backend-as-a-service.

So I not only abandoned the idea of building a desktop app, I also abandoned the idea of using a frontend framework for the job.

In hindsight, this was probably the most frustrating experience of the month. I always have high hopes when I start working with TypeScript. But every time there are issues. Compared to the developer experience that I have with Rust or before that Ruby, I simply feel unproductive and mostly fighting the tools. Maybe that would change after a while, but I never get to the point because it's just too annoying.

Building a Web App with Rails

In a past life, I worked as a Ruby on Rails developer. I love the framework and the productivity it enables. And in December 2021, Rails 7 was released with a bunch of exciting new features. Hotwire is now the default solution for frontend development, combining the responsiveness of single-page applications with the benefits of server-side rendering. And Active Record Encryption makes it easy to protect sensitive data in an application. It checked all the requirements that I had after experimenting with React and TypeScript.

So in week 4–this week–I spun up a new development environment for a Rails app. The goal was still to build a tool for personal goal setting. But a web app enabled a few exciting features that would've been very difficult to add to a desktop app. The app can send notifications much easier, for example when a user hasn't updated their progress in a while. It is also possible to access the app from different devices, allowing users to check in whenever they want.

Progress during the week was slow, though. Other things required my attention, and I wasn't able to focus much on the project. I was also reminded of why I wanted to build a desktop app in the first place. Storing the user's data on the central server requires so much thought about security, privacy, and current regulations. I am not yet convinced that this is the direction that I want to take.

Reflecting on Four Experiments

In four weeks, I tried four different approaches of building a product. It started with a web service, radically pivoted to a desktop app, and from there evolved to the point where it became a web app again.

More than anything else, I am frustrated with my progress. At the beginning of the month, I thought that I would be working on one idea the whole time. And I expected to have a prototype in production after four weeks. But I feel very far away from that goal right now.

I am happy with the experimentation, though. The automation engine wasn't a good idea to pursue. It was too complex, too ambitious, and it would've taken months to get even a prototype out of the door. But I like the idea of the personal goal setting app, because I am passionate about the topic and it is very well scoped. Without experimentation, I wouldn't have ended up here.

I also have to acknowledge that I didn't know enough about building desktop apps at the beginning of the month. Only by trying and failing did I start to dig deeper into the topic. After I finish this post, I will spend the rest of the day reading the paper Local-first software about "a set of principles for software that enables both collaboration and ownership for users". Maybe that will change my goals for next month again, but I will have learned about an interesting new idea no matter what happens.

So I shouldn't feel frustrated with the progress. I did learn a lot and got to try a a bunch of frameworks that were on my to-do-list.

Process Over Outcome

I actually start to think that I am not frustrated with the outcome, but with my process during the month.

A goal of mine was to establish a routine, and I definitely failed to do that. As mentioned in the beginning for example, I really liked writing short(er) blog posts that reflected on the day. But I stopped doing that after the first week. I also wanted to relaunch my podcast, which I haven't done yet. And I wanted to try live-coding on Twitch, which also didn't happen. I set out with an idea for how I want to work, and I failed to follow through on that.

So looking forward to the next months, I want to focus on the process more than on the outcomes.

One aspect of that is certainly to set intentions again. I want to set a goal for the week and then for each day, and check that I am on track towards those goals. Writing a short blog post at the beginning in the morning was definitely helpful in establishing a goal for that day. But it is also a lot of overhead, so I'll have to experiment and see what formats works. Maybe setting the goal is better done by creating tasks in OmniFocus, and then reflecting on the week in a blog post just like this one. I am also considering adopting the Shape Up methodology to plan a week ahead and then focus on execution during the week. I will have to experiment with this and see what works best for me.

This blog post is now a reflection of the whole month, and I feel that time is too long. A weekly or bi-weekly interval would be better suited to spot trends and react in time. There were multiple moments where I could've stopped and reflected on how things were going during the past month. That would've allowed me to react earlier, and possible avoid the feeling of frustration that I feel now.

A week as a unit for planning and reflection, and then days with a certain routine feel like the most promising approach. I do have to be mindful of my other responsibilities, though. I can make plans for my side project, but there will be days were I just won't have the time to work on it.

So let's try to enjoy the process and focus less on outcomes.

Blog