When zx first appeared, it was a tiny esm script that just proposed a new idea for how child_process.spawn API could be enhanced with string template literals.

const result = await $`call smth --with any ${args}`

As we now know, this was exactly what JS folks had been looking for for many years :-). The main goal was to improve the developer experience, so we focused on finding the best combination of utils to solve everyday problems, expand the area of application.

Short history

The project went through 8 major releases, but in the context of the assembly issues, there are only 3 significant ones.

Packed as CommonJS

Despite the fact that the future belongs to ESM, many users have to remain in the notation of their legacy codebase (without any irony). So we have added compatibility with the require API for them.

https://github.com/google/zx/commit/303feea3cd0d95e8d733ee72b96b3316ea5827aa

Dropped CJS support

On the other hand, there were libraries on which zx depend: and only the latest modern major versions received patches, including vulnerabilities fixes. In fact we inherited CVE issues transitively, which was completely unacceptable. And we switched back to ESM.

https://github.com/google/zx/commit/5578d67ebae2049d430bf6489784f6a2aa47b0b1

Migrated to TypeScript

As the code base grew, it became harder for us to ensure consistency of internal contracts. Adding typing was the solution, but it also made us dependent on transpilation and we could no longer control the package artifact in detail.

https://github.com/google/zx/commit/8c2aea89fe7c65767d29c52bfb48f6a46af01864

Problems

Size

Filling the zx with new functionality, we continued to expand the dependency tree, which required their own dependencies, etc. Installation time increased significantly. And at some point it became impossible to ignore: zx#712, zx#669

Compatibility

When we define node engine requirement we limit not users, but ourselves in what platform API we can use. But this in no way prevents our dependencies from pursuing their own policies which affects zx.

Stability & Security

Reproducible setups are important. Especially for utilities of our type. We can try to pin down versions, but if there is a caret (^, ~) or an asterisk (*) at the secondary level, this introduces node_modules variability. Lockfiles can mitigate the problem, but not for npx/yarn dlx execution case.

Bundle

At first we wanted to just get rid of all the helper utilities. Keep only the kernel, but this would mean a loss of backward compatibility. We needed some efficient code processing instead with recomposition and tree-shaking. We needed a bundler. But which one? Our testing approach relies on targets, not sources. We rebuilt the project frequently, speed was critical requirement. In essence, we chose a solution from a couple of among all available alternatives: esbuild and parcel. Esbuild won. Specifically in our case, it proved to be more productive and customizable.

js

Customizable means that we found a way to adapt the tool for our tasks. And a lot had to be improved, including writing our own plugins:

1. Esbuild provides code-splitting with dynamic imports to reuse common code chunks. But not is static form. The manual chunks API is missing.
   esbuild-plugin-entry-chunks

2. By default, esbuild injects helpers into each cjs module. It's fine, when you have just a few, but definitely not when there are many.
   esbuild-plugin-extract-helpers

3. We also wanted to bring back cjs support, but at the same time avoid duplicating code in two versions of the bundles.
   esbuild-plugin-hybrid-export

4. Esbuild and its plugins mostly focus on sources processing, but sometimes additional modification is also necessary for dependencies or bundles:

• Polyfill injects: esbuild#2840, esbuild#3517, esbuild#3099
• Custom patches: esbuild#3360
• Dynamic banners: esbuild#3291
• Importable helpers: esbuild#1230.

esbuild-plugin-transform-hook

d.ts

While we were fighting against the modules, we forgot one small detail - their built-in typings. Esbuild can't do this at all yet. Unbelievable, but the tsc, native TS compiler, also does not provide a typings concat feature. Got around this problem: we've introduced a utility to combine typings of zx own code, and applied some monkey patches for external libdefs squashed via dts-bundle-generator.

Results

There is no silver bullet, all tools and approaches have limitations. Summary:

• Install size reduced: 16Mb → 1Mb
• Reproducible npx invocations
• Hybrid package: ESM and CJS
• Wide range of supported Node.js versions 12...22
• Easy to audit: complete code in one place

Perhaps our efforts are not commensurate with the profit. But we gained invaluable experience. Now we understand the entire zx code in detail down to each char.

zx build setup

#!/usr/bin/env zx

await $`cat package.json | grep name`

const branch = await $`git branch --show-current`
await $`dep deploy --branch=${branch}`

await Promise.all([
  $`sleep 1; echo 1`,
  $`sleep 2; echo 2`,
  $`sleep 3; echo 3`,
])

const name = 'foo bar'
await $`mkdir /tmp/${name}`

npm install zx