Migrating a 150K LOC codebase to Vite and ESBuild: How? (Part 2/3)

noriste

Stefano Magni

Posted on May 26, 2021

Migrating a 150K LOC codebase to Vite and ESBuild: How? (Part 2/3)

The meticulous work behind migrating our codebase to Vite, helpful to fail as soon as possible or to succeed the most brilliant way.


This is part of a three-article series about migrating our React+TypeScript codebase from Webpack to Vite. Part 1 is about why we decided to migrate, Part 3 is about post-mortem considerations.

Migrating the codebase

I could summarize the migration with the following steps:

  1. Compatibility: includes studying Vite, playing with it, and simulating our scenario outside the actual codebase.

  2. Feasibility: does our project works under Vite? Let’s migrate the codebase in the fastest way possible.

  3. Benchmarking: is Vite worthwhile? Are our early assumptions correct?

  4. Reproducibility: repeating the migration without messing up the codebase and reducing the required changes.

  5. Stability: being sure that ESLint, TypeScript, and the tests are happy with the updated codebase for Vite and Webpack.

  6. Automation: preparing the Codemods necessary to jump on Vite automatically.

  7. Migration: reaping the benefits of the previous steps.

  8. Collecting feedbacks: does the team like it? What are the limitations once using it regularly?

In the following chapters, I’m going to deepen each step.

1. Compatibility

Probably the easiest step. Vite’s documentation is pretty concise and clear, and you don’t need anything more to start playing with Vite. My goal was to get familiar with the tool and to check out if and how Vite works well with the critical aspects of our project that are:

  • TypeScript with custom configuration

  • TypeScript aliases

  • Import/export types

  • named exports

  • aggregated exports

  • web workers with internal state

  • Comlink (used to communicate between workers)

  • React Fast Refresh

  • Building the project

  • Browser compatibility

  • React 17’s JSX transform compatibility

Quick and dirty, just creating a starter project through npm init @vitejs/app, experimenting with it, simulating a scenario with all the abovementioned options, and playing with it.

Honestly, I expected more troubles, but all went fine. The first impact with Vite is super positive 😊.

2. Feasibility

Just one and clear goal for this step: adding Vite to our codebase, no matter how. Seriously, no matter if I break TypeScript, ESLint, .env variables, and the tests, I only want to know if there are technicalities that prevent us from moving the project to Vite.

The reason behind this crazy and blind process is not succeeding the most elegant way but failing as soon as possible. With the least amount of work, I must know if we can move our project to Vite or not.

After reading even the ESBuild’s docs, the most impacting changes for us are

  • Adding three more settings to the TypeScript configuration (impacts a lot of imports and prevent from using Enums)
"isolatedModules": true,
"esModuleInterop": true,
"allowSyntheticDefaultImports": true
Enter fullscreen mode Exit fullscreen mode

ESBuild requires the first two ones. You can read why in its documentation. Please remember that ESBuild removes type annotations without validating them. allowSyntheticDefaultImports isn’t mandatory but allows us to keep the codebase compatible with both Vite and Webpack (more on this later)

resolve: {
  alias: {
    '@/defaultIntlV2Messages': '/locales/en/v2.json',
    '@/defaultIntlV3Messages': '/locales/en/v3.json',
    '@/components': '/src/components',
    '@/intl': '/src/intl/index.ts',
    '@/atoms': '/src/atoms/index.ts',
    '@/routing': '/src/routing/index.ts',
    // ...
  },
},
Enter fullscreen mode Exit fullscreen mode
  • Vite’s automatic JSONs conversion into a named-export module. Consider setting Vite’s JSON.stringify in case of troubles.

That’s all. After that, I proceed by fixing errors the fastest way possible with the sole goal of having the codebase working under Vite.

The most annoying part is the new TypeScript configuration because it requires many manual fixes on

  • re-exported types that we didn’t migrate earlier (export type { Props } from instead of export { Props } from)

  • Enums, not supported by ESBuild, replacing them with string unions (UPDATE: const enums aren't supported, thanks Jakub for noticing it)

and then

  • import * as instead of import for some dependencies

  • import instead of import * as for the static assets

Other problems come from the dependencies consumed only by the Web Worker because:

  • Every time the Web Worker imports a dependency, Vite optimizes it and reloads the page. Luckily, Vite exposes an optimizeDeps configuration to handle this situation avoiding a reloading loop.
optimizeDeps: {
  include: [
    'idb',
    'immer',
    'axios',
    // …
  ],
},
Enter fullscreen mode Exit fullscreen mode
  • If something goes wrong when the Web Worker imports a dependency, you don’t have meaningful hints. That’s a significant pain for me but, once discovered, Evan fixed it swiftly.

In the end, after some hours, our project was running on Vite 🎉 it doesn’t care the amount of dirty and temporary hacks I introduced (~ 40 unordered commits) because I am now 100% sure that our project is fully compatible with Vite 😊

3. Benchmarking

Reaching this step as fast as possible has another advantage: we can measure performances to decide if continuing with Vite or bailing out.

Is Vite faster than Webpack for us? These are my early and empiric measurements.

Tool yarn start app loads in React component hot reload ** web-worker change "hot" reload **
Webpack* 150s 6s 13s 17s
Vite* 6s 10s 1s 13s

* Early benchmark where Webpack runs both ESLint and TypeScript while Vite doesn't
** Means from CTRL+S on a file to when the app is ready

Even if the codebase grows up — we are migrating the whole 250K LOC project to a brand new architecture — these early measurements confirm that betting on Vite makes sense.

Notice: We want to reduce risk. Vite attracts us, Vite is faster, Vite is modern… But we aren’t experts yet. Therefore we keep both Vite and Webpack compatibility. If something goes wrong, we can fall back to Webpack whenever we want.

4. Reproducibility

The takeaways of the Feasibility step is a series of changes the codebase needs to migrate to Vite. Now, I look for confidence: if I start from the master branch and re-do the same changes, everything must work again. This phase allows creating a polished branch with about ten isolated and explicit commits. Explicit commits allow moving whatever I can on master, directly into the standard Webpack-based codebase to ease the final migration steps. I’m talking about:

  • adding Vite dependencies: by moving them to master, I can keep them updated during the weekly dependencies update (we installed vite, @vitejs/plugin-react-refresh, and vite-plugin-html)

  • adding Vite types

  • updating the TypeScript configuration with the aforementioned settings (isolatedModules, esModuleInterop, allowSyntheticDefaultImports) and adapting the codebase accordingly

  • transform our static-assets directory into Vite’s public one

Once done, the steps to get Vite up and running are an order of magnitude fewer.

5. Stability

Since most of the required changes are already on master, I can concentrate on the finest ones. That’s why this is the right moment to

  • fix TypeScript (remember, not included in Vite) errors

  • fix ESLint errors

  • fix failing tests (mostly due to failing imports)

  • add Vite’s .env files

  • add the scripts the team is going to use for starting Vite, building the project with Vite, previewing the build, and clearing Vite’s cache (FYI: Vite’s cache is stored in the local node_modules if you use yarn workspaces)

  • create the HTML templates

  • checking that all the Webpack configs have a Vite counterpart

Env variables and files deserve some notes. Our project consumes some process.env-based variables, valorized through Webpack’ Define Plugin. Vite has the same define options and has batteries included for .env files.

I opted for:

  • Using define for the env variables not dependent on the local/dev/production environment. An example
define: {
  'process.env.uuiVersion': JSON.stringify(packageJson.version),
},
Enter fullscreen mode Exit fullscreen mode
  • Supporting import.meta (where Vite stores the env variables) for the remaining ones.

According to our decision of supporting both Webpack and Vite, we ended up with the following type definitions (an example)

declare namespace NodeJS {
  export interface ProcessEnv {
    DISABLE_SENTRY: boolean
  }
}
interface ImportMeta {
  env: {
    VITE_DISABLE_SENTRY: boolean
  }
}
Enter fullscreen mode Exit fullscreen mode

and this Frankenstein-like function to consume the env variables

export function getEnvVariables() {
  switch (detectBundler()) {
    case 'vite':
      return {
        // @ts-ignore
        DISABLE_SENTRY: import.meta.env.VITE_DISABLE_SENTRY,
      }
    case 'webpack':
      return {
        DISABLE_SENTRY: process.env.DISABLE_SENTRY,
      }
  }
}

function detectBundler() {
  try {
    // @ts-expect-error import.meta not allowed under webpack
    !!import.meta.env.MODE
    return 'vite'
  } catch {}
  return 'webpack'
}
Enter fullscreen mode Exit fullscreen mode

I wouldn’t say I like the above code, but it’s temporary and limited to a few cases. We can live with it.

The same is valid for importing the Web Worker script

export async function create() {
  switch (detectBundler()) {
    case 'vite':
      return createViteWorker()
    case 'webpack':
      return createWebpackWorker()
  }
}

async function createViteWorker() {
  // TODO: the dynamic import can be replaced by a simpler, static
  // import ViteWorker from './store/store.web-worker.ts?worker'
  // once the double Webpack+Vite compatibility has been removed
  // @ts-ignore
  const module = await import('./store/store.web-worker.ts?worker')
  const ViteWorker = module.default
  // @ts-ignore
  return Comlink.wrap<uui.domain.api.Store>(ViteWorker())
}

async function createWebpackWorker() {
  if (!process.env.serverDataWorker) {
    throw new Error('Missing `process.env.serverDataWorker`')
  }
  // @ts-ignore
  const worker = new Worker('store.web-worker.ts', {
    name: 'server-data',
  })
  return Comlink.wrap<uui.domain.api.Store>(worker)
}
Enter fullscreen mode Exit fullscreen mode

About the scripts: nothing special here, the package.json now includes

"ts:watch": "tsc -p ./tsconfig.json -w",

// launches both Vite and TSC in parallel
"vite:start": "concurrently - names \"VITE,TSC\" -c \"bgMagenta.bold,bgBlue.bold\" \"yarn vite:dev\" \"yarn ts:watch\"",

"vite:dev": "yarn vite",
"vite:build": "yarn ts && vite build",
"vite:build:preview": "vite preview",
"vite:clearcache": "rimraf ./node_modules/.vite"
Enter fullscreen mode Exit fullscreen mode

Last but not least: I didn’t manage to have Vite ignoring the Webpack’s *.tpl.html files. I ended up removing the html extension to avoid Vite validating them.

6. Automation

Thanks to the previous steps, I can migrate the whole codebase with a couple of cherry-picks and some RegExps. Codemod is perfect for creating a migration script and run the RegExps at blazing speed.

I created a script that

  • remove the node_modules directory

  • transform the code by updating the TypeScript aliases through Codemod

  • re-install the dependencies

  • commit everything

Notice that the script must be idempotent — aka running it once or more times produces the same results — this is crucial when launching the script multiple times and applying it to both the master branch and the open PRs.

Here a small part of the script

# replace aliases pointing to directories (idempotent codemod)

codemod -m -d . - extensions ts,tsx - accept-all \
"'@(resources|components|features|journal)/" \
"'@/\1/"


# replace assets imports (idempotent codemod)

codemod -m -d ./app - extensions ts,tsx - accept-all 'import \* as(.*).(svg|png|jpg|jpeg|json)' 'import\1.\2'


# update some imports (idempotent codemods)

codemod -m -d . - extensions ts,tsx - accept-all 'import \* as tinycolor' 'import tinycolor'

codemod -m -d . - extensions ts,tsx - accept-all 'import \* as classnames' 'import classnames'

codemod -m -d ./apps/route-manager - extensions ts,tsx - accept-all 'import PIXI' 'import * as PIXI'
Enter fullscreen mode Exit fullscreen mode

Here you find the whole script. Again: the more you incorporate changes on master before the final migration, the better.

7. Migration

I designed the script to ease migrating all the open branches, but we opted for closing all the PRs and operate just on master.

Thanks to many prior attempts, and the refinements to the script, migrating the codebase is nothing more than cherry-picking the “special” commit and launching the Codemods.

Pushing the red button

In the end, the 30 hours spent playing with Vite, fixing and refining, paid off: after a couple of minutes, the codebase works both under Vite and Webpack! 🎉🎉🎉

The final vite.config.ts file is the following

import { defineConfig } from 'vite'
import reactRefresh from '@vitejs/plugin-react-refresh'
import { injectHtml } from 'vite-plugin-html'
import packageJson from '../../apps/route-manager/package.json'

// see https://vitejs.dev/config/
export default defineConfig(({ mode }) => {
  return {
    // avoid clearing the bash' output
    clearScreen: false,

    // React 17's JSX transform workaround
    esbuild: { jsxInject: `import * as React from 'react'` },

    define: {
      'process.env.uuiVersion': JSON.stringify(packageJson.version),
    },

    server: {
      port: 3003,
      strictPort: true,
    },

    plugins: [
      reactRefresh(),
      injectHtml({
        injectData: {
          mode,
          title: mode === 'production' ? 'WorkWave RouteManager' : `RM V3 @${packageJson.version}`,
        },
      }),
    ],

    json: {
      // improve JSON performances and avoid transforming them into named exports above all
      stringify: true,
    },

    resolve: {
      alias: {
        '@/defaultIntlV2Messages': '/locales/en/v2.json',
        '@/defaultIntlV3Messages': '/locales/en/v3.json',
        '@/components': '/src/components',
        '@/intl': '/src/intl/index.ts',
        '@/atoms': '/src/atoms/index.ts',
        '@/routing': '/src/routing/index.ts',
        // ...
      },
    },

    // the dependencies consumed by the worker must be early included by Vite's pre-bundling.
    // Otherwise, as soon as the Worker consumes it, Vite reloads the page because detects a new dependency.
    // @see https://vitejs.dev/guide/dep-pre-bundling.html#automatic-dependency-discovery
    optimizeDeps: {
      include: [
        'idb',
        'immer',
        'axios',
        // ...
      ],
    },

    build: {
      target: ['es2019', 'chrome61', 'edge18', 'firefox60', 'safari16'], // default esbuild config with edge18 instead of edge16

      minify: true,
      brotliSize: true,
      chunkSizeWarningLimit: 20000, // allow compressing large files (default is 500) by slowing the build. Please consider that Brotli reduces bundles size by 80%!
      sourcemap: true,

      rollupOptions: {
        output: {
          // having a single vendor chunk doesn't work because pixi access the `window` and it throws an error in server-data.
          // TODO: by splitting axios, everything works but it's luck, not a designed and expected behavior…
          manualChunks: { axios: ['axios'] },
        },
      },
    },
  }
})
Enter fullscreen mode Exit fullscreen mode

Please note that this

esbuild: { jsxInject: `import * as React from 'react'` }
Enter fullscreen mode Exit fullscreen mode

is helpful only if you, like us, have already upgraded your codebase to new React 17’s JSX Transform. The gist of the upgrade is removing import * as React from 'react' from jsx/tsx files. ESBuild doesn’t support new JSX Transform, and React must be injected. Vite exposes jsxInjecton purpose. Alternatively, Alec Larson has just released vite-react-jsx, and it works like a charm.

Last but not least: for now, I can’t leverage vite-tsconfig-paths to avoid hardcoding the TypeScript aliases in Vite’s config yet because, until we support Webpack too, the presence of “public” in the path makes Vite complaining

// Webpack version:
"@/defaultIntlV2Messages": ["./apps/route-manager/public/locales/en/v2.json"]

// Vite version:
'@/defaultIntlV2Messages': '/locales/en/v2.json'
Enter fullscreen mode Exit fullscreen mode

Cypress tests

Unrelated but useful: if you have Cypress-based Component Tests in your codebase, you can jump on Vite without any issue, take a look at this tweet of mine where I explain how to do that.

Benchmarks and conclusions

The final benchmarks confirm the overall speed of Vite

Tool 1st yarn start, app loads in 2nd yarn start, app loads in browser reload (with cache), app loads in React component hot reload ** server-data change "hot" reload **
Webpack 185s 182s 7s 10s 18s
Vite 48s 31s * 11s 1s 14s

* Vite has an internal cache that speeds up initial loading
** Means from CTRL+S on a file to when the app is ready

The comparison is merciless, but is it fair? Not really. Vite outperforms Webpack, but, as said earlier, we run TypeScript and ESLint inside Webpack, while Vite doesn’t allow us to do the same.

How does Webpack perform with a lighter configuration? Could we leverage the speed of ESBuild without Vite? Which one offers the best Developer Experience? I address these questions in part 3.

💖 💪 🙅 🚩
noriste
Stefano Magni

Posted on May 26, 2021

Join Our Newsletter. No Spam, Only the good stuff.

Sign up to receive the latest update from our blog.

Related