How to test publishing your JavaScript package locally

Since I've started using npm as a JS package manager (maybe back in 2015), I always wanted to publish my own packages which could be either a very particular package I'd use in my side projects or a package that attempts to solve a common problem and help the other devs.

Every time I needed to do that I also wanted to test the workflow of publishing and installing locally. Also, I'd like to see a kind of "preview" of how it'll look like when it's published, the "real" npm page, just to see if the README is ok for example.

After a lot of struggle and attempts with various approaches to solving this problem, I think I finally figure out the best (at least for me) of how to solve those 2 problems.

Before diving deep into the final solution, let me tell you about the problems I had with other solutions.

Symlink

Back in 2016, trying to find a way to do that I saw a lot of people talking about Symlink.

In short, Symbolic Link (or Symlink), is when you create a reference link between 2 (or more) files making just a reference to each other.

Imagine you have a library (my-library) and want to use the local files of it in your project (website). A Symlink in this case will be, inside the node_modules, instead of having the production files of my-library, it points to the local folder of it.

But... how to do that?

NPM/Yarn Symlink

Of course, I wasn't the only person in the world desiring a proper way of doing that. And that's why both npm and yarn provide a way of doing symlink out of the box.

I won't explain how to use that in this article, but if you still want to know, you can find a link to how to do that.

In a nutshell, what happens is by doing that, npm or yarn will:

Create a global reference to your package;
Replace the real module with that reference inside your node_modules

Using this approach solves most of the problem of testing packages locally cross any project... until it doesn't.

Symlink problem

My biggest pain-point with global Symlink was with nested node_modules and how the tools (at least back them) resolves what version of a module A (used in both the project and the library) was supposed to be resolved.

The first time I saw that was while writing a React component library. The workflow was:

Go to my library
Run yarn link to create a Symlink
Go to my project
Link my-library
Start my dev server

By only doing that, I started having issues with some React internal rules. That was weird because the error message was really true.

After a couple of hours digging into this problem I finally found an issue on React's repo reporting the exact same problem I had and he pointed about about the Symlink:

The answer from the maintainer was clear:

It's not expected that it would work unless you link react back from your module to your app.

This has actually always been the case (React apps are subtly broken when there are two copies of React module). Hooks surface this immediately which I guess is good.

We do have another issue tracking a better error message for this case.

Of course, it makes a lot of sense. In my component library, React was a peerDependency and I didn't ship that within the bundle. Now, using it via Symlink, React was installed in my library AND my project.

Someone posts a workaround for solving that where you would also need to link the react and react-dom inside the library and use that link in our project. So my flow would be:

Go to my library
Navigate to node_modules/react
Run yarn link to create a react symlink
Navigate to node_modules/react-dom
Run yarn link to create a react symlink
Go back to the root level and run yarn link to symlink my lib
Go to my project
Use the link of my-library, react and react-dom
Start my dev server

By doing that, my problem was gone! But... gosh. Really?

After finishing my tests I had to remove those 3 links from your project and force install the dependencies.

Doing that a couple of times was ok but after 10 times I got really annoyed and created a bash script to execute those steps for me.

Also, now I'm using Docker to run my projects and I've realized that Symlink does not work with my base setup.

Probably because when I run my container, I only create a volume which the current project folder. When the container is up and tries to use that Symlink, it might need to navigate through my file system and I think that's not possible.

It might be possible to do that by adding some extra configs but I just don't want to. I want an easy way of doing something in my lib, push and install it whenever I need to use WITHOUT polluting my real package at npm.

Also, using Symlink you can't tell for sure if you're shipping all files your application will need to work.

Luckily, I found a very simple way to solve that and I want to share it with you.

NPM Proxy Registry

Companies also want to grasp package management in their projects, but maybe some of them need to be private to protect their business and intellectual property.

NPM offers the service of using private packages but as you can imagine, it charges the company for that.

A solution for that would be using an npm proxy.

An npm proxy is just a server that sits in front of the official npm registry server and resolves the dependencies to you.

You can publish an internal package using it and instead of the proxy pushing your package to npm server, it'll store it in its own server.

By running npm install using a proxy server, under-the-hood you'll pass a list of packages you want to install. If the proxy has a package published in its "database", it'll return to you that package. If not, it'll ask for the NPM server that package and return it to you:

Private packages are one of the capabilities of npm proxies.

Imagine you fork axios and publish to your proxy server that modified version. When you run install, the proxy instead of returning axios from NPM, will return to you the version you've published.

You might have been thinking:

Hmm... If we can have this set up in a cloud server... what if we use that system locally?

Yes... that was the conclusion I've made while observing that solution and that's how we'll tackle the problem of testing packages locally.

There are a couple of options to perform this workflow, but in this article, I'm gonna show you how to do that using Verdaccio.

Verdaccio

Verdaccio is:

A lightweight open-source private npm proxy registry

It's tremendously powerful and in version 4 it has NO config to start using it, which does not necessarily mean we won't need to do configs, but not really in the proxy itself.

Running a local server

The easiest way of using Verdaccio locally is installing as a global npm dependency:

yarn global add verdaccio

## Or with npm

npm install -g verdaccio

After that, you can start the server by running:

verdaccio

If you prefer, you can also run this server with docker:

docker run -it --rm --name verdaccio -p 4873:4873 verdaccio/verdaccio

Keep in mind that when the container will be stopped, the packages you've to publish would be removed.

After running the server, you can check the website at http://localhost:4873 (or http://0.0.0.0:4873)

Adding your npm user

To be able to publish a package into your local Verdaccio, you first have to register an npm user there. To do that, run:

npm adduser --registry http://localhost:4873 # OR http://0.0.0.0:4873

The information does not need to be secure or accurate. Remember, it's only a local thing! :)

Publishing and consuming

For both publishing and consuming your local package, you have to always specify what's the registry URL. In other words, what's the server npm must find the packages.

One way of doing that is by creating in the root level of the repo you want to consume/publish a file called .npmrc and specify the registry link there:

# /my-project/.npmrc

registry=http://localhost:4873 # OR http://0.0.0.0:4873

I strongly recommend this method for npm users. The reason is that npm asks you to sets a global registry via npm config set or via publishConfig.registry in your package.json. Both ways are a hassle to rollback when you want to use the regular npmjs.org registry.

With a custom .npmrc per project when you wanted to use from the official registry all you have to do is comment out the registry line in that file.

The other option is for Yarn users which consists in specifying the flag --registry:

# For publishing
yarn publish --registry http://localhost:4873 # OR http://0.0.0.0:4873

# For consuming
yarn add my-private-pkg --registry http://localhost:4873 # OR http://0.0.0.0:4873

By doing that, yarn will resolve the registry without any extra file nor config. If you eventually get annoyed with having to write the registry flag, you can also create a .npmrc file and yarn will also be able to resolve the registry URL from there.

After this configuration, when you publish or install your local package, npm or yarn will first ask your Verdaccio local server for that package, and Verdaccio will do all the job to store or retrieve local packages and resolve public packages at NPM.

Caveats

When we install a dependency, a bunch of information about it is added in package.lock.json or yarn.lock file and one of them is resolved, the URL where we got that dependency:

// package.lock
{
  // ....
 "node_modules/my-package": {
      "version": "1.6.0",
      "resolved": "http://localhost:4873/my-package-1.6.0.tgz",
   // ....
  }
}

This means that if you commit and push the lock file with the local server URL, wherever your project will be built or tested (like a CI/CD), it will fail because there's no localhost:4887 there.

In that sense, we always want to remember to clean this change before pushing any code.

Also, differently from Symlink where you can just turn a watch mode to compile the files and see these changes directly in your project, using this approach, you'll need to publish a new version and install that version every new change you've made.