hel-micro, module federation sdk, build-free, hot update, toolchain-independent micro-module solution, welcome to pay attention and understand

The origin of the module federation

Since the sudden emergence of Google's chrome browser and the official announcement of the release of the v8 js engine on September 2, 2008, it has swept the Internet world with extremely high operating efficiency, and has also captured a large number of users. The momentum of blocking made other major technology companies (apple, moliza, microsoft) feel a huge murderousness, and then everyone began to recruit troops and sharpen their knives to fight a bloody path. Since then, the js engine has entered the period of the arms race, among which Microsoft even He did not hesitate to abandon IE and began to push the edge browser with Microsoft's new js engine Chakra behind it. It is conceivable how much everyone attaches great importance to the cake of js engine, and The birth of v8 catalyzed a large number of well-known open source works, so that the js ecosystem has always maintained a very strong vitality, the most famous of which isnodejsborn in 2009, a server-side js runtime based onv8`, allowing The language js began to bloom from the front to the back, so that the following ridicule that was born very early is still circulating:

Any application that can be written in JavaScript, will eventually be written in JavaScript.

Modular Specifications

nodejs brings the modularization of commonjs to the server side, which makes the organization of large-scale js projects more orderly. At the same time, it also brings the super killer of npm, which improves the efficiency of module distribution and sharing to an unprecedented height.

The front-end application has also doubled with the complexity of network applications, which has led to a period of rapid expansion of code volume. At this time, an effective modular solution is urgently needed to solve how to split modules gracefully, and how to improve code reuse and scalability. maintenance and other issues.

At this time, the two mainstream modular solutions amd and cmd began to compete in the front end and finally established a large position. Their representative implementations are requirejs and seajs respectively. Little friends know or use it.

Engineering system

Although requirejs and seajs bring the implementation of modular specifications to the front end and inject a solid foundation for large-scale js projects, the modular specification alone still cannot solve the problem of how to communicate with npm ecology, and how to manage increasingly Complex module dependencies, if compatible with a series of problems such as new js features, in the final analysis, this involves a keyword engineering system, and then webpack and babel were born, and their goals are very clear , which solves the 4 major problems in the following figure

Then it became the de facto infrastructure standard for the front-end to start engineering system development.

webpack relies on excellent plug-in and loader mechanism, so that its ecology can continue to grow bigger and stronger, and eliminate other opponents such as gulp, grunt that are more tool-oriented in the process

The curse of npm

webpack and npm almost form a perfect partner state, but the resources originally obtained from cdn in the front end are merged into one package body by the packaging tool, which brings fatal update and deployment efficiency problems.

In some scenarios that need to be dynamically updated, this all in one packaging mechanism greatly reduces the deployment efficiency of the package body. This is not a problem of webpack and npm, but people's natural needs for the web environment Typical scenario requirements brought about by rapid iteration and high requirements for express delivery experiments.

Note: externals itself cannot completely solve the demands of dynamic update, it is only suitable for externally linking the underlying PR dependency package to cdn

bundless is coming

At the same time, as the project volume of webpack becomes larger and larger, new problems such as poor development experience (slow hot update), increased package size, and slow build speed (node_modules black hole) are also born. At this time, the development of the new generation The tools snow and vite are starting to eat away at the webpack market in the name of not packing.

They all use the browser's native modularization capability esm, skip the dependency analysis and packaging process required by webpack, and achieve millisecond-level debugging and startup under this design.

However, the extremely fast experience they bring cannot shake the deep siege of the entire webpack ecology. In fact, everyone is in the dual-engine drive mode of debugging based on vite and using webpack for production and packaging. After all, esm is popular It will take time.

The module federation sounded the horn of counterattack

Since everyone complains that wepack is slow to build, is there a way to skip the build step and allow users to combine multi-level dependency modules in their own way?

Of course there are, that is to take the pre-built path

Therefore, module federation was born. Its greatness lies in maintaining the current high-efficiency system of modularization, componentization and engineering of front-end development, allowing independent development and deployment of modules, and direct sharing through CDN, thus breaking free from npm packages. The shackles that cannot be updated dynamically, thus promoting the development and operation experience of the entire front-end world to a new level.

As long as there are more modules that can be promoted to the federation, the local startup speed will be faster!

Moreover, federated modules are born with dual identities, that is, they can be module consumers or module providers, which makes a natural grid relationship between module federation applications, and module distribution efficiency, deployment efficiency, and sharing efficiency are unprecedented. Boost!

The Achilles Heel of the Modular Federation

Is the implementation of module federation really perfect with webpack 5 or other tools? It does solve the problems of free build, dynamic update, cross-project sharing module

However, based on the existing compile-time plug-in mechanism, it is impossible to avoid the problems of strong binding of tool chain and remote module consumption relationship can only be determined at compile time

Just imagine, you need to use the technology of module federation, how many preconditions you need to do, and you need to upgrade a real toolchain! And the federation modules before different toolchains are mutually exclusive! The liquidity of modules is tied to the toolchain of your choice.

Module Federation New Revolution

The only solution to these two major problems is to sdk them, which is hel-micro's new thinking on the realization of module federation, and it is also the secret weapon to initiate a new revolution of module federation.

After sdkization, any technology stack and any tool chain can access the module federation technology losslessly and painlessly.

Module consumption relationship at runtime

The return from the tool chain to the js language itself means that the module consumption relationship is upgraded from compile time to runtime, which will greatly improve the flexibility of dynamic loading of remote modules and enable more complex services.

dimensionality reduction strike

Compared with the module federation implemented by relying on tool plug-ins, the implementation of hel-micro from the language level will cause a dimensionality reduction blow to other module federation implementations.

Compared with the traditional npm sharing method, hel-micro also has more efficient code sharing capabilities (runtime sharing)

Decrypt the sdkized core implementation

To achieve sdkization means that we must tap the hidden capabilities of the js language itself, and jump out of the traditional packaging process thinking, in order to achieve our ultimate goal.

Asynchronous import hidden capabilities

Usually we use the import keyword to statically import other modules in the header file, but in fact, import can be used as a function call to import a module asynchronously and return a promise object

ts
const mod = await import('./some-mod');
`

So we can fine-tune the loading order of modules to achieve the effect of injecting new code into a module before it is statically imported by other modules

The early injection effect brought by this asynchronous import has become a key implementation point for hel-micro to inject remote runtime code into the module proxy object, so that hel-micro can provide users with lazy loading and preloading two loading methods .

There are two core interfaces in the above figure: libReady interface is responsible for exposing modules, preFetchLib interface is responsible for pulling modules, and each module acts as a provider or consumer by calling the interface behavior.

Runtime Dependency Analysis

hel-micro makes remote modules with multiple levels of dependencies efficient, safe and orderly through the internally maintained event bus, module pool, style pool, metadata pool four data structures load.

The module pool can ensure that the module is not loaded repeatedly and reused by the upper-layer callers.

Metadata - the soul of modules

The essence of a module is a collection of build product files. hel-micro provides build-time plug-ins, collects the network path of the product and stores it according to the protocol specified by the sdk, so that the sdk can download and execute all the files on the network later. remote module.

Dual build mechanism

hel-micro uses rollup to package local statically importable proxy files, and uses webpack to package the actual running code of remote injection, so that the proxy module objects in node_modules can be statically imported locally and get complete type hints. Let users get the ultimate development experience of using remote modules as if they were using local modules

There are 4 default directories hel_dist, hel_proxy, hel_proxy_es, hel_bundle to carry different products for package.json to configure different entries.

Among them, the files in the hel_proxy, hel_proxy_es directories are the entry files of the module proxy object we mentioned. We can see that the file is almost an empty shell, so its impact on the package size of the module user can almost be omitted. Excluding.

Platform and Ecology

Hel-mirco sdk mainly relies on the standardized metadata format for remote module loading, so as long as any user provides the metadata of the module according to the standardized format, it can be loaded by hel-micro, which will be very beneficial to the development of hel-micro The construction and development of the upper ecology.

Supports arbitrary deployment of modules

The sdk is decoupled from the platform. By default, we provide support with hel pack and npm unpkg, but it allows you to publish modules to any file service on the network, as long as you know the deployment address.

If the user saves the hel-meta.json metadata to the background database (which can be combined with the devops pipeline), a centralized module management and control platform similar to hel pack can be quickly built

The centralized module management and control platform will be particularly convenient for modules to implement version preview, grayscale volume, second-level rollback, etc., but it does not prevent the sdk from de-neutralizing. Loading multi-platform packages, because sdk inherently supports pulling and using remote modules from multiple different platforms at the same time, such as loading modules from two platforms, unpkg and hel pack at the same time, the platform value will be used as a namespace to isolate different platforms. named module.

Upper-level ecological construction

hel-micro itself only provides the ability to load remote modules. The specific ui adaptation layer also needs to be implemented in the upper encapsulation library area. The current plan is as follows

Taking hel-micro-react as an example, it provides lazy loading of remote components in the form of react hook function, and provides shadow dom style isolation function at the same time

When to adopt hel-micro

At the moment when you encounter any of the following situations, using hel-micro is definitely worth your effort to try.

roadmap 2022~2023

Conclusion

The module federation can play an even more powerful role in the construction of super-large js projects, and the module deployment efficiency and sharing efficiency of giant applications will be easily solved. At the same time, it is matched with micro-container related frameworks (such as wujie, rame), etc., to ensure your isolation operation requirements. Escort, welcome to star hel-micro, understand and use it.

My other open source works friend chain (welcome to follow and understand):

concent, a react data flow solution with built-in dependency collection and setup features

limu, a more efficient js immutable operation library than immer