The webworkers driven UI framework neo.mjs version 2 release announcement
Tobias Uhlig
Posted on May 8, 2021
Concepts for new products need to be disruptive, otherwise there is little need for them. However, the way to use them should be as consistent as possible.
For a framework this means that the API on how to use it should be rock solid, especially when it comes to new versions. Migrating from neo.mjs v1 to v2 is a piece of cake.
This article assumes that you are not familiar with the project yet, so we will cover some of the basics as well.
Content
Eye candy
Design Goals
What is new in version 2?
Statistics
How can I set up my first neo.mjs app?
How to migrate from v1 to v2?
How difficult is it to learn neo.mjs?
When should i use neo.mjs?
What is a config system?
Roadmap
Final thoughts
Special thanks to
1. Eye candy
Let us take a look into use cases which would be difficult to achieve without using neo.mjs:
This is a single page app expanded into multiple browser windows. The app can communicate without the need for a backend.
Desktop browser mandatory, best in Chrome or Edge, runs in Firefox too: neo.mjs/dist/production/apps/sharedcovid/
To get an impression on the performance, try the helix: using a magic mouse or trackpad you can scroll horizontally.
This results in > 30.000 dom manipulations per second locally. The helix is intentionally built to stress out the browser → each item is a dom node (no canvas, svg, webGL).
What basically happens on drop is:
dialog.unmount();
dialog.appName = 'SharedDialog2'; // name of the other window app
dialog.mount();
Yes, we are re-using the same dialog JS instance.
Medium
We can easily lazy-load complex dependencies into the application worker.
This also works in dist/production. Even better, we do get split chunks across different apps. Meaning: you can add multiple apps on the same page with very little overhead.
2. Design Goals
The neo.mjs concepts are disruptive:
Multithreading: most parts of the framework as well as your own application code run inside the app worker. This leaves the main thread mostly idle, leading to an amazing rendering performance, especially when dynamically manipulating the DOM.
Extensibility and Scalability: You can extend and change pretty much anything. You will only get the code you actually use. You can lazy load modules and even get cross app split chunks.
Build on web standards: neo.mjs uses the latest ECMAScript x features as soon as they are available in all major browsers.
“Bring UI development back into the browser”
neo.mjs is one of the very few projects where UI development does not happen inside node and you don’t need any builds or transpilations when changing your code. The development mode can run the framework and your application code as it is directly inside the browser. Right now still limited to Chromium, since Safari (getting close) and Firefox do not support using JS modules inside the worker scope yet. Debugging the real code has saved me a lot of time already.dist/production (webpack based) has to run in all major browsers.
Persistent json based virtual dom structures. There are no templates at all. You define your component trees in a json based format as well. The JS side is in charge. You can control if you want to destroy instances or if you want to keep them for later re-use.
Using the virtual dom engine is optional, you can manually construct delta updates in case you exactly know what should change to further push the performance.
You get documentation views for your own app code inside the neo.mjs docs app out of the box
3. What is new in version 2?
The neo.mjs v2 release is mostly about providing you with new ways for making the state management of your apps easier, as well as enhancing your frontend architectures.
I created two separate articles to cover the view model implementation in detail:
The way to use view models got improved from string based binding formatters to functions containing template literals.
As one of the first breaking changes I used the chance to finally wrap up the long overdue version 2 release.
View models become incredibly powerful for multi window apps, since you can point the parent model of your MainContainer in one app (window) to a view model inside a different app (window). This is an extremely beautiful way to make data accessible across browser windows.
Do not use view models or view controllers for creating custom components, even in case they are complex and use multiple views (like a calendar or table implementation).
The way view controllers internally work has changed for v2. Before, only components who have an own controller triggered parseConfig() and they parsed their items downwards until they found one which had an own controller. In v2 every component will trigger parseConfig() on the closest controller inside the parent tree and only the top level does get parsed. Advantage: you can now dynamically add new components containing string based listeners.
I am constantly trying to keep the project dependencies (npm packages) up to date. Now with the node v16 release out there, a couple of things needed to change.
neo.mjs was still using node-sass (libsass) since a c based compiler sounded charming. dart-sass got renamed to sass and has become the official standard:
Install Sass
good news: the switch worked extremely smooth and we can continue to use the latest scss features.
node-sass as well as the webpack-dev-server had dependencies for a package called “fibers”. This one can no longer work in node v16+.
Switching to
'webpack-dev-server': '4.0.0-beta.2'
removed the last “fibers” dependency, so we should be good for now.
4. Statistics
Out of curiosity I checked the amount of commits a couple of days ago:
neomjs/neo 5196
neomjs/* 6347
pre GA 3720
So, in total the neo.mjs ecosystem is at 10.067 commits.
sloc apps Physical 17133, Source: 11054
sloc buildScripts Physical 2036, Source: 1709
sloc examples Physical 11005, Source: 8805
sloc resources/scss Physical 6697, Source: 5842
sloc src Physical 42032, Source: 22393
Just to get an impression.
5. How can I set up my first neo.mjs app?
The neo.mjs command line interface got you covered:
Open your terminal (or cmd).
Enter a folder where you want to store your project
(I only used Desktop for this demo)Enter “npx neo-app”
You can hit enter on all questions
Open the new generated workspace folder in an IDE
Optional: deploy it to a repository (e.g. GitHub)
Open the MainContainer.mjs file
Change code
Reload the browser window (the dev mode does not require any builds)
More infos here:
neomjs/create-app
In case npx neo-app does not work inside your local environment, the default output is stored here:
neomjs/workspace
Option 2:
You can fork the neomjs/neo repository. The package.json contains a “create-app” program, which will create a new app shell inside the apps folder of the framework.
This approach is especially useful in case you want to work on your app code as well as the framework code at the same time.
6. How to migrate from v1 to v2?
I still remember migrating a client project from Sencha ExtJS version 2 to 3. It was a super intense sprint and took 6 full weeks. I did not have to deal with Angular migrations on my own, but heard many stories that it was close to a rewrite for early versions.
Rich Waters and I kept this in mind when designing the neo.mjs API.
Migrating from v1 to v2 can be done in 10 minutes.
In case your app is on v1.5+, simply run npm update and the build-all program.
For earlier versions, there are 2 strategies:
The first one (recommended) is to run npx neo-app with the same app name. you can then replace the content of the app folder with your old app folder, run build-all and you are done.
The other strategy is to run npx neo-app and manually adjust the content of your old app shell. In detail: delete the node_modules, dist and docs folders. Manually copy the new docs folder of the npx output over. adjust the neo.mjs version inside your package.json (e.g. ^2.0.4). run npm install. run build-all .
make sure your app.mjs file is using the following format:
import MainContainer from './view/MainContainer.mjs';
const onStart = () => Neo.app({
mainView: MainContainer,
name : 'MyApp'
});
export {onStart as onStart};
7. How difficult is it to learn neo.mjs?
I spent quite some time getting friends and former colleagues up to speed.
The feedback I got in general was that getting to the point where you “get flying” takes a bit longer compared to other libraries / frameworks. Depending on your skill level, it could take a week or two.
However, I also got the feedback that once you get to this point, neo.mjs is way easier to master and you can do more.
For v1 we have a tutorial series on how to create the covid demo app (2 parts). It could be helpful to rewrite them for v2. This is a lot of work on my end, so please give me a ping in case you need it:
Rewrite 'How to create a webworkers driven multithreading App - Part 1' · Issue #1817 · neomjs/neo
Every single developer so far asked me:
“Tobi, window and window.document are undefined, what is happening?”
Yes, your app code does really run inside a web worker.
Web Workers API
“There are some exceptions: for example, you can’t directly manipulate the DOM from inside a worker, or use some default methods and properties of the window object.”
The German language has the wonderful word “Kindersicherung”.
Meaning: “Mechanics to prevent children from hurting themselves”
In a way, this applies for Javascript devs and the real dom.
For most use cases, you really don’t need it and you can stick to working with the virtual dom.
What really helps is to look into your generated app files:
The index file will not include your app files, but the main thread starting point (this one will create the worker setup). In dist/production, the main thread file is only 42KB.
Once the setup is ready, the application worker will import your app.mjs file for which you specified the appPath .
You can create multiple apps here if you like to and render them into different div nodes of an already existing page.
The application worker will consume all app.mjs files it can find as dynamic imports. This is how we get the cross app split chunks.
I recently got the question: “I would like to add a loading spinner directly into my index.html file and use my neo.mjs app to remove it once the MainView renders. It is outside of our virtual dom. Can I do it?”
Neo.currentWorker.sendMessage('main', {
action: 'updateDom',
appName: this.appName,
deltas: {
action: 'removeNode',
id : 'my-loading-spinner'
}
});
You can manually create delta updates and send them to main. You can use promiseMessage() in case you want to do something once this async operation is done. Use it with care!
You can also create your own main thread addons, e.g. for working with other libraries. Examples:
As mentioned at the bottom of the repo readme:
You are welcome to join the neo.mjs Slack Channel!
Although my time is limited, I will try my best to point you into the right direction.
8. When should i use neo.mjs?
One of my famous quotes is:
“You don’t need a sports car for driving to the supermarket.”
For rather simple and mostly static websites or apps, neo.mjs might not be a good fit.
The bigger and more complex your apps get, the more value can using this framework generate for you.
This also applies for creating complex (custom) components, like a helix, buffered grid, calendar.
A good use case would for example be a banking / trading app where you want to use web sockets to hammer the dom with realtime updates.
In case you want to create multi window apps based on SharedWorkers, the benefits of using neo.mjs are huge and unique.
Although the framework is still focussing on desktop apps, the general setup can really shine for mobile too. There is more work to be done (see: 10. Roadmap).
9. What is a config system?
A question which frequently pops up is:
“Why is there a static getConfig() method?
Are configs the same thing as class fields?”
Let us take a quick look into the following example:
className actually could be a class field, since the value does not change. It also could be static. The reason it is not static is for debugging purposes: In case you log a component tree into the console and click through the items, it is extremely helpful to see the classNameto know what you are dealing with.
What is the benefit of a config system?
In case we define a config with a trailing underscore, like here a_ , we can optionally use beforeGetA() , beforeSetA() and afterSetA() . Obviously the same applies for b_ .
We define a & b as null values for simplicity reasons. We are using this.down() inside both afterSet() methods which is not available before items got constructed.
So, to keep the logic short, I am using onConstructed() to call:
this.set({
a: 5,
b: 5
});
afterSetA() will set the text config for label1 to value + this.b .
afterSetB() will set the text config for label2 to value + this.a .
→ We are dealing with a cross dependency
Let us assume we had defined a & b via get() and set() and we would call the custom afterSet() methods inside the real setters.
Object.assign(this, {
a: 5,
b: 5
});
In this case, a would get assigned first, the afterSet() call would set the label1 text to 5 (5+null).
Then b would get assigned and the afterSet() call would set the label2 text to 10 (5+5).
The difference with using this.set() is, that we can access both new values right away. this.a as well as this.b are pretty smart getters.
label1 and label2 will both get 10 (5+5) as their text.
A click on the button will trigger:
this.set({
a: 10,
b: 10
});
Afterwards both label texts have the value 20 (10+10).
Try it: dist/production/examples/core/config/index.html (Online Demo)
Config values do get stored with a leading underscore. In case you want to do a silent update (not triggering beforeSet() and afterSet()), you can e.g. use this._a = 7 .
The second benefit is that component.Base extends the logic of this.set()
myButton.set({
iconCls: 'fas fa-globe',
text : 'Hello world!'
});
afterSetIconCls() as well as afterSetText() getting executed on their own trigger a re-rendering (passing the vdom object to the vdom worker to check for delta updates).
In case you change both configs using this.set() , there will be just 1 check for delta updates.
The third benefit of a config system applies when we extend classes.
A tab.Container is using an activeIndex_ config. You can use:
class MyTabContainer extends TabContainer {
static getConfig() {return {
//...
activeIndex: 2
}}
}
This will not override the get() and set() logic, but assign the new value on instantiation.
A little bit outdated, but here is more input:
10. Roadmap
I am happy about how far the project has grown already.
However, the list of things I would love to add is even bigger.
Now that v2 is released, it feels important to structure the priority of the next items / sprints. You have the chance to make an impact!
Meaning: In case there are specific items you would love to see, add a comment to the related feature request:
https://github.com/neomjs/neo/issues
No ticket there yet? Create one! As simple as this.
I personally think it is best to further polish the desktop side of things, since the majority of big and complex apps is still here. Once this is done, we can focus on mobile. This is not set in stone though.
Again: The following list of items is not ordered.
Theme build: the CSS output is still a monolith, which does not honor the really impressive split chunks on the Javascript side of things. I would love to break down the output on a per file basis. To do this, we need a dependency tree of used app & framework components. Ideally similar to the JS split chunks, so that we can lazy load tiny css files as needed. Epic.
Theming: not all sizes are em based yet. We need to adjust the remaining px values.
Buffered grid: while the table implementation is neat, including locked columns and sticky headers, it is not really sufficient for “big data”. Only rendering the rows you see (plus one) can make a big impact. Epic.
Grid editing: “records” are a super lightweight extension of JS objects to get change events. In case you change values of a record, a table view will already update. A cell editor for the table view & buffered grid would be sweet.
Calendar: the current state is still “sneak preview”. it is super impressive since the demo does remove non active views from the real dom, while we can still alter their state and re-use the same JS instances.
More complex examples for view models: so far, all demo apps work without view models. it can be helpful to have one big demo app using them.
core.Observable: The framework is still using an early and never finished version. It is sufficient for adding listeners and firing events, but it does not honor the current state of the framework. Epic.
MessageBus (PubSub): As an alternative for view models, it would be nice to have an additional way to communicate. I probably won’t find the time to evolve it as far as MobX.
Data worker: Rich had the concept, that stores should live within this thread. The idea was that ajax calls / socket connections happen from here, then local filtering & sorting happens here as well (to remove weight from the app worker) and only send the needed data to the app worker. It is more complicated: for stores containing little data the concept makes no sense, so we need to polish the concept. Epic.
Socket connections: We still need an implementation for it.
View models v3: we have a discussion to support “2 way bindings”. Imo only relevant for form fields and it could remove some boiler plate code. To do this, config setters need to fire a change event (or use a MessageBus) to let vms know about changes. Tricky one, since we need to be careful not to create a massive amount of change events which have no receivers.
Support for public class fields: once they are stage4 and work inside webpack (acorn parser) based builds, I would love to add them. breaking change → neo.mjs v3. Epic.
Enhancing form field components: definitely room for improvement here. Epic.
Slider component & slider field: now with a beautiful drag&drop implementation in place, it would be nice to add it.
Color picker field: needs the slider component.
Mobile: we need to enhance components for mobile usage and or create new ones. We need more examples as well. Epic++
More examples for lazy loading: with the cross app split chunks in place, we should either refactor the covid or the real world app to start with an empty viewport and lazy load modules as needed. Epic.
Virtual dom worker v2: more precisely vdom.Helper v2. The results for delta updates are truly impressive. However, the delta update algorithm contains a couple too many tree structure generations and can get improved. Epic.
Docs app v2: we still need to enable lazy loading example apps in dist/prod. there are many aspects on this one that could get enhanced. Epic++.
Website app v2: Since more and more examples get into the framework, the example lists are no longer sufficient. The website app could use a re-design in general, but my capabilities as a designer are limited.
GitHub Pages: At some point I would love to change the logic entirely. Right now we fetch the neo.mjs repo and show examples and apps inside of it. It would be nicer to pull the content of repos inside neomjs/* on their own and deploy them with their own neo.mjs version. Then we can move more examples & demo apps out of the framework “core” repo. Epic++.
More tests: I am waiting for the next major release of Bryntum’s Siesta. Once more contributors join the project, it will be more and more important to have more user interaction tests in place. Not real “unit tests”, but rather loading example apps and simulate users to verify that the business logic does not break when changing code. Epic++.
This list is not complete, but I make a cut here. Not even mentioning long term goals like creating a middleware running the neo core (this one would be amazing). In case you are a Javascript expert you most likely have an idea, about the size of this scope in “hours” → months, years.
While I can work on each item one by one on my own, more contributors could really make an impact here.
Don’t forget that I still need to polish parts of the core, write blog posts and that it is impossible to put my full time into the project (more about this in the next section).
11. Final thoughts
Even in case you consider yourself to be an expert in Javascript, you can most likely learn a lot in case you take a deep dive into the neo.mjs code base. I would say you can reach an entirely new level and “get flying”.
neomjs/neo
You can find all online examples, a blog section and the docs here:
https://neomjs.github.io/pages/
I am definitely one of those guys who thinks “code speaks for itself” and prefer to convince others with actions rather than words.
However, I got the feedback: “People love people, not products.”
I have to admit that there is wisdom and truth inside this quote.
In case you want to learn more about me, feel free to take a look at my LinkedIn profile.
In short: I was an expert when it came to Sencha ExtJS back in the days. After working on client projects, I did join the Sencha Professional Services Team for 2.5 years. It was an amazing time! Since the team was limited to around 8 guys, I was literally flying around the globe all the time to help out on client projects which were completely on fire (fixing framework and app related issues in parallel). The hourly rates were extremely high, so the level of expectation on the clients side was challenging as well. I enjoyed my “cash cow” role and am thankful for the opportunity to dive into a lot of client projects. This helped me a lot with developing a feeling on what the industry actually needs when it comes to UI development.
Afterwards I helped the PlayStation Team for more than 4 years on a massive project from the early inception phase up to the release. My role involved taking care of the client side architecture as well as implementing challenging parts.
This enabled me to save some money which I literally completely burned with getting the neo.mjs project to the point where it is now.
Open source is still considered “charity”, which is a problem.
Companies (especially here in Germany) are very hesitant to even touch new technology, no matter how good it is. I am not even mentioning sponsorships or official Government programs.
The neo.mjs business model is Business as a Service (BaaS), meaning the entire code base is free to use (MIT licensed). In case your company does require help regarding support or need to get some neo.mjs experts involved for e.g. setting up a prototype app you are welcome to get in touch.
I would deeply enjoy pushing the framework full time to generate even more value for you, but this is simply impossible as long as the project is not on a sustainable level.
Best regards & happy coding,
Tobias
12. Special thanks to
Rich Waters, the other co-founder of the project. I am still sad that Rich literally vanished out of existence and have not heard from him in years.
Durlabh Jain for being the first and still only sponsor of the project!
Gerard Horan, my former PlayStation mentor, Pat Sheridan (Modus Create), as well as Hyle Campbell (my former team leader at Sencha). Without you keeping an eye on the project and your morale support, I would have given up already a long time ago!
Also a big “Thank you!” to many friends & previous co-workers from the former Sencha Community! Too many to list them all, but I hope you know I meant you when you read this :)
Posted on May 8, 2021
Join Our Newsletter. No Spam, Only the good stuff.
Sign up to receive the latest update from our blog.
Related