Hi there!

My name is Miquel Camps, I'm a freelance web developer based in Spain, you can find me on twitter and linkedin.

Information

My idea was to combine these two cool projects to be able to play Mario with hand gestures.

Video

Short - Just gameplay

What problems I found?

• There are some factors that can affect the hand tracking: light, position of hands, distance from webcam.

• Sometimes handtrack.js detect your arm or face as a hand.

• There is some lag, you can notice it when you jump sometimes it can take 1-2 seconds.

Play

I recommend you to use a computer with firefox/chrome browser.

https://vivirenremoto.com/mario/

Credits

• Full Screen Mario by Josh Goldberg

• Handtrack.js by Victor Dibia

How I made this

By pure chance I found these two amazing projects Full Screen Mario and Handtrack.js, then the idea came to me.

The handtrack.js library returns the position of the hands found from the webcam with their current position x-y and size height-width in pixels.

Then I read the code of Full Screen Mario to know how I can simulate the move left, right, jump and stop.

I did a little dirty trick with the jump move, maybe is not the best solution but it usually works, the handtrack.js returns the height of the hand if I detect the hand is over 140 pixels then Mario jumps, but it doesn't work well in some conditions like if you are too close or far from the webcam.

I divided virtually the screen width into three parts: move left, neutral, move right.

Finally I implemented the controls to connect the game with the hand gestures.

Here sounds easy but it took me all night long haha.

Final message

I hope you like it, get some inspiration and it would be great if you can share it to your family and friends who ever play once Mario Bros to bring back some good memories.

Disclaimer

Mario, Super Mario Brothers, and all associated games and media are property of Nintendo and/or Nintendo of America Inc., and are protected by United States and international copyright, trademark and other intellectual property laws.