TheLazy.Dev
Portfolio GitHub

Blog

javascript |react |threejs |vrm

How to display 3D humanoid avatar with React

saitoeku3

Tadao Iseki

Posted on December 1, 2019

How to display 3D humanoid avatar with React

Introduction

3DCG and VR technology are used in various places and become familiar with us. And the same phenomenon occurs on web browser. Today I introduce VRM and how to display it with React and @pixiv/three-vrm.

What is VRM?

VRM is a file format for handling 3D humanoid avatar (3D model) data for VR applications. If you have an avatar that conforms to VRM, you can enjoy various applications that require a 3D avatar.

What is @pixiv/three-vrm?

GitHub logo pixiv / three-vrm

Use VRM on Three.js

@pixiv/three-vrm is a JavaScript library to use VRM on Three.js. This enables to render VRM on web applications like VRoid Hub.

VRoid Hub

Prepare VRM

At first, you need to download VRM from VRoid Hub.

  1. Search VRM models by tags.
  2. Select your favorite model.
  3. Move to model page and download by clicking "Use this model"

Set up project 

$ npx create-react-app three-vrm-sample
$ cd three-vrm-sample/
$ yarn add @pixiv/three-vrm three react-three-fiber
Enter fullscreen mode Exit fullscreen mode 
<!DOCTYPE html>
<html>
  <head>
    <title>@pixiv/three-vrm sample</title>
    <style>
      html,
      body {
        background-color: #000;
        color: #fff;
        margin: 0;
        width: 100vw;
        height: 100vh;
      }

      #root {
        width: 100%;
        height: 100%;
      }
    </style>
  </head>
  <body>
    <div id="root"></div>
  </body>
</html>
Enter fullscreen mode Exit fullscreen mode 
import React from 'react'
import ReactDOM from 'react-dom'

const App = () => null

ReactDOM.render(<App />, document.getElementById('root'))
Enter fullscreen mode Exit fullscreen mode

Add VRM Loader

We can load VRM with GLTFLoader because VRM is similar format to GLTF.

import { VRM } from '@pixiv/three-vrm'
import { useRef, useState } from 'react'
import { GLTFLoader } from 'three/examples/jsm/loaders/GLTFLoader'

const useVrm = () => {
  const { current: loader } = useRef(new GLTFLoader())
  const [vrm, setVrm] = useState(null)

  const loadVrm = url => {
    loader.load(url, async gltf => {
      const vrm = await VRM.from(gltf)
      setVrm(vrm)
    })
  }

  return { vrm, loadVrm }
}
Enter fullscreen mode Exit fullscreen mode

Display VRM with react-three-fiber

react-three-fiber is a React renderer for Three.js. You can use Three.js declaratively with it. I use the following three elements this time.

  • <Canvas>: Wrapper element for react-three-fiber elements
  • <spotLight>: Light element to illuminate objects
  • <primitive>: 3D object element

When you input a VRM file, handleFileChange() create object url and load VRM.

import React from 'react'
import { Canvas } from 'react-three-fiber'
import * as THREE from 'three'

const App = () => {
  const { vrm, loadVrm } = useVrm()

  const handleFileChange = event => {
    const url = URL.createObjectURL(event.target.files[0])
    loadVrm(url)
  }

  return (
    <>
      <input type="file" accept=".vrm" onChange={handleFileChange} />
      <Canvas>
        <spotLight position={[0, 0, 50]} />
        {vrm && <primitive object={vrm.scene} />}
      </Canvas>
    </>
  )
}
Enter fullscreen mode Exit fullscreen mode

Result:

Result

Improve appearance

VRM model in the result is small and facing the other side. You want to see it closer and its face, don't you?

1. Create a new camera from THREE.PerspectiveCamera and set its position.

Note:
useThree gives you access to all the basic THREE objects like gl, scene, camera, clock and so on...

import React, { useEffect, useRef } from 'react'
import { useThree, Canvas } from 'react-three-fiber'
import * as THREE from 'three'

const App = () => {
  const { aspect } = useThree()
  const { current: camera } = useRef(new THREE.PerspectiveCamera(30, aspect, 0.01, 20))
  const { vrm, loadVrm } = useVrm()

  const handleFileChange = event => {
    const url = URL.createObjectURL(event.target.files[0])
    loadVrm(url)
  }

  // Set camera position
  useEffect(() => {
    camera.position.set(0, 0.6, 4)
  }, [camera])

  return (
    <>
      <input type="file" accept=".vrm" onChange={handleFileChange} />
      <Canvas camera={camera}>
        <spotLight position={[0, 0, 50]} />
        {vrm && <primitive object={vrm.scene} />}
      </Canvas>
    </>
  )
}
Enter fullscreen mode Exit fullscreen mode

2. Rotate and look at camera

Assign camera to vrm.lookAt.target and rotate vrm 180°.

import { VRM } from '@pixiv/three-vrm'
import { useEffect, useRef, useState } from 'react'
import { useThree } from 'react-three-fiber'
import { GLTFLoader } from 'three/examples/jsm/loaders/GLTFLoader'

const useVrm = () => {
  const { camera } = useThree()
  const { current: loader } = useRef(new GLTFLoader())
  const [vrm, setVrm] = useState(null)

  const loadVrm = url => {
    loader.load(url, async gltf => {
      const vrm = await VRM.from(gltf)
      vrm.scene.rotation.y = Math.PI
      setVrm(vrm)
    })
  }

  // Look at camera
  useEffect(() => {
    if (!vrm || !vrm.lookAt) return
    vrm.lookAt.target = camera
  }, [camera, vrm])

  return { vrm, loadVrm }
}
Enter fullscreen mode Exit fullscreen mode

Final Code:

import { VRM } from '@pixiv/three-vrm'
import ReactDOM from 'react-dom'
import React, { useEffect, useRef, useState } from 'react'
import { useThree, Canvas } from 'react-three-fiber'
import * as THREE from 'three'
import { GLTFLoader } from 'three/examples/jsm/loaders/GLTFLoader'

const useVrm = () => {
  const { camera } = useThree()
  const { current: loader } = useRef(new GLTFLoader())
  const [vrm, setVrm] = useState(null)

  const loadVrm = url => {
    loader.load(url, async gltf => {
      const vrm = await VRM.from(gltf)
      vrm.scene.rotation.y = Math.PI
      setVrm(vrm)
    })
  }

  // Look at camera
  useEffect(() => {
    if (!vrm || !vrm.lookAt) return
    vrm.lookAt.target = camera
  }, [camera, vrm])

  return { vrm, loadVrm }
}

const App = () => {
  const { aspect } = useThree()
  const { current: camera } = useRef(new THREE.PerspectiveCamera(30, aspect, 0.01, 20))
  const { vrm, loadVrm } = useVrm()

  const handleFileChange = event => {
    const url = URL.createObjectURL(event.target.files[0])
    loadVrm(url)
  }

  // Set camera position
  useEffect(() => {
    camera.position.set(0, 0.6, 4)
  }, [camera])

  return (
    <>
      <input type="file" accept=".vrm" onChange={handleFileChange} />
      <Canvas camera={camera}>
        <spotLight position={[0, 0, 50]} />
        {vrm && <primitive object={vrm.scene} />}
      </Canvas>
    </>
  )
}

ReactDOM.render(<App />, document.getElementById('root'))
Enter fullscreen mode Exit fullscreen mode

Result:

Result

Looks good 👍

Conclusion

VRM will be used more widely in the future. I hope this article will help you when you need to use VRM with React.

@pixiv/three-vrm has more features, so if you are interested, please read the documentation and try it out.

If you have any problems or questions, please write comment or reply to my Twitter account.

Sample Repository:

GitHub logo saitolume / three-vrm-sample

👤Sample repository using @pixiv/three-vrm with React

💖 💪 🙅 🚩
saitoeku3
Tadao Iseki

Posted on December 1, 2019

Join Our Newsletter. No Spam, Only the good stuff.

Sign up to receive the latest update from our blog.

Related

How to display 3D humanoid avatar with React
javascript How to display 3D humanoid avatar with React

December 1, 2019