Posted on April 15, 2021
In this demo, we will try to reproduce one of the main interface of the Death Stranding game.
Death Stranding is a game produced by Hideo Kojima (especially known for its Metal Gear series games). The game takes place in a post-apocalyptic future where an unknown phenomenon has ravaged most of the world. You play a character, Sam, responsible for making merchandise deliveries to the scattered remains of the population in a world that became quite dangerous. If Sam looks familiar to you it’s because its model is based on the actor who played Daryl in Walking Dead.
On this interface, the player must arrange the merchandise he will carry from point A to point B.
The arrangement done by the player will have a significant consequence on the success of the delivery.
This interface is really interesting for a number of reasons:
For the sake of this article, I reduced the scope of the interface but I tried to keep the essence of what makes it interesting. Our goal will be to reproduce:
For the 3D scene, there will be 3 different positions to display the merchandises:
This article requires some knowledge about threejs and react-three-fiber.
If you have no experience in threejs the best resource on the web to get started is the course made by Bruno Simon: ThreejsJourney
If you’re looking for resources on react-three-fiber you can take a look at this repository
There are 2 possibilities to consume this article. You can simply read it to get a global understanding of how the demo works or you can try to reproduce the demo to have a deeper understanding.
If you choose the latter, I created a starter project on codesanbox with all the assets to get started more easily. You can also download it if you prefer to work locally.
Feel free to choose what suits you best.
flagrede
/
death-stranding-ui
https://deathstranding.gameuionweb.com/
https://dev.to/flagrede/how-to-reproduce-death-stranding-ui-with-react-and-react-three-fiber-cif
The base project is a classic create-react-app. Here’s the list of the additional libraries used in it:
A note on Twind:
This library is a CSS-in-JS version of TailwindJS. If you're more comfortable with another styling solution don't hesitate to replace it. If you prefer vanilla Tailwind, Twind can be used just like that by using the following shim (already included in the starter).
We're going to start building our interface with the 3D part. First, we will create the 3D grid of the private locker. The grid cell delimitations will be done using particles.
Then we will create two smaller grids (for shared locker and sam cargo) without particles. Finally, we need to be able to move the camera between these 3 positions.
This component will be responsible for loading and displaying the model. We will go through the whole process but some parts are already done in the starter.
/public folder
At this point, we should be able to see the model. Now we have to position/rotate/scale the model correctly so it fits the original UI.
We will also handle a secondary display for the model. It will be useful later on to separate the selected item from the other. For this secondary display, we will try to display it with a translucent blue color and a wireframe on top of it.
const SelectedMaterial = ({ blue = 0.2, ...props }) => {
return (
<>
<Material
{...props}
uniforms={{
r: { value: 0.0, type: 'float' },
g: { value: 0.0, type: 'float' },
b: { value: blue, type: 'float' },
}}
transparent
>
<Material.Frag.Body children={`gl_FragColor = vec4(r, g, b, blue);`} />
</Material>
</>
)
}
To simulate the selected state you can try to use react-three-a11y. By wrapping our model with the <A11y> component we will have access to hover, focus, and pressed state through useA11y() hook. We can try to display a SelectedMaterial based on the hover state for example.
Since we will have a 2D overlay on top of the 3D scene we won’t need react-three-a11y afterward but it’s good to know that you can bring accessibility to your 3D scene quite easily with it.
This will be the most complex part of the demo.
To recreate this grid we will need 2 components:
There are 2 different kinds of particles which are called smallCross and bigCross. In the end, we will have to compute these 2 position arrays plus the one for the briefcases.
First, we will start with the Grid component.
const Grid = ({ texture, positions = [], ...props }) => (
<points {...props}>
<pointsMaterial
size={0.6}
opacity={0.5}
color="#316B74"
alphaMap={texture}
transparent
depthWrite={false}
blending={THREE.AdditiveBlending}
/>
<bufferGeometry attach="geometry">
<bufferAttribute attachObject={['attributes', 'position']} count={positions.length / 3} array={positions} itemSize={3} />
</bufferGeometry>
</points>
)
Here we’re using an alpha map texture to recreate the “cross” particle effect. We’re also tweaking a few parameters for the colors and the transparency. The particle’s positions and count are given to the bufferAttribute tag. The positions array needs to have the following format [x1, y1, z1, x2, y2, z2, ...].
Let’s continue with the GridsContainer.
We said that we have 3 position arrays to compute but we can do the 3 of them at the same time.
First question, how many particles do we need for the small cross particles array?
Let’s say we want
Also for one particle weed 3 values (x, y, z).
In the end, we will need an array 720 values (20 * 6 * 2 * 3) to display a grid of 20 columns, 6 lines, and 2 layers.
This is only for the small cross particles position array, the big cross array has 2 times less coordinate and the briefcases one 4 times less.
This is because for each cell we want to display:
There are probably several ways of doing this. Here’s one method:
At the end of the loop, we can filter the coordinates we don’t need for the big cross and briefcases arrays (remember that we have 2 times and 4 times fewer coordinates for these too).
Don’t hesitate to put every configuration variable (columns, lines, layers, spacing …) for this grid in a tool like leva to make it look like what you want.
In the actual render, we need to:
positionsBriefcases valuespositionsSmallCross valuespositionsBigCross values
This one is simpler than the grid we just build since it doesn’t use any particles.
Here we just want to display briefcases on the same Z value, 3 columns, and any number of lines. In our new ExternalGrid component we will map just the briefcases list and call a util function to get the position.
Our util function to get the position could look like this:
const X_SPACING = 2
const Y_SPACING = -1
export const getPositionExternalGrid = (index, columnWidth = 3) => {
const x = (index % columnWidth) * X_SPACING
const y = Math.floor(index / columnWidth) * Y_SPACING
return [x, y, 0]
}
To make the scene look right color-wise on the background we have to add a floor and a fog.
Floor:
<Plane rotation={[-Math.PI * 0.5, 0, 0]} position={[0, -6, 0]}>
<planeBufferGeometry attach="geometry" args={[100, 100]} />
<meshStandardMaterial attach="material" color="#1D2832" />
</Plane>
Fog:
<fog attach="fog" args={['#2A3C47', 10, 20]} />
Add these 2 elements to the main canvas.
Before going into building the HTML UI we need to create our state with the data.
For this demo, I wanted to give a try to valtio as the state manager.
We will need to create a state with proxyWithComputed, because we will have to computed values based on the state.
In the actual state we have only two values:
To populate it we need a function to generate data. This function already exists in the starter.
So our state looks like this for now:
proxyWithComputed(
{
selectedItem: 0,
allItems: [...generateItems(9, 'private'), ...generateItems(3, 'share'), ...generateItems(3, 'sam')],
},
The second parameter takes an object and is used to define the computed values.
Here’s the list of computed values we will need:
This is the component that will display our list of briefcases. As we saw on the schema it uses the following child components:
The component should also handle the following events:
In this modal, we add actions to move the selected briefcase from one category to another.
To do that we just need to update the category of the selected item in the store. Since we’re using computed values to display the lists, everything should update automatically.
We will also need to handle keyboard navigation in this modal.
This is the right side part of the UI. We just need to display all the data of the selected item here.
The only interaction is about the like button. Whenever the user clicks on it we should update the likes count of the selected briefcase. This is simple to do thanks to Valtio, we just update allItems[selectedItem].likes in the state directly and the likes counts should update in the Inventory.
We now have a 2D UI and a 3D scene, it would be nice to make them interact with each other.
Currently, we just highlight the selected item in the UI part. We need to reflect this to the 3D briefcase as well. We already made the selected material, we just need to use it inside the Briefcase component.
From now on, our camera was only looking at the main grid, the private locker. We will create 3 components to move the camera and display them based on the properties isPrivateLocker, isShareLocker, and isSamCargo that we created earlier in the state.
Here for example the code that look at the main grid:
function ZoomPrivateLocker() {
const vec = new THREE.Vector3(0, 1.5, 4)
return useFrame((state) => {
state.camera.position.lerp(vec, 0.075)
state.camera.lookAt(0, 0, 0)
state.camera.updateProjectionMatrix()
})
}
To give our UI a more realistic look we must make it look like it is slightly rotated from the camera. We can do that with the following CSS:
body{
perspective 800px;
}
.htmlOverlay {
transform: rotate3d(0, 1, 0, 357deg);
}
We’re now going to add some animations to both the UI and the 3D scene.
All animations has been done using react-spring.
This is the animation that happens inside Inventory whenever the selected item changes.
There are actually 3 parts to this animation:
We will go through each of them and combine them together with the useChain hook.
Sliding animation
To reproduce this animation we will need custom SVGs (they are already available in the starter). I used the tool https://yqnn.github.io/svg-path-editor/ to make 3 SVGs.
I think we could have an even better effect with more SVGs, feel free to try adding more frames to animation.
To animate these 3 SVGs, we will declare a x property inside a useSpring going from 0 to to 2 and in the render we will have this:
<a.path
d={
x &&
x.to({
range: [0, 1, 2],
output: [
'M 0 0 l 16 0 l 0 3 l -16 0 l 0 -3',
'M 0 0 l 25 0 l -10 3 l -15 0 l 0 -3',
'M 0 0 l 16 0 L 16 3 l -5 0 l -11 -3 m 11 3',
],
})
}
/>
</a.svg>
Now we just need to animate the opacity and the width and we should have a good sliding animation effect.
background height
Here we’re just expending the item’s background with a default spring:
const [{ height }] = useSpring(() => ({
from: { height: 0 },
to: { height: 24 },
ref: heightRef,
}))
glowing color animation
To reproduce this part we will make a spring between 2 colors and play with the opacity at the same time:
const [{ bgOpacity, color }] = useSpring(() => ({
from: { bgOpacity: 1, color: '#456798' },
to: { bgOpacity: 0.5, color: '#3E5E8D' },
ref: bgOpacityRef,
loop: true,
easing: (t) => t * t,
config: config.slow,
}))
All together
Finally, we just have to use these 3 animations with the useChain hook
useChain([opacityRef, heightRef, bgOpacityRef], [0, 0.2, 0])
The SideMenu animation will use the same technique we just saw. It will be a spring that goes through 3 SVGs. Again I was a bit lazy on the number of SVG frames, feel free to try with more.
Here are the 3 SVGs I used for the demo:
output: [
'M 0 0 l 16 0 l 0 3 l -16 0 l 0 -3',
'M 0 0 l 25 0 l -10 3 l -15 0 l 0 -3',
'M 0 0 l 16 0 L 16 3 l -5 0 l -11 -3 m 11 3',
],
Here our OuterBox component:
const OuterBox = () => (
<div>
<div className="h-1 w-2 bg-gray-200 absolute top-0 left-0" />
<div className="h-1 w-2 bg-gray-200 absolute top-0 right-0" />
<div className="h-1 w-2 bg-gray-200 absolute bottom-0 left-0" />
<div className="h-1 w-2 bg-gray-200 absolute bottom-0 right-0" />
</div>
)
This component is displayed inside ItemDescription one. It shows four little white stripes at the edges of ItemDescription.
On the animation side, we will have to animate the height property of the component from 0 to 100%.
For the bar that shows an item's durability, we will make an animated bar (like a loader).
We need to animate the width property based on the damage attribute of the item.
For the 3D scene, we will add just one animation that will be triggered whenever a briefcase is changed from one category to another. We will make it seem like the briefcases, those that have changed, are falling from above.
We can handle this animation in the Briefcase component. Whenever the position of a briefcase will change, we will animate the new value on the Y-axis from the new value plus a delta to the new value.
Until now the spring animations were triggered whenever a component was mounted. Here we need to animate briefcases that are already mounted.
To trigger a spring that has already been played once we need the second parameter received from the useSpring hook.
const [{ position: animatedPosition }, set] = useSpring(() => ({
from: { position: [position[0], position[1] + 5, position[2]] },
to: { position },
}))
Be careful to use @react-spring/three instead of @react-spring/web here.
For the sounds part we’re going to create a sound manager component using useSound hook from Joshua Comeau. After that, we’re going to put our sound functions newly-created into our state so that we can everywhere in the app.
Here’s the list of sounds we need to handle:
We’re done with the tutorial, I hope you liked it. If you’re trying to make your own version of the Death Stranding UI, don’t hesitate to share it with me on twitter. If you're interested in more GameUI on web demos I share updates on the upcoming demos on this newsletter.
Posted on April 15, 2021
Sign up to receive the latest update from our blog.
