Bennett Upfield
Posted on March 10, 2022
Description
This post will cover a function I've created in Unity VR with the SteamVR plugin to create a system where a player can create an adjustable wall. I decided to create this to use VR systems to what I think are they're full capabilities, creating engaging systems that are not really possible on a normal control scheme of mouse and keyboard. I have tried to keep the controls for the creator as simple as possible, with some added flair for fun, and I think I have succeeded in creating an easy to use system for creating a wall archetype object.
Control Scheme
As I said, my control scheme does have some flair to it and is not wholly practical but other than a single system that could be a series of buttons instead of giving player interaction through the capabilities of the Index controllers. In the Official Game Dev Post; Nameless: 1 I go over a control scheme I use here to choose what action to take. There are Three different Main-Modes containing three Sub-Modes with this controller scheme. For this project I utilize one Main Mode, and below is the switch simplified for the action.
if(handposition("//mainmode 1//"){
switch(handposition("//submode//")){
case "InitialMode": //this is what the hand position is in upon entering the mainmode
//draw mode refer to Initial Mesh Creation
break;
case "OpenPalm":
//initiate case selection and mesh adjustability, refer to Wall Mesh Creation and Adjustability
break;
case "ClosedFist":
//unused
break;
}
}
However, I do use two more specific controller actions that I will go over in their relevant sections, the first a Draw Function for Initial Mesh Creation, the second a General Purpose Hand Movement in the Direction the Palm is Pointing Action Controller that is used in the Wall Mesh Creation and Adjustability.
Initial Mesh Creation
I divided the generation of the wall into three parts for two reasons. One is that it allows for the use of this draw function in unison with other Archetypes. Another is that drawing a wall is physically impossible on a flat canvas. The draw function only creates a "surface" mesh (a better name for a 2d mesh used in creating a wall") which I then use to crown the wall with. To get this surface all I had to do was make the player outline a shape with his controller. To do this I have a three step system. When first entering InitialMode the player creates a plane that they can "paint" their shape on.
function drawStart(GameObject hand){
Vector3 doubleReferenceMeansVariable = hand.transform.forward;
Vector3 pointMeter = hand.transform.position + doubleReferenceMeansVariable;
DrawPlane = createPlane(pointMeter, doubleReferenceMeansVariable);
}
plane createPlane(Vector3 point, Vector3 vector){
float d = -(-(point.x * vector.x) - (point.y * vector.y) - (point.z * vector.z));
return new plane{x = vector.x, y = vector.y, z = vector.z, d = d};
}
Draw start takes the hand that is currently entering InitialMode and creates a plane based on the direction its pointing (hand.transform.forward) a meter away (unity.transform.direction gives a normalized vector meaning that adding one to a point makes the new point one unit away) and on the normal of that point and line. The next section of code takes the forward position of the hand controllers again, but instead creates a line from that forward and checks for points that intersect with the plane past the controller, and adds those points to an array. As its pretty simple to imagine I'll save you the code. The Third step of the system is the creation of surface mesh once the hand enters the Open Palm Mode, which takes the Vector3 array and uses the program triangulator to create a mesh from the array, and also using the scripts created by Jens-Kristian Nielsen to use this program and understand it Link Here.
triangulateMesh createSurface(Vector3[] vects){
List<Vector3> vectorList = new List<Vector3>(vects);
Vector3 firstPoint = vectorList[0];
vectorList = vectorList.ConvertAll(new System.Converter<Vector3, Vector3>((point)=>{
return point - firstPoint;
}));
Vector3 linea = vectorList[1] - vectorList[0];
Vector3 lineb = vectorList[2] - vectorList[0];
Vector3 cross = Vector3.Cross(linea, lineb);
linea = linea.normalized;
cross = cross.normalized;
Vector3 cross2 = Vector3.Cross(linea, cross);
Vector3 point1 = vectorList[0];
Vector3 point2 = point1 + linea;
Vector3 point3 = point1 + cross2;
Vector3 point4 = point1 + cross;
Matrix4x4 pointMatrix = new Matrix4x4(
new Vector4(point1.x, point1.y, point1.z, 1),
new Vector4(point2.x, point2.y, point2.z, 1),
new Vector4(point3.x, point3.y, point3.z, 1),
new Vector4(point4.x, point4.y, point4.z, 1));
Matrix4x4 conversionMatrix = new Matrix4x4(
new Vector4(0, 0 ,0 ,1),
new Vector4(1, 0, 0, 1),
new Vector4(0, 1, 0, 1),
new Vector4(0, 0, 1, 1));
Matrix4x4 b = conversionMatrix * pointMatrix.inverse;
List<Vector2> vectorLi = vectorList.ConvertAll<Vector2>(new System.Converter<Vector3, Vector2>((point)=>{
Vector3 result;
result.x = b.m00 * point.x + b.m01 * point.y + b.m02 * point.z + b.m03;
result.y = b.m10 * point.x + b.m11 * point.y + b.m12 * point.z + b.m13;
float num = b.m30 * point.x + b.m31 * point.y + b.m32 * point.z + b.m33;
num = 1 / num;
result.x *= num;
result.y *= num;
Vector2 xy = new Vector2(result.x, result.y);
return xy;
}));
MeshFinder.PSLG myPoints = new MeshFinder.PSLG(vectorLi);
MeshFinder.Polygon2D finished = binstance.Triangulate(myPoints);
Mesh newMesh = new Mesh();
List<Vector3> meshVertexs = new List<Vector2>(finished.vertices).ConvertAll<Vector3>(new System.Converter<Vector2, Vector3>((xy)=>(new Vector3(xy.x, xy.y, 0))));
newMesh.vertices = meshVertexs.ToArray();
newMesh.triangles = finished.triangles;
return new triangulateMesh(){
mesh = new Mesh(){
vertices = meshVertexs.ToArray(),
triangles = finished.triangles,
},
surfaceLength = newMesh.vertexCount,
outlineLength = vects.Length,
};
}
As you can see here I simply use a transform matrix to change my 3d plane into a 2d plane, as that is also a second reason you cannot draw a 3d object (the Triangulator program only will fill in the blanks for 2d objects). I then throw these newly created 2d points into Jens and Triangulator written part of the program which gives me a lovely 2d mesh with no giant holes in it. I then save the two variables surfaceLength and outlineLength as knowing the size of the surface and of the original outline is useful further down the line. From this I create a surface mesh.
(Just a demonstration of the creation of the plane)
Wall Mesh Creation
After the creation of the initial surface mesh there is another selection tree to choose which type of object archetype you would like to take. This is chosen from the other new controller, the General Purpose Hand Movement in the Direction the Palm is Pointing Action Controller. This code checks if your palm is open and if your palm is open it checks if the hand moves in the direction your palm is moving.
if(leftOpen){
leftPalm = leftHand.transform.right.normalized;
leftCurrMovement = (leftHand.transform.position - leftPrePos).normalized;
if(Mathf.Abs((leftPalm + leftCurrMovement).magnitude) >= 1.85f){ //This checks if velocity is reasonably aligned with the direction the palm points
goingForward(exportLeft, false);
wroteLeft = false;
}
}
Above checks first if the palm is open, then checks the palm direction and its movement adds the two normalized vectors together and checks if the magnitude is greater than 1.85. This reasonably checks if the two vectors align, while giving that much needed human wiggle room. Once the script checks for if the controller is going in the palm open direction it runs another check that first adds sets the current controller movement class and then checks if the class on the whole has gone a set distance while aligned.
if(testRequiredLength <= Mathf.Abs((info.currPos - info.startPos).magnitude) && (which ? rightTypeChoosen == false : leftTypeChoosen == false))
This simply checks if the from the beginning of the movement you have gone further than a required length. The reset of this movement is initiated if
(wroteLeft && exportLeft.type == "None")
but if you reach the designated length a type is assigned to the class depending on how you moved the controller, mostly up, mostly down, or simply "forward". Once this selection is made the wall creation begins.
GameObject currCreation = new GameObject();
IceWall c = currCreation.AddComponent<IceWall>();
triangulateMesh toMake = c.wallFunction(initialMesh);
c.wallMesh = toMake;
currCreation.AddComponent<MeshFilter>().mesh = toMake.mesh;
currCreation.AddComponent<MeshRenderer>();
triangulateMesh wallFuncion(triangulateMesh surface){
List<Vector3> vectorList = new List<Vector3>(surface.mesh.vertices).ConvertAll<Vector3>(new System.Converter<Vector3, Vector3>((vector)=>(new Vector3(vector.x, 0, vector.y)))); //converts to a z by x surface instead of x and y
for(int b = 0; b < 10; ++b){
float number = b;
for(int i = 0; i < surface.outlineLength; ++i){
Vector3 outlineVector = vectorList[i];
vectorList.Add(new Vector3(outlineVector.x, -number / 10f, outlineVector.z));
}
} // adding new vertexes
List<int> triangleList = new List<int>(surface.mesh.triangles);
for(int curr = 0; curr < triangleList.Count; curr += 3){
int reverse1 = triangleList[curr];
int reverse2 = triangleList[curr + 2];
triangleList[curr] = reverse2;
triangleList[curr + 2] = reverse1;
}
for(int b = 0; b < 10; ++b){
for(int i = 0; i < surface.outlineLength; ++i){
int whichLayer = surface.outlineLength * b + surface.surfaceLength; // decides where the added vertexes from the loop above are
int whereTop = (b > 0 ? surface.surfaceLength : 0) + Mathf.Max(b - 1, 0) * surface.outlineLength; //decides where the vertexes above the added vertexes are
if(i == surface.outlineLength - 1){
triangleList.AddRange(new int[]{i + whichLayer, i + whereTop, whereTop,});
triangleList.AddRange(new int[]{i + whichLayer, whereTop, whichLayer});
}
else{
triangleList.AddRange(new int[]{i + whichLayer, i + whereTop, i + 1 + whereTop,});
triangleList.AddRange(new int[]{i + whichLayer, i + 1 + whereTop, i + 1 + whichLayer});
}
}
}
Mesh meshToCreateWall = new Mesh();
meshToCreateWall.vertices = vectorList.ToArray();
meshToCreateWall.triangles = triangleList.ToArray();
return new IceHandler.triangulateMesh(){
mesh = meshToCreateWall,
outlineLength = surface.outlineLength,
surfaceLength = surface.surfaceLength,
};
}
This simply takes the triangulated mesh that was created in the Initial Mesh Creation which uses the two extra values OutlineLength and SurfaceLength to create the walls walls. It of course does this by using a for loop that adds new vertexes under each original outline Vertex (the vertexes that the player draws originally). The surfaces triangles also have to be reversed as the triangles set by the Triangulator will be upside down if not reversed. With this we finally have our Wall created, and now only need to adjust the wall dynamically.
(Demonstration of Wall Creation)
Wall Mesh Adjustability
As you might of noticed the wall creation above does not have any adjustability of the height of the wall. Because of this there must be a further way to adjust the wall, which is once again done in the "openpalm" section of the switch, but only after the wall has been created. In each wall GameObject there is a script that takes the Palm Movement class and interprets what each class Movement is saying to result in different adjustments. The two useful Palm Movement types are the up and down types, which both initiate the wall adjustment "lift".
void simpleLift(actionPalmMovement lift, bool which){
OpenPalmHandler.palmMovement mov = which ? lift.rightPalm : lift.leftPalm;
Debug.Log("Before: " + wallMesh.mesh.vertices[0].y);
Vector3[] newVertexes = wallMesh.mesh.vertices;
if(mov.fin){
for(int i = 0; i < wallMesh.surfaceLength; ++i){//for the top
newVertexes[i].y += mov.velocity.y * Time.deltaTime;
}
for(int i = 0; i < 10; ++i){//this means all the sides
float d = i;
for(int j = wallMesh.surfaceLength + wallMesh.outlineLength * i; j < wallMesh.outlineLength * i + wallMesh.surfaceLength; ++j){
newVertexes[i].y += mov.velocity.y * Time.deltaTime * ((1f + d) / 11f);
}
}
}
else{
for(int i = 0; i < wallMesh.surfaceLength; ++i){//for the top
newVertexes[i].y += mov.currMov.y;
}
for(int i = 0; i < 10; ++i){//this means all the sides
float d = i;
for(int j = wallMesh.surfaceLength + wallMesh.outlineLength * i; j < wallMesh.outlineLength * i + wallMesh.surfaceLength; ++j){
newVertexes[i].y += mov.currMov.y * ((1f + d) / 11f);
}
}
}
wallMesh.mesh.vertices = newVertexes;
}
This adds the movement of the Palm Movement to the wall, by method of OutlineLength and SurfaceLength used in a for loop again to adjust the right Vectors, each moving distance depending on which layer the vertex is on, to spread the object stretching along as many vertexes as possible. With this we have a fully adjustable wall when using a single controller.
(Demonstration of Simple Lift)
Dynamic Actions
All these actions I have presented have been single controller actions, but Humans have two hands. So, I have made more the system capable of handling both hands inputting at once, and here I'll present the most player visible one. For adjusting the wall with two hands there is a complexLift. This Action taking two inputs does two things differently from the first. First it test for which hand is further along the z axis of the wall,
Transform wall = this.gameObject.transform;
Vector3 leftZCheck = action.lefthand.x * wall.right + action.lefthand.z * wall.forward;
Vector3 rightZCheck = action.righthand.x * wall.right + action.righthand.z * wall.forward;
Vector3 zFurther = leftZCheck.z >= rightZCheck.z ? action.leftPalm.fin ? action.leftPalm.velocity * Time.deltaTime : action.leftPalm.currMov : action.rightPalm.fin ? action.rightPalm.velocity * Time.deltaTime : action.rightPalm.currMov;
Vector3 zShorter = leftZCheck.z >= rightZCheck.z ? action.rightPalm.fin ? action.rightPalm.velocity * Time.deltaTime : action.rightPalm.currMov : action.leftPalm.fin ? action.leftPalm.velocity * Time.deltaTime : action.leftPalm.currMov;
then it creates a slope dependent on the hand movement and the length of the wall,
float m = furthestZ - closestZ == 0 ? 0 : (zFurther.y - zShorter.y) / (furthestZ - closestZ);
After creating this slope it adds the slope * the vector z Position to each Vector, along wtih the layer position adjustment to.
newVertices[j].y += isnoteven ? m * newVertices[j].z * ((1f + d) / 11f) : action.leftPalm.currMov.y * ((1f + d) / 11f);
This allows the player to create a sloped wall with the use of two hands.
(Demonstration of ComplexLift)
Final Thoughts
This project is Unfinished. As you can see from all the presentations from it, the wall is untextured, the height adjustments are hard to control, and the wall is a solid surface. The next blog post on this will probably be me adding all these features, finding a natural adjustment multiplier, adding a noise function for the added vectors, and texturing based on the noise function. However, this feature is just a demonstration of the capabilities of the VR medium. Unlike using a mouse and keyboard you can create 3d models from relatively simple feeling commands. For anything but VR controllers the creation of a wall like this would take the use of something like blender, or a serious of sliders similar to Character Creation screen, but with the use of VR controllers and the complex hand-like inputs you can receive from them the creation of more interesting functions like this is possible.
Posted on March 10, 2022
Join Our Newsletter. No Spam, Only the good stuff.
Sign up to receive the latest update from our blog.