33:null 34:["$","title",null,{"children":"How to create a 3D engine from scratch using Vega part 2 | Mathias Tiberghien | Observable"}] 3a:T500,md`# How to create a 3D engine from scratch using Vega part 2 In the previous post, I discovered the quaternions and their wonderfull application in the 3D world. I wanted to know a little more about it, so I started to follow some tutorials and discovered another concept that I never heard before: tesselation. This the technique transforms 3D shapes into a lot of triangles. These triangles are composed with 3 vectors which are defining the location and the orientation (which side of the triangle is exposed to the user) of the triangle. A 3D engine analyses the normal (the vector perpendicular to the triangle surface) of these triangles to check if the triangle is hidden or shown. the z component of the normal allows also order the triangles to avoid overlapping. The normal is also used to calculate the light that each triangle should receive. This collection of triangles is named data model, and is generally built with CAD software. A common format to store these model is the .stl format. It stores the minimal information to be able to represents 3D objects. Our goal will be to try to represent the content of a .stl file using Vega and our 3D engine...`3b:T1ce4,embed({ "$schema": "https://vega.github.io/schema/vega/v5.json", "width":400, "height":400, "signals":[ { "name":"size", "init":"min(width,height)/2" }, { "name":"angle_HOR", "on":[{"events":{"source":"view", "type": "mousemove", "filter":"event.buttons === 1"}, "update":"rotation_HOR + (event.movementX > 0 ? 5 : event.movementX < 0 ? -5 : 0)"}] }, { "name":"validAngle_HOR", "update":"angle_HOR % 360 <0 ? 360 + angle_HOR % 360 : angle_HOR % 360" }, { "name":"angle_VER", "on":[{"events":{"source":"view", "type": "mousemove", "filter":"event.buttons === 1"}, "update":"rotation_VER + (event.movementY < 0 ? 5 : event.movementY > 0 ? -5 : 0)"}] }, { "name":"validAngle_VER", "update":"angle_VER % 360 <0 ? 360 + angle_VER % 360 : angle_VER % 360" }, { "name":"rotation_HOR", "value":45, "bind":{"input":"range", "min": 0, "max": 360, "step": 1}, "on":[{"events":{"signal":"validAngle_HOR"}, "update":"validAngle_HOR"}] }, { "name":"rotation_VER", "value":65, "bind":{"input":"range", "min": 0, "max": 360, "step": 1}, "on":[{"events":{"signal":"validAngle_VER"}, "update":"validAngle_VER"}] }, { "name":"qrAngle_HOR", "init":"-PI*(rotation_HOR)/360", "on": [{"events":{"signal":"-rotation_HOR"}, "update":"-PI*(rotation_HOR)/360"}] }, { "name":"qrAngle_VER", "init":"PI*(rotation_VER)/360", "on": [{"events":{"signal":"rotation_VER"}, "update":"PI*(rotation_VER)/360"}] }, { "name":"qrw0", "update":"cos(qrAngle_VER)" }, { "name":"qrw1", "update":"cos(qrAngle_HOR)" }, { "name":"qrx0", "update":"sin(qrAngle_VER)" }, { "name":"qrz1", "update":"sin(qrAngle_HOR)" }, { "name":"q", "update":"{x:qrx0*qrw1, y:-qrx0*qrz1, z:qrw0*qrz1, w: qrw0*qrw1}" }, { "name":"q_1", "update":"{x:-q.x, y:-q.y, z: -q.z, w: q.w}" }, { "name":"color", "value":"steelblue" }, { "name":"pyramid_data", "init":"[[[1,-1,-1],[-1,1,-1],[1,1,1]],[[-1,1,-1],[1,-1,-1],[-1,-1,1]],[[1,1,1],[-1,1,-1],[-1,-1,1]],[[-1,-1,1],[1,-1,-1],[1,1,1]]]" }, { "name":"scale", "value":1, "bind":{"input":"range", "min": 0, "max": 2, "step": 0.1}, "on": [{"events":"view:mousewheel", "update":"max(0, min(2, scale + (event.wheelDelta > 0 ? 0.1 : -0.1)))"}] } ], "data":[ { "name":"pyramid", "values":[{}], "transform":[ { "type":"formula", "expr": "pyramid_data", "as":"verts" }, { "type":"flatten", "fields":["verts"], "as":["verts"] }, { "type":"formula", "expr":"[datum.verts[0],datum.verts[1],datum.verts[2],datum.verts[0]]", "as":"verts" }, { "type":"window", "ops":["row_number"], "as":["id"] }, { "type":"formula", "expr":"[datum.verts[1][0] - datum.verts[0][0],datum.verts[1][1] - datum.verts[0][1],datum.verts[1][2] - datum.verts[0][2]]", "as":"v1" }, { "type":"formula", "expr":"[datum.verts[2][0] - datum.verts[1][0],datum.verts[2][1] - datum.verts[1][1],datum.verts[2][2] - datum.verts[1][2]]", "as":"v2" }, { "type":"formula", "expr":"{x:datum.v1[1]*datum.v2[2] - datum.v1[2]*datum.v2[1],y:datum.v1[0]*datum.v2[2] - datum.v1[2]*datum.v2[0],z:datum.v1[0]*datum.v2[1] - datum.v1[1]*datum.v2[0]}", "as":"normal" }, { "type":"formula", "expr":"{x:datum.v1[1]*datum.v2[2] - datum.v1[2]*datum.v2[1],y:datum.v1[0]*datum.v2[2] - datum.v1[2]*datum.v2[0],z:datum.v1[0]*datum.v2[1] - datum.v1[1]*datum.v2[0]}", "as":"normal" }, { "type":"formula", "expr":"sqrt(pow(datum.normal.x,2)+pow(datum.normal.y,2)+pow(datum.normal.z,2))", "as":"normalLength" }, { "type":"formula", "expr":"{x:datum.normal.x/datum.normalLength, y:datum.normal.y/datum.normalLength, z:datum.normal.z/datum.normalLength, w:0}", "as":"normal" } ] }, { "name":"projections", "source":"pyramid", "transform":[ { "type":"formula", "expr":"{x:q.w * datum.normal.x + q.y*datum.normal.z- q.z * datum.normal.y, y:q.w * datum.normal.y - q.x * datum.normal.z + q.z*datum.normal.x, z:q.w * datum.normal.z + q.x*datum.normal.y - q.y * datum.normal.x, w: -q.x * datum.normal.x - q.y * datum.normal.y - q.z * datum.normal.z }", "as":"q1" }, { "type":"formula", "expr":"{ z:datum.q1.w * q_1.z + datum.q1.x * q_1.y - datum.q1.y * q_1.x + datum.q1.z * q_1.w}", "as":"nq2" }, { "type":"collect", "sort":{"field":["nq2.z"]} }, { "type":"filter", "expr":"datum.nq2.z >=0" }, { "type":"flatten", "fields":["verts"], "as":["verts"] }, { "type":"formula", "expr":"{x:datum.verts[0], y:-datum.verts[1],z:datum.verts[2], w:0}", "as":"q" }, { "type":"formula", "expr":"sqrt(pow(datum.q.x,2)+pow(datum.q.y,2)+pow(datum.q.z,2))", "as":"qLength" }, { "type":"extent", "signal":"qLength_ext", "field":"qLength" }, { "type":"formula", "expr":"{x:datum.q.x/qLength_ext[1], y:datum.q.y/qLength_ext[1],z:datum.q.z/qLength_ext[1], w:0}", "as":"q" }, { "type":"formula", "expr":"{x:q.w * datum.q.x + q.y*datum.q.z- q.z * datum.q.y, y:q.w * datum.q.y - q.x * datum.q.z + q.z*datum.q.x, z:q.w * datum.q.z + q.x*datum.q.y - q.y * datum.q.x, w: -q.x * datum.q.x - q.y * datum.q.y - q.z * datum.q.z }", "as":"q1" }, { "type":"formula", "expr":"{x:datum.q1.w * q_1.x + datum.q1.x * q_1.w + datum.q1.y * q_1.z - datum.q1.z * q_1.y, y:datum.q1.w * q_1.y - datum.q1.x * q_1.z + datum.q1.y * q_1.w + datum.q1.z * q_1.x}", "as":"q2" }, { "type":"formula", "expr":"(datum.q2.x)*size*scale + width/2", "as":"x" }, { "type":"formula", "expr":"(datum.q2.y)*size*scale + width/2", "as":"y" } ] } ], "marks":[ { "type":"group", "from":{"facet":{"data":"projections", "groupby":["id"], "name": "facet"}}, "marks":[ { "type":"area", "from":{"data":"facet"}, "encode":{ "update":{ "x":{"field":"x"}, "y":{"field":"y"}, "y2":{"value":0}, "fill":{"signal":"hcl(hcl(color).h, hcl(color).c, datum.nq2.z * 80)"} } } } ] }, { "type":"text", "encode":{ "enter":{"fontSize":{"value":16}}, "update":{ "text":{"signal":"'number of visible faces: '+length(data('projections'))/4"} }} } ] })3c:T6e0,([{ "type":"formula", "expr": "pyramid_data", "as":"verts" }, { "type":"flatten", "fields":["verts"], "as":["verts"] }, { "type":"formula", "expr":"[datum.verts[0],datum.verts[1],datum.verts[2],datum.verts[0]]", "as":"verts" }, { "type":"formula", "expr":"[datum.verts[1][0] - datum.verts[0][0],datum.verts[1][1] - datum.verts[0][1],datum.verts[1][2] - datum.verts[0][2]]", "as":"v1" }, { "type":"formula", "expr":"[datum.verts[2][0] - datum.verts[1][0],datum.verts[2][1] - datum.verts[1][1],datum.verts[2][2] - datum.verts[1][2]]", "as":"v2" }, { "type":"formula", "expr":"{x:datum.v1[1]*datum.v2[2] - datum.v1[2]*datum.v2[1],y:datum.v1[0]*datum.v2[2] - datum.v1[2]*datum.v2[0],z:datum.v1[0]*datum.v2[1] - datum.v1[1]*datum.v2[0]}", "as":"normal" }, { "type":"formula", "expr":"{x:datum.v1[1]*datum.v2[2] - datum.v1[2]*datum.v2[1],y:datum.v1[0]*datum.v2[2] - datum.v1[2]*datum.v2[0],z:datum.v1[0]*datum.v2[1] - datum.v1[1]*datum.v2[0]}", "as":"normal" }, { "type":"formula", "expr":"sqrt(pow(datum.normal.x,2)+pow(datum.normal.y,2)+pow(datum.normal.z,2))", "as":"normalLength" }, { "type":"formula", "expr":"{x:datum.normal.x/datum.normalLength, y:datum.normal.y/datum.normalLength, z:datum.normal.z/datum.normalLength, w:0}", "as":"normal" }, { "type":"window", "ops":["row_number"], "as":["id"] } ])3d:Tad9,({ "name":"projections", "source":"stl", "transform":[ { "type":"formula", "expr":"{x:q.w * datum.normal.x + q.y*datum.normal.z- q.z * datum.normal.y, y:q.w * datum.normal.y - q.x * datum.normal.z + q.z*datum.normal.x, z:q.w * datum.normal.z + q.x*datum.normal.y - q.y * datum.normal.x, w: -q.x * datum.normal.x - q.y * datum.normal.y - q.z * datum.normal.z }", "as":"q1" }, { "type":"formula", "expr":"{x:datum.q1.w * q_1.x + datum.q1.x * q_1.w + datum.q1.y * q_1.z - datum.q1.z * q_1.y, y:datum.q1.w * q_1.y - datum.q1.x * q_1.z + datum.q1.y * q_1.w + datum.q1.z * q_1.x, z:datum.q1.w * q_1.z + datum.q1.x * q_1.y - datum.q1.y * q_1.x + datum.q1.z * q_1.w}", "as":"nq2" }, { "type":"filter", "expr":"datum.nq2.z >=0" }, { "type":"flatten", "fields":["verts"], "as":["verts"] }, { "type":"formula", "expr":"{x:datum.verts[0], y:-datum.verts[1],z:datum.verts[2], w:0}", "as":"q" }, { "type":"formula", "expr":"sqrt(pow(datum.q.x,2)+pow(datum.q.y,2)+pow(datum.q.z,2))", "as":"qLength" }, { "type":"extent", "signal":"qLength_ext", "field":"qLength" }, { "type":"formula", "expr":"{x:datum.q.x/qLength_ext[1], y:datum.q.y/qLength_ext[1],z:datum.q.z/qLength_ext[1], w:0}", "as":"q" }, { "type":"formula", "expr":"{x:q.w * datum.q.x + q.y*datum.q.z- q.z * datum.q.y, y:q.w * datum.q.y - q.x * datum.q.z + q.z*datum.q.x, z:q.w * datum.q.z + q.x*datum.q.y - q.y * datum.q.x, w: -q.x * datum.q.x - q.y * datum.q.y - q.z * datum.q.z }", "as":"q1" }, { "type":"formula", "expr":"{x:datum.q1.w * q_1.x + datum.q1.x * q_1.w + datum.q1.y * q_1.z - datum.q1.z * q_1.y, y:datum.q1.w * q_1.y - datum.q1.x * q_1.z + datum.q1.y * q_1.w + datum.q1.z * q_1.x,z:datum.q1.w * q_1.z + datum.q1.x * q_1.y - datum.q1.y * q_1.x + datum.q1.z * q_1.w}", "as":"q2" }, { "type":"window", "fields":["q2.z"], "ops":["max"], "groupby":["id"], "frame": [ null, null ], "as":["z_order"] }, { "type":"collect", "sort":{"field":["z_order"]} }, { "type":"formula", "expr":"(datum.q2.x)*size*scale + width/2", "as":"x" }, { "type":"formula", "expr":"(datum.q2.y)*size*scale + width/2", "as":"y" } ] })3e:T6d7,md`- One "group" mark will facet each triangle using 'id' column - One "line" mark the group is representing each triangle - On "text" mark will display the number of displayed polygons which is the size of projection table divided by four because we have 4 points for each triangle (the fourth being the same as the first to close the area) **NOTE:**You can notice that the fill color is built using a hcl color definition which change the light factor of the color base on the project normal z paramter. It means the triangles close to the viewer will be brighter that the far ones. Once we have checked that the maths is working we can now proceed to the final test. Use a 3D data model... I felt personnaly really thrilled at this point. ## Let's draw a 3D Model This is what I did to get my 3D Model: - I went there to get a free model - I've downloaded the .stl file - I use an online converter to convert the binary .stl to .json stl - I had to remove the last object ('description') of the .json stl because it breaks the uniformity of the array - I create the 'stl' data table in my vega specifification passing as values of the table the content of the stl json file - Added a few transformations to the table in order to: - create the 'verts' column - create the 'normal' (the normal provided in the .st file gave me wrong result, so I calculated it based on triangle vectors) - create the 'id' column - Et voilà! **NOTE:** We can consider that the 'stl' table is now generalized for .stl files. All we have to do to change what we want to represent another object is the content of 'values' using other .stl files` 3f:T1ce6,spec=({ "$schema": "https://vega.github.io/schema/vega/v5.json", "width":400, "height":400, "signals":[ { "name":"size", "init":"min(width,height)/2" }, { "name":"angle_HOR", "on":[{"events":{"source":"view", "type": "mousemove", "filter":"event.buttons === 1"}, "update":"rotation_HOR + (event.movementX > 0 ? 5 : event.movementX < 0 ? -5 : 0)"}] }, { "name":"validAngle_HOR", "update":"angle_HOR % 360 <0 ? 360 + angle_HOR % 360 : angle_HOR % 360" }, { "name":"angle_VER", "on":[{"events":{"source":"view", "type": "mousemove", "filter":"event.buttons === 1"}, "update":"rotation_VER + (event.movementY < 0 ? 5 : event.movementY > 0 ? -5 : 0)"}] }, { "name":"validAngle_VER", "update":"angle_VER % 360 <0 ? 360 + angle_VER % 360 : angle_VER % 360" }, { "name":"rotation_HOR", "value":35, "bind":{"input":"range", "min": 0, "max": 360, "step": 1}, "on":[{"events":{"signal":"validAngle_HOR"}, "update":"validAngle_HOR"}] }, { "name":"rotation_VER", "value":60, "bind":{"input":"range", "min": 0, "max": 360, "step": 1}, "on":[{"events":{"signal":"validAngle_VER"}, "update":"validAngle_VER"}] }, { "name":"qrAngle_HOR", "init":"-PI*(rotation_HOR)/360", "on": [{"events":{"signal":"-rotation_HOR"}, "update":"-PI*(rotation_HOR)/360"}] }, { "name":"qrAngle_VER", "init":"PI*(rotation_VER)/360", "on": [{"events":{"signal":"rotation_VER"}, "update":"PI*(rotation_VER)/360"}] }, { "name":"qrw0", "update":"cos(qrAngle_VER)" }, { "name":"qrw1", "update":"cos(qrAngle_HOR)" }, { "name":"qrx0", "update":"sin(qrAngle_VER)" }, { "name":"qrz1", "update":"sin(qrAngle_HOR)" }, { "name":"q", "update":"{x:qrx0*qrw1, y:-qrx0*qrz1, z:qrw0*qrz1, w: qrw0*qrw1}" }, { "name":"q_1", "update":"{x:-q.x, y:-q.y, z: -q.z, w: q.w}" }, { "name":"color", "value":"steelblue" }, { "name":"scale", "value":1, "bind":{"input":"range", "min": 0, "max": 2, "step": 0.1}, "on": [{"events":"view:mousewheel", "update":"max(0, min(2, scale + (event.wheelDelta > 0 ? 0.1 : -0.1)))"}] } ], "data":[ { "name":"stl", "values": stl , "transform":[ { "type":"formula", "expr":"[datum.verts[0],datum.verts[1],datum.verts[2]]", "as":"verts" }, { "type":"window", "ops":["row_number"], "as":["id"] }, { "type":"formula", "expr":"[datum.verts[1][0] - datum.verts[0][0],datum.verts[1][1] - datum.verts[0][1],datum.verts[1][2] - datum.verts[0][2]]", "as":"v1" }, { "type":"formula", "expr":"[datum.verts[2][0] - datum.verts[1][0],datum.verts[2][1] - datum.verts[1][1],datum.verts[2][2] - datum.verts[1][2]]", "as":"v2" }, { "type":"formula", "expr":"{x:datum.v1[1]*datum.v2[2] - datum.v1[2]*datum.v2[1],y:datum.v1[0]*datum.v2[2] - datum.v1[2]*datum.v2[0],z:datum.v1[0]*datum.v2[1] - datum.v1[1]*datum.v2[0]}", "as":"normal" }, { "type":"formula", "expr":"sqrt(pow(datum.normal.x,2)+pow(datum.normal.y,2)+pow(datum.normal.z,2))", "as":"normalLength" }, { "type":"formula", "expr":"{x:datum.normal.x/datum.normalLength, y:datum.normal.y/datum.normalLength, z:datum.normal.z/datum.normalLength, w:0}", "as":"normal" } ] }, { "name":"projections", "source":"stl", "transform":[ { "type":"formula", "expr":"{x:q.w * datum.normal.x + q.y*datum.normal.z- q.z * datum.normal.y, y:q.w * datum.normal.y - q.x * datum.normal.z + q.z*datum.normal.x, z:q.w * datum.normal.z + q.x*datum.normal.y - q.y * datum.normal.x, w: -q.x * datum.normal.x - q.y * datum.normal.y - q.z * datum.normal.z }", "as":"q1" }, { "type":"formula", "expr":"{x:datum.q1.w * q_1.x + datum.q1.x * q_1.w + datum.q1.y * q_1.z - datum.q1.z * q_1.y, y:datum.q1.w * q_1.y - datum.q1.x * q_1.z + datum.q1.y * q_1.w + datum.q1.z * q_1.x, z:datum.q1.w * q_1.z + datum.q1.x * q_1.y - datum.q1.y * q_1.x + datum.q1.z * q_1.w}", "as":"nq2" }, { "type":"filter", "expr":"datum.nq2.z >=0" }, { "type":"flatten", "fields":["verts"], "as":["verts"] }, { "type":"formula", "expr":"{x:datum.verts[0], y:-datum.verts[1],z:datum.verts[2], w:0}", "as":"q" }, { "type":"formula", "expr":"sqrt(pow(datum.q.x,2)+pow(datum.q.y,2)+pow(datum.q.z,2))", "as":"qLength" }, { "type":"extent", "signal":"qLength_ext", "field":"qLength" }, { "type":"formula", "expr":"{x:datum.q.x/qLength_ext[1], y:datum.q.y/qLength_ext[1],z:datum.q.z/qLength_ext[1], w:0}", "as":"q" }, { "type":"formula", "expr":"{x:q.w * datum.q.x + q.y*datum.q.z- q.z * datum.q.y, y:q.w * datum.q.y - q.x * datum.q.z + q.z*datum.q.x, z:q.w * datum.q.z + q.x*datum.q.y - q.y * datum.q.x, w: -q.x * datum.q.x - q.y * datum.q.y - q.z * datum.q.z }", "as":"q1" }, { "type":"formula", "expr":"{x:datum.q1.w * q_1.x + datum.q1.x * q_1.w + datum.q1.y * q_1.z - datum.q1.z * q_1.y, y:datum.q1.w * q_1.y - datum.q1.x * q_1.z + datum.q1.y * q_1.w + datum.q1.z * q_1.x,z:datum.q1.w * q_1.z + datum.q1.x * q_1.y - datum.q1.y * q_1.x + datum.q1.z * q_1.w}", "as":"q2" }, { "type":"window", "fields":["q2.z"], "ops":["max"], "groupby":["id"], "frame": [ null, null ], "as":["z_order"] }, { "type":"collect", "sort":{"field":["z_order"]} }, { "type":"formula", "expr":"(datum.q2.x)*size*scale + width/2", "as":"x" }, { "type":"formula", "expr":"(datum.q2.y)*size*scale + width/2", "as":"y" } ] } ], "marks":[ { "type":"group", "from":{"facet":{"data":"projections", "groupby":["z_order","id"], "name": "facet"}} , "marks":[ { "type":"line", "from":{"data":"facet"}, "encode":{ "enter":{"interpolate":{"value":"linear-closed"}}, "update":{ "x":{"field":"x"}, "y":{"field":"y"}, "fill":{"signal":"hcl(hcl(color).h, hcl(color).c, datum.nq2.z * 80)"}, "stroke":{"signal":"hcl(hcl(color).h, hcl(color).c, datum.nq2.z * 80)"}, "strokeWidth":{"value":0.65} } } } ] }, { "type":"text", "encode":{ "enter":{"fontSize":{"value":16}}, "update":{ "text":{"signal":"'number of visible faces: '+length(data('projections'))/3"} }} } ] })40:T50d,([ { "type":"formula", "expr":"[datum.verts[0],datum.verts[1],datum.verts[2]]", "as":"verts" }, { "type":"window", "ops":["row_number"], "as":["id"] }, { "type":"formula", "expr":"[datum.verts[1][0] - datum.verts[0][0],datum.verts[1][1] - datum.verts[0][1],datum.verts[1][2] - datum.verts[0][2]]", "as":"v1" }, { "type":"formula", "expr":"[datum.verts[2][0] - datum.verts[1][0],datum.verts[2][1] - datum.verts[1][1],datum.verts[2][2] - datum.verts[1][2]]", "as":"v2" }, { "type":"formula", "expr":"{x:datum.v1[1]*datum.v2[2] - datum.v1[2]*datum.v2[1],y:datum.v1[0]*datum.v2[2] - datum.v1[2]*datum.v2[0],z:datum.v1[0]*datum.v2[1] - datum.v1[1]*datum.v2[0]}", "as":"normal" }, { "type":"formula", "expr":"sqrt(pow(datum.normal.x,2)+pow(datum.normal.y,2)+pow(datum.normal.z,2))", "as":"normalLength" }, { "type":"formula", "expr":"{x:datum.normal.x/datum.normalLength, y:datum.normal.y/datum.normalLength, z:datum.normal.z/datum.normalLength, w:0}", "as":"normal" } ])41:Ta36,md`## If you want to know more When you deal with 3D transformations and projections, you have to consider a lot of paramters and concepts. To avoid some calculations, and to be able to do something fast, I have skipped a lot of these concepts. Now that we have experimented, it's maybe time to try to talk a little a bit about these concepts: - In 3D, you have to consider different coordinates systems which represent the objects you want to render: - The local system is the raw data containing the definition of your model - The world system is the local system expressed in a more global system: this is important to be able to render objects in relations to others - The view system are the coordinates of the objects from the viewer point of view. The z paramter in that system represents the distance to the eye or camera - The clipping system represents the field of view of the camera: it is usefull to remove all the objects your camera is not able to see - The screen system represents the coordinates of your object on the screen - You can transform an object from one system to another using translations, rotations and scales. The transformations from one system to another can be expressed by matrix. A good practice is to have one matrices by transisition to be able to define the config of each system more intuitively (and avoid or detect errors) - In the experiments above, I mixed almost everything to avoid calculations: - I skipped the world system because or consider that my local system was the world system, I always applied translations to my local coordinates to ensure the center of the 3 axis are the center of the object - I used the quaternion to represent the transformation from world system to the view system: instead of moving the camera, you move the object - I skipped the clipping system because I wanted to render everthing - I added a scale factor based on the size and translate my view coordinates to the center of the view to perform the transofmation from view system to to screen system - The rotations parameters in the expermients are defining the transformations from world system to view system. I've only added 2 parameters: rotation_HOR and rotation_VER. As we have 3 axis, I should have added rotation_Z, but it was less intuitive and as the mouse uses 2 coordinates, I prefered to skip it. Another think to know is that I should have named rotation_HOR by yaw angle, and rotation_VER by pitch angle. rotation_Z is the roll angle Here's my source of information: LearnOpenGL`42:[{"id":"6a524e57a225b7b899201ee262a00b199fbead057e3656ea1c2c90eda36885a451d383bc4877b465569d50959c51d68b2001363974a1dbc51fe5f8ec6e9ad448","href":"https://static.observableusercontent.com/files/6a524e57a225b7b899201ee262a00b199fbead057e3656ea1c2c90eda36885a451d383bc4877b465569d50959c51d68b2001363974a1dbc51fe5f8ec6e9ad448","name":"dolphin_3D_2.json","type":"application/json","lastModified":1582914126460.858,"size":246128}] 38:{"id":"0f26dc18913db663","slug":"how-to-create-a-3d-engine-from-scratch-using-vega-part-2","title":"How to create a 3D engine from scratch using Vega part 2","thumbnail":"f2fb88cc84b161b2494b2e640eaea4a27742a3672e761cf734ffb234aba27952","access_level":"public","create_time":"$D2020-02-28T14:48:35.295Z","update_time":"$D2020-03-19T15:00:07.157Z","view_time":null,"first_public_time":"$D2020-02-28T19:12:25.757Z","has_unpublished_edits":null,"fork":false,"stars":8,"starred":false,"owner":{"type":"team","id":"1ab1463344b266e0","login":"mathiastiberghien","name":"Mathias Tiberghien","avatar_url":"https://avatars-next.observableusercontent.com/a16241a634f3076e093beeb4c0c0a672737bd033a393f97b3242b427679fc82e","tier":"starter_2024","bio":""},"version":230,"publish_version":230,"role":null,"authored":false,"authors":[{"type":"individual","id":"0161860a595b9637","login":"mathiastiberghien","name":"Mathias Tiberghien","avatar_url":"https://avatars-next.observableusercontent.com/a16241a634f3076e093beeb4c0c0a672737bd033a393f97b3242b427679fc82e","tier":null,"bio":""}],"members":[{"type":"individual","id":"0161860a595b9637","login":"mathiastiberghien","name":"Mathias Tiberghien","avatar_url":"https://avatars-next.observableusercontent.com/a16241a634f3076e093beeb4c0c0a672737bd033a393f97b3242b427679fc82e","tier":null,"role":"editor"}],"comments":[],"custom_thumbnail":false,"trashed":false,"lapsed":null,"subscribed":null,"forks":3,"collection":{"id":"0bfa1471ddeaa408","slug":"vega","title":"Vega","owner":{"id":"1ab1463344b266e0","login":"mathiastiberghien","name":"Mathias Tiberghien"}},"type":"notebook","body":{"titling":"auto","title":"How to create a 3D engine from scratch using Vega part 2","cells":[{"id":0,"value":"$3a","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":13,"value":"embed = require(\"vega-embed@6\")","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":110,"value":"embed(spec,{\"renderer\":\"svg\"})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":6,"value":"md`## 3D Algebra part 2\nWe know how to project 3D points on a 2D plane using quaternions. Now we need to calculate the normal of a triangle.\n- A triangle is determined by 3 points p1, p2 and p3\n- These points can be decomposed in 2 vectors v1 and v2:\n - v1 = (p2.x -p1.x, p2.y - p1.y, p2.z - p1.z)\n - v2 = (p3.x -p2.x, p3.y - p2.y, p3.z - p2.z\n- The normal n is the product of v1 by v2:\n - n = (v1.y*v2.z - v1.z*v2.y, v1.x*v2.z - v1.z*v2.x, v1.x*v2.y - v1.y*v2.x)\n - you should divide each member x,y,z of n with the square root of the sum of the square of x,y and z to have each value being within -1 and 1 range.\n\nWe have what we need to move forward. Let's draw the simplest triangle based shape you can think of: a pyramid!\n\n## Let's draw a pyramid` ","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":16,"value":"$3b","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":22,"value":"({\n \"name\":\"pyramid_data\",\n \"init\":\"[[[1,-1,-1],[-1,1,-1],[1,1,1]],[[-1,1,-1],[1,-1,-1],[-1,-1,1]],[[1,1,1],[-1,1,-1],[-1,-1,1]],[[-1,-1,1],[1,-1,-1],[1,1,1]]]\"\n })","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":25,"value":"md`The unique signal that was added from the first 3D exercices is the pyramid data: an array of 4 triplets which each triplets representing a face of the pyramid.\n\nFrom the side of data transformations we added the following (in the data table named 'pyramid'):\n- Create a line by triangle\n- Create the normal of each triangle using the product of 2 vectors of the triangle\n- Normalize the triangle\n- Add an id to be able to group each triangle`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":35,"value":"$3c","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":41,"value":"md`**NOTE:** I used that specific format for the pyramid because it is the same that we have in stl files. This will allow us to create a projection table that will deal with:\n- a \"verts\" column containing the 3 sommits of each triangle\n- a \"normal\" column representing the normal of each triangle\n- an \"id\" column\" allowing to identify each triangle\n\nThe 'projections' table have the following structure:` ","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":49,"value":"$3d","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":52,"value":"md`The transformations in the table will:\n- project the normal using our projection quaternion\n- nomalize the normal to have value between -1 and 1\n- filter the triangles using the z (depth) parameter of the normal (we will do calculations on the triangle which have a projected normal z paramter greather than 0 (in front of us)\n- Create one row per sommit of each triangle\n- Project and scale each point in 2D using the projection quaternion\n- Get the max z parameter of each projected point (grouping by triangle id) and sort the triangles using that parameter\n\n**NOTE:** Aside the normal calculation this is very similar that what we have done in previous experimentations.\n\nThe marks part is as following:`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":59,"value":"([{\n \"type\":\"group\",\n \"from\":{\"facet\":{\"data\":\"projections\", \"groupby\":[\"z_order\",\"id\"], \"name\": \"facet\"}}\n ,\n \"marks\":[\n {\n \"type\":\"line\",\n \"from\":{\"data\":\"facet\"},\n \"encode\":{\n \"enter\":{\"interpolate\":{\"value\":\"linear-closed\"}},\n \"update\":{\n \"x\":{\"field\":\"x\"},\n \"y\":{\"field\":\"y\"},\n \"fill\":{\"signal\":\"hcl(hcl(color).h, hcl(color).c, datum.nq2.z * 80)\"},\n \"stroke\":{\"signal\":\"hcl(hcl(color).h, hcl(color).c, datum.nq2.z * 80)\"},\n \"strokeWidth\":{\"value\":0.65}\n }\n }\n }\n ]\n },\n {\n \"type\":\"text\",\n \"encode\":{\n \"enter\":{\"fontSize\":{\"value\":16}},\n \"update\":{\n \"text\":{\"signal\":\"'number of visible faces: '+length(data('projections'))/3\"}\n }}\n }])","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":62,"value":"$3e","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":94,"value":"stl = FileAttachment(\"dolphin_3D_2.json\").json()","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":97,"value":"$3f","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":103,"value":"embed(spec,{\"renderer\":\"svg\"})","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":118,"value":"md`Here are the transformations on table 'stl':`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":121,"value":"$40","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":179,"value":"md`You can now try to display other .stl files. You will able to watch the performances when there are more polygons.\n\n## OOPS\nI had several issues with the z order and it was difficult for me to understand it because of the facetting process.\nHere's the trick to solve it. In projections table you calculate the max z projection point for each triangle. You order your points using the that z parameter. When facetting, you first group by the z order parameter then by id (if you first group by id, you are loosing the z order...).\n\nHere's the trick:\n- in projections table transforms:`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":217,"value":"([{\n \"type\":\"formula\",\n \"expr\":\"{x:datum.q1.w * q_1.x + datum.q1.x * q_1.w + datum.q1.y * q_1.z - datum.q1.z * q_1.y, y:datum.q1.w * q_1.y - datum.q1.x * q_1.z + datum.q1.y * q_1.w + datum.q1.z * q_1.x,z:datum.q1.w * q_1.z + datum.q1.x * q_1.y - datum.q1.y * q_1.x + datum.q1.z * q_1.w}\",\n \"as\":\"q2\"\n },\n {\n \"type\":\"window\",\n \"fields\":[\"q2.z\"],\n \"ops\":[\"max\"],\n \"groupby\":[\"id\"],\n \"frame\": [\n null,\n null\n ],\n \"as\":[\"z_order\"]\n },\n {\n \"type\":\"collect\",\n \"sort\":{\"field\":[\"z_order\"]}\n }])","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":221,"value":"md`- in marks:`","pinned":true,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":180,"value":"( {\n \"type\":\"group\",\n \"from\":{\"facet\":{\"data\":\"projections\", \"groupby\":[\"z_order\",\"id\"], \"name\": \"facet\"}}\n ,\n \"marks\":[\n {\n \"type\":\"line\",\n \"from\":{\"data\":\"facet\"},\n \"encode\":{\n \"enter\":{\"interpolate\":{\"value\":\"linear-closed\"}},\n \"update\":{\n \"x\":{\"field\":\"x\"},\n \"y\":{\"field\":\"y\"},\n \"fill\":{\"signal\":\"hcl(hcl(color).h, hcl(color).c, datum.nq2.z * 80)\"},\n \"stroke\":{\"signal\":\"hcl(hcl(color).h, hcl(color).c, datum.nq2.z * 80)\"},\n \"strokeWidth\":{\"value\":0.65}\n }\n }\n }\n ]\n })","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":202,"value":"$41","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":132,"value":"md`## Conclusion\nWhen I began that 3D journey, I was just hoping to be able to project a square. I never imagined that playing with 3D projections was so accessible. Off course the rendering is far to be perfect and this looks more like 90's 3D but I was once again amazed by the reaction of Vega to handle that you challenge. For the record, I discovered Vega this summer and I what I love with the tech, is that it is so robust and easy that you can focus on what you wanna do instead of how you will gonna do it. I'm not sure if I will continue this 3D journey, but I'm interested to discover now how to deal with light...\n`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"},{"id":185,"value":"md`In the facet, the 'z_order' column was added in the groupby and that was it!`","pinned":false,"hidden":false,"mode":"ojs","author":"$undefined","summary":"$undefined","tool":"$undefined","providerMetadata":"$undefined","attachments":"$undefined","output":"$undefined","database":"$undefined"}],"files":"$W42","stdlib":"1"}} 39:"$38:body"