Sign in to Leia Developer Portal

Part 2: Three.js Project

Creating Three.js WebGL Project

If you have a project based on Three.js with bundler like webpack you can skip to Adding Leia WebGL SDK

Prerequisites: you will need node installed. You can get it on https://nodejs.org/ or use your package installer.

NPM and Webpack Setup

Open a terminal, cd to a new dir and initialize a new project:

mkdir myproject 
cd myproject 
npm init

You can press Enter multiple times to fill default values for your project. A file named package.json would be created, feel free to examine it.

Make a dir for your source files, named src:

mkdir src

Make a dir for the "build" of your project, named dist:

mkdir dist

This directory would be used by server to serve files.

Install dependencies: a bundler that will resolve imports and update dist dir, a server and popular webgl framework three.js.

npm install webpack express three

Create empty main script file src/index.js and run webpack: 

npx webpack

It will ask you about installation of webpack-cli but only first time.

Notice that dist dir got new file: dist/main.js, it's empty just like the src/index.js

You will write your code in src/index.js, running npx webpack will transform it into dist/main.js (do not edit it!) while resolving dependencies. The server you will configure will serve files from the dist directory. Let's configure the server now.

In the root of your project create a file named app.js with next content:

var express = require('express');
var app = express();
app.use(express.static(__dirname + '/dist'));
app.listen(3000);
console.log('open 127.0.0.1:3000 in your browser');

Since there's nothing to serve yet, create index.html under dist directory with some basic content like:

<h1>Hello, world!</h1>

Now run the server:

node app.js

After opening http://127.0.0.1:3000/ in your browser you will see a "Hello, world!" message. Press Ctrl+C to stop the server.

Creating A Rotating Cube

Let's reference our script, put these lines into dist/index.html:

<script type="module" src="main.js"></script>
<body>
   <canvas id="myCanvas"></canvas>
</body>

A canvas is the place where you will be rendering something using WebGL.

Let's write our src/index.js now, start from importing three.js:

import * as THREE from 'three';

function main() {
	// setup:
	const canvas = document.querySelector('#myCanvas');
	const renderer = new THREE.WebGLRenderer({canvas});
	const fov = 60, aspect = 1.6, near = 0.1, far = 100;
	const camera = new THREE.PerspectiveCamera(fov, aspect, near, far);
	const scene = new THREE.Scene();
	scene.background = new THREE.Color("rgb(0, 128, 256)");
	
	// creating example textured cube	
	const loader = new THREE.TextureLoader();
	const geometry = new THREE.BoxGeometry(1, 1, 1);
	const material = new THREE.MeshBasicMaterial({
		map: loader.load('texture.jpg'),
  	});
	const cube = new THREE.Mesh(geometry, material);
	cube.position.z = -2; // important, cube and camera must not be at same place
	scene.add(cube);

	// render loop
	function render(time) {
		// rotation animation:
		cube.rotation.x = time * 0.001;
		cube.rotation.y = time * 0.003;

		// actual rendering:
		renderer.render(scene, camera);

		// asking for next frame:
		requestAnimationFrame(render);
	}

	// asking for initial frame:
	requestAnimationFrame(render);
}

window.onload = main

Build it with

npx webpack

And run server:

node app.js

In browser you will see a black spinning cube on blue background:

You can see we are referencing texture.jpg but not providing any. Put some jpeg image named texture.jpg into dist/ and reload browser page. Cube should be textured now:

Configuring Webpack

Sometimes errors are hard to track due to obfuscation. To avoid this, you'll need to create webpack.config.js file and disable "minimize" parameter:

const path = require('path');
module.exports = {
  entry: './src/index.js',
  optimization: {
    minimize: false
  },
  output: {
    path: path.resolve(__dirname, 'dist'),
    filename: 'main.js',
  },
};

Now let's enlarge our rendering view, change index.html to:

<script type="module" src="main.js"></script>
<body style="margin:0">
   <canvas id="myCanvas" style="width:100%;height:100%"></canvas>
</body>

Resizing The Canvas

You will see that canvas is enlarged but the rendering is pixelated. This is because it needs to be resized. In the src/index.js add a function before the render function

function resizeRendererToDisplaySize(renderer) {
    const canvas = renderer.domElement;
    const width = canvas.clientWidth;
    const height = canvas.clientHeight;
    const needResize = canvas.width !== width || canvas.height !== height;
    if (needResize) {
      renderer.setSize(width, height, false);
    }
    return needResize;
}

And in the beginning of render function add:

if (resizeRendererToDisplaySize(renderer)) {
  const canvas = renderer.domElement;
  camera.aspect = canvas.clientWidth / canvas.clientHeight;
  camera.updateProjectionMatrix();
}

Now after calling npx webpack and node app.js you can refresh your browser page and see that it's not pixelated anymore.

Note: you must do npx webpack each time you are changing src/index.js.

For your convenience you can edit package.json by adding scripts for frequently used actions:

"scripts": {
    "build" : "npx webpack", 
    "serve" : "node app.js",
    ...

To run a script you do:

npm run build
npm run serve

To enable full-screen you can set click handler for the canvas:

function main() {
    // setup:
    const canvas = document.querySelector('#myCanvas');
    canvas.onclick = ()=>canvas.requestFullscreen()

If everything above worked you can proceed by installing Leia WebGL SDK .

Backup With Git

Before doing this you might want to backup your project. If you have git installed:

git init

It is good to avoid storing unwanted files in your repository by adding .gitignore (starting with dot) file with list of files to ignore: 

dist/main.js
main.js.LICENSE.txt 
dist/main.js 
node_modules/ 
git add .
git status

You should see in the terminal:

Changes to be committed: new file: .gitignore new file: app.js new file: dist/index.html new file: dist/texture.jpg new file: package-lock.json new file: package.json new file: src/index.js new file: webpack.config.js

git commit -m 'Add rotating cube project'

Testing The Page

Before proceeding it's also good to test this page on your device. You can do this by accessing same WiFi point, knowing your PC/laptop IP address you do something like (requires adb):

adb shell am start \
-n com.android.chrome/com.google.android.apps.chrome.Main \
-a android.intent.action.VIEW -d "192.168.1.106:3000"

Of course you must start your server first and you'll have to run this command from separate terminal tab.

Alternatively you can manually enter <yourIP>:3000 address into the browser's address bar on your device.

MacOS

On macOS usually ipconfig getifaddr en0 helps to get your local IP: 

adb shell am start \
-n com.android.chrome/com.google.android.apps.chrome.Main \
-a android.intent.action.VIEW \
-d $(ipconfig getifaddr en0)":3000"

Or you can open System Preferences, Network to see the IP.

Windows

Run ipconfig in a command-prompt and use the "IPv4 Address" on your Leia device to access the server.

Adding Leia WebGL SDK

Importing

To install Leia WebGL SDK : 

npm install leiawebglsdk

Add next imports to your src/index.js (right after the THREE import):

import RenderController from 'leiawebglsdk/src/RenderController.js';
import BackLightController from 'leiawebglsdk/src/BackLightController.js';
import { BacklightMode } from 'leiawebglsdk/src/Constants.js';

Canvas Setup

Now Leia WebGL SDK is accessible in your code.

Change beginning of your main function to enable Leia Backlight on first click (helper apk must be installed) and enable full-screen on second click.

function main() {
	// setup:
	const canvas = document.querySelector('#myCanvas');
	var clickCount = 0
	canvas.onclick = () => {
		if (clickCount++ == 0) 
			BackLightController.requestBacklightMode(BacklightMode.ON);
		else 
			canvas.requestFullscreen()
	}

That's only one option to organize backlight/full-screen control.

When you request a new backlight mode, onBlur (focus lost) and onFocus events happen. So if you are writing your handlers for onBlur and onFocus events you should ignore these events if you have been invoking requestBacklightMode recently (<500ms ago).

When backlight is OFF it is better to show regular rendering.

Important: Leia Web GL SDK requires separate canvas.

dist/index.html changes to:

<script type="module" src="main.js"></script>
<body style="margin:0">
   <canvas id="hiddenCanvas" style="display: none;"></canvas>
   <canvas id="myCanvas" style="width:100%;height:100%"></canvas>
</body>

Canvas for Three.js is now accessed by searching for hidden canvas id and used only to initialize Three.js. A new mainCanvas will be used for Leia WebGL SDK.

function main() {
    // setup:
    const canvas = document.querySelector('#hiddenCanvas'); // for three.js
    const mainCanvas = document.querySelector("#myCanvas"); // for leia webgl sdk

Now there's no rendering visible and you need to change code that deals with renderer and camera (you will need to use multiple cameras). Leia WebGL SDK RenderController will provide required settings for these cameras.

Initialization

After the onClick handler declaration, put initialization code:

var gl = mainCanvas.getContext("webgl", { preserveDrawingBuffer : true });
const controller = RenderController;
var convergenceDistance = 20; // distance from camera to the focus point
controller.initialize(mainCanvas, gl, window);
controller.setConvergence(convergenceDistance);

Note the distance here is 20. While in our scene we set cube position to 2. We need to update cube position (and scaling, to make it bigger):

	...
	const geometry = new THREE.BoxGeometry(5, 5, 5);
	...
	cube.position.z = -20;
	...

If scaling of your scene is small, in other words if you have small objects, small distances this will create huge disparity because default baseline in RenderController is 1.

You can setConvergence but you'll need to invoke camera recalculations updateProjMats() in order to see effect of changing convergence.

Instead of a single camera declaration (comment or remove that line), use RenderController to create multiple cameras (and also place scene declaration above it):

//const camera = new THREE.PerspectiveCamera(fov, aspect, near, far);
const scene = new THREE.Scene(); 
controller.setupCanvas(gl); 
const rtWidth = controller.getRenderTextureWidth(); 
const rtHeight = controller.getRenderTextureHeight(); 
const renderTarget = new THREE.WebGLRenderTarget(rtWidth,rtHeight);
var cameras = []
function updateProjMats() {
    let projectionMatrices = controller.getProjectionMatrices("perspective");
    for (let i = 0; i < projectionMatrices.length; i++) {
      let matrix = new THREE.Matrix4();
      matrix.elements = projectionMatrices[i];
      cameras[i].projectionMatrix = matrix.clone();
      cameras[i].projectionMatrixInverse = matrix.clone().invert();
    }
}
var cameraPositions = controller.getCameraPositions();
for (var i = 0 ; i < cameraPositions.length ; i++) {
    // note, fov, aspect, near, far will be ignored, 
    // you need to set them through controller:
    const camera = new THREE.PerspectiveCamera(fov, aspect, near, far);
     // all cameras centered around zero: 
     camera.position.set(cameraPositions[i], 0, 0);
     cameras.push(camera); 
     scene.add(camera); 
}
screen.orientation.addEventListener("change", function(e) { 
    controller.adaptToOrientation(screen.orientation.type); 
    updateProjMats(); 
}, false);
controller.adaptToOrientation(screen.orientation.type);
updateProjMats();
controller.setupTextures(gl, rtWidth, rtHeight);

Rendering

Continuing to set up Leia WebGL SDK, you'll need to change your renderer function, comment renderer.render line and add:

    // actual rendering:
		//renderer.render(scene, camera);
    const tempBuffer = new Uint8Array(rtWidth * rtHeight * 4);
    cameras.forEach((camera,index) => {
      renderer.clear();
      renderer.setRenderTarget(renderTarget);
      renderer.render(scene, camera);
      renderer.readRenderTargetPixels(renderTarget, 0, 0, rtWidth, rtHeight, tempBuffer);
      controller.saveTexture(tempBuffer, index, rtWidth, rtHeight, gl); 
    });
    controller.update(gl);

Our canvas resize code must be commented out or removed too, at the top of the render function:

//if (resizeRendererToDisplaySize(renderer)) {
//  const canvas = renderer.domElement;
//  camera.aspect = canvas.clientWidth / canvas.clientHeight;
//  camera.updateProjectionMatrix();
//}

Because or canvas is now hidden, only mainCanvas will receive clicks, change canvas.onclick = () => { to:

mainCanvas.onclick = () => {

Disable canvas fullscreen as it is currently unsupported. This is a known issue in the WebGL SDK.

//else
//    canvas.requestFullscreen()

At top of your html file add:

<meta name="viewport" content="...">

And in your main function set scaling once and disable user scaling to prevent 3D effect from the damage:

function main() {
	// ensure pixel perfect scaling for different display settings
	document.querySelector("meta[name=viewport]").setAttribute
    	('content', 'initial-scale=' + 
    	(1.0/window.devicePixelRatio) + ', minimum-scale=0.01, user-scalable=0');
    ...

Result

That's it, rendering must change now (visible interlacing lines) and after clicking on your page with rotating cube you'll see Leia backlight in action.

Complete source code of dist/index.html:

<meta name="viewport" content="...">
<script type="module" src="main.js"></script>
<body style="margin:0">
   <canvas id="hiddenCanvas" style="display: none;"></canvas>
   <canvas id="myCanvas" style="width:100%;height:100%"></canvas>
</body>

Complete source code of src/index.js (rotating cube with Leia WebGL SDK enabled):

import * as THREE from 'three';
import RenderController from 'leiawebglsdk/src/RenderController.js';
import BackLightController from 'leiawebglsdk/src/BackLightController.js';
import { BacklightMode } from 'leiawebglsdk/src/Constants.js';

function main() {
	// ensure pixel perfect scaling for different display settings
	document.querySelector("meta[name=viewport]").setAttribute
    	('content', 'initial-scale=' + 
    	(1.0/window.devicePixelRatio) + ', minimum-scale=0.01, user-scalable=0');

	// setup:
	const canvas = document.querySelector('#hiddenCanvas'); // for three.js
    const mainCanvas = document.querySelector("#myCanvas"); // for leia webgl sdk
	var clickCount = 0
	mainCanvas.onclick = () => {
		if (clickCount++ == 0) 
			BackLightController.requestBacklightMode(BacklightMode.ON);
		//else 
		//	canvas.requestFullscreen() // known issue in Leia WebGL SDK, resolution is in development
	}
	var gl = mainCanvas.getContext("webgl", { preserveDrawingBuffer : true });
	const controller = RenderController;
	var convergenceDistance = 20; // distance from camera to the focus point
	controller.initialize(mainCanvas, gl, window);
	controller.setConvergence(convergenceDistance);
	const renderer = new THREE.WebGLRenderer({canvas});
	const fov = 60, aspect = 1.6, near = 0.1, far = 100;
	//const camera = new THREE.PerspectiveCamera(fov, aspect, near, far);
	const scene = new THREE.Scene(); 
	controller.setupCanvas(gl); 
	const rtWidth = controller.getRenderTextureWidth(); 
	const rtHeight = controller.getRenderTextureHeight(); 
	const renderTarget = new THREE.WebGLRenderTarget(rtWidth,rtHeight);
	var cameras = []
	function updateProjMats() {
	    let projectionMatrices = controller.getProjectionMatrices("perspective");
	    for (let i = 0; i < projectionMatrices.length; i++) {
	      let matrix = new THREE.Matrix4();
	      matrix.elements = projectionMatrices[i];
	      cameras[i].projectionMatrix = matrix.clone();
	      cameras[i].projectionMatrixInverse = matrix.clone().invert();
	    }
	}
	var cameraPositions = controller.getCameraPositions();
	for (var i = 0 ; i < cameraPositions.length ; i++) {
	    // note, fov, aspect, near, far will be ignored, 
	    // you need to set them through controller:
	    const camera = new THREE.PerspectiveCamera(fov, aspect, near, far);
	     // all cameras centered around zero: 
	     camera.position.set(cameraPositions[i], 0, 0);
	     cameras.push(camera); 
	     scene.add(camera); 
	}
	screen.orientation.addEventListener("change", function(e) { 
	    controller.adaptToOrientation(screen.orientation.type); 
	    updateProjMats(); 
	}, false);
	controller.adaptToOrientation(screen.orientation.type); 
	updateProjMats();
	controller.setupTextures(gl, rtWidth, rtHeight);
	
	scene.background = new THREE.Color("rgb(0, 128, 256)");
	
	// creating example textured cube	
	const loader = new THREE.TextureLoader();
	const geometry = new THREE.BoxGeometry(5, 5, 5);
	const material = new THREE.MeshBasicMaterial({
		map: loader.load('texture.jpg'),
  	});
	const cube = new THREE.Mesh(geometry, material);
	cube.position.z = -20; // important, cube and camera must not be at same place
	scene.add(cube);

	function resizeRendererToDisplaySize(renderer) {
	    const canvas = renderer.domElement;
	    const width = canvas.clientWidth;
	    const height = canvas.clientHeight;
	    const needResize = canvas.width !== width || canvas.height !== height;
	    if (needResize) {
	      renderer.setSize(width, height, false);
	    }
	    return needResize;
	}

	// render loop
	function render(time) {
// if (resizeRendererToDisplaySize(renderer)) {
//   const canvas = renderer.domElement;
//   camera.aspect = canvas.clientWidth / canvas.clientHeight;
//   camera.updateProjectionMatrix();
// }
	// rotation animation:
	cube.rotation.x = time * 0.001;
	cube.rotation.y = time * 0.003;

// actual rendering:
	//renderer.render(scene, camera);
    const tempBuffer = new Uint8Array(rtWidth * rtHeight * 4);
    cameras.forEach((camera,index) => {
      renderer.clear();
      renderer.setRenderTarget(renderTarget);
      renderer.render(scene, camera);
      renderer.readRenderTargetPixels(renderTarget, 0, 0, rtWidth, rtHeight, tempBuffer);
      controller.saveTexture(tempBuffer, index, rtWidth, rtHeight, gl); 
    });
    
    controller.update(gl);

		// asking for next frame:
		requestAnimationFrame(render);
	}

	// asking for initial frame:
	requestAnimationFrame(render);
}

window.onload = main

Download Source Code

This project source code can be downloaded with our GitHub repo, under Example 1: https://github.com/LeiaInc/Leia_WebGL_SDK https://github.com/LeiaInc/Leia_WebGL_SDK/tree/main/example1

PreviousPart 1: Server SetupNextExample 2: VSC with Three.js

Last updated

Copyright © 2023 Leia Inc. All Rights Reserved