Building 3-D Web Experiences With react-three-fiber

If you need to build a 3-D experience on the web, and plain old CSS, SVG, or canvas aren't going to cut it, using WebGL (the browser's API for 3-D rendering) is the way to go. Working with the raw WebGL API is hard and time consuming, though.

Three.js is a popular library that makes working with WebGL easier. Most 3-D experiences you see on the web are built with Three.js. Their homepage features a ton of great real-world examples, from NASA to Github to games:

Despite being an abstraction on top of WebGL, even Three.js can feel pretty low-level, especially if you're used to using libraries like React or Vue.

Consider how large web UIs aren't often built with hand-crafted imperative DOM manipulation code like this:

const element = document.createElement('div');
element.innerText = 'Hello world';
document.body.appendChild(element);

Three.js has a similar imperative API:

const geometry = new THREE.BoxGeometry();
const material = new THREE.MeshBasicMaterial({ color: 'orange' });
const cube = new THREE.Mesh(geometry, material);
scene.add(cube);

This imperative programming style can get unwieldy once you have enough state and lifecycle to manage, which is exactly why frameworks with a declarative API, like React and Vue, are so popular.

What if you could build 3-D scenes with React? You can! react-three-fiber is a library that lets you do exactly that.

react-three-fiber

Rather than using divs and spans, react-three-fiber (R3F) lets you render Three.js objects like meshes, lights, cameras, and shaders. Your code looks more like this:

const MyScene = () => {
  return (
    <Canvas>
      <PointLight position={[10, 10, 10]} />
      <Camera position={[0, 0, 10]} />
      <Product color={selectedColor} />
    </Canvas>
  );
};

You declare the 3-D objects you want in a scene and R3F takes care of running the imperative Three.js code under the hood. And, in the same way that React gives you hooks to access the primitive DOM elements via refs, R3F lets you access the primitive Three.js objects for when you need a little more control.

You also have easy access to the existing, massive Three.js ecosystem of examples and libraries. The extend API from R3F lets you render third-party objects using JSX:

import { Canvas, extend } from 'react-three-fiber'
import { OrbitControls } from 'three/examples/jsm/controls/OrbitControls'

extend({ OrbitControls })

...

<Canvas>
  <orbitControls enableDamping />
</Canvas>

At the end of this post you'll find resources and tutorials on how to use react-three-fiber.

Use Cases

Three.js is great for all kinds of 3-D experiences on the web, and I'd say you should always consider using react-three-fiber to manage a Three.js scene, whether that's for art, adding a 3-D product viewer to an e-commerce site, VR/AR experiences, productivity tools, or even games.

If you're already building a web page in React and need to add 3-D functionality, you can use all the same state management libraries, data fetching abstractions, and pass data and props between your 3-D scene and other parts of your React project.

Here are some R3F examples:

https://codesandbox.io/embed/r3f-game-i2160

https://codesandbox.io/embed/r3f-moksha-f1ixt

https://codesandbox.io/embed/r3f-train-l900i

Performance

R3F on top of Three.js has the same performance characteristics as React on top of the DOM. Any abstraction has some overhead, and theoretically it's possible to write faster lower-level code yourself, but in practice and at scale, React's component model makes balancing complexity and performance significantly easier.

But, building performant 3-D experiences on the web can be challenging regardless of the abstraction you're using. Once you reach a certain level of complexity, you may need to start digging in to how the GPU and WebGL work in order to achieve your performance goals. You're significantly more likely to be bottlenecked by your specific approach rather than by the abstractions provided by R3F or Three.js.

For example, if React is a tool for building UIs on the web, can you use R3F to build 3-D UIs? I explored this from the perspective of futuristic, sci-fi UIs, and you absolutely can build 3-D UIs with R3F:

https://formidable.com/blog/2021/future-ui/

You can easily render custom fonts, SVGs, 3-D objects, rounded rectangles, handle user interactions, and even make it all accessible. But, many of us take for granted the performance optimizations that browsers and the DOM take care of automatically.

If you take a naive approach to rendering a few hundred flat rectangles in raw WebGL, Three.js, or R3F, it's going to use a lot more CPU and GPU power than if you did that with the DOM. A naive approach will mean that every frame (every 16 milliseconds on most monitors), the CPU will send unique instructions to the GPU for every rectangle. It can get slow. Hundreds of divs, rectangles, borders, icons, and blocks of text are common in UIs.

To make performant UIs using the GPU, you need to minimize the number of instructions sent to the GPU, known as "draw calls". "Instancing" is the technique for reducing draw calls, which means that wherever possible, you send a single mesh to the GPU, (e.g., a rectangle), then tell it how many instances you want, where they should be located, and what scale. Three.js and R3F support instancing, but it's up to you to think of how to apply it for your type of UI. Neither Three.js nor R3F are a "UI framework", but you can build UIs with them if you're comfortable tackling these performance hurdles yourself, which may very well be required if you're building a VR/AR experience.

What You Need to Know

The world of 3-D can be overwhelming at first. These libraries and abstractions help significantly, but building 3-D experiences can require understanding a few new concepts.

In the same way that you need to understand HTML when using React, you need to understand Three.js when using R3F, since ultimately, R3F is just a way to manage a Three.js scene. All of the actual rendering is done by Three.js. You will need to reference the Three.js docs when working with different types of concepts.

If you're new to Three.js you should get comfortable with these core Three.js concepts first:

• Cameras
• Lights
• Materials, textures, shaders
• Geometry
• Meshes
• 3-D model file formats

Once you're comfortable with these, the react-three-fiber documentation will make a lot more sense.

The last section of this article is a roundup of documentation, tutorials, and community libraries that will help get you started. Hopefully this overview and these resources give you everything you need to start building 3-D experiences on the web. Good luck!

Resources

Three.js:

• https://threejsfundamentals.org
• https://threejs.org/docs/

React

• https://reactjs.org/docs/getting-started.html
• https://reactjs.org/docs/hooks-intro.html

react-three-fiber tutorials:

• https://www.smashingmagazine.com/2020/11/threejs-react-three-fiber/
• https://blog.logrocket.com/3d-rendering-in-the-browser-with-react-three-fiber/

react-three-fiber documentation:

• https://docs.pmnd.rs/react-three-fiber/api/canvas

Libraries built on top of react-three-fiber

• @react-three/drei Tons of helpful components and hooks
• @react-three/flex Use flexbox to layout 3-D objects
• @react-three/postprocessing Shader effects like bloom, depth of field, and more
• @react-three/a11y Expose and describe your 3-D objects to assistive technologies