Inside Sketch: A deep dive into our Canvas tech

If you’re familiar with Sketch, you’ve probably looked at your documents both in the Mac app and the browser. Though the touch points are different, you see the same content. But have you ever wondered how we make that happen?

When you view your Sketch document in a browser, you’re actually looking at a preview of the document we’ve prepared in advance. It’s not just any preview — it’s a fully zoomable, scrollable preview of the entire page with potentially hundreds of Artboards on them. Using this preview, we can make documents accessible to anyone in your team, even if they don’t have a Mac, and still offer them a great experience.

I want to share some more insights about how we did this and how we keep rendering accurate between platforms. I’ll also walk you through some of the performance improvements we’ve recently unlocked.

A note on measuring performance

Before we start, a brief note about measuring performance, both at Sketch and as an industry. It’s also hard to pin down performance to a single number. It might sound good on paper but often doesn’t mean much in practice. For example, because Sketch documents are incredibly varied, some of the changes we made were very impactful to some yet nearly immaterial to others. Ultimately, we opted for improving performance across the board by focusing on our fundamental architecture rather than on a specific metric or chasing individual edge cases.

On a secondary note, the chief difficulty for large tech projects such as these is that you have to swap out the engine while keeping the car running. This brings its own set of challenges both for the team and the product.

Introducing: scene graph

For the past two years, we’ve been working on a long-term plan in the Mac app to switch away from our straight-to-pixels rendering pipeline. Instead, we opted to create an intermediary representation between Sketch layers and the pixels you see on your screen which we felt could unlock some good performance optimisations for us.

This graph doesn’t think in terms of layers with styles, groups and Artboards but in more basic terms such as paths you need to fill or stroke — which is a good level at which to analyze redundant drawing operations and speed things up. After all, the key to faster rendering is to draw as little as possible between passes.

In 2022, we started building out our scene graph — a data structure made up of nodes that helps organize and represent a graphical scene. In our case, we’re talking about representing typical objects in Sketch, such as layers, styles and Artboards in a more fundamental way.

Here’s an example of what that looks like:

One step at a time

We started out by using our scene graph only when rendering individual layers, effectively creating hundreds of tiny scene graphs during each rendering pass. We then moved on to groups of layers and entire pages, and finally to sharing a single scene graph across all threads. With every release, we steadily increased the scene graph’s scope.

With the release of Sketch 95, we’ve crossed another major milestone. Rather than reconstructing a scene graph from scratch every time, we’re now updating it in place. For example, when a user moves a layer, now we only need to update one or two properties in the tree, rather than recreate it entirely.

It may seem like an obvious thing to do, and we would agree. But as we said earlier, having to keep the engine running while you make changes means that we need to make our improvements step by careful step — and we’re super happy to have taken the last big step in Sketch 95.

Improvements on the Mac app

Building the scene graph has been a success for us, and we’re seeing faster performance in many of our own documents. Thanks to this leap in technology, we’ve also been able to make the following feature improvements, among others:

• Better shadow rendering when zoomed in
• Improved background blurs inside Symbols and in masks
• Ability to turn any layer into an alpha mask and apply blur to it
• Support negative shadow spreads (the casting shape can be smaller or larger than the shadow)

But that’s not the only measure of success. Thanks to rolling out the scene graph technology on the Mac, we’ve also drastically improved Sketch’s fidelity and capabilities in the browser.

Taking it to the Web

If you’ve been using Sketch for a while, you probably noticed that we switched from a rigid grid of Artboards to an infinite Canvas view in our web app. This mirrors the experience we’re all familiar with on the Mac and really helps streamline conversations between designers and stakeholders because we’re all looking at the same thing.

How did we build this new full Canvas experience on the web? You may have guessed the answer by now: the new Canvas experience is, of course, powered by the same scene graph we built for the Mac app. The great thing about our new scene graph is that it’s zoom-independent, and we can render it live in the front-end without having to use any pre-rendered assets. This meant we could theoretically offer a fluid panning and zooming experience in a browser.

That doesn’t mean it was an easy feature to build, though. The scene graph has already taken care of complex logic like resolving Symbols, overrides, Boolean operations, outlined text and more. But there was still the not-so-trivial part left of interpreting these commands and getting pixels on the screen quickly, accurately, and in a way that’s supported by the wide variety of browsers out there.

Overcoming the challenges

Let’s start at the beginning: the scene graph is generated by our render farm and stored on the server for later use. When a user tries to view a document in the browser we then pass the generated scene graph on to the frontend, which parses it and renders it to pixels. Thanks to WebAssembly and Emscripten we don’t need to do this in Javascript but we can use C++ and the Skia rendering library — with bits of WebGL sprinkled in — to render paths, fills, filters at full fidelity and at high speed. And because we’re not just displaying images, we can render partially, or fully, at any zoom level, allowing you to pan and zoom to your heart’s content.

Getting the Canvas on the web to pan and zoom smoothly was quite a challenge though, especially on complex Sketch documents. Web browser limitations and the variety of devices it needs to work on really made this a challenging problem. Crucially for example, effective multithreading was only recently enabled by all browsers, so the renderer has to work correctly even when only the main thread is available.

How do you render fast and fluently while not blocking user interaction if you have only one thread available to work on? We won’t go into detail here, but we came up with our own strategies, like carefully scheduling and executing time-consuming tasks only when the user is not interacting, and we built our own system to properly cache the Canvas content and speed up operations like panning and zooming.

What started as a project to improve render speed on the Mac unlocked great features in another area, even though the improvements seemed invisible to the end user at the start. It’s enormously satisfying to see these long-term bets play out — especially when we’ve managed to increase the parity between the web and Mac experiences.

This has, in turn, unlocked several other long-awaited features for Sketch, but we’ll save that for another post. Until then, stay tuned 😉