PAINTING WITH DATA

1854

London is suffering one of its worst ever cholera outbreaks. The disease has ripped through Soho, killing over 600 people, and causing many of the local residents to flee the area.

At the time there were two competing theories on the cause of cholera’s spread: Some people argued that it was spread by rank-smelling air or “miasma”, that came out of rotten, decaying matter.

Meanwhile, others said that it was spread through some sort of unidentified microscopic organism, or “germ”. If you’ve heard of germ theory but haven’t heard of miasma theory then at this point you might have an idea which side was right, but please don’t spoil it for your neighbours if they haven't figured it out yet.

Now every good story needs to have a hero, and this one is no exception. He was an English physician, and a skeptic of miasma theory. His name was John Snow.

Good charts
save lives 💪

A good chart
should have a story

But thirdly and most importantly, it tells us that a good chart should have a story.

Most data visualization is done for the purpose of communication. We have an insight about a dataset, and we have a potential audience, and we would like to convey our insight to our audience. To communicate our insight successfully, we will have to present the audience with a clear and exciting story.

The need for a story may sound a bit disturbing to engineers, who may equate it with making things up, putting a spin on things, or overselling results. After all, surely the point of a chart is to persuade people with cold hard numbers, not wishy washy emotions. However, we must not overlook the important role that stories play in reasoning and memory. We get excited when we hear a good story, and we get bored when the story is bad or when there is none.

What is your
chart's purpose?

Who are your audience?

Aesthetics

Continuous

Discrete

Continuous data values are values for which arbitrarily fine intermediates exist. For example, distance is a continuous value, because you can break that down into as many intermediates as you like, from kilometres down to nanometers. On the other hand, number of persons in a room is a discrete value. A room can hold 5 persons, or 6 persons, but not 5 and a half persons.

You can use position, size, colour or line width to represent continuous data, while shape and line type should usually only be used to represent discrete data.

Each of these can have advantages and disadvantages for different types of data. e.g. size can be misleading because it's tricky for the human eye to accurately tell how much bigger one thing is than another, while certain colour scales don’t work well for colour-blind people. So you’ll want to be careful how you use these scales.

Animated aesthetics

⚠️

Photosensitive
seizure warning

The following slide contains
rapidly flashing images

Every second
8,796 photos
are shared on Snapchat.

In 2015, I was working with a creative team making interactive web content. And we did a piece about this hot new app called Snapchat. At the time, it was growing very quickly, and it was amazing to think there might be nearly 9 thousand photos being shared per second. We wanted to impress upon people just how many photos this was, so we visualised it by using the Snapchat logo or an emoji to represent each photo.

I thought it was effective, and it did okay on Reddit, but that many photos per second was a lot for most computer’s GPUs to handle.

So, predictably, the frame-rate suffered on all but the most powerful computers. Mobile phones did not handle it well at all. It was a good learning experience about performance limits on different devices.

By the way, the number of Snapchats being shared these days is closer to 34 thousand per second, which would probably be enough to completely ruin the frame rate for everyone, if not completely crash your browser.

So that was a fun project :). But you can also use animation in more subtle ways to communicate about your data!

Common dataviz variable types

Type of variable	Examples	Appropriate scale
quantitative/numerical continuous	1.3, 5.7, 83, 1.5x10^-2	continuous
quantitative/numerical discrete	1, 2, 3, 4	discrete
qualitative/categorical unordered	dog, cat, fish	discrete
qualitative/categorical ordered	good, fair, poor	discrete
date or time	Jan. 5 2018, 8:03am	continuous or discrete
text	The quick brown fox jumps over the lazy dog.	none, or discrete

Source: https://serialmentor.com/dataviz/aesthetic-mapping.html

Scales map data values
onto aesthetics

	1	2	3	4
Position
Shape
Colour

Scale functions

Domain (pixels):
Range (hue):
Input value (pixels):
Output value (hue):

Choosing a
chart type

A couple of years ago, I did some work for a client project, with a team who asked us to make a bubble chart. We asked why they wanted a bubble chart specifically, and they said it’s mainly because the client likes bubble charts. You can show a lot of different dimensions at once with a bubble chart, and it’s what they’ve always used, so the client has grown to expect them. So we built it for them. And I think it ended up being a really good bubble chart, with lots of filters and animated transitions between states and so on, and the client was impressed and the team were happy. But I can’t help thinking that we were going in with one arm tied behind our backs, and that we could have chosen a chart that would have communicated so much better if we’d been allowed more flexibility to choose a chart that would suit the dataset.

Don’t choose a chart type just because you think it looks cool

Choosing a web technology

<svg />

SVG is basically an alternate universe version of HTML focused on graphics instead of documents, but where everything is set to position absolute. You get some new element types like circles and polygons and lines, but you have to use a bizarro-world version of CSS which has features like stroke and fill, but you can’t use most of the regular useful features that you’d normally depend on, like background-image and border and box-shadow and so on.

Also, text won’t wrap automatically which even more annoying than it sounds, so you need to use JavaScript to calculate text wrap with weird hacks like rendering to the DOM, then measuring the text width, then deleting it and re-rendering it with extra tags inserted to add line breaks, which is basically a nightmare.

SVG has a DOM just like HTML, so it’s not great for JavaScript animations on large numbers of elements, but you can use CSS animations which are super performant, so I try to use those over JavaScript animations wherever possible.

<canvas />


	const canvas = document.querySelector('canvas');
	const context = canvas.getContext('2d');
	context.clearRect(0, 0, width, height);
	context.drawImage(object.image, x, y, object.width, object.height);
	context.fillStyle = object.fillStyle;
	context.fillRect(x, y, object.width, object.height);

WebGL

Just because you can
doesn't mean you should

For that reason, I don't think many people actually write native WebGL code. You're better off using a library like Three.js, which is what I used to code this lovely spinning bar chart.

Down 👇 to show title text

I use this example to raise a very real issue with 3D charts, which is that most of the time, they're completely gratuitous and don't add anything useful. In fact by projecting data into a third dimension then representing it on a 2D screen, you're introducing visual distortion. Our eyes can try to correct for visual ambiguity, but we can only do so much.

Much like all those crappy clip-art-looking 3D pie charts generated with Microsoft Excel for early 2000s Powerpoint presentations, many WebGL charts are gratuitous decoration that do nothing to help the viewer understand the data. So I'd avoid them if at all possible. The best exceptions to this rule, are visualisations of actual 3D objects, like topographical maps with mountainous terrain. But in this case, you'll still want to allow users to rotate the view, to reduce the distortion effect by providing different perspectives.

Don't forget about

HTML & CSS!

Choosing a
charting library

But some of you might be wondering what D3 is, and why I didn't include it in the list. D3 is a data visualisation library too, in a way. But unlike the others, it's much more low-level. It's a bit more like jQuery or Underscore than a chart library, in the sense that, rather than giving you ready-built charts or chart components that you can configure, it gives you the tools to build your own custom charts from scratch, and a set of best practices to follow.

And in fact the syntax looks pretty similar to jQuery too. So if you like method chaining, you'll love it!

D3 is designed to enforce separation of concerns between data and presentation, just like the web was designed to do with HTML & CSS. I know that's not a popular idea lately, what with CSS in JS and all, but I reckon it's still a useful model in the right context. Fight me.

	D3	Charting libraries
Development time	🐌	⚡️
Flexibility	💃	👴🏻
Learning curve	😖	😌
Performance	🏎	🚗
Code readability	🙅‍	🤷‍
Popularity	👯‍	📭

So instead of asking “which library should I use”, the real question is, “should I use D3, or another charting library”? And this one really depends on your situation, because each has a set of pros and cons.

DOWN

1. For instance, making a chart with a charting library is typically much quicker than writing something from scratch with D3. That’s the whole point: They’re an abstraction that saves you having to repeat the same basic steps every time you make a new chart.

DOWN

2. But when it comes to power and flexibility, that’s when D3 can really dance around the competition. It gives you full control over how things look, and has so many different tools and algorithms that you can mix and match to create really cool effects. I've been working with D3 for many years and I still regularly come across things where I'm like, whoa, D3 can do that?

It also works seamlessly with existing web technologies, and can be used in conjunction with other tools and frameworks. You can manipulate any part of the DOM, or use it with SVG, Canvas, WebGL, or whatever.

By contrast, most charting libraries are fairly opinionated, so they don't always do everything you want them to. If you have a specific design in mind, then the abstractions they’ve chosen may not fit your use case.

I often find that it's not until I’ve nearly finished building a chart, that I discover the limitations of a given library's API, and realise that it it's not able to implement certain quirks of the design I've been given. So then I have to start all over again in something else.

So what I sometimes do is make a basic prototype in a charting library, then build a more fleshed-out version in D3 once I've got more time to invest.

DOWN

3. But when it comes to the learning curve, that’s definitely a downside for D3. Most charting libraries are designed to be fairly easy to get started - often you just need to modify an example config, and it’s usually pretty intuitive.

Whereas D3 can take a bit longer to get your head around. I found D3’s data binding and element creation especially tricky to understand when I was making my first chart with it: First, you need to create a new Selection, to select all the DOM elements on the page. But they don’t exist yet, because you haven't created them. Then you bind data to them, even though they still don’t exist. Then you ‘enter’ them, which means specifically selecting the ones that exist in the data, but not in the DOM. Then you append DOM nodes to this selection, and only then do they actually get created on the page. It makes sense once you understand why it works that way, but it did my head in.

DOWN

4. Now when it comes to performance, whether animation performance or download size, that’s a mixed bag. I’ve found that some charting libraries I've used have been a little janky at executing complex animations, but it really depends how far you’re pushing it beyond what it was designed to do. Whereas D3 is usually great for animation, and makes it surprisingly easy to do some really cool transition effects like staggered delays, and you can use CSS animations with it too.

In terms of download size, it really depends on the library. If you only need one bar chart but you have to include the whole kitchen sink, that’s obviously not ideal. D3 used to be the worst culprit for this: It has dozens of different APIs, and if you import the whole thing it’s about half a megabyte unminified. But as of version 4, it's now completely modular, so you can just import the bits you want. Win.

DOWN

5. Next up, code readability and maintainability. Of course this depends on the library and on your preferences, after all code beauty is in the eye of the beholder. But I think most of us would agree that D3’s more imperative code style and chained functions are a bit more complex to understand. Which means that you need to put more thought into documenting them, and avoid writing sprawling spaghetti code. Most libraries are gonna beat D3 on this one.

DOWN

6. Finally, D3 is far more popular than any other JavaScript charting library, at least if you’re going by Github stars. It has a large community, and lots of examples, tutorials, stackoverflow questions, demos and even a few books written about it.

In summary

PAINTING WITH DATA

1854

Good charts
save lives 💪

A good chart
should have a story

What is your
chart's purpose?