How the Eye Reads a Chart: The Encoding Effectiveness Ranking

1Four measurement scales

Chapter 1 classified data by what it holds. This classifies it by what arithmetic is legal, which is the distinction that stops you drawing false conclusions.

Fig 4.1Which operations each scale allows

The interval trap is the one that catches people. Interval scales have no true zero, so differences are meaningful but proportions are not. You can say Hyderabad is 25 degrees warmer than Shimla. You cannot say it is twice as hot. Identify the scale before you compute, otherwise you apply arithmetic the data does not support and produce conclusions that look rigorous and are false.

Fig 4.2Why 50 °C is not twice as hot as 25 °C

same two temperatures, measured in

The ratio changes when you change the unit, which is exactly what proves the ratio was never meaningful. In Kelvin, which does have a true zero, the ratio stays put whatever else you do. A true zero is what makes proportional statements valid, and it is the only difference between interval and ratio.

How the eye decodes a graph

Cleveland's three operations, performed automatically every time somebody looks at a chart.

Detection

Find the geometric object carrying the quantity. Bar length is the quantity. Pie area is the share.

Assembly

Group the marks and find structure, for example following an ordered set of bars as a trend.

Estimation

Judge magnitude and difference. "This is roughly five times that."

Fig 4.3Sorting, and what it does to assembly

Ordering a chart logically speeds up assembly enormously, which is why sorting bars by value is such a cheap improvement. Most weak charts fail at estimation while passing detection, so when you design, check the reader can perform all three.

The encoding effectiveness ranking

This is the core of Chapter 4. Visual channels ranked by how accurately people read them.

#	Encoding	Discriminate	Rank	Estimate
1	Position on a common aligned scale	Yes	Yes	Yes
2	Position on identical but non-aligned scales	Yes	Yes	Yes
3	Length	Yes	Yes	Yes
4	Angle / slope	Yes	Yes	Partly, needs a derived metric
5	Volume / density / saturation	Yes	Partly	Weakly
6	Colour	Yes	Barely	No

Colour separates categories and stops there. You can say which is highest and lowest, but never by how much.
Volume beats colour. Bubble sizes rank reasonably, but comparing circle areas by eye is imprecise, so you end up saying "slightly more" instead of giving a number.
Angle shows direction of change well. For rate of change, plot the rate itself rather than asking the audience to judge slopes.
Length does all three, which is why the humble bar chart is so hard to beat.
Position on a common scale is the most accurate channel available, because the eye measures distance against one shared axis directly.

Test yourself on the ranking

Fig 4.4Two marks, one question: how many times bigger is B than A?

channel

Run the same guess down the ranking and watch your own error grow. On position and length most people land within a few percent. On area the error typically doubles, and on colour saturation there is nothing to estimate with at all. That growing error is the ranking, measured on yourself.

The practical limit on colour

Fig 4.5Discrimination failing past six colours

Avoid more than six colours in one chart. Past that, discrimination itself starts to fail, and matching a slice to its legend entry becomes the reader's main activity. The fix is never a seventh hue: it is folding the small categories into "other", or splitting the chart.

Expressiveness and effectiveness

Test 1

Expressiveness

Does it show all the facts in the data, and nothing the data does not contain?

Test 2

Effectiveness

Is it perceived as easily as it possibly could be? The working definition: information is presented effectively if you cannot improve it further.

A chart can be fully expressive and still ineffective. A saturated colour map contains every value and lets you estimate none of them, which is exactly the case the ranking predicts.

Structured data: choosing the chart

Analysis	Purpose	Charts
Univariate, counts	One variable	Bar chart, big number, pie or donut
Univariate, distribution	Spread and outliers	Histogram, boxplot
Multivariate, comparison	Groups against each other	Stacked bar, box chart
Multivariate, relationships	Do variables move together	Scatter, heat map, parallel coordinates
Multivariate, trends	Change over time	Line chart, dual-axis chart

Big number is the one addition worth noting: a single figure set large is a legitimate chart type when one number is the finding. Boxplot construction is repeated here, and correlation against causation is restated, because the course returns to both repeatedly.

Unstructured data: the pre-processing pipeline

Structured data is a matrix with defined variables. Text has no structure, so it must be converted into something computable before any chart is possible.

Fig 4.6One review, taken through all seven steps

step

Stemming against lemmatisation is the pair worth knowing. Stemming chops characters and sometimes destroys meaning, so "happiness" and "happily" reduce to a stem that means nothing. Lemmatisation resolves to a real root word and does not, so "increased", "increasing" and "increasingly" all give "increase".

Charts for text

Fig 4.7Why a word cloud needs a frequency bar chart beside it

word cloud · size carries frequency

frequency bar chart · length carries frequency

The cloud shows which word is largest and supports no estimation, exactly as the ranking predicts for size. Try to say from the cloud alone how much more often "delivery" appears than "packaging". The bar chart is the cloud's necessary companion, because length does the estimating the cloud cannot.

Chart	Shows
Word cloud	Frequency as font size. The fastest way to see dominant terms
Frequency bar chart	The same counts, but readable as numbers
Word tree	Branches from a root node to show which words follow which, revealing relationships between entities
Sentiment score	Tone of a passage: positive, neutral or negative
Joint plot	Polarity against subjectivity, with both distributions on the axes

Polarity and subjectivity

Polarity

−1 to +1

How negative or positive the text is.

Subjectivity

0 to 1

How much opinion and emotion it carries, against fact. "I feel hungry" is subjective, "I earn a salary" is objective.

Fig 4.8A joint plot, and what averaging destroys

Aggregate sentiment can cancel out. A body of text with strongly positive and strongly negative sections averages to neutral, which hides both. Always segment before averaging sentiment, otherwise the summary destroys the finding.

Analysing conversations

Five questions define any conversation, and each maps to a visual: when it happened, who was involved, from whom to whom, how dense the network is, and what was said.

Fig 4.9Timing and network, from the same message log

Timing is a simple count of messages on a timeline, and a sudden spike in volume is a finding on its own, before anybody reads a word. Density asks whether it is two people talking or one person addressing many, and content is where topic modelling comes in.

Finding topics

Technique	What it does
Word embedding	Groups words used in similar linguistic contexts. The classic relation is king to man as queen is to woman
Topic modelling	Unsupervised learning, using LDA (latent Dirichlet allocation) or LSA (latent semantic analysis), that splits documents into a chosen number of topics
t-SNE clustering	t-distributed stochastic neighbour embedding. Reduces high-dimensional text to a plottable space

Fig 4.10The topic model output: bubbles, and the words inside one

Click any bubble. Distance between bubbles is dissimilarity between topics, so close bubbles are related and distant ones are unrelated, and bubble size is how much of the corpus that topic accounts for. Alongside it sits the word frequency within the selected topic, because the bubble alone never tells you what the topic is about.

Key points

Four measurement scales with the operations each permits.
Why 50 °C is not twice as hot as 25 °C, and what a true zero has to do with it.
Cleveland's three visual operations: detection, assembly, estimation.
The encoding effectiveness ranking, in order, from position on a common scale down to colour.
The three audience functions: discriminate, rank, estimate, and which encodings support which.
Why colour cannot support estimation, and the six-colour limit.
When to plot rate of change instead of asking the audience to judge slopes.
Expressiveness against effectiveness, and the "cannot improve it further" test.
The seven-step text pre-processing pipeline in order.
Stemming against lemmatisation, with an example where stemming fails.
Why a word cloud needs a frequency bar chart beside it.
Polarity and subjectivity, their ranges, and why aggregate sentiment can mislead.
The five aspects of a conversation.
Word embedding, topic modelling with LDA or LSA, and t-SNE, and what each contributes.