08

Chapter 8 · Python

Matplotlib in the Real World

Chapter 7 drew charts from data you typed in. Chapter 8 connects Matplotlib to real data: files, URLs, maps, and data that does not exist yet because it is in the future.

1Getting data in

Pandas reads directly from a URL. You never have to download the file manually.

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt

plt.style.use('seaborn-v0_8-whitegrid')   # consistent clean look for every plot

df = pd.read_csv("https://example.com/data.csv")     # CSV
df = pd.read_json("https://example.com/data.json")   # JSON
FormatWhat it isReader
CSV, comma-separated valuesOne row per line, values separated by commaspd.read_csv()
JSON, JavaScript object notationHierarchical, nested structurepd.read_json()

Everything in Python goes through a DataFrame. That is the shape pandas and Matplotlib both expect, so the first job with any source is getting it into one.

Time series

Fig 8.1What happens when you forget parse_dates

Without parse_dates, dates are strings, and strings sort alphabetically rather than chronologically. "10-Feb" comes before "2-Jan" in an alphabetical sort, so the line jumps between random points and the chart is meaningless. set_index('date') then makes the date the axis, so time-based plotting and slicing work.

Selecting one column, df['temp_max'], is called subsetting, and it is how you plot one series out of a wide table.

Multiple plots in one figure

Fig 8.2The grid, and how you index into it
plt.subplots()

Indexing starts at 0. With a single column, one index works: axes[0], axes[1]. With a grid you must give both, row then column: axes[0, 1], or the code cannot tell where the plot goes. Hover any cell in the grid to see its index.
ElementMeaning
plt.subplots(rows, cols)Creates a grid of axes
axes[0], axes[1]Select a subplot when there is a single column
axes[0, 1]Select a subplot in a grid, row then column
sharex=TrueAll subplots share one x-axis, so dates align vertically
fig.suptitle()Title for the whole figure, not one subplot
plt.tight_layout()Fixes overlapping labels and titles

Why stacked time series need sharex

Fig 8.3Two panels, aligned and not aligned

Stacking related series is how you show a relationship between them, and high temperature against low precipitation reads immediately when the axes line up. Misaligned x-axes make two panels impossible to read together, and the reader cannot tell that it happened.

Plotting from a DataFrame, and the random walk

Fig 8.4Pure noise, before and after cumsum()

The classic description is a drunkard's walk: at each step you toss a coin and move one step either way. After a thousand steps the path looks purposeful, and it is not. Keep this in mind before reading meaning into any trending line. Press the button a few times and count how many of the walks you would have described as a trend.

Encoding a third variable on a scatter

Fig 8.5s= turns a scatter into a bubble chart

Scale the size variable so bubbles stay readable, because raw population values produce bubbles that swamp the plot. Choose x and y from the relationship you expect, since the independent variable belongs on x and the dependent on y. Chapter 4's ranking still applies: position is read accurately, area is not, so bubble size ranks but does not let anyone estimate.

Geographic data with GeoPandas

Fig 8.6One extra column is the whole difference

The geometry column is the whole difference. Without it you have a table; with it you have a map. column= colours regions by a data value, which produces a choropleth, and Chapter 2's population caveat applies here too. legend_kwds controls legend orientation and label, and a horizontal legend often fits better than a vertical one that eats width.
cities = pd.DataFrame({'city': ['New Delhi', 'New York', 'Tokyo'],
                       'lat':  [28.6, 40.7, 35.7],
                       'lon':  [77.2, -74.0, 139.7]})

gdf_cities = gpd.GeoDataFrame(
    cities, geometry=gpd.points_from_xy(cities['lon'], cities['lat']))

ax = world.plot(color='lightgrey')      # base layer
gdf_cities.plot(ax=ax, color='red')     # overlay on the same axes
plt.show()
  1. Build the geometry from longitude and latitude with points_from_xy. Longitude first, since it is the x coordinate.
  2. Pass ax=ax to draw the second layer onto the same axes. Without it you get two separate maps.
  3. You do not always need the full map outline, because plotting the points alone often reads as the region anyway, through the closure principle from Chapter 1.

Curve fitting and extrapolation

Forecasting is a visualisation problem as much as a modelling one: you fit a functional form to existing data and extend it forward.

Fig 8.7Same twelve months. Two assumed shapes. Forty per cent apart at month 18
degree

Roughly a 40% difference in the forecast, from the same data. The only change was the assumed shape. Extrapolation creates data that does not exist, there is always error, and the error grows the further out you go. No functional form is universally best, and one that fits this dataset may fit the next one badly.
Why plot the fit at all

You could run the model and report a number. Showing actual and predicted on one chart does something the number cannot: it lets the audience see how closely the model tracks reality. That is what earns acceptance for the forecast, which is Chapter 2's acceptance principle applied to modelling.

Key points

  1. Reading CSV and JSON, including directly from a URL.
  2. Why parse_dates and set_index are needed before plotting a time series.
  3. plt.subplots(rows, cols), and indexing with one index against two.
  4. What sharex=True does and why stacked time series need it.
  5. fig.suptitle() against ax.set_title(), and what tight_layout() fixes.
  6. cumsum() and the random walk, and why an apparent trend can be pure chance.
  7. Encoding a third variable through marker size, and why the size must be scaled.
  8. What makes a GeoDataFrame different from a DataFrame.
  9. gpd.read_file(), colouring by column=, and building points with points_from_xy.
  10. Passing ax=ax to overlay layers on one map.
  11. np.polyfit and np.poly1d, and what the degree controls.
  12. The polynomial against linear comparison, and why the choice of form changes the forecast so much.
  13. Why the fitted curve is plotted alongside the actual data rather than reported as a number.