Descriptive statistics in geology

Quick refresher

A sample is a tiny part of the population. Core makes up a few billionths of the reservoir volume, and the plugs (small cylinders cut from the core) on which the properties are measured are another hundred times smaller. Any average calculated from them is an estimate with an error. It is not a property of the reservoir.
First we look at the data: we build a histogram and a cumulative curve. Only then do we calculate. A single number will not show two peaks or a long “tail”.
For symmetrical data (the porosity of one rock type), the mean, median and mode are close. For skewed data (data stretched out towards high values, such as permeability), the arithmetic mean is noticeably larger than the median, and the geometric mean is close to the median.
Always $\bar k_{\text{harm}} \le \bar k_{\text{geom}} \le \bar k_{\text{arith}}$ . The arithmetic mean describes flow along layers, the harmonic mean describes flow across layers, and the geometric mean describes a random patchwork (patches of different permeability mixed at random).
We give the mean together with a measure of spread. For permeability, the coefficient of variation $C_v = s/\bar x$ works like this: below 0.5 the rock is homogeneous (nearly the same everywhere), from 0.5 to 1 it is heterogeneous, and above 1 it is very heterogeneous.
P50 is the median. P10 and P90 are read in two opposite ways: in a reserves estimate, P90 is the cautious (small) value, but in statistics the 90th percentile is a large value. Always check which of the two is meant.

Key relationships

\bar x = \dfrac{1}{N}\sum_{i=1}^{N} x_i

arithmetic mean

\bar k_{\text{geom}} = \exp\left(\dfrac{1}{N}\sum_{i=1}^{N}\ln k_i\right)

geometric mean

\bar k_{\text{harm}} = N\left(\sum_{i=1}^{N}\dfrac{1}{k_i}\right)^{-1}

harmonic mean

s = \sqrt{\dfrac{1}{N-1}\sum_{i=1}^{N}(x_i - \bar x)^2}

standard deviation of a sample

C_v = \dfrac{s}{\bar x}

coefficient of variation

Common pitfalls

Averaging permeability arithmetically “out of habit”, without finding out how the flow runs relative to the layers.
Quoting a mean without a measure of spread and without the number of measurements.
Using someone else's P10 and P90 without finding out whether they are read as cumulative shares or as exceedance probabilities (the chance that a value is exceeded).
Replacing values “below the detection limit” (the smallest value the instrument can measure) with zeros, or quietly dropping them.
Applying the “mean ± one deviation” rule to permeability in millidarcies: its distribution is not symmetrical.

After this lesson you can

distinguish a population (everything we want to judge) from a sample (the part we measure), and a property of the reservoir from its estimate from plugs (small cylinders cut from core)
build a histogram (a bar chart of the data) and a cumulative curve, and read the median (the middle value) and percentiles from them
explain the two opposite meanings of the labels P10 and P90, and find out which one is being used
calculate the arithmetic, geometric and harmonic means, the median and the mode (the most common value), and explain why they differ for data that vary a lot
calculate the variance, the standard deviation and the coefficient of variation (measures of how widely the values scatter), and use the coefficient to judge how heterogeneous (uneven) the rock is
choose the measure of the centre that suits the problem, and explain why permeability is viewed on a logarithmic scale

Sample and population: what we have actually measured

Everything that we want to judge, for example the permeability at every point of a reservoir, is called the population in statistics. We cannot measure all of it. We measure a sample: a few dozen or a few hundred core plugs (small cylinders cut from the core), or the log readings in a few wells.

The true mean and the true spread of the population are its parameters, and they stay unknown. From the sample we calculate their estimates: the mean of the plugs, and their standard deviation (how widely they scatter around that mean). An estimate is not a property of the reservoir. It is our approximation to it, and a different set of plugs would give a slightly different number. How different is the subject of the next lesson (Distributions and sample size).

We must name the population before we average anything: the whole accumulation (the oil or gas pool), one layer, or one rock type within it. If sandstone and shale fall into one sample, its mean describes neither of them.

Worked exampleWhat share of the reservoir have we measured

Given (assumed data)

Accumulation: area 10 km², reservoir thickness 20 m
Core: taken in 5 wells through the whole thickness of the reservoir, diameter 100 mm
Plugs: 60 cylinders from each well, diameter and length 30 mm

Solution

Rock volume: $10\cdot10^{6}\cdot20 = 2\cdot10^{8}$ m³.
Core volume: $5\cdot\pi\cdot0.05^{2}\cdot20 = 0.785$ m³. This is $3.9\cdot10^{-9}$ of the reservoir volume.
Volume of one plug: $\pi\cdot1.5^{2}\cdot3.0 = 21.2$ cm³. Three hundred plugs make 6.36 L, or $3.2\cdot10^{-11}$ of the reservoir volume.
To picture it: the same shares of an Olympic swimming pool (2500 m³) are 9.8 mL for the core and 0.08 mL for the plugs.

Answer: The core is about four billionths of the reservoir volume. The plugs on which the properties were measured are about three parts in a hundred billion.

What it means. Two teaspoons of water will tell you everything about a swimming pool, because the water is mixed. Rock is not mixed: its properties change from layer to layer and from well to well. So we must be careful when we judge the whole from such a small part. We also need an estimate of the spread and the error, and we come to both below. The data in this example are assumed for illustration.

There are five things worth finding out about any set of data. This lesson is about the first three.

Centre: the value around which the measurements group.
Spread: how much the measurements differ from one another.
Shape: whether the data are symmetrical, or have a long “tail” or two peaks.
Reliability: how precise these estimates are with the number of plugs we have (Distributions and sample size).
Variation in space: how the property changes with depth and across the area (Spatial correlation and the variogram).

Types of data: what can be averaged

Not every kind of data can be added up and divided. Before we calculate a mean, we should understand which type the quantity belongs to.

Types of quantities and the measures that suit them

Type: Names (nominal)
Examples: rock type (lithology), facies (a set of rocks formed in one environment), name of a formation
What makes sense: proportions and the mode, the most frequent category

Type: Ranks (ordinal)
Examples: grain sorting class (how even the grain sizes are), degree of cementation in points
What makes sense: median and mode; an “average score of 2.3” means nothing

Type: Counts (discrete)
Examples: number of fractures per metre of core, number of thin layers
What makes sense: mean and spread; the values themselves are whole numbers

Type: Continuous
Examples: porosity, permeability, thickness, saturation
What makes sense: all measures of centre and spread

Type	Examples	What makes sense
Names (nominal)	rock type (lithology), facies (a set of rocks formed in one environment), name of a formation	proportions and the mode, the most frequent category
Ranks (ordinal)	grain sorting class (how even the grain sizes are), degree of cementation in points	median and mode; an “average score of 2.3” means nothing
Counts (discrete)	number of fractures per metre of core, number of thin layers	mean and spread; the values themselves are whole numbers
Continuous	porosity, permeability, thickness, saturation	all measures of centre and spread

Among continuous quantities there is one more important difference. Pore volume is additive: the pore volumes of two pieces of rock simply add up. So the mean porosity, weighted by volume, has an exact physical meaning: it preserves the pore volume. For a layer of constant area, we weight by thickness instead (mean porosity and hydrocarbons in place).

Permeability is not additive. Take two plugs of the same size, with permeabilities of 5 and 500 mD (millidarcies). Join them side by side (in parallel). Together they pass almost as much fluid as the good plug alone. Over the doubled cross-section, that is a rock of about 250 mD. Now join them one after the other (in series). Together they resist the flow almost as much as the bad plug alone. Over the doubled length, that is about 10 mD. Permeability has no single “correct” average: it depends on how the parts are placed relative to the flow.

The same applies to ratios, such as the anisotropy ratio $k_v/k_h$ (vertical permeability divided by horizontal permeability) or the coefficient of variation. We do not average them directly. We calculate them again from the averaged quantities.

The histogram, the cumulative curve and percentiles

Calculating averages without looking at the data is like describing the people in a room by their average height, without knowing that half of them are children. The first step is the histogram: we divide the range of values into equal intervals (classes), count how many measurements fall into each, and draw a bar of the matching height for each class. We follow a few rules when we build it.

We usually take from 7 to 25 classes. With too few, the shape of the distribution is smoothed out. With too many, each class holds only one or two values, and the picture breaks up into random spikes.
We choose round class boundaries and decide in advance where to put a value that falls exactly on a boundary, for example always into the class on the right.
On the vertical axis we can plot the share of plugs (the relative frequency) instead of their number. This lets us compare histograms of samples of different sizes.

The cumulative curve (also called the cumulative frequency curve) shows, for each value, what share of the measurements does not exceed it. It rises from 0 to 100%. We read percentiles from it: the values below which a given share of the data lies. The 50th percentile is the median: half of the measurements are smaller than it and half are larger. The 25th and 75th percentiles are called quartiles.

Worked exampleHistogram, cumulative curve and percentiles of porosity

Given (assumed data)

Sample: 40 plugs of one reservoir
Porosity classes, %: 10–12, 12–14, 14–16, 16–18, 18–20, 20–22, 22–24, 24–26
Number of plugs in the classes: 1, 2, 5, 8, 10, 8, 4, 2

Solution

Cumulative number of plugs at the upper boundary of each class: 1, 3, 8, 16, 26, 34, 38, 40. As shares of 40 these are 2.5, 7.5, 20, 40, 65, 85, 95 and 100%.
Mean of the grouped data: the class midpoints weighted by the number of plugs. The sum is $11\cdot1 + 13\cdot2 + 15\cdot5 + 17\cdot8 + 19\cdot10 + 21\cdot8 + 23\cdot4 + 25\cdot2 = 748$ . We divide by 40: $\bar\phi = 18.7$ %.
Median. The 50% mark is reached inside the class 18–20%, where the cumulative share grows from 40 to 65%. We assume the plugs are spread evenly inside the class, so: $18 + 2\cdot(50 - 40)/(65 - 40) = 18.8$ %.
In the same way we find the value below which 10% of the plugs lie, $14 + 2\cdot(10 - 7.5)/(20 - 7.5) = 14.4$ %, and the value below which 90% lie, $22 + 2\cdot(90 - 85)/(95 - 85) = 23.0$ %.
The tallest bar is the class 18–20%. This is the modal class (the class with the most plugs).

Answer: Mean 18.7%, median 18.8%, mode in the class 18–20%; 80% of the plugs have a porosity from 14.4 to 23.0%.

What it means. The mean, median and mode almost coincide, and the bars fall away on both sides of the peak in nearly the same way. The distribution is close to symmetrical, so a mean and a measure of spread are enough to describe it. For the permeability of the same plugs, as shown below, this is no longer so. The data in this example are assumed for illustration.

CalculatedHistogram and cumulative curve of the porosity of the 40 plugs of the example. The dots on the curve mark the values below which 10, 50 and 90% of the plugs lie.

On a histogram we look for what no average can show. Two peaks mean that two rocks are mixed in the sample and have to be described separately. A long “tail” on one side means asymmetry. Single bars far from the rest may be outliers, that is, values far from all the others. They can be measurement errors, or rare but real values. A sharp cut-off at the edge shows the measurement limit of the instrument.

One of the most common mix-ups in petroleum statistics is about percentiles. The labels P10, P50 and P90 are read in two ways. For P10 and P90, the two ways give opposite numbers.

One label, two meanings (the porosity from the example above)

Label: P10
Cumulative share (“not more than”): porosity 14.4%, a small value: 10% of the plugs lie below it
Exceedance probability (“not less than”): porosity 23.0%, a large value: only 10% of the plugs exceed it

Label: P50
Cumulative share (“not more than”): 18.8%, the median
Exceedance probability (“not less than”): 18.8%, the median

Label: P90
Cumulative share (“not more than”): porosity 23.0%, a large value: 90% of the plugs lie below it
Exceedance probability (“not less than”): porosity 14.4%, a small value: 90% of the plugs exceed it

Label	Cumulative share (“not more than”)	Exceedance probability (“not less than”)
P10	porosity 14.4%, a small value: 10% of the plugs lie below it	porosity 23.0%, a large value: only 10% of the plugs exceed it
P50	18.8%, the median	18.8%, the median
P90	porosity 23.0%, a large value: 90% of the plugs lie below it	porosity 14.4%, a small value: 90% of the plugs exceed it

The first reading is used in mathematical statistics and in many programs. The second is used when estimating resources and reserves, where P90 is the cautious estimate and P10 is the optimistic one (Volumetric estimation of hydrocarbons in place).

Measures of the centre: mean, median and mode

A measure of the centre (also called central tendency) tells us which value the data group around. There are three such measures, and each one understands the “centre” in its own way.

Arithmetic mean: the sum of the values divided by their number; the “centre of gravity” of the data. It counts every value, so rare, very large or very small values move it a lot.
Median: the value that splits the sorted measurements into two halves. With an even number of measurements, we take the average of the two values in the middle. It does not depend on how large the extreme values are.
Mode: the most frequent value; for continuous data, the midpoint of the tallest bar of the histogram. It is the only one of the three measures that works for names (for example, the most common rock type).

Arithmetic mean of a sample

\bar x = \frac{1}{N}\sum_{i=1}^{N} x_i

where:

$\bar x$ arithmetic mean, an estimate of the true mean of the population, units of the quantity
$x_i$ the $i$ th measurement, units of the quantity
$N$ number of measurements, —

Units: the same as those of the measured quantity

Holds when

Each measurement represents the same share of the studied volume (the plugs are taken evenly). Otherwise we need weights, such as thicknesses or volumes.
All the measurements belong to one population: one layer, one rock type.

For additive quantities the arithmetic mean preserves the sum: the mean porosity multiplied by the rock volume gives the pore volume. For quantities that are not additive it is only one of the possible means.

Ten core plugs from one reservoir (assumed data)

Depth, m: 2450.15
Porosity, %: 22
Permeability, mD: 120

Depth, m: 2450.45
Porosity, %: 25
Permeability, mD: 400

Depth, m: 2450.75
Porosity, %: 20
Permeability, mD: 168

Depth, m: 2451.05
Porosity, %: 21
Permeability, mD: 90

Depth, m: 2451.35
Porosity, %: 19
Permeability, mD: 55

Depth, m: 2451.65
Porosity, %: 18
Permeability, mD: 65

Depth, m: 2451.95
Porosity, %: 18
Permeability, mD: 30

Depth, m: 2452.25
Porosity, %: 17
Permeability, mD: 40

Depth, m: 2452.55
Porosity, %: 14
Permeability, mD: 20

Depth, m: 2452.85
Porosity, %: 16
Permeability, mD: 12

Depth, m	Porosity, %	Permeability, mD
2450.15	22	120
2450.45	25	400
2450.75	20	168
2451.05	21	90
2451.35	19	55
2451.65	18	65
2451.95	18	30
2452.25	17	40
2452.55	14	20
2452.85	16	12

The plugs were taken every 0.3 m in the interval 2450.0–2453.0 m. We use this set in all the examples below and in the next lesson.

Worked exampleMean, median and mode of ten plugs

Given (assumed data)

Porosity in ascending order, %: 14, 16, 17, 18, 18, 19, 20, 21, 22, 25
Permeability in ascending order, mD: 12, 20, 30, 40, 55, 65, 90, 120, 168, 400

Solution

Porosity. The sum is 190, and the mean is $\bar\phi = 190/10 = 19.0$ %. The median is the average of the fifth and sixth values: $(18 + 19)/2 = 18.5$ %. The mode is 18% (it occurs twice), but with ten values it is not reliable.
Permeability. The sum is 1000, and the mean is $\bar k = 100$ mD. The median is $(55 + 65)/2 = 60$ mD. The mean is 1.7 times larger than the median, and 7 of the ten plugs are below the mean.
Let us remove one plug, the most permeable one (400 mD). The mean of the nine that remain is $600/9 = 66.7$ mD, which is a third less. The median is the fifth value out of nine, 55 mD, and it has hardly changed.

Answer: Porosity: mean 19.0%, median 18.5%. Permeability: mean 100 mD, median 60 mD.

What it means. For porosity the mean and the median are close: the data are almost symmetrical. For permeability the mean is “pulled” upwards: one plug out of ten gives 40% of the sum. This does not mean that the arithmetic mean is wrong here. It answers its own question (see the next section). But we should not read it as the “typical plug”. The median describes the typical plug. The data in this example are assumed for illustration.

If the plugs represent different thicknesses or volumes, we replace the simple mean with a weighted one: we multiply each value by its thickness and divide the sum by the total thickness. If core was sampled more densely in the good layers, the mean without weights comes out too high. Such an example is worked in the lesson on porosity.

Three averages for permeability

Permeability is not additive, so we use three different averages for it: the arithmetic, the geometric and the harmonic. All three are calculated from the same plugs, but they answer different questions.

Geometric mean

\bar k_{\text{geom}} = \left(\prod_{i=1}^{N} k_i\right)^{1/N} = \exp\left(\frac{1}{N}\sum_{i=1}^{N}\ln k_i\right)

where:

$\bar k_{\text{geom}}$ geometric mean, mD
$k_i$ permeability of the $i$ th plug, mD
$N$ number of plugs, —

Units: the same as for $k_i$ ; $\ln$ is the natural logarithm

Holds when

All the values are positive. A single zero turns the mean into zero, so instead of “zeros” we put in the measurement limit of the instrument.
The plugs represent equal volumes of rock. Otherwise the logarithms are weighted by thickness.

The geometric mean is the arithmetic mean taken on a logarithmic scale: we average not the values themselves but their logarithms (roughly, their orders of magnitude, or powers of ten). The base of the logarithm does not matter: ten raised to the average of the base-10 logarithms gives the same number. We usually calculate with the second form, because adding logarithms is easier than multiplying hundreds of values.

Harmonic mean

\bar k_{\text{harm}} = N\left(\sum_{i=1}^{N}\frac{1}{k_i}\right)^{-1}

where:

$\bar k_{\text{harm}}$ harmonic mean, mD
$k_i$ permeability of the $i$ th plug, mD
$N$ number of plugs, —

Units: the same as for $k_i$

Holds when

All the values are positive.
The plugs represent equal thicknesses. Otherwise $\bar k_{\text{harm}} = \sum h_i / \sum (h_i/k_i)$ .

Here we average not the permeabilities but their reciprocals $1/k$ (one divided by each value). These are the resistances to flow. Then we go back from the mean resistance to permeability. So the result is decided by the smallest values: one tight (low-permeability) plug “weighs” more than ten good ones.

Relation between the three means

\bar k_{\text{harm}} \le \bar k_{\text{geom}} \le \bar k_{\text{arith}}

where:

$\bar k_{\text{harm}}$ harmonic mean, mD
$\bar k_{\text{geom}}$ geometric mean, mD
$\bar k_{\text{arith}}$ arithmetic mean, mD

Units: any units, the same for all three

Holds when

All the values are positive and are taken with equal weights.
Equality holds only if all the values are equal.

The more heterogeneous the rock, the further apart the three means are. For a homogeneous rock they almost coincide. A discrepancy between the means is itself a sign of heterogeneity.

SchematicWhich average goes with which arrangement of parts with different permeability, relative to the flow. The arrows show the flow. In the top sketch, the length of an arrow shows roughly how much fluid flows along that layer.

Which average goes with which flow

Arrangement relative to the flow: Flow along continuous layers: the layers work in parallel
Average: arithmetic, weighted by thickness
What decides the result: the most permeable layers

Arrangement relative to the flow: Flow across the layers: the layers work in series
Average: harmonic, weighted by thickness
What decides the result: the tightest layers

Arrangement relative to the flow: The parts are mixed at random and there are no continuous layers
Average: geometric
What decides the result: a “typical” rock; close to the median

Arrangement relative to the flow	Average	What decides the result
Flow along continuous layers: the layers work in parallel	arithmetic, weighted by thickness	the most permeable layers
Flow across the layers: the layers work in series	harmonic, weighted by thickness	the tightest layers
The parts are mixed at random and there are no continuous layers	geometric	a “typical” rock; close to the median

The rules hold for single-phase flow (only one fluid flows). The first two are worked out in the lesson on permeability. The third holds strictly for two-dimensional flow in an isotropic random medium (a random rock whose properties do not depend on direction) when permeability is log-normally distributed (its logarithm follows the bell-shaped normal curve). In three dimensions the effective permeability (the permeability of the equivalent uniform rock) is somewhat higher than the geometric mean.

Worked exampleThree averages of the permeability of ten plugs

Given (assumed data)

Permeability, mD: 12, 20, 30, 40, 55, 65, 90, 120, 168, 400
Thickness per plug: the same, 0.3 m

Solution

Arithmetic: $\bar k_{\text{arith}} = 1000/10 = 100$ mD.
Geometric. Natural logarithms: 2.4849, 2.9957, 3.4012, 3.6889, 4.0073, 4.1744, 4.4998, 4.7875, 5.1240, 5.9915. Their sum is 41.1552 and their mean is 4.1155; $\bar k_{\text{geom}} = \e^{4.1155} = 61.3$ mD.
Harmonic. The sum of the reciprocals is $1/12 + 1/20 + \dots + 1/400 = 0.2531$ mD⁻¹; $\bar k_{\text{harm}} = 10/0.2531 = 39.5$ mD.
The arithmetic mean is 2.5 times larger than the harmonic mean. The geometric mean (61.3 mD) almost coincides with the median (60 mD).

Answer: Harmonic 39.5 mD, geometric 61.3 mD, arithmetic 100 mD.

What it means. If these ten layers continue across the area, the unit (the whole set of layers) passes flow along the bedding (the layering of the rock) like a rock of about 100 mD, and across the bedding like a rock of about 40 mD. The ratio $k_v/k_h$ for the unit is about 0.4, even though each plug on its own is isotropic. If the layers cannot be traced between wells and the parts are mixed, the geometric mean, about 60 mD, is closer to the truth. The choice of the average is a decision about the geological structure of the reservoir, not arithmetic. The data in this example are assumed for illustration.

Worked exampleOne missed thin shale layer

Given (assumed data)

Interval: 3.0 m; from ten plugs $\bar k_{\text{arith}} = 100$ mD, $\bar k_{\text{harm}} = 39.5$ mD
Layer that did not get into the plugs: shale, thickness 2 cm, permeability 0.002 mD

Solution

The resistance to vertical flow is the sum of the ratios $h_i/k_i$ . For the sandstone: $2.98/39.5 = 0.0754$ m/mD. For the thin layer: $0.02/0.002 = 10$ m/mD, which is 133 times more than for all the rest of the interval.
The vertical permeability of the interval is $k_v = 3.0/(0.0754 + 10) = 0.30$ mD instead of 39.5 mD.
The horizontal permeability is $k_h = (2.98\cdot100 + 0.02\cdot0.002)/3.0 = 99.3$ mD, practically unchanged.

Answer: A shale layer 2 cm thick lowered the vertical permeability of the interval by more than a hundred times, from 39.5 to 0.30 mD, and left the horizontal permeability unchanged.

What it means. Such layers almost never get into the plugs: a cylinder is drilled out of firm sandstone, not out of a shale seam as thin as a finger. For this reason the harmonic mean of the plugs usually overestimates the vertical permeability, sometimes by orders of magnitude. It is more reliable to estimate it in other ways. We can use the description of the core (are there continuous layers?), well tests (tests that record the pressure and flow of a whole well) that measure the vertical connection between intervals, and the behaviour of water and gas cones (the cone-shaped rise of water or gas towards the well). The data in this example are assumed for illustration.

Three notes of caution apply to these rules.

Tilted (dipping) beds. What matters is not the vertical and the horizontal but the direction of the flow relative to the layers. In gently dipping beds, flow along the layers is horizontal and flow across the layers is vertical. In steeply dipping beds (angles over 70°), vertical flow and horizontal flow along the strike (the direction of a horizontal line on the bed) run along the layers and are described by the arithmetic mean. Horizontal flow across the strike crosses the layers, and here we need the harmonic mean.
Limits. However the parts are arranged, the effective permeability cannot be lower than the harmonic mean or higher than the arithmetic mean. If these differ by tens of times, the structure of the reservoir is more important than the precision of single measurements.
Multiphase flow. The rules were derived for one phase (one fluid). When water displaces oil, a high-permeability layer does not just add flow capacity. It lets the water through sooner than the other layers, and no single average can describe such a system (Reservoir heterogeneity and permeability averaging).

Values outside the measurement range

Core reports contain entries such as “< 0.01 mD”, “> 5000 mD” and “no measurement” (the plug broke). These are not zeros, and they are not gaps that we can quietly throw away.
If we throw away small values, we overestimate the harmonic and geometric means. If we throw away large values, we underestimate the arithmetic mean. We cannot replace “< 0.01” with zero either: with a zero, the geometric and harmonic means become zero. Usually we put in the measurement limit itself, and check how much the result changes with this choice.
The median does not depend on such entries as long as there are fewer than half of them. This is one more reason to prefer it.
If the answer is decided by the extreme values (vertical permeability by the tightest layers, inflow by the most permeable ones), no substitution will restore them. We measure such intervals again, with a method that has a suitable range.

Measures of spread: variance, standard deviation and coefficient of variation

Two reservoirs with the same average permeability can behave quite differently. In one, all the plugs are close to the mean. In the other, layers of 5 and 500 mD lie side by side. A measure of spread (variability) tells them apart. It also shows how many plugs we need for a reliable mean, and it hints at how evenly water will push oil out of the rock. A mean without a measure of spread is half an answer.

The simplest measure is the range: the difference between the largest and the smallest value. It depends on only two measurements and grows with the number of plugs, so we use measures that include all the values: the variance, the standard deviation and the coefficient of variation.

Variance and standard deviation of a sample

s^2 = \frac{1}{N-1}\sum_{i=1}^{N}\left(x_i - \bar x\right)^2, \qquad s = \sqrt{s^2}

where:

$s^2$ variance of the sample, square of the units of the quantity
$s$ standard deviation (SD), also called the root-mean-square deviation, units of the quantity
$x_i$ the $i$ th measurement, units of the quantity
$\bar x$ arithmetic mean of the same measurements, units of the quantity
$N$ number of measurements, —

Units: $s$ is in the units of the measured quantity, $s^2$ is in their squares

Holds when

The measurements belong to one population.
The denominator is $N - 1$ , not $N$ . The deviations are measured from the mean of the same sample, so on average they are slightly smaller than from the true mean. For $N$ above a few dozen, the difference does not matter.

The variance is the average of the squared deviations from the mean. The standard deviation brings it back to the units of measurement. If the distribution is close to normal (the symmetrical bell shape), about 68% of the values lie within $\bar x \pm s$ , and about 95% lie within $\bar x \pm 2s$ . For skewed data this rule does not work.

Coefficient of variation

C_v = \frac{s}{\bar x}

where:

$C_v$ coefficient of variation, —
$s$ standard deviation, units of the quantity
$\bar x$ arithmetic mean of the same measurements, units of the quantity

Units: dimensionless; sometimes it is expressed in per cent

Holds when

The quantity is positive and is measured from a natural zero (permeability, porosity, thickness). For temperature in degrees Celsius or for elevations above sea level, the coefficient of variation has no meaning.
The mean and the deviation are calculated from the same measurements on the ordinary scale, not the logarithmic one.

A spread of ±50 mD is very large for a reservoir with a mean permeability of 60 mD and almost nothing for a reservoir with 2000 mD. The coefficient of variation shows the spread as a fraction of the mean, so it lets us compare different properties and different reservoirs.

Worked exampleSpread of the porosity and permeability of ten plugs

Given (assumed data)

Porosity, %: 14, 16, 17, 18, 18, 19, 20, 21, 22, 25; mean 19.0
Permeability, mD: 12, 20, 30, 40, 55, 65, 90, 120, 168, 400; mean 100

Solution

Porosity. Deviations from the mean: −5, −3, −2, −1, −1, 0, 1, 2, 3, 6. The sum of their squares is $25 + 9 + 4 + 1 + 1 + 0 + 1 + 4 + 9 + 36 = 90$ .
The variance is $s^2 = 90/9 = 10.0$ , the standard deviation is $s = 3.16$ %, and the coefficient of variation is $C_v = 3.16/19.0 = 0.166$ .
Permeability. Deviations: −88, −80, −70, −60, −45, −35, −10, 20, 68, 300. The sum of squares is 121,018, of which 90,000 comes from one plug.
The variance is $s^2 = 121{,}018/9 = 13{,}446$ mD², the standard deviation is $s = 116$ mD, and $C_v = 116/100 = 1.16$ .

Answer: Porosity:

s = 3.16

C_v = 0.166

. Permeability:

s = 116

mD,

C_v = 1.16

What it means. As a fraction of the mean, the spread of permeability is seven times that of porosity. Judged by porosity, the reservoir is homogeneous. Judged by permeability, it is very heterogeneous. Notice that the difference $\bar k - s$ comes out negative (100 − 116 mD), which cannot be true for permeability. This is a sign of an asymmetrical distribution: we cannot describe it as “the mean plus or minus the deviation”. The data in this example are assumed for illustration.

Degree of heterogeneity by the coefficient of variation of permeability

$C_v$: below 0.5
Assessment: homogeneous rock
What it means: the plugs are close to one another; the three means almost coincide

$C_v$: 0.5–1.0
Assessment: heterogeneous
What it means: the means differ noticeably; the distribution is skewed

$C_v$: above 1.0
Assessment: very heterogeneous
What it means: the arithmetic mean is decided by a few of the best plugs; a reliable estimate needs hundreds of measurements

$C_v$	Assessment	What it means
below 0.5	homogeneous rock	the plugs are close to one another; the three means almost coincide
0.5–1.0	heterogeneous	the means differ noticeably; the distribution is skewed
above 1.0	very heterogeneous	the arithmetic mean is decided by a few of the best plugs; a reliable estimate needs hundreds of measurements

The boundaries are a convention, adopted for permeability. For the porosity of one rock type, $C_v$ is usually 0.1–0.3.

Constant (uniform) property: the same value at every point. This is an idealisation used in the simplest formulas.
Homogeneity: the property changes little from point to point ( $C_v$ below 0.5).
Heterogeneity: the property changes a lot from point to point. Do not confuse it with interbedding (alternating layers of different rocks): a clean sandstone without a single shale layer can also be very heterogeneous in permeability.
Anisotropy: the property depends on direction (usually $k_v < k_h$ ). This is a different quality: a rock can be homogeneous and anisotropic, or heterogeneous and isotropic.

Besides the coefficient of variation, two more measures describe the heterogeneity of permeability. The Dykstra–Parsons coefficient is found from a probability plot (a graph on which a chosen distribution becomes a straight line, see Distributions and sample size). The Lorenz coefficient relates flow capacity to storage capacity (the advanced part of the lesson covers it).

Why we look at permeability on a logarithmic scale

The permeability of a single reservoir often changes by two or three orders of magnitude (a factor of 100 to 1000), while porosity changes by a factor of one and a half or two. The reason is that the influences on permeability are not added but multiplied. Grain size enters it squared and porosity enters it roughly cubed. Sorting, cement and clay minerals add further factors (what permeability depends on, where reservoir quality comes from). On an ordinary scale such data are awkward: almost all the values crowd near zero, and the few large ones stretch the axis.

The logarithm turns multiplication into addition and ratios into differences. On a logarithmic scale the distance from 10 to 100 mD is the same as from 100 to 1000 mD, and the histogram of permeability usually becomes almost symmetrical.

CalculatedPermeability of the same 40 plugs as on the porosity histogram above. Top: classes of 50 mD; the distribution is sharply skewed. Bottom: classes whose boundaries double each time; the distribution is almost symmetrical, and the three means line up in the order harmonic, geometric, arithmetic.

Three practical consequences follow.

We plot permeability graphs (histograms, plots against porosity, well-log curves) on a logarithmic scale.
The mean on a logarithmic scale is the geometric mean. For such data it is close to the median and describes the “typical” plug.
We express the spread by the standard deviation of the logarithm, $s_{\ln k}$ . It is dimensionless, and it means not “plus or minus so many millidarcies” but “so many times larger or smaller”.

Worked exampleSpread of permeability in logarithms

Given (assumed data)

Natural logarithms of the permeability of ten plugs: 2.4849, 2.9957, 3.4012, 3.6889, 4.0073, 4.1744, 4.4998, 4.7875, 5.1240, 5.9915
Their mean: 4.1155 (example above)

Solution

The sum of squared deviations of the logarithms from their mean is 9.756. The variance is $9.756/9 = 1.084$ , and the standard deviation is $s_{\ln k} = 1.041$ .
“Plus or minus one deviation” in logarithms is the interval from $\e^{4.1155 - 1.041} = 22$ to $\e^{4.1155 + 1.041} = 174$ mD: a factor of $\e^{1.041} \approx 2.8$ smaller and larger than the geometric mean.
Of the ten plugs, 7 fall into this interval. This is close to the 68% that we would expect of a normal distribution.
In base-10 logarithms the same deviation is $s_{\lg k} = 1.041/2.303 = 0.45$ , which is less than half an order of magnitude.

Answer:

s_{\ln k} = 1.041

: a typical plug differs from the geometric mean (61.3 mD) by a factor of about 2.8, in one direction or the other.

What it means. In logarithms the data behave well: the interval of one standard deviation does not go into the negative region, and it contains the expected share of plugs. This is why we calculate the statistics of permeability in logarithms and return to millidarcies only at the end. Which distribution lies behind this, and how to check it on a probability plot, is covered in the lesson “Distributions and sample size”. The data in this example are assumed for illustration.

The scale of measurement: core, logs and well tests

Every measurement averages the property over its own volume. A probe permeameter (an instrument that measures permeability at one point) presses its tip against the surface of the core and covers less than a cubic centimetre of rock. A cylindrical plug covers two or three dozen cubic centimetres. A well log covers tens of centimetres around the wellbore. Well tests (tests that record the pressure and flow rate of a whole well) cover tens and hundreds of metres of the reservoir. Statistics always refer to the scale at which the data were collected. Several consequences follow.

The spread usually falls as the measured volume grows. A large volume already contains good and poor parts inside it. So log values are usually less scattered than plug values, and plug values are less scattered than point measurements. We can compare coefficients of variation only for data of the same scale.
Anisotropy grows with scale. A single plug of homogeneous sandstone is almost isotropic. The unit of ten layers in the example above has $k_v/k_h \approx 0.4$ , and with one shale layer it has about 0.003. For a model cell a few metres thick, this ratio is set by the layers and thin beds, not by measurements on plugs.
Core means and well-test permeability are compared by choosing the mean that suits the structure of the reservoir. If permeable layers continue for hundreds of metres, the test “sees” the arithmetic mean weighted by thickness. If they are small lenses (lens-shaped bodies) in less permeable rock, the result is closer to the geometric mean. The difference tells us about the connectivity of the reservoir (how well its parts join up). It is not an error of one of the methods.
The conditions of measurement differ. Plugs are tested clean and dry, under less compression than in the reservoir, and most often with gas. Well tests give the effective permeability for the reservoir fluid at reservoir conditions (how permeability is measured).

Terms and symbols

Population and sample: The population is all the values of the property that we want to judge. The sample is the part of them that we have measured.
Histogram: A bar chart of the number (or share) of measurements in equal intervals of values.
Percentile: The value below which a given share of the data lies. The 50th percentile is the median.
Median: The value that splits the sorted data into two halves. It does not depend on the extreme values.
Mode: The most frequent value or class.
$\bar k_{\text{arith}}$ , $\bar k_{\text{geom}}$ , $\bar k_{\text{harm}}$: The arithmetic, geometric and harmonic means. The first goes with flow along the layers, the third with flow across the layers, and the second with a random patchwork.
$s^2$ , $s$: The variance and the standard deviation of a sample: measures of spread in squared units and in the units themselves.
$C_v$: The coefficient of variation: the standard deviation as a fraction of the arithmetic mean.
$L_c$: The Lorenz coefficient: twice the area between the curve of cumulative flow capacity against cumulative storage capacity and the diagonal. It runs from 0 to 1.

Common mistakes and misconceptions

Wrong

“P90 is a large value: 90% is more than 10%.”

In fact

It depends on the convention. In a reserves estimate, P90 is the value that is exceeded with a probability of 90%, that is, a small, cautious value. In statistics, the 90th percentile is a large value. If we do not find out which convention is used, we cannot use the number.

Wrong

“The arithmetic mean of permeability is the permeability of the typical plug.”

In fact

For skewed data most plugs are below the arithmetic mean (in the example of the lesson, seven out of ten). The median and the geometric mean describe the typical plug. The arithmetic mean answers a different question: how much fluid the layers pass when the flow runs along them.

Wrong

“Plugs marked ‘below the detection limit’ can be left out.”

In fact

It is the tight layers that decide the vertical permeability. If we leave them out, we overestimate the harmonic and geometric means. We cannot replace them with zeros either.

Wrong

“A standard deviation of 116 mD with a mean of 100 mD means that permeability lies within 100 ± 116 mD.”

In fact

The rule “the mean ± one deviation holds two thirds of the data” applies to symmetrical distributions close to normal. We express the spread of permeability in logarithms. There, one deviation means “about 2.8 times larger or smaller”.

Wrong

“A heterogeneous reservoir is a reservoir with shale layers.”

In fact

Heterogeneity is the variability of a property within one rock, and its measure is the coefficient of variation. A clean sandstone can be very heterogeneous in permeability. Sandstone and shale are different populations, and we describe them separately.

In depth: advanced detail

The Lorenz curve and coefficient

The coefficient of variation describes the spread of one quantity. The Lorenz curve relates two. It shows what share of the flow capacity of the reservoir (how much fluid it can carry) sits in what share of its storage capacity (how much fluid it can hold). The method comes from economics. There, since the early twentieth century, the same kind of curve has shown what share of income belongs to what share of the population.

Construction

1
Order the layers
Arrange the layers (plugs) in descending order of the ratio $k/\phi$ , from the best to the worst.
2
Accumulate the sums
At each step calculate the cumulative share of flow capacity $F$ and the cumulative share of storage capacity $C$ .
3
Plot the curve
Plot $F$ against $C$ . The curve leaves the point (0, 0), arrives at the point (1, 1) and lies above the diagonal.
4
Measure the area
The Lorenz coefficient $L_c$ is twice the area between the curve and the diagonal.

Cumulative shares of flow capacity and storage capacity (Lorenz curve)

F_J = \frac{\sum_{j=1}^{J} k_j h_j}{\sum_{i=1}^{I} k_i h_i}, \qquad C_J = \frac{\sum_{j=1}^{J} \phi_j h_j}{\sum_{i=1}^{I} \phi_i h_i}

where:

$k_j,\ \phi_j,\ h_j$ permeability, porosity and thickness of the $j$ th layer, in the order of decreasing $k/\phi$ , mD, fraction, m
$I$ total number of layers; the sums in the denominators are taken over all the layers, —
$J$ number of layers counted up to this step, —
$F_J$ cumulative share of flow capacity, fraction
$C_J$ cumulative share of storage capacity, fraction

Units: fractions; $k$ , $\phi$ and $h$ are in any units that are the same for all the layers

Holds when

The layers are ordered by decreasing $k/\phi$ .
Each plug represents its own layer. For equal thicknesses, $h$ cancels.

The product $k h$ is the flow capacity of a layer, and $\phi h$ is its storage capacity. For a homogeneous reservoir the curve coincides with the diagonal: each share of the storage capacity gives the same share of the flow capacity.

Lorenz coefficient

L_c = 2A

where:

$L_c$ Lorenz coefficient, —
$A$ area between the Lorenz curve and the diagonal, —

Units: dimensionless, from 0 to 1

Holds when

We find the area from the plotted curve, for example by the trapezoid rule (adding up the areas of the trapezoids between neighbouring points).

$L_c = 0$ is a homogeneous reservoir. The closer the coefficient is to 1, the larger the share of the flow capacity that falls on a small share of the storage capacity. Values close to 1 are typical of naturally fractured reservoirs, where the fractures carry the flow, while the oil and gas in place sit in the rock matrix (the rock between the fractures).

Worked exampleLorenz curve of ten plugs

Given (assumed data)

Plugs in descending order of $k/\phi$ : $k$ , mD / $\phi$ , %: 400 / 25; 168 / 20; 120 / 22; 90 / 21; 65 / 18; 55 / 19; 40 / 17; 30 / 18; 20 / 14; 12 / 16
Thicknesses: equal
Sums: $\sum k = 1000$ mD, $\sum\phi = 190$ %

Solution

After the best plug: $F = 400/1000 = 0.400$ , $C = 25/190 = 0.132$ . After the two best: $F = 568/1000 = 0.568$ , $C = 45/190 = 0.237$ .
Continuing in the same way, we get ten points; the last is (1, 1). After five plugs, $F = 0.843$ at $C = 0.558$ .
The area under the curve by the trapezoid rule is 0.723; under the diagonal it is 0.5.
$L_c = 2\cdot(0.723 - 0.5) = 0.446$ .

Answer:

L_c \approx 0.45

; the two best plugs, which hold 24% of the storage capacity, account for 57% of the flow capacity.

What it means. The coefficient is in the middle of the scale. The heterogeneity is clear, but it is far from the values close to 1. Suppose we plot the same curve without rearranging the plugs, in the order of depth (a modified Lorenz curve). The steep parts then show the intervals that give the main inflow, and the flat parts show the weak ones. Such a graph is compared with the inflow profile from production logging (a log run in a producing well that shows where the fluid comes from). In our set the three top plugs, which hold 35% of the storage capacity, give 69% of the flow capacity. The data in this example are assumed for illustration.

CalculatedLorenz curve for the ten plugs of the lesson. The area

A

between the curve and the diagonal is shaded; the orange dot marks the two best plugs.

Why the denominator is N − 1, and what to do with very small samples

The deviations are measured from the mean calculated from the same plugs. This mean is “fitted” to the sample. The sum of the deviations from it is always zero, and the sum of their squares is smaller than it would be from the true mean of the population. There are not $N$ independent deviations but $N - 1$ (the last one is fixed by the others). This is the number of degrees of freedom of the sample. Dividing by $N - 1$ corrects the underestimate, and on average $s^2$ then equals the variance of the population.

A small inaccuracy remains in the standard deviation itself. The square root of the unbiased variance (the one with $N - 1$ , which has no systematic error) is, on average, slightly smaller than the true value. For normally distributed data the correction factor is close to $1 + \dfrac{1}{4(N-1)}$ : for five plugs it is 1.06, for ten it is 1.03, and for thirty it is less than 1.01. We apply the correction only for very small samples, with fewer than ten values.

Another point is much more important: from a small sample, the spread itself is known only roughly. For normally distributed data, the relative error of the standard deviation is about $1/\sqrt{2(N-1)}$ : 35% for five plugs, almost a quarter for ten, and 13% for thirty. A coefficient of variation calculated from five plugs is good only for the roughest estimate.

Connections to other topics

Distributions and sample size — the normal and log-normal distributions, the probability plot, and the number of plugs needed for a reliable mean.
Rock permeability — Darcy's law, and the averaging of the permeability of layers that work in parallel and in series.
Porosity — averaging porosity over thickness, and the effect of uneven sampling.
Core analysis — how porosity and permeability are measured on core, and what limits these measurements.
Volumetric estimation of hydrocarbons in place — the average values of parameters in the calculation of hydrocarbons in place, and the range of uncertainty.
Reservoir and seal — which rocks become reservoirs, and what their quality depends on.

Check yourself

1
The permeabilities of five plugs are 2, 40, 50, 60 and 848 mD. Which measure describes the “typical” plug best?
1. A.Range: 846 mD
2. B.Largest value: 848 mD
3. C.Arithmetic mean: 200 mD
4. D.Median: 50 mD
Show the answer
Correct answer: D. Median: 50 mD
The sum is 1000 and the mean is 200 mD, but four plugs out of five are several times lower than it: the mean is decided by a single plug. The median is the third value out of five, 50 mD. It does not depend on the extreme values and describes the typical plug.
2
Three layers of equal thickness have permeabilities of 10, 100 and 1000 mD. Find the three means, and say which of them goes with flow along the layers and which with flow across the layers.
Show the answer
Arithmetic: $(10 + 100 + 1000)/3 = 370$ mD, for flow along the layers. Harmonic: $3/(0.1 + 0.01 + 0.001) \approx 27$ mD, for flow across the layers. Geometric: $\sqrt[3]{10\cdot100\cdot1000} = 100$ mD, for randomly mixed parts. The ratio $k_v/k_h$ of the unit is about 0.07.
3
A reserves report says: P90 = 12 million t, P10 = 31 million t. In what sense are the labels used?
1. A.As cumulative shares: 90% of the outcomes are smaller than 12 million t
2. B.It is impossible to tell
3. C.As exceedance probabilities: with a probability of 90%, the reserves are not less than 12 million t
Show the answer
Correct answer: C. As exceedance probabilities: with a probability of 90%, the reserves are not less than 12 million t
P90 is smaller than P10, so these are exceedance probabilities, the convention used when estimating reserves: 12 million t is the cautious estimate and 31 million t is the optimistic one. With cumulative shares it would be the other way round, because the 90th percentile is larger than the 10th.
4
Two reservoirs have the same mean permeability, 150 mD. The first has $C_v = 0.3$ , and the second has $C_v = 1.4$ . How do the reservoirs differ, and what does this mean for development?
Show the answer
The first is homogeneous: the plugs are close to the mean, the three means almost coincide, and the displacement will be relatively even. The second is very heterogeneous. A few layers give the main flow capacity, the harmonic and geometric means are much lower than the arithmetic mean, and the injected water will go first along the best layers. To estimate the mean reliably in the second reservoir, we need far more plugs.
5
The harmonic mean of the permeability of core plugs is taken as an estimate of the vertical permeability of a unit. In which direction is this estimate usually wrong?
1. A.It is not wrong if there are more than thirty plugs
2. B.Too high: thin tight layers do not get into the plugs
3. C.Too low: the plugs are taken from the worst layers
Show the answer
Correct answer: B. Too high: thin tight layers do not get into the plugs
The harmonic mean is decided by the tightest parts, and these (thin shale and cemented layers) usually do not get into the plugs. In the example of the lesson, one missed layer 2 cm thick lowered the vertical permeability of a three-metre unit by more than a hundred times. The number of plugs does not correct this error.
6
Why do we build a histogram of permeability on a logarithmic scale, and which average goes with its centre?
Show the answer
Permeability is formed by multiplying factors and changes by orders of magnitude. On an ordinary scale almost all the values collect in the first classes, and the rare large ones stretch the axis. On a logarithmic scale the distribution becomes close to symmetrical. Its centre is the mean of the logarithms, that is, the logarithm of the geometric mean, which for such data is close to the median.

Updated: 2 October 2026