Distributions and sample size

Quick refresher

A histogram of a sample approximates the probability density of the population. The area under the density curve between two values is the proportion of rock whose values lie in that range.
The normal distribution (the bell curve) is defined by its mean $\mu$ and standard deviation $\sigma$ (the typical spread around the mean). 68% of the values lie within $\mu \pm \sigma$ and 95% lie within $\mu \pm 2\sigma$ . The porosity of a single rock type and measurement errors are close to a normal distribution.
A quantity is log-normally distributed if its logarithm is normally distributed. Permeability is like this, and so is anything that results from multiplying many factors. For such a distribution the mode (the most common value) is smaller than the median (the middle value), and the median is smaller than the mean.
On a probability plot a normal distribution appears as a straight line. Bends and kinks show that the scale is unsuitable or that the data are a mixture of rock types.
The standard error of the mean, $s/\sqrt N$ , falls slowly: to halve the uncertainty of the mean you need four times as many samples. For a precision of ±20% at 95% probability, you need about $(10\,C_v)^2$ samples. Here $C_v$ is the coefficient of variation (the standard deviation divided by the mean).
Bias, a systematic error of sampling, does not depend on the number of samples. Samples taken only from hard, good rock give a precise but wrong mean.

Key relationships

z = \dfrac{x - \mu}{\sigma}

the standardised value (how many standard deviations $x$ lies from the mean); the proportion of values below $x$ equals $\Phi(z)$

x_{50} = \e^{\mu},\quad \bar x = \e^{\mu + \sigma^2/2}

median and mean of a log-normal distribution ( $\mu$ and $\sigma$ refer to $\ln x$ )

SE = \dfrac{s}{\sqrt{N}}

standard error of the mean

\bar x \pm t\,\dfrac{s}{\sqrt{N}}

confidence interval of the mean; $t \approx 2$ at 95% and $N > 30$

N_0 = (10\,C_v)^2

number of samples for a precision of ±20% (95%)

P_s = \dfrac{200\,C_v}{\sqrt{N_s}}

the precision of the mean, in %, reached with $N_s$ samples

Common pitfalls

Taking the standard deviation of the data for the error of the mean, or the other way round.
Applying the “±2σ” rule to permeability in millidarcies rather than to its logarithm.
Treating P50 as the expected value. For a right-skewed distribution (one with a long tail towards large values) the mean is larger than the median.
Treating a hundred samples from one well as a hundred independent pieces of information about the reservoir.
Believing that a large number of samples makes up for poorly chosen sampling.

After this lesson you can

explain what a probability density and a distribution function are, and how they relate to the histogram and the cumulative curve
describe the normal and log-normal distributions, say where each occurs in reservoir data, and use them to find proportions and percentiles (the value below which a given share of the data lies)
construct a probability plot (a graph on which a chosen distribution becomes a straight line) and use it to check the type of distribution and to find the median and the spread
calculate the standard error and the confidence interval of the mean (how uncertain the mean is) and distinguish them from the standard deviation of the data (how scattered the values are)
estimate how many samples are needed for a given precision of the mean, and what precision the samples in hand give
recognise the sources of sampling bias (a systematic error in how the samples are chosen) and explain why it does not shrink as the number of samples grows

From the histogram to the distribution

A histogram describes a sample: forty plugs (small cylinders cut from core), a hundred, a thousand (how it is built). Imagine more and more plugs and narrower and narrower classes (the ranges that the bars cover). The stepped outline of the histogram would turn into a smooth curve. This curve no longer describes a sample but the population, and it is called the probability density function.

The height of the density curve is not itself a probability. Probability is the area under the curve. The proportion of values between $a$ and $b$ equals the area above the segment from $a$ to $b$ , and the whole area under the curve equals one. Probability here is a number from 0 (impossible) to 1 (certain). For rock it is convenient to think of it as the proportion of the volume with such properties.

The cumulative curve of a sample (it adds up the shares of values as $x$ grows) corresponds to the cumulative distribution function $F(x)$ , the probability that the quantity does not exceed $x$ . It rises from 0 to 1. The exceedance probability is $1 - F(x)$ . This is what we mean when we call an estimate of hydrocarbons in place P90: there is a 90% probability that the true value is at least this large.

Sample and population: what corresponds to what

From a sample (estimate): histogram
Of the population (parameter): probability density $f(x)$

From a sample (estimate): cumulative curve
Of the population (parameter): distribution function $F(x)$

From a sample (estimate): mean $\bar x$
Of the population (parameter): mean $\mu$

From a sample (estimate): standard deviation $s$
Of the population (parameter): standard deviation $\sigma$

From a sample (estimate)	Of the population (parameter)
histogram	probability density $f(x)$
cumulative curve	distribution function $F(x)$
mean $\bar x$	mean $\mu$
standard deviation $s$	standard deviation $\sigma$

We also use the distribution function in reverse: for a given probability we find the value, which is called the quantile. Monte Carlo simulation is built on this. We take a random number between 0 and 1 as the cumulative probability. We then find the porosity, thickness or area that corresponds to it from the distribution function. We repeat this thousands of times (Probabilistic volumetrics by Monte Carlo simulation).

The normal distribution

The normal distribution (also called the Gaussian distribution) is a symmetrical bell-shaped curve. It arises where a quantity is the sum of many small independent contributions, none of which dominates. Examples are random measurement errors and the means of a large number of samples.

Probability density of the normal distribution

f(x) = \frac{1}{\sigma\sqrt{2\pi}}\,\exp\left[-\frac{(x - \mu)^2}{2\sigma^2}\right]

where:

$f(x)$ probability density, 1/units of the quantity
$x$ value of the quantity, units of the quantity
$\mu$ mean; for the normal distribution it is also the median and the mode, units of the quantity
$\sigma$ standard deviation, units of the quantity

Units: $x$ , $\mu$ and $\sigma$ are in the units of the measured quantity

Holds when

The quantity can take any values, including negative ones. For porosity or thickness this is only an approximation, and it is good as long as $\mu$ is greater than about $3\sigma$ .
The distribution is unimodal (it has one peak) and symmetrical.

Two numbers define the curve completely: $\mu$ shifts the bell along the axis, and $\sigma$ stretches it. From a sample we estimate them by the arithmetic mean $\bar x$ and the standard deviation $s$ .

Standardised value

z = \frac{x - \mu}{\sigma}

where:

$z$ distance from the mean in standard deviations, —
$x$ value of the quantity, units of the quantity
$\mu,\ \sigma$ mean and standard deviation of the distribution, units of the quantity

Units: dimensionless

Holds when

The distribution is normal. Then $z$ follows the standard normal distribution, which has a mean of 0 and a standard deviation of 1.

Any problem about a normal distribution reduces to a single curve. The proportion of values below $x$ equals $\Phi(z)$ , the function of the standard normal distribution. We take it from tables or calculate it in a spreadsheet. In reverse, the value with cumulative probability $p$ equals $\mu + z_p\sigma$ .

What proportion of the values of a normal distribution lies in an interval

Interval: $\mu \pm 0.674\sigma$
Proportion of values: 50%
Note: between the quartiles (the 25th and 75th percentiles)

Interval: $\mu \pm \sigma$
Proportion of values: 68.3%
Note: about two thirds

Interval: $\mu \pm 1.282\sigma$
Proportion of values: 80%
Note: between the 10th and 90th percentiles

Interval: $\mu \pm 1.645\sigma$
Proportion of values: 90%
Note: between the 5th and 95th percentiles

Interval: $\mu \pm 1.960\sigma$
Proportion of values: 95%
Note: basis of 95% intervals

Interval: $\mu \pm 2\sigma$
Proportion of values: 95.4%
Note: a convenient rounding

Interval: $\mu \pm 3\sigma$
Proportion of values: 99.7%
Note: almost all values

Interval	Proportion of values	Note
$\mu \pm 0.674\sigma$	50%	between the quartiles (the 25th and 75th percentiles)
$\mu \pm \sigma$	68.3%	about two thirds
$\mu \pm 1.282\sigma$	80%	between the 10th and 90th percentiles
$\mu \pm 1.645\sigma$	90%	between the 5th and 95th percentiles
$\mu \pm 1.960\sigma$	95%	basis of 95% intervals
$\mu \pm 2\sigma$	95.4%	a convenient rounding
$\mu \pm 3\sigma$	99.7%	almost all values

CalculatedA normal curve with a mean of 19.0% and a standard deviation of 3.16%. These are the parameters of the porosity of the ten plugs in the example below. The dark band is the interval

\mu \pm \sigma

, and the light band is

\mu \pm 2\sigma

In reservoir data, four kinds of quantities are close to the normal distribution.

The porosity of a single rock type is often close to normal.
Measurement errors in pressure, depth or a log reading. This is the basis for judging the quality of a history match (how well a model reproduces the past behaviour of the field). We compare the difference between the calculation and the measurement with the standard deviation of the measurement error (objective function).
Mean values are distributed almost normally even if the data themselves are distributed differently (more on this below).
The logarithm of permeability and the logarithms of other quantities that are formed by multiplying factors.

The normal curve is not suitable for quantities squeezed between limits (saturation or net-to-gross ratio near 0 or 1). It is not suitable for mixtures either. If a sample contains two rock types, the histogram has two peaks, and we must describe them separately.

Worked exampleThe proportion of rock below a porosity cut-off

Given (assumed data)

Porosity of the reservoir: close to normal; for ten plugs $\bar\phi = 19.0$ %, $s = 3.162$ %
Cut-off (the limiting value): 15%

Solution

Standardise the cut-off: $z = (15 - 19.0)/3.162 = -1.265$ .
From the table of the normal distribution, $\Phi(-1.265) = 0.103$ . So about 10% of the rock has a porosity below 15%.
The 10th and 90th percentiles: $19.0 - 1.282\cdot3.162 = 14.9$ % and $19.0 + 1.282\cdot3.162 = 23.1$ %.

Answer: About 10% of the rock lies below the cut-off. 80% of the values lie between 14.9 and 23.1%.

What it means. In the sample itself, one plug in ten lies below 15%. But we cannot judge a proportion from a single plug, and the distribution curve uses all ten values at once. Here is an independent check: for forty plugs of the same reservoir, the 10th and 90th percentiles came out at 14.4 and 23.0% (the histogram example). The estimate is good as long as the distribution really is close to normal and the plugs represent the whole reservoir. The data in this example are assumed for illustration.

The log-normal distribution

A quantity is log-normally distributed if its logarithm is normally distributed. The normal distribution arises when many independent influences are added. The log-normal one arises when they are multiplied, because the logarithm of a product equals the sum of the logarithms.

Many of the quantities that geologists and reservoir engineers work with are log-normally distributed, exactly or approximately.

Permeability is the main example: it depends on a product of factors (grain size, porosity, sorting, cementation).
Sizes of grains and of pore throats (the narrow openings that connect pores). This is why, in grain-size analysis, we plot sizes on a logarithmic scale.
Hydrocarbons in place are the result of multiplying the rock volume, the net-to-gross ratio, porosity, saturation and the shrinkage factor, which converts reservoir volume to surface volume (Volumetric estimation of hydrocarbons in place). Even if each factor is distributed symmetrically, the distribution of the product is skewed to the right (it has a long tail of large values).
Sizes of accumulations in a petroleum basin, approximately: there are many small ones and few large ones.

Median, mean and mode of the log-normal distribution

x_{50} = \e^{\mu}, \qquad \bar x = \e^{\mu + \sigma^2/2}, \qquad x_{\text{mode}} = \e^{\mu - \sigma^2}

where:

$\mu$ mean of the natural logarithm of the quantity, —
$\sigma$ standard deviation of the natural logarithm of the quantity, —
$x_{50}$ median; also the geometric mean, units of the quantity
$\bar x$ arithmetic mean, units of the quantity
$x_{\text{mode}}$ mode — the most probable value, units of the quantity

Units: $x$ is in the units of the quantity (for example, mD); $\mu$ and $\sigma$ refer to $\ln x$ and are dimensionless

Holds when

The logarithm of the quantity is normally distributed.
We calculate $\mu$ and $\sigma$ from natural logarithms (base e). For decimal logarithms, $\mu_{\ln} = 2.303\,\mu_{\lg}$ and $\sigma_{\ln} = 2.303\,\sigma_{\lg}$ .

The mode is always smaller than the median, and the median is smaller than the mean. The gap between them grows with $\sigma$ . At $\sigma = 1$ the mean is $\e^{0.5} \approx 1.65$ times larger than the median, and the mode is $\e \approx 2.7$ times smaller than the median. From a sample, we estimate $\mu$ and $\sigma$ by the mean and standard deviation of the logarithms.

Percentiles and coefficient of variation of the log-normal distribution

x_p = \e^{\mu + z_p\sigma}, \qquad C_v = \sqrt{\e^{\sigma^2} - 1}

where:

$x_p$ value below which the proportion $p$ of the values lies, units of the quantity
$z_p$ quantile of the standard normal distribution: −1.282 for 10%, 0 for 50%, +1.282 for 90%, —
$C_v$ coefficient of variation of the quantity itself (not of its logarithm), —
$\mu,\ \sigma$ mean and standard deviation of the natural logarithm, —

Units: $x_p$ is in the units of the quantity; the rest is dimensionless

Holds when

The logarithm of the quantity is normally distributed.

The percentiles are placed symmetrically not by difference but by ratio. The 90th percentile is as many times larger than the median as the 10th percentile is smaller than it: $\e^{1.282\sigma}$ times. The coefficient of variation depends only on $\sigma$ .

CalculatedA log-normal curve with the parameters of the permeability of the ten plugs in the example below (

\mu = 4.116

and

\sigma = 1.041

for the natural logarithm of permeability in mD). The mode, the median and the mean differ from one another by several times.

Worked exampleA log-normal model of permeability from ten plugs

Given (assumed data)

Natural logarithms of the permeability (mD) of ten plugs: mean $m = 4.1155$ , standard deviation $s = 1.041$
From the plugs themselves: arithmetic mean 100 mD, median 60 mD

Solution

Median: $\e^{4.1155} = 61.3$ mD.
Mean: the exponent is $4.1155 + 1.041^2/2 = 4.657$ , so $\e^{4.657} = 105$ mD.
Mode: the exponent is $4.1155 - 1.041^2 = 3.032$ , so $\e^{3.032} = 20.7$ mD.
The 10th and 90th percentiles: $\e^{4.1155 - 1.282\cdot1.041} = 16.1$ mD and $\e^{4.1155 + 1.282\cdot1.041} = 233$ mD. Their ratio is about 14.
Proportion of rock with a permeability below 10 mD: $z = (\ln 10 - 4.1155)/1.041 = -1.74$ and $\Phi(z) = 0.041$ , which is about 4%.

Answer: The median is 61.3 mD, the mean is 105 mD and the mode is 20.7 mD. 80% of the values lie between 16.1 and 233 mD.

What it means. The model agrees with the sample: a median of 61 against 60 mD, and a mean of 105 against 100 mD. The most probable value (the mode, about 21 mD) is five times smaller than the mean. For such a rock, “occurs most often” and “on average” are quite different numbers. A tenth of the rock is worse than 16 mD, and a tenth is better than 230 mD. These “tails” decide where the stagnant zones are (zones where fluid hardly moves) and which paths the water takes when it breaks through. The data in this example are assumed for illustration.

The probability plot: checking the distribution

A histogram of ten or twenty samples is too coarse to judge the shape of a distribution. A more reliable tool is the probability plot. Its axis of cumulative probability is stretched so that a normal distribution is drawn as a straight line. In the past, special probability paper was printed for this. Now the computer calculates the scale.

How to construct it

1
Sort
Arrange the $N$ values in increasing order and number them: $i = 1, 2, \dots, N$ .
2
Assign probabilities
Give each value the cumulative probability $p_i = (i - 0.5)/N$ , which is the middle of its “own” share $1/N$ . Other formulas are also used, for example $i/(N + 1)$ . The difference is noticeable only at the extreme points.
3
Find the quantiles
For each $p_i$ find $z_i$ , the quantile of the standard normal distribution (from a table or with a spreadsheet function).
4
Plot
Plot the values against $z_i$ . To test for normality, plot the values themselves. To test for log-normality, plot the values on a logarithmic axis.
5
Read
If the points fall on a straight line, the distribution fits. The value at $z = 0$ is the median. A change of one unit of $z$ is one standard deviation.

Data for the probability plot of the permeability of ten plugs (assumed data)

$i$: 1
$k_i$ , mD: 12
$p_i$: 0.05
$z_i$: −1.645

$i$: 2
$k_i$ , mD: 20
$p_i$: 0.15
$z_i$: −1.036

$i$: 3
$k_i$ , mD: 30
$p_i$: 0.25
$z_i$: −0.674

$i$: 4
$k_i$ , mD: 40
$p_i$: 0.35
$z_i$: −0.385

$i$: 5
$k_i$ , mD: 55
$p_i$: 0.45
$z_i$: −0.126

$i$: 6
$k_i$ , mD: 65
$p_i$: 0.55
$z_i$: 0.126

$i$: 7
$k_i$ , mD: 90
$p_i$: 0.65
$z_i$: 0.385

$i$: 8
$k_i$ , mD: 120
$p_i$: 0.75
$z_i$: 0.674

$i$: 9
$k_i$ , mD: 168
$p_i$: 0.85
$z_i$: 1.036

$i$: 10
$k_i$ , mD: 400
$p_i$: 0.95
$z_i$: 1.645

$i$	$k_i$ , mD	$p_i$	$z_i$
1	12	0.05	−1.645
2	20	0.15	−1.036
3	30	0.25	−0.674
4	40	0.35	−0.385
5	55	0.45	−0.126
6	65	0.55	0.126
7	90	0.65	0.385
8	120	0.75	0.674
9	168	0.85	1.036
10	400	0.95	1.645

Worked exampleProbability plot of the permeability of ten plugs

Given (assumed data)

Permeability in increasing order, mD: 12; 20; 30; 40; 55; 65; 90; 120; 168; 400
Mean and standard deviation of the logarithms: $m = 4.1155$ , $s = 1.041$

Solution

Probabilities $p_i = (i - 0.5)/10$ : 0.05, 0.15, 0.25 and so on up to 0.95.
Quantiles for the first half of the series: −1.645, −1.036, −0.674, −0.385, −0.126. For the second half, they are the same numbers with a plus sign, in reverse order.
The straight line that corresponds to the mean and standard deviation of the logarithms is $\ln k = 4.1155 + 1.041\,z$ . At $z = 0$ it gives 61 mD, at $z = -1$ it gives 22 mD, and at $z = +1$ it gives 174 mD.
The point that deviates most from the line is the most permeable plug: it lies a factor of 1.2 above the line. The other nine points lie within 9% of the line.

Answer: The points fall on a straight line, so the log-normal distribution fits. The median is 61 mD. The “plus or minus one standard deviation” interval runs from 22 to 174 mD.

What it means. On an ordinary, non-logarithmic axis the same points would fall on a steep arc. That is a sign that the normal distribution is unsuitable for permeability itself. Ten points are too few to tell similar distributions apart with confidence, but they are enough to reject a distribution that is clearly unsuitable. The data in this example are assumed for illustration.

CalculatedProbability plot of the permeability of ten plugs. The axis of values is logarithmic, and the axis of cumulative proportion is graduated according to the normal distribution. The line is drawn from the mean and standard deviation of the logarithms. The dashed lines mark the median and the values one standard deviation below and above it.

Deviations of the points from the line carry information too.

A smooth bend along the whole length means the wrong scale was chosen: for example, permeability plotted without its logarithm.
A kink (two straight lines with different slopes) means that two populations are mixed in the sample, for example two rock types, or matrix and fractures. We separate them and describe each one individually.
A horizontal “shelf” at the edge means the values are cut off at the measurement limit of the instrument.
One or two points off the line at the edges may be outliers (values far from all the others). Before discarding them, we check the sample and the measurement. In general, the extreme points are the least reliable.

How precisely the mean is known

The mean of the samples is an estimate. If we had taken another ten samples from the same reservoir, the mean would have come out different. The question is how widely such means are scattered around the true one.

The answer comes from the central limit theorem. The means of independent samples of size $N$ are distributed approximately normally around the true mean $\mu$ . The larger $N$ is, the more tightly they cluster. This holds for almost any distribution of the data themselves. The standard deviation of such means is called the standard error (SE).

Standard error of the mean

SE = \frac{s}{\sqrt{N}}

where:

$SE$ standard error of the mean, units of the quantity
$s$ standard deviation of the data, units of the quantity
$N$ number of independent measurements, —

Units: the same as those of the measured quantity

Holds when

The measurements are independent: the value of one sample does not predict the value of another.
The sample is representative: every part of the population had an equal chance of being included.
For strongly skewed data, the normal approximation for the mean works only with tens of samples.

The standard deviation $s$ describes the spread of the data, and it does not decrease as the number of samples grows. The standard error describes the uncertainty of the mean, and it decreases as $1/\sqrt N$ . To halve it, we need four times as many samples.

Confidence interval of the mean

\bar x - t\,\frac{s}{\sqrt{N}} \;\le\; \mu \;\le\; \bar x + t\,\frac{s}{\sqrt{N}}

where:

$\mu$ true mean of the population, units of the quantity
$\bar x$ mean of the sample, units of the quantity
$t$ Student's coefficient (a table value) for the chosen confidence probability and $N - 1$ degrees of freedom (the number of independent values left after the mean has been found), —
$s,\ N$ standard deviation of the data and number of measurements, units of the quantity, —

Units: the same as those of the measured quantity

Holds when

The same as for the standard error.
The data are distributed approximately normally, or there are enough samples.
A confidence probability of 95% means this: if we build a hundred such intervals from different samples, about 95 of them will cover the true mean.

With a large number of samples and a probability of 95%, the coefficient $t$ is close to 2 (more precisely, 1.96). With a small number of samples it is larger, because $s$ itself is estimated imprecisely.

Student's coefficient for a confidence probability of 95%

Number of samples $N$: 5
$t$: 2.78

Number of samples $N$: 10
$t$: 2.26

Number of samples $N$: 15
$t$: 2.14

Number of samples $N$: 20
$t$: 2.09

Number of samples $N$: 30
$t$: 2.05

Number of samples $N$: 60
$t$: 2.00

Number of samples $N$: very many
$t$: 1.96

Number of samples $N$	$t$
5	2.78
10	2.26
15	2.14
20	2.09
30	2.05
60	2.00
very many	1.96

Two-sided interval (above and below the mean), $N - 1$ degrees of freedom.

Worked exampleConfidence interval of the mean porosity and the mean permeability

Given (assumed data)

Porosity, 10 plugs: $\bar\phi = 19.0$ %, $s = 3.162$ %
Permeability, the same plugs: $\bar k = 100$ mD, $s = 116$ mD
Student's coefficient: $t = 2.262$ (95%, 9 degrees of freedom)

Solution

Porosity: $SE = 3.162/\sqrt{10} = 1.00$ %. Half-width of the interval: $2.262\cdot1.00 = 2.26$ %.
The interval for the mean porosity is from 16.7 to 21.3%, that is, ±12% of the mean itself.
Permeability: $SE = 116/\sqrt{10} = 36.7$ mD. Half-width: $2.262\cdot36.7 = 83$ mD, which is ±83% of the mean.

Answer: The mean porosity is

19.0 \pm 2.26

%, and the mean permeability is

100 \pm 83

mD.

What it means. The same ten plugs fix the mean porosity to within about 12%, but the mean permeability only to ±83%. For permeability, the formal interval (from 17 to 183 mD) is also unreliable. With so skewed a distribution, the mean of ten samples is not normally distributed. We can estimate the mean porosity of the interval from these plugs, but we practically cannot estimate the mean permeability. The data in this example are assumed for illustration.

How many samples are enough

We can turn the formula for the confidence interval around: we choose the precision we want and find the number of samples. It is convenient to state the precision as a tolerance $P$ in per cent of the mean: “the mean of the samples differs from the true mean by no more than $P$ % with a probability of 95%”.

Number of samples needed for a given precision of the mean

N_0 = \left(\frac{100\,t\,C_v}{P}\right)^2

where:

$N_0$ number of samples needed; the result is rounded up, —
$t$ Student's coefficient; about 2 at a probability of 95% and more than thirty samples, —
$C_v$ coefficient of variation $s/\bar x$ , —
$P$ tolerance — the permissible deviation of the mean of the samples from the true mean, % of the mean

Units: $P$ is in per cent; the other quantities are dimensionless

Holds when

We are estimating the arithmetic mean.
The measurements are independent and the sample is representative.
There are enough samples for the mean to be distributed approximately normally.
$C_v$ is known at least approximately, from a trial series of measurements or by analogy with similar rocks.

The formula follows from the condition that the half-width of the confidence interval, $t\,s/\sqrt{N_0}$ , equals the tolerance $P\,\bar x/100$ . The number of samples grows as the square of the coefficient of variation and as the square of the inverse tolerance. A rock that is twice as heterogeneous, or a tolerance that is twice as tight, needs four times as many samples.

Rule for the number of samples: precision of the mean ±20% at 95% probability

N_0 = (10\,C_v)^2, \qquad P_s = \frac{200\,C_v}{\sqrt{N_s}}

where:

$N_0$ number of samples for a tolerance of ±20%, —
$C_v$ coefficient of variation, —
$N_s$ actual number of samples, —
$P_s$ precision of the mean reached with $N_s$ samples, %

Units: $P_s$ is in per cent of the mean

Holds when

$t = 2$ : probability 95%, more than thirty samples.
The first formula refers to a tolerance of 20%, the second to any number of samples at $t = 2$ .
For very heterogeneous rocks ( $C_v$ noticeably greater than 1, for example complex carbonates), a tolerance of ±20% is often unattainable. Then we accept ±50%, and $N_0 = (4\,C_v)^2$ .

The rule lets us answer two questions in a minute: how many measurements we need, and what the ones we already have are worth. It was derived and tested mainly for permeability.

Number of samples for different rocks and tolerances

$C_v$: 0.2
±50%: 1
±20%: 4
±10%: 16

$C_v$: 0.5
±50%: 4
±20%: 25
±10%: 100

$C_v$: 1.0
±50%: 16
±20%: 100
±10%: 400

$C_v$: 1.5
±50%: 36
±20%: 225
±10%: 900

$C_v$: 2.0
±50%: 64
±20%: 400
±10%: 1600

$C_v$: 3.0
±50%: 144
±20%: 900
±10%: 3600

$C_v$	±50%	±20%	±10%
0.2	1	4	16
0.5	4	25	100
1.0	16	100	400
1.5	36	225	900
2.0	64	400	1600
3.0	144	900	3600

Calculated at $t = 2$ . Numbers below 30 are underestimates. For small samples $t$ is greater than two, and we cannot estimate the spread itself from two or three samples.

CalculatedPrecision of the mean

P_s = 200\,C_v/\sqrt{N_s}

against the number of samples for four values of the coefficient of variation. The dashed line is the tolerance of ±20%, and the points on it are the numbers of samples needed,

N_0 = (10\,C_v)^2

Worked exampleAre ten plugs enough

Given (assumed data)

Interval: 3.0 m, 10 plugs at 0.3 m spacing
Coefficients of variation: porosity 0.166; permeability 1.16

Solution

Porosity: $N_0 = (10\cdot0.166)^2 = 2.8$ , so formally 3 plugs would suffice. Ten is enough with a margin. They give a precision of $200\cdot0.166/\sqrt{10} = 10.5$ %.
Permeability: $N_0 = (10\cdot1.16)^2 = 134.6$ , that is 135 plugs. Ten plugs give $200\cdot1.16/\sqrt{10} = 73$ %.
Sampling step for 135 measurements in a 3.0 m interval: $3.0/135 = 0.022$ m, or 2.2 cm.

Answer: For the mean porosity, ten plugs are enough (±10.5%). For the mean permeability with a tolerance of ±20%, we need about 135 measurements, which is more than thirteen times as many.

What it means. A step of 2 cm is smaller than the diameter of a plug itself, so we cannot drill that many plugs from three metres of core. Dense permeability measurements are made with a probe permeameter (also called a profile permeameter or minipermeameter). We press its tip against the surface of the slabbed core (core sawn lengthwise), and we find the permeability at that point from the gas flow rate. The measurements are quick and do not destroy the core, so a hundred points per interval is a feasible programme. The precisions of ±10.5 and ±73% come from the rule with $t = 2$ . With the exact $t = 2.26$ for ten samples they are worse: ±12 and ±83%, as in the previous example. The data in this example are assumed for illustration.

How to plan sampling

1
A trial series
Make about 25 evenly spaced measurements in the interval under study: a reservoir, a unit or an individual rock type.
2
Estimate the spread
Calculate $C_v$ . If it is not greater than 0.5, twenty-five measurements are already enough: $(10\cdot0.5)^2 = 25$ .
3
Make the network denser
If $C_v$ is greater, place new points between those already made, as many as are needed for the total number of measurements to reach $N_0$ . The step of an even network equals the length of the interval divided by $N_0$ .
4
Check
Recalculate $C_v$ and the precision $P_s$ reached. If the number of measurements you need is out of reach, state a wider tolerance openly.

Bias: when more samples do not help

Everything said so far about precision referred to random error, which decreases as the number of samples grows. But there is another error: bias, the systematic error of sampling. Suppose the samples are taken not from the whole population but from a convenient part of it. Then their mean settles on the wrong value, and the more samples there are, the more confidently it settles there.

In core data, bias builds up at every step, from the choice of well to the measurement in the laboratory.

Which wells are cored. These are exploration wells and the first production wells, usually at the crest of the accumulation. On the flanks and in the water zone there is almost no core, although the rock there may be different. Secondary changes (changes in the rock after it was deposited) in the oil-saturated and water-saturated parts of a reservoir may proceed differently.
What can be recovered. Loose, fractured and vuggy (full of small cavities) intervals are destroyed during drilling and while the core is brought to the surface. Core recovery (the share of the cored interval that comes up) from the best reservoirs is often the lowest.
What the plugs are drilled from. We can cut a cylinder only from hard, uniform rock. Loose sand crumbles, so the highest permeability values are lost. We bypass thin dense layers, so the lowest values are lost.
Where sampling is denser. Productive sandstones are sampled more densely than siltstones and shales. Then the simple mean of the samples describes the reservoir rock rather than the section as a whole.
Measurement conditions. The samples are cleaned, dried and released from the overburden pressure (the weight of the rock above). So the porosity, and especially the permeability, measured on core are usually somewhat higher than at reservoir conditions.
Instrument limits. Values below and above the measurement range are cut off.

Worked exampleDense in the good rock, empty in the best

Given (assumed data)

Hard clean sandstone: 40% of the thickness of the reservoir; 24 plugs; mean permeability 300 mD
Shaly sandstone: 45% of the thickness; 6 plugs; 40 mD
Loose coarse-grained sand: 15% of the thickness; plugs could not be drilled; about 2000 mD from other data

Solution

Simple mean of the 30 plugs: $(24\cdot300 + 6\cdot40)/30 = 248$ mD.
Mean of the same two rock types, but weighted by their thicknesses: $(0.40\cdot300 + 0.45\cdot40)/0.85 = 162$ mD. The simple mean is too high by a factor of 1.5 because the hard sandstone is sampled more densely.
Mean over the whole reservoir, including the loose sand: $0.40\cdot300 + 0.45\cdot40 + 0.15\cdot2000 = 438$ mD.
If we double the number of plugs but sample in the same way (48 and 12), the simple mean stays equal to 248 mD. Only its standard error decreases.

Answer: The plugs give 248 mD, the sampled part of the section gives 162 mD, and the reservoir as a whole has about 438 mD.

What it means. There are two biases, and they act in opposite directions. The first is the uneven density of sampling. We correct it with weights: we multiply the means of the rock types by their proportions in the section, which we know from logs. The second is a rock that did not get into the samples at all. No weights can correct this, and here 15% of the thickness provides 68% of the flow capacity along the bedding (the ability of the section to carry flow along its layers). We need other data: core cut with the loose intervals preserved, sidewall cores, logs and well testing. The data in this example are assumed for illustration.

CalculatedThe shares of the three rock types of the example in the thickness of the reservoir (from logs) and in the total number of core plugs. The mean permeability of each rock type is shown on the right.

Bias cannot be removed completely, but it can be reduced and, above all, noticed in good time.

Describe all the core, calculate the means for each rock type, and weight them by the proportions of the rock types in the section taken from logs.
Mark the intervals with no core recovery and no plugs, and find out what rock they consist of.
Compare the core with logs and with the results of well tests. A systematic discrepancy is a sign of bias.
Plan sampling in advance: an even network and extra points in variable layers, rather than “wherever it is convenient to drill”.

Terms and symbols

Probability density $f(x)$: A curve whose area between two values equals the proportion of the population with such values.
Distribution function $F(x)$: The probability that the quantity does not exceed $x$ . It rises from 0 to 1.
$\mu$ , $\sigma$: The mean and standard deviation of the population. For the log-normal distribution, they refer to the logarithm of the quantity.
$z$ , $\Phi(z)$: The standardised value, and the cumulative probability of the standard normal distribution.
Probability plot: A plot of the values against the quantiles of the normal distribution. Normal data fall on a straight line.
$SE$: The standard error of the mean, $s/\sqrt N$ . It measures the uncertainty of the mean, not the spread of the data.
$t$: Student's coefficient: the number of standard errors in the half-width of the confidence interval. It is about 2 at a probability of 95% and a large number of samples.
$N_0$ , $P_s$: The number of samples needed for a given tolerance, and the precision of the mean reached with the number of samples in hand.
Bias: A systematic difference between the sample and the population. It does not decrease as the number of samples grows.
$V_{DP}$: The Dykstra–Parsons coefficient: a measure of the heterogeneity of permeability, from 0 to 1, found from the probability plot.

Common mistakes and misconceptions

Wrong

“If we take more samples, the standard deviation will decrease.”

In fact

The standard deviation is a property of the data themselves. As the sample grows we know it more precisely, but it does not decrease. What decreases is the standard error of the mean, $s/\sqrt N$ .

Wrong

“A 95% confidence interval contains 95% of the samples.”

In fact

It refers to the mean, not to individual values. It is the range that covers the true mean with a probability of 95%. 95% of the individual values of a normal distribution occupy a much wider interval, $\bar x \pm 2s$ .

Wrong

“There are many samples, so the mean is reliable.”

In fact

The number of samples reduces only the random error. If the sampling is biased (core only from the crest, plugs only from hard rock), the mean settles precisely on the wrong value.

Wrong

“Permeability is normally distributed, like everything in nature.”

In fact

It is the logarithm of permeability that is approximately normally distributed. In millidarcies the distribution is sharply skewed: the mode, the median and the mean differ from one another by several times.

Wrong

“P50 is the expected, that is the mean, value.”

In fact

P50 is the median. For a right-skewed distribution the mean is larger than the median. For a log-normal distribution it is larger by a factor of $\e^{\sigma^2/2}$ . For hydrocarbons in place this means that the expected value is higher than the P50 estimate.

In depth: advanced detail

The Dykstra–Parsons coefficient

In the petroleum literature, the heterogeneity of permeability (how unevenly it varies from place to place) is often expressed by the Dykstra–Parsons coefficient. It was proposed in the middle of the twentieth century for waterflood calculations in layered reservoirs. It is based on the log-normal distribution, and we read it directly from the probability plot.

Dykstra–Parsons coefficient

V_{DP} = \frac{k_{50} - k_{84.1}}{k_{50}} = 1 - \e^{-\sigma_{\ln k}}

where:

$V_{DP}$ Dykstra–Parsons coefficient, —
$k_{50}$ median permeability, mD
$k_{84.1}$ permeability exceeded by 84.1% of the samples — one standard deviation of the logarithm below the median, mD
$\sigma_{\ln k}$ standard deviation of the natural logarithm of permeability, —

Units: dimensionless, from 0 to 1

Holds when

Permeability is log-normally distributed. For other distributions the coefficient can be misleading.
We take $k_{50}$ and $k_{84.1}$ from the line drawn through the points of the probability plot, not from individual samples.
The percentage 84.1 is the proportion of samples with a greater permeability. It is the exceedance probability, not the cumulative proportion. On a plot with an axis of cumulative proportion, as in this lesson, we read $k_{84.1}$ at the 16% mark ( $z = -1$ ).

$V_{DP} = 0$ means a homogeneous reservoir. The closer the coefficient is to 1, the stronger the heterogeneity. The second form follows from $k_{84.1} = k_{50}\,\e^{-\sigma}$ . In terms of decimal logarithms, $V_{DP} = 1 - 10^{-\sigma_{\lg k}}$ .

Worked exampleThe Dykstra–Parsons coefficient from ten plugs

Given (assumed data)

The line on the probability plot: $\ln k = 4.1155 + 1.041\,z$

Solution

Median: $k_{50} = \e^{4.1155} = 61.3$ mD. One standard deviation lower: $k_{84.1} = \e^{4.1155 - 1.041} = 21.6$ mD.
$V_{DP} = (61.3 - 21.6)/61.3 = 0.65$ . Check: $1 - \e^{-1.041} = 0.65$ .
The coefficient of variation of the same log-normal model is $\sqrt{\e^{1.041^2} - 1} = 1.40$ . From the ten plugs themselves it came out as 1.16. In small samples from skewed distributions the coefficient of variation is usually underestimated.

Answer:

V_{DP} \approx 0.65

What it means. For most reservoirs the coefficient lies between 0.5 and 0.9, so our reservoir is in the moderate part of the range. The scale is compressed at the upper end. $V_{DP}$ values of 0.6, 0.8 and 0.9 correspond to $\sigma_{\ln k}$ of about 0.9, 1.6 and 2.3. So reservoirs differ far more than the numbers themselves suggest. The lesson “Reservoir heterogeneity and permeability averaging” covers how the coefficient is used in waterflood calculations. The data in this example are assumed for illustration.

Measures of heterogeneity of the log-normal distribution

$\sigma_{\ln k}$: 0.5
Mean / median: 1.13
$C_v$: 0.53
$V_{DP}$: 0.39
$x_{90}/x_{10}$: 3.6

$\sigma_{\ln k}$: 1.0
Mean / median: 1.65
$C_v$: 1.31
$V_{DP}$: 0.63
$x_{90}/x_{10}$: 13

$\sigma_{\ln k}$: 1.5
Mean / median: 3.08
$C_v$: 2.91
$V_{DP}$: 0.78
$x_{90}/x_{10}$: 47

$\sigma_{\ln k}$: 2.0
Mean / median: 7.39
$C_v$: 7.32
$V_{DP}$: 0.86
$x_{90}/x_{10}$: 168

$\sigma_{\ln k}$	Mean / median	$C_v$	$V_{DP}$	$x_{90}/x_{10}$
0.5	1.13	0.53	0.39	3.6
1.0	1.65	1.31	0.63	13
1.5	3.08	2.91	0.78	47
2.0	7.39	7.32	0.86	168

Here $x_{10}$ and $x_{90}$ are the 10th and 90th percentiles by cumulative proportion. All four measures are related to $\sigma_{\ln k}$ in one fixed way, so for log-normal data it is enough to know any one of them.

Confidence interval for log-normal data

For strongly skewed data we construct the interval in logarithms, where the distribution is close to normal. The result is an interval for the mean logarithm, that is, for the geometric mean (the median).

Confidence interval of the geometric mean

\exp\left(m - t\,\frac{s_{\ln}}{\sqrt N}\right) \;\le\; x_{50} \;\le\; \exp\left(m + t\,\frac{s_{\ln}}{\sqrt N}\right)

where:

$x_{50}$ median (geometric mean) of the population, units of the quantity
$m$ mean of the natural logarithms of the sample, —
$s_{\ln}$ standard deviation of the natural logarithms, —
$t,\ N$ Student's coefficient and number of measurements, —

Units: the limits are in the units of the quantity

Holds when

The quantity is log-normally distributed.
The measurements are independent and representative.

On the ordinary scale the interval is asymmetric, but “in factors” it is symmetric. We multiply and divide the estimate by the same factor, $\exp(t\,s_{\ln}/\sqrt N)$ .

Worked exampleInterval for the geometric mean of permeability

Given (assumed data)

Logarithms of the permeability of ten plugs: $m = 4.1155$ , $s_{\ln} = 1.041$
Student's coefficient: $t = 2.262$

Solution

Half-width in logarithms: $2.262\cdot1.041/\sqrt{10} = 0.745$ . Factor: $\e^{0.745} = 2.11$ .
Interval: from $61.3/2.11 = 29$ mD to $61.3\cdot2.11 = 129$ mD.

Answer: With a probability of 95%, the geometric mean lies between 29 and 129 mD.

What it means. The geometric mean of ten plugs is known to within a factor of 2.1. For the arithmetic mean of log-normal data, the interval is even wider and also asymmetric. To the uncertainty of the median we must add the uncertainty of the spread, because the mean depends on the spread through $\e^{\sigma^2/2}$ . The conclusion is the same as in the main text: we cannot determine the mean permeability of a heterogeneous reservoir from ten plugs. The data in this example are assumed for illustration.

Dependent samples

The formula for the standard error assumes independent measurements. In a reservoir, neighbouring points are alike. If one plug is highly permeable, the plug ten centimetres from it most likely is too. Such measurements partly repeat one another, and they contain less new information about the population than their number suggests.

Standard error of the mean for dependent measurements

SE = \frac{s}{\sqrt{N}}\,\sqrt{1 + (N - 1)\,\bar\rho}, \qquad N_{\text{eff}} = \frac{N}{1 + (N - 1)\,\bar\rho}

where:

$SE$ standard error of the mean, units of the quantity
$\bar\rho$ mean correlation coefficient between pairs of measurements (0 means independent, 1 means that they repeat one another), —
$N_{\text{eff}}$ effective number of independent measurements, —
$s,\ N$ standard deviation of the data and number of measurements, units of the quantity, —

Units: $SE$ is in the units of the measured quantity

Holds when

We are estimating the mean of a population that is wider than the sampled stretch: of a reservoir from one well, or of an accumulation from several wells.
All the measurements have the same variance (the same spread), and the correlation between them is not negative.

At $\bar\rho = 0$ the formula reduces to the ordinary one. With positive correlation the standard error does not tend to zero as $N$ grows. It levels off at $s\sqrt{\bar\rho}$ . However many samples we take in one well, they say only a limited amount about the reservoir as a whole.

Worked exampleWhat ten similar plugs are worth

Given (assumed data)

Number of samples: 10
Mean correlation between the samples $\bar\rho$: 0.3

Solution

Effective number of independent measurements: $10/(1 + 9\cdot0.3) = 2.7$ .
The standard error is larger than the one from the formula for independent samples, by a factor of $\sqrt{1 + 9\cdot0.3} = 1.9$ .

Answer: Ten samples with a mean correlation of 0.3 are equivalent to 2.7 independent samples.

What it means. The variogram describes how fast similarity falls off with distance (Spatial correlation and the variogram). The practical conclusion is simple: for judging a reservoir as a whole, the number of wells with core matters more than the number of plugs in each of them. The data in this example are assumed for illustration.

Connections to other topics

Descriptive statistics in geology — the mean, the median, the three means for permeability, the standard deviation and the coefficient of variation are the starting material of this lesson.
Volumetric estimation of hydrocarbons in place — the range of the estimate of hydrocarbons in place, and why the product of the extreme values is not a probabilistic estimate.
History matching — measurement error as a standard deviation, and the objective function normalised by it.
Rock permeability — what permeability depends on and why core and a well test give different numbers.
Porosity — how uneven sampling shifts the mean porosity and the hydrocarbons in place.
Core analysis — the sampling and preparation of core plugs are the source of most biases.

Check yourself

1
The porosity of a reservoir is normally distributed with a mean of 20% and a standard deviation of 3%. What proportion of the rock has a porosity between 17 and 23%?
1. A.About 50%
2. B.About 68%
3. C.About 95%
4. D.About 99.7%
Show the answer
Correct answer: B. About 68%
The interval from 17 to 23% is the mean plus or minus one standard deviation, and 68% of the values of a normal distribution lie within $\mu \pm \sigma$ . 95% lie within two standard deviations, which is from 14 to 26%.
2
From 25 samples, the coefficient of variation of permeability is 0.8. How many samples do we need for a precision of ±20%, and what precision do the samples in hand give?
Show the answer
The number needed is $N_0 = (10\cdot0.8)^2 = 64$ samples. The 25 in hand give $P_s = 200\cdot0.8/\sqrt{25} = 32$ %, so the mean is known to within about a third. To reach ±20%, we must make the network two and a half times denser.
3
On a probability plot of the logarithms of permeability, the points form two straight lines with a kink between them. What does this mean?
1. A.There are too many samples
2. B.Two rock types with different distributions are mixed in the sample
3. C.One should return from the logarithms to the values themselves
4. D.The distribution is normal
Show the answer
Correct answer: B. Two rock types with different distributions are mixed in the sample
One straight line is one population with one mean and one spread. A kink means a mixture of two populations, for example two rock types, or matrix and fractures. We must separate them and describe each one individually. The number of samples does not affect the shape of the plot.
4
The logarithm of permeability is normally distributed: $\mu = \ln 50$ , $\sigma = 1.2$ (natural logarithms, permeability in mD). Find the median and the arithmetic mean.
Show the answer
Median: $\e^{\mu} = 50$ mD. Mean: $\e^{\mu + \sigma^2/2} = 50\cdot\e^{0.72} \approx 103$ mD, which is twice the median. The wider the spread, the further the mean moves away from the “typical” value.
5
Core was taken only in crestal wells, and the plugs only from hard sandstone. The number of plugs was increased from 100 to 400. What changed?
1. A.The standard error halved, the bias stayed the same
2. B.Nothing changed
3. C.The bias halved
4. D.Both the standard error and the bias halved
Show the answer
Correct answer: A. The standard error halved, the bias stayed the same
The standard error decreases as $1/\sqrt N$ : four times as many samples give half the error. But this is the precision of the estimate of the mean of the sampled rock. The loose layers and the flanks of the accumulation still did not get into the sample, so the bias stayed the same.
6
The mean porosity of 16 samples is 18%, the standard deviation is 2%. Find the standard error and the 95% confidence interval of the mean ( $t = 2.13$ ).
Show the answer
$SE = 2/\sqrt{16} = 0.5$ %. Half-width of the interval: $2.13\cdot0.5 \approx 1.1$ %. With a probability of 95%, the true mean porosity lies between 16.9 and 19.1%. The individual samples are scattered much more widely, roughly from 14 to 22%.

Updated: 2 October 2026