Floating-point behaviour is systematic: the format, the scale of the values, and the order of operations constrain what a computation can retain. This tutorial makes those constraints observable and uses them to explain the variance discrepancy from Module 1.
Compare common binary formats with two revealing values.
Inspect a decimal value that is not exactly representable.
Map spacing at several magnitudes.
Observe when an increment has no effect.
Classify values at the edges of the format.
Compare evaluation orders.
Explain the variance discrepancy.
Confirm the working format
The published course environment uses IEEE 754 binary64 for Python’s float. The assertions make that assumption explicit: if this notebook runs in a different environment, it should stop instead of silently attaching binary64 claims to another format. Radix is another name for the base of the number system, so a radix of 2 means that this activity is using binary floating point.
radix: 2
significand precision: 53 bits
largest finite value: 1.797693e+308
smallest normal value: 2.225074e-308
epsilon near one: 2.220446e-16
The significand precision describes how much binary detail a normal value can retain. The exponent range allows magnitudes from approximately \(10^{-308}\) to \(10^{308}\), but it does not make every number in that interval representable.
Compare common binary formats
Binary16 and bfloat16 both use 16 storage bits but allocate them differently. Binary16 has 11 bits of significand precision and 5 exponent bits; bfloat16 has 8 bits of significand precision and 8 exponent bits. Binary32 and binary64 have 24 and 53 bits of significand precision, respectively.
The helpers below emulate a declared conversion path without requiring a third-party numerical library. Python’s struct module performs binary16 and binary32 conversions. The bfloat16 helper first converts the finite input to binary32 and then rounds that bit pattern to bfloat16 using round-to-nearest, ties-to-even. It is a format-conversion experiment, not an emulator for a particular CPU, GPU, compiler, or arithmetic kernel.
def round_binary16(value):"""Round a finite value to binary16, then return it as binary64."""return struct.unpack(">e", struct.pack(">e", value))[0]def round_binary32(value):"""Round a finite value to binary32, then return it as binary64."""return struct.unpack(">f", struct.pack(">f", value))[0]def round_bfloat16_from_binary32(value):"""Round a finite binary32 conversion to bfloat16, returning binary64."""ifnot math.isfinite(value):raiseValueError("This teaching helper expects a finite value.") binary32_bits = struct.unpack(">I", struct.pack(">f", value))[0] rounding_bias =0x7FFF+ ((binary32_bits >>16) &1) bfloat16_bits = (binary32_bits + rounding_bias) &0xFFFF0000return struct.unpack(">f", struct.pack(">I", bfloat16_bits))[0]format_rounders = {"binary16": round_binary16,"bfloat16": round_bfloat16_from_binary32,"binary32": round_binary32,"binary64": float,}
Before running the next cell, make two predictions:
Which formats distinguish 1.001 from 1.0?
Which formats retain \(10^{-20}\) as a nonzero value?
These probes separate local precision near one from exponent range.
Binary16 changes 1.001 to its nearby value 1.0009765625, while bfloat16 rounds it to 1. Bfloat16 has coarser spacing near one because it retains fewer significand bits. Conversely, binary16 rounds \(10^{-20}\) to zero, while bfloat16 retains a normal value because its wider exponent field provides approximately the normal range of binary32.
The result is a trade-off, not a quality ranking. Whether either approximation is adequate depends on the required accuracy and on the complete computation.
Predict the first lost unit increment
If a binary format has \(p\) bits of significand precision, all consecutive integers through \(2^p\) are representable. At \(2^p\), the upward spacing is two, so the halfway sum \(2^p+1\) can round back to \(2^p\). Some larger integers remain representable, but not every integer does. Predict the threshold for each format before running the check.
The same finite-set model predicts every row; only the format parameters change. This is why a result record should name the format instead of saying only “floating point,” “single precision,” or “16 bit.”
Inspect a stored decimal fraction
Before running the cell, predict whether the exact stored value of 0.1 is the rational number \(1/10\). The hexadecimal form is a compact, exact description of the stored binary floating-point value.
The stored ratio is close to, but not equal to, \(1/10\). Python’s usual display uses the shortest decimal text that converts back to the same stored value. Showing more digits reveals the approximation; it does not alter it.
Map the local spacing
math.nextafter(x, math.inf) gives the next representable value above x. Predict whether the absolute gap will remain constant as x grows.
x next value upward gap
9.9999999999999995e-07 1.0000000000000002e-06 2.11758e-22
1 1.0000000000000002 2.22045e-16
1000000 1000000.0000000001 1.16415e-10
9007199254740992 9007199254740994 2
10000000000000000 10000000000000002 2
The absolute gap grows with magnitude. Near one, it is approximately \(2.22\times10^{-16}\); at \(2^{53}\) and \(10^{16}\), it is 2. Values that differ by less than a local gap may map to the same floating-point value.
math.ulp(x) reports a related local spacing measure. An ulp is meaningful at the scale of x; it is not one global comparison tolerance.
Observe a lost increment
At \(2^{53}\), adjacent binary64 values are two units apart. Predict the result of adding each increment before running the cell.
base =float(2**53)increments = [0.5, 1.0, 2.0, 3.0, 4.0]print(f"base = {base:.0f}, upward gap = {math.ulp(base):.0f}")print(f"{'increment':>10s}{'computed sum':>18s}{'changed?':>8s}")for increment in increments: computed_sum = base + incrementprint(f"{increment:10.1f}{computed_sum:18.0f} "f"{str(computed_sum != base):>8s}" )
base = 9007199254740992, upward gap = 2
increment computed sum changed?
0.5 9007199254740992 False
1.0 9007199254740992 False
2.0 9007199254740994 True
3.0 9007199254740996 True
4.0 9007199254740996 True
base + 1.0 is exactly halfway between two representable values. Under the default round-to-nearest, ties-to-even rule, it rounds back to base. The increment is not zero; its contribution is absent from this stored sum.
Exercise: find an effective increment
Change test_increment and find the smallest positive integer increment for which the computed sum differs from base. Predict the answer from the local spacing before rerunning the cell.
test_increment =1.0# Change this value.test_sum = base + test_incrementprint(f"base: {base:.0f}")print(f"increment: {test_increment:.1f}")print(f"sum: {test_sum:.0f}")print(f"changed: {test_sum != base}")
source stored value classification
ordinary 1 normal finite
overflowed inf +infinity
smallest normal 2.2250738585072014e-308 normal finite
half smallest normal 1.1125369292536007e-308 subnormal
smallest subnormal 4.9406564584124654e-324 subnormal
underflowed 0 +zero
negative zero -0 -zero
indeterminate nan NaN
Subnormal values fill part of the gap between the smallest normal magnitude and zero, but with progressively fewer significant bits. The transition through that region is gradual underflow. A still smaller result rounds to signed zero.
Do not test for NaN with equality. NaN is unordered and is not equal to itself; use a classification operation such as math.isnan.
not_a_number =float("nan")print(f"NaN equals itself: {not_a_number == not_a_number}")print(f"math.isnan(NaN): {math.isnan(not_a_number)}")print(f"positive and negative zero compare equal: {0.0==-0.0}")print(f"sign of +0.0: {math.copysign(1.0, 0.0):+.0f}")print(f"sign of -0.0: {math.copysign(1.0, -0.0):+.0f}")
NaN equals itself: False
math.isnan(NaN): True
positive and negative zero compare equal: True
sign of +0.0: +1
sign of -0.0: -1
Change the evaluation order
Both expressions below equal 1 in real arithmetic. Predict their floating-point results by considering the first addition in each expression.
a =1.0e16b =-1.0e16c =1.0left_grouped = (a + b) + cright_grouped = a + (b + c)print(f"(a + b) + c = {left_grouped:.1f}")print(f"a + (b + c) = {right_grouped:.1f}")print(f"results equal: {left_grouped == right_grouped}")
(a + b) + c = 1.0
a + (b + c) = 0.0
results equal: False
The first grouping cancels a and b exactly before adding c. In the second grouping, adding c to b is rounded back to b, so the later sum is zero. Floating-point addition is therefore not generally associative.
Return to the variance experiment
The shortcut formula subtracts two intermediate values near \(10^{16}\ \mathrm{ns}^2\). Inspect their local spacing and compare the result with the centered calculation and the exact reference established in Module 1.
measurements_ns = [100_000_004.0,100_000_007.0,100_000_013.0,100_000_016.0,]mean_ns =sum(measurements_ns) /len(measurements_ns)centered_variance_ns2 =sum( (value - mean_ns) **2for value in measurements_ns) /len(measurements_ns)mean_of_squares_ns2 =sum( value**2for value in measurements_ns) /len(measurements_ns)square_of_mean_ns2 = mean_ns**2shortcut_variance_ns2 = mean_of_squares_ns2 - square_of_mean_ns2print(f"mean: {mean_ns:.1f} ns")print(f"centered values: {[value - mean_ns for value in measurements_ns]} ns")print(f"mean of squares: {mean_of_squares_ns2:.17g} ns²")print(f"square of mean: {square_of_mean_ns2:.17g} ns²")print(f"spacing at mean of squares: {math.ulp(mean_of_squares_ns2):.1f} ns²")print(f"centered variance: {centered_variance_ns2:.1f} ns²")print(f"shortcut variance: {shortcut_variance_ns2:.1f} ns²")print(f"exact reference from Module 1: {22.5:.1f} ns²")
mean: 100000010.0 ns
centered values: [-6.0, -3.0, 3.0, 6.0] ns
mean of squares: 10000002000000122 ns²
square of mean: 10000002000000100 ns²
spacing at mean of squares: 2.0 ns²
centered variance: 22.5 ns²
shortcut variance: 22.0 ns²
exact reference from Module 1: 22.5 ns²
The centered calculation works with values near the spread of the data. The shortcut calculation first rounds much larger intermediate values whose local spacing is \(2\ \mathrm{ns}^2\), then subtracts them. The rounded intermediates no longer contain enough information to recover \(22.5\ \mathrm{ns}^2\).
This explains the observation; it does not prove that every centered variance calculation is reliable or determine an acceptable error for another use case.
Evidence record
Record the environment assumption, observations, explanation, and limitation together. This makes clear which claims may need to be revisited in another language or floating-point format.
evidence = {"primary_arithmetic_format": "IEEE 754 binary64","rounding_assumption": "round to nearest, ties to even","format_probe_records": format_probe_records,"integer_threshold_records": integer_threshold_records,"spacing_at_2**53": math.ulp(float(2**53)),"lost_increment": float(2**53) +1.0==float(2**53),"addition_is_associative_for_test": left_grouped == right_grouped,"variance_reference_ns2": 22.5,"centered_variance_ns2": centered_variance_ns2,"shortcut_variance_ns2": shortcut_variance_ns2,"limitation": ("No acceptable-error criterion or measurement uncertainty was assessed." ),}evidence