Step 6 of 8

Floating-point rounding

double stores numbers in binary floating point: a sign, an exponent and about 53 bits of precision. Whole numbers up to about 9 quadrillion are exact, but most decimal fractions have no exact binary form, just as 1/3 has no exact decimal form. 0.1 is stored as the nearest binary fraction, slightly off. Arithmetic then accumulates those tiny errors:

#include <stdio.h>

int main(void) {
    double a = 0.1 + 0.2;
    printf("%d\n", a == 0.3);
    printf("%.17f\n", a);
    double total = 0;
    for (int i = 0; i < 10; i++) total += 0.1;
    printf("%d %.17f\n", total == 1.0, total);
    long long cents = 0;
    for (int i = 0; i < 10; i++) cents += 10;
    printf("%lld.%02lld\n", cents / 100, cents % 100);
    return 0;
}

0
0.30000000000000004
0 0.99999999999999989
1.00

Two rules professionals follow

1. Never compare doubles with == after arithmetic. Ask whether they're close enough instead. The tolerance should scale with the size of the numbers: an error of 0.000001 is huge for values near 0.000001 but negligible for values near a million. A good test is |a - b| <= eps * max(1, |a|, |b|), which behaves like an absolute tolerance for small numbers and a relative one for big numbers.
2. Never store money in double. Store whole cents in an integer type (as in the last part of the example), and only format as dollars for display. Banks and payment systems do exactly this.

Printing with %.2f hides the error but doesn't remove it; comparisons and repeated sums still see it.

Your turn: write int nearly_equal(double a, double b, double eps) using that formula. Don't use <math.h>; write the absolute value yourself.

Previous: The signed/unsigned comparison trap Next: Type punning and strict aliasing