Kahan Summation Algorithm

Prerequisites: Rounding off errors, Introduction to the floating-point representation
Kahan summation algorithm, also known as compensated summation and summation with the carry algorithm, is used to minimize the loss of significance in the total result obtained by adding a sequence of finite-precision floating-point numbers. This is done by keeping a separate running compensation (a variable to accumulate small errors). 

Reason for Loss of Significance:

• As we know in Java language we have two primitive floating-point types, float and double, with single-precision 32-bit and double-precision 64-bit format values and operations specified by IEEE 754. That is, they are represented in a form like:

SIGN FRACTION * 2EXP

• For example, 0.15625 = (0.00101)₂, which in floating-point format is represented as: 1.01 * 2^-3. However, not all fractions can be represented exactly as a fraction of a power of two. For example, 0.1 = (0.000110011001100110011001100110011001100110011001100110011001… )₂ and thus cannot be stored inside a floating-point variable.
• Therefore, floating-point error/loss of significance refers to when a number that cannot be stored as it is in the IEEE floating-point representation and repetitively some arithmetic operation is performed on it. This leads to some unexpected value and the difference between the expected and the obtained value is the error.

Below is an implementation that simulates the significance error:

C++

// C++ program to illustrate the
// floating-point error
#include<bits/stdc++.h>
using namespace std;

double floatError(double no)
{
    double sum = 0.0;
    for(int i = 0; i < 10; i++)
    {
        sum = sum + no;
    }
    return sum;
}

// Driver code
int main()
{
    cout << setprecision(16);
    cout << floatError(0.1);
}

// This code is contributed by rutvik_56

Java

// Java program to illustrate the
// floating-point error

public class GFG {

    public static double floatError(double no)
    {
        double sum = 0.0;
        for (int i = 0; i < 10; i++) {
            sum = sum + no;
        }
        return sum;
    }

    public static void main(String[] args)
    {

        System.out.println(floatError(0.1));
    }
}

Python3

# Python3 program to illustrate the
# floating-point error

def floatError(no):
    sum = 0.0
    for i in range(10):
        sum = sum + no
    return sum

if __name__ == '__main__':
    print(floatError(0.1))

# This code is contributed by mohit kumar 29

C#

// C++ program to illustrate the
// floating-point error
using System;

public class Program
{
    public static double FloatError(double no)
    {
        double sum = 0.0;
        for (int i = 0; i < 10; i++)
        {
            sum += no;
        }
        return Math.Round(sum - 1E-15, 15);
    }

  // Driver code
    public static void Main()
    {
        Console.WriteLine($"{FloatError(0.1):F15}");
    }
}

Javascript

<script>
// Javascript program to illustrate the
// floating-point error

function floatError(no)
{
    let sum = 0.0;
        for (let i = 0; i < 10; i++) {
            sum = sum + no;
        }
        return sum;
}

document.write(floatError(0.1));



// This code is contributed by patel2127
</script>

Output

0.9999999999999999

Note: The expected value for the above implementation is 1.0 but the value returned is 0.9999999999999999. Therefore, in this article, a method to reduce this error by using Kahan’s Summation algorithm is discussed. 

Kahan Summation Algorithm: The idea of the Kahan summation algorithm is to compensate for the floating-point error by keeping a separate variable to store the running time errors as the arithmetic operations are being performed. This can be visualised by the following pseudocode: 

function KahanSum(input)
    var sum = 0.0                    
    var c = 0.0                      
    for i = 1 to input.length do     
                                     
        var y = input[i] - c         
        var t = sum + y              
        c = (t - sum) - y            
                                     
        sum = t                      
                                     
    next i                          

    return sum

In the above pseudocode, algebraically, the variable c in which the error is stored is always 0. However, when there is a loss of significance, it stores the error in it. 

Below is the implementation of the above approach:

C++

// C++ program to illustrate the
// Kahan summation algorithm
#include <bits/stdc++.h>
using namespace std;

// Function to implement the Kahan
// summation algorithm
double kahanSum(vector<double> &fa)
{
    double sum = 0.0;

    // Variable to store the error
    double c = 0.0;

    // Loop to iterate over the array
    for(double f : fa)
    {
        double y = f - c;
        double t = sum + y;

        // Algebraically, c is always 0
        // when t is replaced by its
        // value from the above expression.
        // But, when there is a loss,
        // the higher-order y is cancelled
        // out by subtracting y from c and
        // all that remains is the
        // lower-order error in c
        c = (t - sum) - y;
        sum = t;
    }
    return sum;
}

// Function to implement the sum
// of an array
double sum(vector<double> &fa)
{
    double sum = 0.0;

    // Loop to find the sum of the array
    for(double f : fa)
    {
        sum = sum + f;
    }
    return sum;
}

// Driver code
int main()
{
    vector<double> no(10);
    for(int i = 0; i < 10; i++)
    {
        no[i] = 0.1;
    }

    // Comparing the results
      cout << setprecision(16);
    cout << "Normal sum: " << sum(no) << " \n";
    cout << "Kahan sum: " << kahanSum(no);
}

// This code is contributed by ajaykr00kj

Java

// Java program to illustrate the
// Kahan summation algorithm

public class GFG {

    // Function to implement the Kahan
    // summation algorithm
    private static double kahanSum(double... fa)
    {
        double sum = 0.0;

        // Variable to store the error
        double c = 0.0;

        // Loop to iterate over the array
        for (double f : fa) {

            double y = f - c;
            double t = sum + y;

            // Algebraically, c is always 0
            // when t is replaced by its
            // value from the above expression.
            // But, when there is a loss,
            // the higher-order y is cancelled
            // out by subtracting y from c and
            // all that remains is the
            // lower-order error in c
            c = (t - sum) - y;

            sum = t;
        }

        return sum;
    }

    // Function to implement the sum
    // of an array
    private static double sum(double... fa)
    {
        double sum = 0.0;

        // Loop to find the sum of the array
        for (double f : fa) {
            sum = sum + f;
        }

        return sum;
    }

    // Driver code
    public static void main(String[] args)
    {
        double[] no = new double[10];
        for (int i = 0; i < no.length; i++) {
            no[i] = 0.1;
        }

        // Comparing the results
        System.out.println("Normal sum: " + sum(no));
        System.out.println("Kahan sum: " + kahanSum(no));
    }
}

Javascript

<script>

// Javascript program to illustrate the
// Kahan summation algorithm

// Function to implement the Kahan
// summation algorithm
function kahanSum(fa)
{
    let sum = 0.0;

    // Variable to store the error
    let c = 0.0;

    // Loop to iterate over the array
    for(let f = 0; f < fa.length; f++)
    {
        let y = fa[f] - c;
        let t = sum + y;

        // Algebraically, c is always 0
        // when t is replaced by its
        // value from the above expression.
        // But, when there is a loss,
        // the higher-order y is cancelled
        // out by subtracting y from c and
        // all that remains is the
        // lower-order error in c
        c = (t - sum) - y;

        sum = t;
    }
    return sum;
}

// Function to implement the sum
// of an array
function sum(fa)
{
    let sum = 0.0;

    // Loop to find the sum of the array
    for(let f = 0; f < fa.length; f++)
    {
        sum = sum + fa[f];
    }
    return sum;
}

// Driver code
let no = new Array(10);
for(let i = 0; i < no.length; i++)
{
    no[i] = 0.1;
}

// Comparing the results
document.write("Normal sum: " +
               sum(no) + "<br>");
document.write("Kahan sum: " +
               kahanSum(no).toFixed(1) + "<br>");

// This code is contributed by unknown2108

</script>

Python3

# Python3 program to illustrate the
# Kahan summation algorithm

# Function to implement the Kahan
# summation algorithm
def kahanSum(fa):
    sum = 0.0

    # Variable to store the error
    c = 0.0

    # Loop to iterate over the array
    for f in fa:
        y = f - c
        t = sum + y

        # Algebraically, c is always 0
        # when t is replaced by its
        # value from the above expression.
        # But, when there is a loss,
        # the higher-order y is cancelled
        # out by subtracting y from c and
        # all that remains is the
        # lower-order error in c
        c = (t - sum) - y
        sum = t

    return sum


# Driver code
if __name__ == "__main__":
    no = [0.0] * 10
    for i in range(10):
        no[i] = 0.1

    # Comparing the results
    print("Normal sum: ", sum(no))
    print("Kahan sum: ", kahanSum(no))

C#

// Python3 program to illustrate the
// Kahan summation algorithm

// Function to implement the Kahan
// summation algorithm
using System;

class Program {
    // Driver code
    static void Main(string[] args) {
        double[] no = new double[10];
        for (int i = 0; i < 10; i++) {
            no[i] = 0.1;
        }
        // Comparing the results
        Console.WriteLine("Normal sum: {0}", Sum(no));
        Console.WriteLine("Kahan sum: {0}", KahanSum(no));
    }

    static double Sum(double[] fa) {
        double sum = 0.0;

        foreach (double f in fa) {
            sum += f;
        }

        return sum;
    }

    static double KahanSum(double[] fa) {
        double sum = 0.0;

        // Variable to store the error
        double c = 0.0;

        foreach (double f in fa) {
            double y = f - c;
            double t = sum + y;

            // Algebraically, c is always 0
            // when t is replaced by its
            // value from the above expression.
            // But, when there is a loss,
            // the higher-order y is cancelled
            // out by subtracting y from c and
            // all that remains is the
            // lower-order error in c
            c = (t - sum) - y;
            sum = t;
        }

        return sum;
    }
}

// This code is contributed by Shiv1043g

Output:

Normal sum: 0.9999999999999999
Kahan sum: 1.0

Time Complexity: O(N), where N is the length of the list.
Auxiliary Space: O(1)