Kahan Summation Algorithm

Prerequisites: Rounding off errors, Introduction to floating point representation

Kahan summation algorithm, also known as compensated summation and summation with the carry algorithm, is used to minimize the loss of significance in the total result obtained by adding a sequence of finite-precision floating-point numbers. This is done by keeping a separate running compensation (a variable to accumulate small errors).

Reason for Loss of Significance:

  • As we know in Java language we have two primitive floating-point types, float and double, with single-precision 32-bit and double-precision 64-bit format values and operations specified by IEEE 754. That is, they are represented in a form like:
    SIGN FRACTION * 2EXP 
    
  • For example, 0.15625 = (0.00101)2, which in floating-point format is represented as: 1.01 * 2-3. However, not all fractions can be represented exactly as a fraction of a power of two. For example, 0.1 = (0.000110011001100110011001100110011001100110011001100110011001… )2 and thus cannot be stored inside a floating-point variable.
  • Therefore, floating-point error/loss of significance refers to when a number that cannot be stored as it is in the IEEE floating-point representation and repetitively some arithmetic operation is performed on it. This leads to some unexpected value and the difference between the expected and the obtained value is the error.

Below is an implementation that simulates the significance error:

Java

filter_none

edit
close

play_arrow

link
brightness_4
code

// Java program to illustrate the
// floating-point error
  
public class GFG {
  
    public static double floatError(double no)
    {
        double sum = 0.0;
        for (int i = 0; i < 10; i++) {
            sum = sum + no;
        }
        return sum;
    }
  
    public static void main(String[] args)
    {
  
        System.out.println(floatError(0.1));
    }
}

chevron_right


Python3

filter_none

edit
close

play_arrow

link
brightness_4
code

# Python3 program to illustrate the
# floating-point error
  
def floatError(no):
    sum = 0.0
    for i in range(10):
        sum = sum + no
    return sum
  
if __name__ == '__main__':
    print(floatError(0.1))
  
# This code is contributed by mohit kumar 29

chevron_right


Output:



0.9999999999999999

Note: The expected value for the above implementation is 1.0 but the value returned is 0.9999999999999999. Therefore, in this article, a method to reduce this error by using Kahan’s Summation algorithm is discussed.

Kahan Summation Algorithm: The idea of the Kahan summation algorithm is to compensate for the floating-point error by keeping a separate variable to store the running time errors as the arithmetic operations are being performed. This can be visualised by the following pseudocode:

function KahanSum(input)
    var sum = 0.0                    
    var c = 0.0                      
    for i = 1 to input.length do     
                                     
        var y = input[i] - c         
        var t = sum + y              
        c = (t - sum) - y            
                                     
        sum = t                      
                                     
    next i                          

    return sum

In the above pseudocode, algebraically, the variable c in which the error is stored is always 0. However, when there is a loss of significance, it stores the error in it.

Below is the implementation of the above approach:

filter_none

edit
close

play_arrow

link
brightness_4
code

// Java program to illustrate the
// Kahan summation algorithm
  
public class GFG {
  
    // Function to implement the Kahan
    // summation algorithm
    private static double kahanSum(double... fa)
    {
        double sum = 0.0;
  
        // Variable to store the error
        double c = 0.0;
  
        // Loop to iterate over the array
        for (double f : fa) {
  
            double y = f - c;
            double t = sum + y;
  
            // Algebraically, c is always 0
            // when t is replaced by its
            // value from the above expression.
            // But, when there is a loss,
            // the higher-order y is cancelled
            // out by subtracting y from c and
            // all that remains is the
            // lower-order error in c
            c = (t - sum) - y;
  
            sum = t;
        }
  
        return sum;
    }
  
    // Function to implement the sum
    // of an array
    private static double sum(double... fa)
    {
        double sum = 0.0;
  
        // Loop to find the sum of the array
        for (double f : fa) {
            sum = sum + f;
        }
  
        return sum;
    }
  
    // Driver code
    public static void main(String[] args)
    {
        double[] no = new double[10];
        for (int i = 0; i < no.length; i++) {
            no[i] = 0.1;
        }
  
        // Comparing the results
        System.out.println("Normal sum: " + sum(no));
        System.out.println("Kahan sum: " + kahanSum(no));
    }
}

chevron_right


Output:

Normal sum: 0.9999999999999999
Kahan sum: 1.0

Attention reader! Don’t stop learning now. Get hold of all the important DSA concepts with the DSA Self Paced Course at a student-friendly price and become industry ready.




My Personal Notes arrow_drop_up


If you like GeeksforGeeks and would like to contribute, you can also write an article using contribute.geeksforgeeks.org or mail your article to contribute@geeksforgeeks.org. See your article appearing on the GeeksforGeeks main page and help other Geeks.

Please Improve this article if you find anything incorrect by clicking on the "Improve Article" button below.



Improved By : mohit kumar 29

Article Tags :
Practice Tags :


Be the First to upvote.


Please write to us at contribute@geeksforgeeks.org to report any issue with the above content.