Python | Consuming an Iterable and Diagnosing faults in C

The article aims to write C extension code that consumes items from any iterable object such as a list, tuple, file, or generator.

Code #1 : C extension function that shows how to consume the items on an iterable.

static PyObject* py_consume_iterable(
    PyObject* self, PyObject* args)
{
    PyObject* obj;
    PyObject* iter;
    PyObject* item;

    if (!PyArg_ParseTuple(args, "O", &obj)) {
        return NULL;
    }

    if ((iter = PyObject_GetIter(obj)) == NULL) {
        return NULL;
    }

    while ((item = PyIter_Next(iter)) != NULL) {
        /* Use item */
        ... Py_DECREF(item);
    }

    Py_DECREF(iter);
    return Py_BuildValue("");
}

The code in above mirrors similar code in Python. The PyObject_GetIter() call is the same as calling iter() to get an iterator. The PyIter_Next() function invokes the next method on the iterator returning the next item or NULL if there are no more items. Make sure you’re careful with memory management — Py_DECREF() needs to be called on both the produced items and the iterator object itself to avoid leaking memory.

Diagnosing Segmentation Faults –
The interpreter violently crashes with a segmentation fault, bus error, access violation, or other fatal error. One likes to get a Python traceback that shows where the program was running at the point of failure.
The faulthandler module can be used to help you solve this problem. Include the following code in your program.

Code #2 :

import faulthandler
faulthandler.enable()

Alternatively, run Python with the -Xfaulthandler option such as this:

bash % python3 -Xfaulthandler program.py

Last, but not least, the PYTHONFAULTHANDLER environment variable can be set accordingly. With faulthandler enabled, fatal errors in C extensions will result in a Python traceback being printed on failures. For example:

Fatal Python error: Segmentation fault
Current thread 0x00007fff71106cc0:
File "example.py", line 6 in foo
File "example.py", line 10 in bar
File "example.py", line 14 in spam
File "example.py", line 19 in 
Segmentation fault

Although this won’t tell where in the C code things went wrong, at least it can tell how it got there from Python.

The faulthandler will show the stack traceback of the Python code executing at the time of failure. At the very least, this will show the top-level extension function that was invoked. With the aid of pdb or other Python debugger, one can investigate the flow of the Python code leading to the error.