Object Interning In Python

Object interning is a technique used in Python to optimize memory usage and improve performance by reusing immutable objects instead of creating new instances. It is particularly useful for strings, integers and user-defined objects. By interning objects, Python can store only one copy of each distinct object in memory reducing memory consumption and speeding up operations that rely on object comparisons.

Syntax :

import sys

interned_string = sys.intern(“string_to_intern”)

• We import the sys module using import a statement.
• The sys.intern() the function is called with a string argument that you want to the intern.
• The interned string is assigned to a variable, typically with a different name to the indicate that it is interned.

Example :

In this code, you have two pairs of string variables: string1 and string2, and interned_string1 and interned_string2.

Python3

import sys
string1 = "Hello"
string2 = "Hello"
interned_string1 = sys.intern(string1)
interned_string2 = sys.intern(string2)
print(string1 is string2)
print(interned_string1 is interned_string2)

Output :

True
True

Why String Interning is Important?

The String interning is important because strings are frequently used and can consume a significant amount of memory. Interning allows Python to store only one copy of each distinct string value in memory regardless of how many times it is referenced. This reduces memory consumption and speeds up string comparisons since they can be compared by memory address instead of comparing each character.

Examples :

In this case, both variables a and b are assigned the string “hi“, which is an immutable object in Python. Since strings are immutable, Python optimizes memory usage by reusing the same object when the same string value is assigned to multiple variables.

Python

a = "hi"
b = "hi"
print(id(a))
print(id(b))
print(a is b)

Output :

2267159609840
2267159609840
True

In this case, the variables a and b are assigned different string values. Although the strings “hi” and “hello” contain the same number of characters, they are different objects in memory.

Python

a = "hi"
b = "hello"
print(id(a))
print(id(b))
print(a is b)

Output :

2310655656432
2310655671024
False

Integer Interning

Similar to string interning Python interns certain integers in a specific range (-5 to 256) for performance optimization. Integers within this range are cached and reused ensuring that multiple references to the same integer point to the same memory location. Although, CPython, object interning is used for certain integers beyond the range of -5 to 256.

Python3

a = 1000
b = 1000
print(id(a))
print(id(b))
print(a is b)

Output :

2693705015568
2693705015568
True

In this example, both a and b refer to the same memory location because the integer value falls within the interned range.

User-Defined Objects

By default, user-defined objects in Python are not interned. However, you can implement your own interning mechanism by overriding the __new__ method of the class. By doing so you can control object instantiation and reuse existing instances based on specific criteria.

Example :

In this code, you create two instances of MyClass with different values: 20 and 50. Since the instances have different values, they are considered different objects, and therefore a is b returns False.

However, if you were to create another instance of MyClass with the value 20, it would return the existing instance created earlier, and a is b would then return True.

Python3

class MyClass:
    _instances = {}
    def __new__(cls, value):
        if value in cls._instances:
            return cls._instances[value]
        else:
            instance = super().__new__(cls)
            cls._instances[value] = instance
            return instance

    def __init__(self, value):
        self.value = value
a = MyClass(20)
b = MyClass(50)
print(a is b)

Output :

1764283301312
1764283301216
False