Gradual typing is a type system developed by Jeremy Siek and Walid Taha in 2006 which allows parts of a program to be dynamically typed and other parts to be statically typed. That means, the programmer can choose which part of the program he/she want to type check.
Gradual type checker is a static type checker that checks for type errors in statically typed part of a gradually typed program. The type checker deal with dynamically typed part of a gradually typed program by assigning variables and function parameters to special kind of type called Any. A static type checker will treat every type as being compatible with Any and Any as being compatible with every type. This means that it is possible to perform any operation or method call on a value of type on Any and assign it to any variable, the static type checker will not complain.
Why do we need a static type checker?
- To find bugs sooner in statically typed part of the program
- Larger the project, it becomes more difficult to debug a runtime type error
- It aids in understanding the program for a new engineer in the team as the flow of objects is hard to follow in Python
Background Information:
In 2014, Guido van Rossum along with Jukka Lehtosalo and Lukasz Langa made a proposal PEP 484 for Type Hints. The goal was to provide standard syntax for type annotations, opening up Python code to easier static analysis. In 2006, PEP 3107 had already introduced syntax for function annotations, but the semantics were deliberately left undefined as there was no clear idea on how a third party tool would make use of it.
Proposal Outline:
- Developer chooses whether to use or not a separate program called Static type Checker
- Function annotations are provisional, only be used by Static Type Checkers
- Third party libraries, Standard Libraries, C Extensions, Code where owner choose not to annotate, for PY2 Compatibility, it takes time to annotate, in such cases where it is not feasible to use type annotation, the developers can make dummy declarations of classes and functions in a separate file called Stub file which is seen only by Static Type Checker
- Strongly inspired by Mypy, a static type checker developed by Jukka Lehtosalo
Why do we need Type Hints?
- To help Type Checkers
- To serve as additional documentation
- To help IDEs improve suggestions and code checks
Few basic examples for function annotations:
Example 1:
# Python program to demonstrate # function annotations # Setting the arguments type and # return type to int def sum(num1: int, num2: int) -> int:
return num1 + num2
# will not throw an error print(sum(2, 3))
# will raise a TypeError print(sum(1, 'Geeks'))
|
Output:
5
Traceback (most recent call last):
File "/home/1c75a5171763b2dd0ca35c567f855c61.py", line 13, in
print(sum(1, 'Geeks'))
File "/home/1c75a5171763b2dd0ca35c567f855c61.py", line 7, in sum
return num1 + num2
TypeError: unsupported operand type(s) for +: 'int' and 'str'
Example 1 is a simple function whose argument and return type are declared in the annotations. This states that the expected type of the arguments num1 and num2 is int and the expected return type is int. Expressions whose type is a subtype of a specific argument type are also accepted for that argument.
Example 2:
# Python program to demonstrate # function annotations # Setting the arguments type and # return type to str def say_hello(name: str) -> str:
return 'Hello ' + name
# will not throw an error print(say_hello("Geeks"))
# will raise a TypeError print(say_hello(1))
|
Output:
Hello Geeks
Traceback (most recent call last):
File "/home/1ff73389e9ad8a9adb854b65716e6ab6.py", line 13, in
print(say_hello(1))
File "/home/1ff73389e9ad8a9adb854b65716e6ab6.py", line 7, in say_hello
return 'Hello ' + name
TypeError: Can't convert 'int' object to str implicitly
In Example 2, the expected type of the name argument is str. Analogically, the expected return type is str.
Few basic examples for variable annotations:
PEP 484 introduced type hints, a.k.a. type annotations. While its main focus was function annotations, it also introduced the notion of type comments to annotate variables:
Example 1:
# Python program to demonstrate # variable annotations # declaring the list to be # of int type l = [] # type: List[int]
# declaring the variable to # be str type name = None # type: str
|
Variable types are inferred by the initializer, although there are ambiguous cases. For example in Example 1, if we don’t annotate the variable l which is an empty list, a static type checker will throw an error. Similarly, for an uninitialized variable name, one needs to assign it to type none along with type annotation, otherwise, a static type checker will throw an error.
PEP 526 introduced a standard syntax for annotating the types of variables (including class variables and instance variables), instead of expressing them through comments:
Example 2:
# Python program to demonstrate # variable annotations l: List[int] = []
name: str
|
Example 2 is same as Example 1 but with the standard syntax introduced in PEP 526 instead of comment style of type annotation for variables. Notice that, in this syntax, there is no need to assign variable name to type none.
Example of a static type checker:
Mypy is a static type checker for Python 3 and Python 2.7. Using the Python 3 function annotation syntax (using the PEP 484 notation) or a comment-based annotation syntax for Python 2 code, you will be able to efficiently annotate your code and use mypy to check the code for common errors.
Mypy requires Python 3.5 or later to run. Once you’ve installed Python 3, install mypy using pip:
$ python3 -m pip install mypy
Once mypy is installed, run it by using the mypy tool:
$ mypy program.py
This command makes mypy type check your program.py file and print out any errors it finds. Mypy will type check your code statically: this means that it will check for errors without ever running your code, just like a linter.
Although you must install Python 3 to run mypy, mypy is fully capable of type checking Python 2 code as well: just pass in the –py2 flag.
$ mypy --py2 program.py
Examples of type errors thrown by Mypy:
# Python program to demonstrate # mypy def sum(a: int, b: int) -> int:
return a + b
sum( 1, '2') # Argument 2 to "sum" has incompatible type "str"; expected "int"
sum(1, b '2') # Argument 2 to "sum" has incompatible type "bytes"; expected "int"
|
Gradual Typing in Production Applications:
Lukasz Langa gave a talk on Gradual Typing in Production Applications at PyCascade-2018. He gave workflow suggestions as below:
- Workflow Suggestion #1: Find the most critical functions and start typing those first. For example, typing the widely used functions and widely imported modules first, since this allows code using these modules and functions to be type checked more effectively.
- Workflow Suggestion #2: Enable file-level linter type checking early. For example, flake8-mypy only presents type errors related to the current file and the standard library.
- Workflow Suggestion #3: Enable full program type checking in continuous integration to fight regressions. For example, Do full codebase checks with mypy as part of continuous integration to prevent type errors in existing code due to new code.
- Workflow Suggestion #4: Measure function coverage and the number of TypeError/AttributeErrorexceptions in production. This gives clear idea on how to proceed with gradual typing for the remaining codebase.
Conclusion:
- Only annotated functions are type checked and unannotated functions are ignored by a static type checker.
- A clean run of the type checker doesn’t prove there are no errors. As you gradually annotate the remaining codebase new type errors may show up.
- New annotations can discover errors in other functions for which type checker did not throw any error previously.