Python in the Enterprise Part 2: Static Type Checking

Due to the size of the code bases involved and often the lack of time to implement unit test, static typing is a must-have when developing enterprise systems.

Thankfully, Python now has support for optional static type checking in the form of the mypy static type checker, and the addition of type annotations to the Python language.

In this article, I’m going to cover the features of mypy and Python type annotations that I use the most.

Install mypy

pip install mypy

Basics

def add(x: int, y: int) -> int:
    return x + y

total = add(1, 2)

$ mypy example.py
Success: no issues found in 1 source file

Checking arguments

def add(x: int, y: int) -> int:
    return x + y

# error: Argument 2 to "add" has incompatible type "str"; expected "int"
total = add(1, "abc")

Type hinting lists

from typing import List

# type inferred as List[int] (even though Python supports heterogeneous lists)
l = [1, 2, 3]

# mypy error: Argument 1 to "append" of "list" has incompatible type "str"; expected "int"
l.append("abc")

# we can be explicit as well
from typing import List
l1: List[int] = [1, 2, 3]

# in function signatures
def do_something(stuff: List[int]):
  pass

Type hinting dicts

from typing import Dict

# inferred as Dict[str, int]
d = {"bob": 20}

# error: Incompatible types in assignment (expression has type "str", target has type "int")
d["alice"] = "30"

# in function signatures
from typing import Dict
def process(data: Dict[str, int]):
  pass

process(d)

# We can be explicit with Dict variables as well
d1: Dict[str, int] = {"bob": 30}

Typing hinting lists (abstract collection types)

# python documentation recommends, using duck types for function parameters
# so that the type checker will allow the use of list-like objects rather than just
# lists
from typing import Sequence
def do_something(stuff: Sequence[int]):
  pass

do_something([1, 2, 3])
do_something((1, 2, 3))

# error: List item 0 has incompatible type "str"; expected "int"
# error: List item 1 has incompatible type "str"; expected "int"
do_something(["abc", "def"])

# error: Argument 1 to "do_something" has incompatible type "Iterator[int]"; expected "Sequence[int]"
do_something(iter([1,2,3]))

# similarly use Mapping instead of Dict whenever possible so that Dict-like objects
# can be used in place of the concrete Dict type
from typing import Mapping
def process(stuff: Mapping[str, int]):
  pass

process({"name": 123})

mypy infers types whenever possible

def add(x: int, y: int) -> int:
    return x + y

# mypy infers that total is of type "int"
total = add(1, 2)

# error: Unsupported operand types for + ("str" and "int")
print("Total: " + total)

# we can also be explicit
total: int = add(1, 2)

Union and Optional (1)

For when a function argument/return value can be of more than one type

from typing import Union, Sequence

# support both a single recipient and a List
def send_email(recipients: Union[str, Sequence[str]]):
  pass

send_email('bob@example.com')
send_email(['bob@example.com', 'example.com'])

class Record:
  pass

# return None or the record depending on whether the record exists or not
def get_record_from_db() -> Union[Record, None]:
  pass

Union and Optional (2)

Union[K, None] is equivalent to Optional[K]. The previous example can be rewritten as:

from typing import Optional

class Record:
  name: str 

def get_record_from_db() -> Optional[Record]:
  pass

Using Union/Optional for return types isn’t ideal because the caller needs to check the return value before use:

record = get_record_from_db()
# error: Item "None" of "Optional[Record]" has no attribute "name"
print(record.name)

# check for value to make mypy happy
if record:
  print(record.name)

Or use mypy with the --no-strict-optional option.

Classes

from typing import List 

class Person:
  def __init__(self, name: str, nationalities: List[str]):
    # type inferred from arguments
    self.name = name
    self.nationalities = nationalities
    # error: Need type annotation for 'houses' (hint: "houses: List[<type>] = ..."
    self.houses = []

# error: Argument 2 to "Person" has incompatible type "int"; expected "List[str]
p = Person("bob", 123)

p = Person("bob", ["World"])

# error: "Person" has no attribute "age"
p.age = 30

Dataclasses

Similarly, with dataclasses:

# same as the previous example but using a dataclass
from dataclasses import dataclass

from typing import List

@dataclass
class Person:
  name: str
  nationalities: List[str]
  # error: Need type annotation for 'houses' (hint: "houses: List[<type>] = ...")
  houses = []

# error: Argument 2 to "Person" has incompatible type "int"; expected "List[str]
p = Person("bob", 123)

p = Person("bob", ["World"])

# error: "Person" has no attribute "age"
p.age = 30

Escape Hatches

Any is a special type that will never cause a type error. Can be used for heterogeneous collections, working with legacy code or working with libraries that produce their object model at runtime e.g. some RPC/Protocol generation libraries like thriftpy and protobuf 1.

from typing import List, Any
l: List[Any] = ["bob", 1, 2.0]

obj: Any
obj.blah_blah()

# type: ignore

Used to workaround bugs/limitations of mypy, work with legacy code or implement framework level logic that monkey patches user defined objects.

s: str = 123  # type: ignore

Tips

Other capabilities

I tried to cover most of the features that I use on a daily basis. Some features that I did not cover include:

  1. https://github.com/protocolbuffers/protobuf/issues/2638 ↩︎

Comments

comments powered by Disqus