Modules and Packages

Step 0Writing files from a notebook Done

A notebook is not a project, but it sits on a machine with a filesystem, and that is enough. Write a module the way you would write any other file :

from pathlib import Path

Path("hello.py").write_text('''
def greet(name):
    return f"Hello, {name}"
''')

Then import it like any module :

import hello

print(hello.greet("Achmea"))

Three things to know before they bite you :

• Write files with write_text. The cell magic %%writefile is a Jupyter convenience that Fabric does not always accept, and write_text is plain Python that works anywhere
• A directory has to exist before you can write into it. os.makedirs("calcu/single", exist_ok=True) comes first
• import runs a module once and remembers it. Edit the file, rerun the import, and nothing changes. Either importlib.reload(hello), or restart the session, which in this notebook costs seconds because nothing else is loaded

import importlib

importlib.reload(hello)

Use a triple-quoted string for the file contents, and mind the quotes inside it : if the code you are writing contains ''', switch the outer quotes to """.

Check where Python is looking, because the rest of the exercise depends on it :

import os
import sys

print(os.getcwd())
print(sys.path[:3])

The empty string or the current directory at the front of sys.path is why a file you just wrote can be imported at all.

Step 1One script becomes two files Done

Open solutions/01.python-toolchain/starter/calculator.py from the course bundle and copy it into a cell. Run it once, so you know what it does before you change it.

Now write the function definitions to a module of their own :

from pathlib import Path

Path("calculations.py").write_text('''
# the function definitions from calculator.py, and nothing else
''')

Take those definitions out of the calculator and leave the calls behind. Every call now fails, because the names are gone. Resolve that with an import on the first line :

import calculations

That alone is not enough. Prefix every call with the module it now lives in :

addresult = calculations.add(f1, f2)

Run it. Does the calculator behave as it did before? It should ; nothing about the program changed except where its parts live.

Step 2Import the names themselves Done

Prefixing every call is tiresome. The from <module> import <name> form brings the names into your own namespace :

from calculations import faculty, convertointlist, summate, average, add, subtract, multiply, divide

Remove the prefixes and run it again.

Which of the two forms would you rather read in six months? The prefixed one says where add comes from at the point where it is used, and a module with a long name is usually imported as something short instead.

Step 3A package with subpackages Done

Make a calcu package with three subpackages :

• single, for functions that take one parameter, such as faculty
• double, for functions that take two, such as add
• multiple, for functions that take a list, such as summate

import os

for folder in ["calcu", "calcu/single", "calcu/double", "calcu/multiple"]:
    os.makedirs(folder, exist_ok=True)
    open(os.path.join(folder, "__init__.py"), "a").close()

print(os.listdir("calcu"))

Put the functions into modules inside those subpackages, each written with write_text, and give the modules names that say what is in them, such as double/arithmetic.py. Take them out of the calculator.

Import them in the calculator and check that the program still works, first with plain import, then with from ... import ... so the calls stay readable.

What does __init__.py do here? It marks the directory as a package. Python 3 can import a directory without one, as a namespace package, but the empty file is still how most projects say "this is a package and I meant it".

Step 4A test harness Done

Write a test harness for the calculator and run it.

Path("test_calcu.py").write_text('''
from calcu.double.arithmetic import add, divide


def test_add():
    assert add(2, 3) == 5


def test_divide_by_zero():
    ...
''')

Run the tests. The simplest harness is plain asserts under an if __name__ == "__main__": guard, which you run with exec(Path("test_calcu.py").read_text()). For the real thing, install pytest for this session and start it as a program of its own :

%pip install pytest

import subprocess
import sys

result = subprocess.run(
    [sys.executable, "-m", "pytest", "-q", "test_calcu.py"], capture_output=True, text=True
)
print(result.stdout)

A %pip install lasts as long as the session. Command-line tools such as pytest and mypy end by calling sys.exit. Called inside a notebook cell, that ends the cell with a SystemExit error in place of the report, so run them in a separate process as shown above.

What should a test for divide check besides the answer? That dividing by zero does what you decided it should do, rather than whatever Python does by default.

Try one of these Done

• Print calculations.__file__ and calcu.single.__path__. Where did Python find them?
• Import a module twice in two cells and add a print at the top of the file. How many times does it run?
• Delete calcu/single/__init__.py and import again. Does it still work? What changed?