Block 1 · Appendix A2
21-dataclasses-pattern-matching-starter/lakehouse/default/Files/data/solutions/21.dataclasses-pattern-matching/Open the notebook 21-dataclasses-pattern-matching-starter in your workspace and run it once. It holds a hand-written Flight class and four made-up flights, and a comment marks where the code of each part goes. Steps 1 to 3 are the exercise. Pattern matching is optional and sits under If time permits.
Do one part at a time, 1a, 1b and so on, and compare with the expected output before you go on. The solution is in 21-dataclasses-pattern-matching-solution, under every part on the exercise site, and at the back of the exercises PDF.
Run the Flight cell and read its output. Which field is missing from the printed flights? Are the two identical flights equal?
Flight(flight='KL1001', origin='AMS', dest='LHR')
Flight(flight='KL1002', origin='AMS', dest='CDG')
Flight(flight='DL0045', origin='JFK', dest='AMS')
Flight(flight='BA0431', origin='LHR', dest='AMS')
FalseTurn Flight into a data class. Remove __init__ and __repr__, keep the four fields, and run the cell again. Compare with 1a : does the printed form now show every field, and are the two identical flights equal?
The slides Declaring a Data Class and Automatic Methods show the decorator and the fields.
A data class lists its fields as annotated names in the class body, and the decorator writes __init__, __repr__ and __eq__ from them.
Flight(flight='KL1001', origin='AMS', dest='LHR', dep_delay=12)
Flight(flight='KL1002', origin='AMS', dest='CDG', dep_delay=-3)
Flight(flight='DL0045', origin='JFK', dest='AMS', dep_delay=47)
Flight(flight='BA0431', origin='LHR', dest='AMS', dep_delay=95)
Truefrom dataclasses import dataclass
@dataclass
class Flight:
flight: str
origin: str
dest: str
dep_delay: intGive dep_delay a default of 0. Create Flight("KL1003", "AMS", "OSL") and print it.
A field with a default is written like a function parameter with a default.
Flight(flight='KL1003', origin='AMS', dest='OSL', dep_delay=0) dep_delay: int = 0
print(Flight("KL1003", "AMS", "OSL"))Add a field remarks that holds a list of strings, such as "gate change". Give it [] as its default and run the cell.
ValueError: mutable default <class 'list'> for field remarks is not allowed: use default_factory remarks: list[str] = []Change the default so that every flight gets its own empty list. Run the cell again.
The slides The Mutable Default Trap and Using default_factory cover this.
field comes from the dataclasses module, next to dataclass.
Flight(flight='KL1001', origin='AMS', dest='LHR', dep_delay=12, remarks=[])
Flight(flight='KL1002', origin='AMS', dest='CDG', dep_delay=-3, remarks=[])
Flight(flight='DL0045', origin='JFK', dest='AMS', dep_delay=47, remarks=[])
Flight(flight='BA0431', origin='LHR', dest='AMS', dep_delay=95, remarks=[])
Truefrom dataclasses import dataclass, field
@dataclass
class Flight:
flight: str
origin: str
dest: str
dep_delay: int = 0
remarks: list[str] = field(default_factory=list)Create two flights without remarks. Append a remark to the first, and print the remarks of both. Is the second list still empty?
['gate change'] []first = Flight("KL1003", "AMS", "OSL")
second = Flight("KL1004", "AMS", "BCN")
first.remarks.append("gate change")
print(first.remarks, second.remarks)Add a __post_init__ method that raises a ValueError when origin or dest is not three uppercase letters, such as "ams" or "LONDON". Put the flight number and the bad code in the message.
The slide Post-Initialization shows when the decorator calls __post_init__.
str has len, isalpha and isupper. A loop over (self.origin, self.dest) checks both airports with one piece of code.
def __post_init__(self) -> None:
for code in (self.origin, self.dest):
if not (len(code) == 3 and code.isalpha() and code.isupper()):
raise ValueError(f"{self.flight}: {code!r} is not an IATA airport code")Make __post_init__ raise a ValueError as well when origin and dest are the same airport.
if self.origin == self.dest:
raise ValueError(f"{self.flight}: origin and destination are both {self.origin}")Create these three flights, each inside a try, and print the error : ("KL1005", "ams", "LHR"), ("KL1006", "AMS", "AMS") and ("KL1007", "AMS", "LONDON"). Then write a comment that says where the error appears : when the object is created, or when it is used?
The three records go through the same try, so a loop over them saves two copies of the code.
ValueError: KL1005: 'ams' is not an IATA airport code
ValueError: KL1006: origin and destination are both AMS
ValueError: KL1007: 'LONDON' is not an IATA airport codeYour wording of the messages may differ from these.
for bad in (("KL1005", "ams", "LHR"), ("KL1006", "AMS", "AMS"), ("KL1007", "AMS", "LONDON")):
try:
Flight(*bad)
except ValueError as err:
print("ValueError:", err)
# The error appears when the object is created, inside __post_init__.
# A bad record never exists, so no later code can use it.Steps 4 to 6 are optional. They use the Flight data class from Steps 1 to 3, so finish those first. Run the starter's last cell, optional_pattern_matching, before you start. It holds commands and messages, and a board that has a copy of every flight by flight number.
The commands list holds operator commands as lists of words. Write a function run(command) that uses match. Give it a case for ["list"] that prints one line per flight on the board, with flight number, route and delay. Call run(["list"]).
The slide Matching Sequences shows a match on a list of words.
KL1001 AMS-LHR delay 12
KL1002 AMS-CDG delay -3
DL0045 JFK-AMS delay 47
BA0431 LHR-AMS delay 95Your layout of the lines may differ.
def run(command: list[str]) -> None:
match command:
case ["list"]:
for f in board.values():
print(f" {f.flight} {f.origin}-{f.dest} delay {f.dep_delay}")
run(["list"])Add three cases :
["delay", flight, minutes] adds the minutes to the dep_delay of that flight["cancel", flight] appends "cancelled" to the remarks of that flight["divert", flight, airport] prints that the flight is divertedThe slide Capture Patterns explains how a plain name in a pattern picks up the value at its position.
minutes is text. It has to go through int() before it can be added.
case ["delay", flight, minutes]:
board[flight].dep_delay += int(minutes)
print(f"{flight} delayed to {board[flight].dep_delay} minutes")
case ["cancel", flight]:
board[flight].remarks.append("cancelled")
print(f"{flight} cancelled")
case ["divert", flight, airport]:
print(f"{flight} diverted to {airport}")Add a case for [] that does nothing, and a last case for anything else that prints the verb and the arguments it did not understand. Run every command in commands, and print the command before each run. What does ["delay", "KL1001"] match, and why not the delay case?
The slide Extended Sequence Matching shows how a starred name collects the rest of a sequence.
The order of the cases matters : the first pattern that matches wins.
> list
KL1001 AMS-LHR delay 12
KL1002 AMS-CDG delay -3
DL0045 JFK-AMS delay 47
BA0431 LHR-AMS delay 95
> delay KL1002 25
KL1002 delayed to 22 minutes
> cancel BA0431
BA0431 cancelled
> divert DL0045 RTM
DL0045 diverted to RTM
> delay KL1001
Unknown or incomplete command 'delay' with arguments ['KL1001']
> case []:
pass
case [verb, *rest]:
print(f"Unknown or incomplete command {verb!r} with arguments {rest}")
for command in commands:
print(">", " ".join(command))
run(command)Try run(["delay", "XX9999", "10"]) and read the error. Then add a guard, if flight in board, to the delay and cancel cases, and try it again.
The slide Guards shows the if after a pattern. A case whose guard fails moves on to the next case.
Unknown or incomplete command 'delay' with arguments ['XX9999', '10']Before the guard, the call fails with a KeyError on 'XX9999'.
case ["delay", flight, minutes] if flight in board:
board[flight].dep_delay += int(minutes)
print(f"{flight} delayed to {board[flight].dep_delay} minutes")
case ["cancel", flight] if flight in board:
board[flight].remarks.append("cancelled")
print(f"{flight} cancelled")
run(["delay", "XX9999", "10"])Let the verb "scrap" do the same as "cancel". Try run(["scrap", "KL1001"]).
The slide OR Patterns shows alternatives separated by |.
KL1001 cancelled case ["cancel" | "scrap", flight] if flight in board:
board[flight].remarks.append("cancelled")
print(f"{flight} cancelled")
run(["scrap", "KL1001"])Write a function classify(flight: Flight) -> str that uses class patterns on your data class. A flight whose last remark is "cancelled" is "cancelled", and any other flight is "on time". Print the class of every flight in flights, and of these two extra flights : Flight("KL1009", "AMS", "OSL", 5, ["cancelled"]) and Flight("KL1010", "CDG", "AMS", 4).
The slide Matching Objects shows a class pattern with named attributes.
The remarks are a list of any length. The slide Extended Sequence Matching shows how *_ accepts a remainder without binding it, so [*_, "cancelled"] matches a list that ends in "cancelled".
KL1001: on time
KL1002: on time
DL0045: on time
BA0431: on time
KL1009: cancelled
KL1010: on timedef classify(flight: Flight) -> str:
match flight:
case Flight(remarks=[*_, "cancelled"]):
return "cancelled"
case Flight():
return "on time"
extra = [Flight("KL1009", "AMS", "OSL", 5, ["cancelled"]), Flight("KL1010", "CDG", "AMS", 4)]
for f in flights + extra:
print(f"{f.flight}: {classify(f)}")Add two cases before the last one : a flight with a dep_delay above 60 is "severely late", and a flight with a dep_delay above 15 is "late". Put the delay in the text, for example "late, 47 minutes". Run the loop again. Then move the > 15 case above the > 60 case and run it once more : what happens to BA0431?
A class pattern can capture an attribute, Flight(dep_delay=delay), and take a guard.
The first matching pattern wins, as in a SQL CASE WHEN.
KL1001: on time
KL1002: on time
DL0045: late, 47 minutes
BA0431: severely late, 95 minutes
KL1009: cancelled
KL1010: on time case Flight(dep_delay=delay) if delay > 60:
return f"severely late, {delay} minutes"
case Flight(dep_delay=delay) if delay > 15:
return f"late, {delay} minutes"Add a case for a flight to AMS that is not late : "on time, inbound from" and its origin. Put it before the last case, and run the loop again.
A class pattern can match an attribute on a literal, Flight(dest="AMS"), and capture another one in the same pattern.
KL1001: on time
KL1002: on time
DL0045: late, 47 minutes
BA0431: severely late, 95 minutes
KL1009: cancelled
KL1010: on time, inbound from CDG case Flight(dest="AMS", origin=origin):
return f"on time, inbound from {origin}"The messages list is shaped like the messages a live flight feed sends. Write a function handle(message) that uses mapping patterns. A position message with "on_ground": True prints that the aircraft is on the ground, and any other message prints its type. Run every message in messages.
The slide Matching Dictionaries shows a mapping pattern that names some keys and ignores the rest.
Use {"type": kind} for the last case : it matches any message that has a type key and binds it.
ignored message of type 'position'
ignored message of type 'position'
KLM45 is on the ground
ignored message of type 'landed'
ignored message of type 'error'
ignored message of type 'heartbeat'def handle(message: dict) -> None:
match message:
case {"type": "position", "on_ground": True, "callsign": callsign}:
print(f"{callsign} is on the ground")
case {"type": kind}:
print(f"ignored message of type {kind!r}")
for message in messages:
handle(message)Add two more cases before the last one. A position above 9000 metres prints that the aircraft is cruising, and any other position prints that it is climbing or descending. Run every message again.
The cruising case needs a guard on the altitude. The slide Guards shows the syntax.
Put the cruising case before the general position case, or it never runs.
KLM1001 cruising at 11278 m
EIN602 climbing or descending at 1150 m
KLM45 is on the ground
ignored message of type 'landed'
ignored message of type 'error'
ignored message of type 'heartbeat' case {"type": "position", "callsign": callsign, "altitude": altitude} if altitude > 9000:
print(f"{callsign} cruising at {altitude} m")
case {"type": "position", "callsign": callsign, "altitude": altitude}:
print(f"{callsign} climbing or descending at {altitude} m")Add a case for a landed message before the last one. Print the callsign and the airport, and collect the other keys of the message in rest and print those as well.
The slide Matching Dictionaries shows **rest at the end of a mapping pattern.
DAL45 landed at AMS, also {'runway': '18R'}This is the line for the landed message. Your wording may differ.
case {"type": "landed", "callsign": callsign, "airport": airport, **rest}:
print(f"{callsign} landed at {airport}, also {rest}")Add a case before the last one for an error message with code 429 or 503 that prints a retry message with the code. Run every message once more. Which keys of the messages does no pattern mention?
The slide OR Patterns shows a choice inside a mapping pattern. 429 | 503 as code binds the value that matched.
KLM1001 cruising at 11278 m
EIN602 climbing or descending at 1150 m
KLM45 is on the ground
DAL45 landed at AMS, also {'runway': '18R'}
retry later, status 429
ignored message of type 'heartbeat'Your wording of the lines may differ.
case {"type": "error", "code": 429 | 503 as code}:
print(f"retry later, status {code}")Airport data class with frozen=True, order=True and the fields code and city. Use instances as dictionary keys, try to change a city, and make a changed copy with dataclasses.replace. Sort a list of airports and check which field decides the order.[asdict(f) for f in flights] and sort it by dep_delay.[f.name for f in fields(Flight)], and explain why astuple with those names is the safer way to hand the records to Spark.