Strings in real data: clean carefully, not aggressively

A partner sends " B001 | Mina | 3 " instead of a dictionary. Another sends a customer name with an accent. Our job is to interpret the data according to a format, not remove everything that looks inconvenient.

You need: string methods, lists, conditions, functions, and immutability. By the end: you can parse simple records, create deliberate matching keys, and explain the limits of your cleaning rules. Allow 30–40 minutes.

Split a simple, explicitly defined format

For this exercise, a record has exactly three fields separated by |; the fields cannot contain that separator. Save as partner_record.py:

raw = "  B001 | Mina | 3  "
parts = raw.split("|")

if len(parts) != 3:
    print("Expected exactly three fields.")
else:
    booking_id = parts[0].strip()
    customer = parts[1].strip()
    quantity_text = parts[2].strip()
    print(f"ID: {booking_id}")
    print(f"Customer: {customer}")
    print(f"Quantity text: {quantity_text}")

Output:

ID: B001
Customer: Mina
Quantity text: 3

.split("|") returns a list of substrings. .strip() removes surrounding whitespace from each field. It does not validate the ID, require a nonempty name, or convert the quantity to an integer. Parsing, normalising, and validating are distinct steps.

If this were actual CSV with quoted separators or embedded newlines, manual splitting would be the wrong parser. We will use the csv module for that format later.

Whitespace splitting is a different operation

text = "  Moon   Mice\tReturns  "
print(text.split())
print(" ".join(text.split()))
print("A||C".split("|"))

Output:

['Moon', 'Mice', 'Returns']
Moon Mice Returns
['A', '', 'C']

With no separator, .split() groups whitespace and omits empty fields at the edges. With an explicit separator, adjacent separators can produce empty fields. An empty field may carry important information, so do not casually discard it.

" ".join(...) puts a space between strings. The separator owns the method: ", ".join(names) is the pattern. Items must be strings; a list containing an integer raises TypeError.

Collapsing whitespace is useful for a search key, but can damage formatting in a stored document. Preserve the original text when the original matters.

Match without rewriting display text

display_name = "  Straße Cinema  ".strip()
search_key = display_name.casefold()

print(display_name)
print(search_key)
print(search_key == "STRASSE CINEMA".casefold())

Output:

Straße Cinema
strasse cinema
True

.casefold() is designed for caseless matching and can make changes beyond .lower(), such as treating German ß like ss. Keep the display value separately. A case-insensitive key is a chosen comparison rule, not proof that two business entities are identical.

Do not apply .title() to everyone’s name as a universal correction. Names and brands have their own spelling. Prefer stable source IDs for joining records when they are available.

Unicode: text is not just one byte per character

Python strings represent Unicode text. A character visible on screen may consist of multiple Unicode code points. Try this complete example:

composed = "caf\u00e9"
decomposed = "cafe\u0301"

print(composed == decomposed)
print(len(composed))
print(len(decomposed))

Output:

False
4
5

\u00e9 represents an accented e as one code point; \u0301 is a combining accent following the e. The strings may look the same while their representations differ. len() counts code points here, not bytes or visible symbols.

For a system that must compare canonically equivalent text, we will introduce Unicode normalisation through the standard library. .strip() and .casefold() do not solve every normalisation issue. An encoding such as UTF-8 concerns converting text to or from bytes at a file/network boundary; that is another separate concept.

Remove exactly what the rule says

filename = "sales.csv"
print(filename.removesuffix(".csv"))
print(filename.endswith(".csv"))
print("A-B-C".replace("-", "/", 1))

Output:

sales
True
A/B-C

.removesuffix() removes an exact suffix if present. .strip(".csv") would instead remove any of those individual characters repeatedly from both ends, which is not the same rule. The optional count in .replace() limits the number of replacements.

Performance: avoid rebuilding and rechecking needlessly

Strings are immutable. Transformations can allocate new strings, and splitting creates a list of substrings. Clean stable lookup values once when appropriate rather than normalising the same values inside every comparison.

For assembling many known pieces, collecting strings and joining them expresses the work clearly. Repeated concatenation can require repeatedly copying growing results; exact behaviour depends on the implementation and usage. Neither approach is a reason to hold an enormous export in memory. Later we will write output incrementally.

For three short fields, prioritise readable rules. At scale, measure allocations and I/O as well as CPU time. Removing accents or empty fields for speed is a correctness change, not just an optimisation.

Your turn: a venue lookup key

Write venue_key(text) that strips surrounding whitespace, collapses runs of whitespace to one space, and casefolds the result. Preserve the caller’s original string. Do not remove punctuation or accents.

def venue_key(text):
    words = text.split()
    return " ".join(words).casefold()


original = "  NORTH\tCinema  "
print(venue_key(original))
print(original == "  NORTH\tCinema  ")
print(venue_key("   ") == "")

Checkpoint: explain why splitting CSV manually, stripping a suffix with .strip(), and rewriting display names can each produce plausible-looking but incorrect data. Next we will use comprehensions to express transformations without hiding their meaning.