Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 8 additions & 4 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ Each module defines a plain Python set of known name pieces:
`HumanName` is the single public class. Assigning to `full_name` (or instantiating with a string) triggers `parse_full_name()`.

Parse flow:
1. `pre_process()` — strips nicknames (parenthesis/quotes) and emoji, fixes "Ph.D." variant spellings
1. `pre_process()` — strips nicknames/maiden names (parenthesis/quotes, routed to `nickname_list`/`maiden_list` per `Constants.nickname_delimiters`/`maiden_delimiters`) and emoji, fixes "Ph.D." variant spellings
2. Split on commas → 1 part (no comma), 2 parts (suffix-comma or lastname-comma), 3+ parts
3. `parse_pieces()` — splits on spaces, detects dotted abbreviations like "Lt.Gov." and adds them to constants dynamically
4. `join_on_conjunctions()` — merges pieces adjacent to conjunctions into single tokens (e.g. `['Secretary', 'of', 'State']` → `['Secretary of State']`); also joins prefix particles to the following lastname token
Expand All @@ -110,11 +110,11 @@ Each named attribute (`title`, `first`, etc.) is a `@property` that joins its co
2. Add `x: str | None = None` to `HumanName.__init__` signature after related kwargs
3. Add `self.x = x if x is not None else self.C.x` in body — use `is not None`, not `or`, to allow falsy values like `""`
4. conftest auto-restores scalar CONSTANTS between tests, but tests that *set* CONSTANTS mid-run still need their own try/finally
5. Update `docs/customize.rst`'s constants list (and `docs/usage.rst` if it affects a documented example) — don't wait for the release checklist, these `.rst` docs aren't covered by CI so a stale one can ship silently.
5. Update `docs/customize.rst`'s constants list (and `docs/usage.rst` if it affects a documented example) — don't wait for the release checklist, these `.rst` docs aren't covered by CI so a stale one can ship silently. Check `AGENTS.md` itself too (Extension Patterns, Gotchas, the Architecture section) for now-stale attribute/test names or descriptions — it has the same blind spot as the `.rst` docs and the release checklist only catches it at release time.

**Adding a new mutable/collection `Constants` attribute** (a `SetManager`/`TupleManager`-backed group, e.g. `extra_nickname_delimiters`): add it to `_COLLECTION_CONFIG_ATTRS` in `tests/conftest.py`, or tests that mutate the global `CONSTANTS` copy will leak state into later tests. Contents must be deep-copyable (the snapshot uses `copy.deepcopy`) — already true for the existing manager types.
**Adding a new mutable/collection `Constants` attribute** (a `SetManager`/`TupleManager`-backed group, e.g. `nickname_delimiters`/`maiden_delimiters`): add it to `_COLLECTION_CONFIG_ATTRS` in `tests/conftest.py`, or tests that mutate the global `CONSTANTS` copy will leak state into later tests. Contents must be deep-copyable (the snapshot uses `copy.deepcopy`) — already true for the existing manager types.

Add a dedicated `copy.deepcopy()` round-trip test for it too (see `test_regexes_deepcopy_roundtrip`/`test_extra_nickname_delimiters_deepcopy_roundtrip` in `tests/test_constants.py`), not just reliance on conftest's autouse snapshot/restore exercising it incidentally. `TupleManager`/`RegexTupleManager.__getattr__` answer *any* unknown attribute lookup — including dunder probes like `__deepcopy__` — so a new manager subtype or a `__getattr__` tweak can silently break `copy.deepcopy` (this bit `RegexTupleManager` before the dunder-lookup guard was added). A direct test on the new attribute's own manager instance catches that where the conftest fixture, which never asserts on the copy, would not.
Add a dedicated `copy.deepcopy()` round-trip test for it too (see `test_regexes_deepcopy_roundtrip`/`test_nickname_delimiters_deepcopy_roundtrip` in `tests/test_constants.py`), not just reliance on conftest's autouse snapshot/restore exercising it incidentally. `TupleManager`/`RegexTupleManager.__getattr__` answer *any* unknown attribute lookup — including dunder probes like `__deepcopy__` — so a new manager subtype or a `__getattr__` tweak can silently break `copy.deepcopy` (this bit `RegexTupleManager` before the dunder-lookup guard was added). A direct test on the new attribute's own manager instance catches that where the conftest fixture, which never asserts on the copy, would not. As with the scalar-attribute pattern above, also check `AGENTS.md` itself for now-stale references when you touch this.

**Adding a word to a config set** — first check the *other* sets for the same word (grep `nameparser/config/` or intersect the sets in a `python3 -c`). Real overlaps exist: `do`/`st`/`mc` ∈ `PREFIXES` ∩ `TITLES`/`SUFFIX_ACRONYMS`; `abd` = "ABD" ∈ `SUFFIX_ACRONYMS`; `abu` ∈ `PREFIXES` ∩ `first_name_prefixes` (position-dependent: leading token → first-name join, mid-name → last-name join). Usually position-dependent and harmless, but can force a guard or an exclusion (the `last_base` all-particles guard; dropping `abd` from `first_name_prefixes`).

Expand All @@ -140,6 +140,10 @@ Add a dedicated `copy.deepcopy()` round-trip test for it too (see `test_regexes_

**`is_suffix()`'s period-stripping is asymmetric** — it does `lc(piece).replace('.', '')` (strips *all* periods) before checking `suffix_acronyms`, but only `lc(piece)` (leading/trailing only) before checking `suffix_not_acronyms`. Code that reimplements this check instead of calling `is_suffix()` must mirror both branches or it will misclassify acronym suffixes with internal-only periods (e.g. `"M.D"` with no trailing dot) — bit `parse_nicknames()`'s `handle_match()` in PR #189.

**`nickname_delimiters`/`maiden_delimiters` built-ins are string sentinels, not compiled patterns** — the three default keys (`quoted_word`, `double_quotes`, `parenthesis`) store the *name* of a `Constants.regexes` entry (a plain `str`), resolved via `getattr(self.C.regexes, name)` at parse time in `parse_nicknames()` — not the compiled `re.Pattern` itself. This is what lets `CONSTANTS.regexes.parenthesis = ...` keep affecting nickname/maiden parsing after construction, same as before this mechanism existed. Routing a built-in between buckets must be a `pop()` + assign (`maiden_delimiters['parenthesis'] = nickname_delimiters.pop('parenthesis')`) to carry that string sentinel over — copying `CONSTANTS.regexes['parenthesis']` directly into the new bucket instead would freeze it as a snapshot and silently stop tracking further `regexes` overrides. A key added by a caller for a *custom* delimiter is a real compiled pattern, distinguished at parse time via `isinstance(raw_pattern, re.Pattern)`. (#22)

**`TupleManager.__setattr__`/`__delattr__` guard dunder names too, not just `__getattr__`** — constructing a subscripted generic, e.g. `TupleManager[re.Pattern[str] | str]({...})` (needed so mypy sees the right value type instead of inferring one from the dict literal), makes `typing`'s `GenericAlias.__call__` set `__orig_class__` on the new instance right after `__init__` returns. Before this guard existed, `__setattr__` was a bare `dict.__setitem__` alias, so that assignment silently inserted a bogus `'__orig_class__'` entry into the dict itself, corrupting `.values()`/iteration for *every* `TupleManager`/`RegexTupleManager` instance, not just the one being constructed — this bit `nickname_delimiters`'s construction (#22) before the guard was added. Same fix shape as the `__getattr__` dunder guard above: fall back to `object.__setattr__`/`object.__delattr__` for dunder names, dict-backed storage for everything else.

**`_join_first_name_prefix` guard must exclude trailing suffixes** — suffix tokens are still in `pieces` when the helper runs (suffix detection happens in the assignment loop, later). The `reserve_last` guard must count `if not self.is_suffix(p)` to avoid treating a trailing suffix like "Jr." as a last-name slot; otherwise `"abdul salam jr"` → `last='jr'`.

**Doctests** — docstring examples in `nameparser/*.py` run under `uv run pytest` (`--doctest-modules`; `testpaths` is `tests` + `nameparser` only). The `.rst` doctests in `docs/` (`usage.rst`, `customize.rst`) are **not** run by pytest or CI (CI does `sphinx-build -b html`, not `-b doctest`), so verify `.rst` examples manually: `python3 -c "import doctest; print(doctest.testfile('docs/usage.rst', module_relative=False, optionflags=doctest.NORMALIZE_WHITESPACE))"`. Note `customize.rst` has pre-existing failures under `-b doctest` (CONSTANTS state leaks across examples — no per-example reset like `tests/conftest.py` provides — plus non-deterministic `SetManager` repr).
Expand Down
68 changes: 59 additions & 9 deletions docs/customize.rst
Original file line number Diff line number Diff line change
Expand Up @@ -56,14 +56,17 @@ remove punctuation to normalize them for comparison.
Adding Custom Nickname Delimiters
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

:py:meth:`~nameparser.parser.HumanName.parse_nicknames` recognizes three
built-in delimiters -- ``quoted_word``, ``double_quotes`` and
``parenthesis`` -- read from :py:attr:`~nameparser.config.Constants.regexes`,
so overriding e.g. ``CONSTANTS.regexes.parenthesis`` still works exactly as
before. To recognize an *additional* delimiter without overriding one of the
built-ins, add a pattern to
:py:obj:`~nameparser.config.Constants.extra_nickname_delimiters` (empty by
default) under any key, then re-run
:py:meth:`~nameparser.parser.HumanName.parse_nicknames` recognizes delimiters
through two per-bucket collections:
:py:obj:`~nameparser.config.Constants.nickname_delimiters` (default: the
three built-ins -- ``quoted_word``, ``double_quotes`` and ``parenthesis``,
each resolved live from :py:attr:`~nameparser.config.Constants.regexes`, so
overriding e.g. ``CONSTANTS.regexes.parenthesis`` still works exactly as
before) and :py:obj:`~nameparser.config.Constants.maiden_delimiters` (empty
by default -- see "Routing to Maiden Name" below).

To recognize an *additional* delimiter, add a compiled pattern to
``nickname_delimiters`` under any key, then re-run
:py:meth:`~nameparser.parser.HumanName.parse_full_name` to pick it up:

.. doctest::
Expand All @@ -73,11 +76,52 @@ default) under any key, then re-run
>>> hn = HumanName("Benjamin {Ben} Franklin", constants=None)
>>> hn.nickname
''
>>> hn.C.extra_nickname_delimiters['curly_braces'] = re.compile(r'\{(.*?)\}', re.U)
>>> hn.C.nickname_delimiters['curly_braces'] = re.compile(r'\{(.*?)\}', re.U)
>>> hn.parse_full_name()
>>> hn.nickname
'Ben'

Routing to Maiden Name
~~~~~~~~~~~~~~~~~~~~~~~

Parenthesized (or otherwise delimited) alternate/maiden surnames --
``"Baker (Johnson), Jenny"`` -- go to ``nickname`` by default, same as any
other delimited content. To route a delimiter to the first-class ``maiden``
field instead, move its key from ``nickname_delimiters`` to
``maiden_delimiters`` on a ``Constants`` instance (a plain ``dict.pop()`` +
assign -- this preserves the live link back to ``regexes`` for the three
built-ins) *before* parsing a name with it, the same way you'd configure
``patronymic_name_order`` or ``middle_name_as_last``:

.. doctest::

>>> from nameparser import HumanName
>>> from nameparser.config import Constants
>>> C = Constants()
>>> C.maiden_delimiters['parenthesis'] = C.nickname_delimiters.pop('parenthesis')
>>> hn = HumanName("Baker (Johnson), Jenny", constants=C)
>>> hn.first, hn.last, hn.maiden
('Jenny', 'Baker', 'Johnson')

This also strips the parenthesized maiden name from the no-comma written
form, since routing happens before positional parsing:

.. doctest::

>>> hn = HumanName("Jenny Baker (Johnson)", constants=C)
>>> hn.first, hn.last, hn.maiden
('Jenny', 'Baker', 'Johnson')

Routing an already-active built-in delimiter on an *existing* ``HumanName``
instance and calling ``parse_full_name()`` again will not work: only the
``full_name`` setter resets the working copy of the name string back to the
original input, so re-parsing in place has nothing left for the moved
delimiter to match if it already matched during the first parse. Configure
the ``Constants`` first, as above.

``maiden`` is not included in the default :py:obj:`~nameparser.config.Constants.string_format`,
so ``str(hn)`` is unaffected unless you add ``{maiden}`` to your own format.

Other editable attributes
~~~~~~~~~~~~~~~~~~~~~~~~~~

Expand Down Expand Up @@ -283,6 +327,7 @@ constant so that "Hon" can be parsed as a first name.
last: 'Solo'
suffix: ''
nickname: ''
maiden: ''
]>
>>> from nameparser.config import CONSTANTS
>>> CONSTANTS.titles.remove('hon')
Expand All @@ -296,6 +341,7 @@ constant so that "Hon" can be parsed as a first name.
last: 'Solo'
suffix: ''
nickname: ''
maiden: ''
]>


Expand Down Expand Up @@ -336,6 +382,7 @@ making them lower case and removing periods.
last: 'Johns'
suffix: ''
nickname: ''
maiden: ''
]>


Expand Down Expand Up @@ -363,6 +410,7 @@ the config on one instance could modify the behavior of another instance.
last: 'Johns'
suffix: ''
nickname: ''
maiden: ''
]>


Expand Down Expand Up @@ -390,6 +438,7 @@ reference to the module-level config values with the behavior described above.
last: 'Johns'
suffix: ''
nickname: ''
maiden: ''
]>
>>> other_instance.has_own_config
True
Expand Down Expand Up @@ -461,6 +510,7 @@ directly to the attribute.
last: 'Doe'
suffix: 'Md.'
nickname: ''
maiden: ''
]>


Expand Down
3 changes: 2 additions & 1 deletion docs/release_log.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,9 +7,10 @@ Release Log
to 1.2.1 first (which includes a one-version compatibility shim), load and
re-pickle under 1.2.1, then upgrade to 1.3.0.

- Add a first-class ``maiden`` field and ``maiden_delimiters`` to ``Constants``, so a delimiter (e.g. parenthesis) can be routed to ``maiden`` instead of ``nickname`` for alternate/maiden surnames, e.g. ``"Baker (Johnson), Jenny"`` (closes #22)
- Fix suffix-shaped parenthesized/quoted content (e.g. ``"(Ret)"``, ``"(MBA)"``) being misclassified as a nickname instead of a suffix (closes #111)
- Add ``suffix_acronyms_ambiguous`` to ``Constants`` for acronym suffixes that also read as given-name nicknames (e.g. ``"JD"``, ``"Ed"``), used when disambiguating parenthesized/quoted content (#111)
- Add ``extra_nickname_delimiters`` to ``Constants`` for registering additional nickname-delimiter regex patterns at runtime, without subclassing (closes #110, #112)
- Add ``nickname_delimiters`` to ``Constants`` for registering additional nickname-delimiter regex patterns at runtime, without subclassing (closes #110, #112)
- Fix missing comma between ``'msc'`` and ``'mscmsm'`` in ``suffix_acronyms``, which silently concatenated them into a bogus ``'mscmscmsm'`` entry (#111)
- Add ``given_names`` (and ``given_names_list``) attribute as aggregate of first and middle names, mirroring ``surnames`` (closes #157)
- Add ``suffix_delimiter`` to ``Constants`` and ``HumanName`` for parsing suffixes separated by arbitrary delimiters, e.g. ``"RN - CRNA"`` (#156)
Expand Down
12 changes: 9 additions & 3 deletions docs/usage.rst
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ Requires Python 3.10+.
last: 'Velasquez y Garcia'
suffix: 'Jr.'
nickname: ''
maiden: ''
]>
>>> name.middle = "Jason Alexander"
>>> name.middle
Expand All @@ -54,15 +55,16 @@ Requires Python 3.10+.
last: 'Velasquez y Garcia'
suffix: 'Jr.'
nickname: ''
maiden: ''
]>
>>> name.middle = ["custom","values"]
>>> name.middle
'custom values'
>>> name.full_name = 'Doe-Ray, Jonathan "John" A. Harris'
>>> name.as_dict()
{'last': 'Doe-Ray', 'suffix': '', 'title': '', 'middle': 'A. Harris', 'nickname': 'John', 'first': 'Jonathan'}
{'title': '', 'first': 'Jonathan', 'middle': 'A. Harris', 'last': 'Doe-Ray', 'suffix': '', 'nickname': 'John', 'maiden': ''}
>>> name.as_dict(False) # add False to hide keys with empty values
{'middle': 'A. Harris', 'nickname': 'John', 'last': 'Doe-Ray', 'first': 'Jonathan'}
{'first': 'Jonathan', 'middle': 'A. Harris', 'last': 'Doe-Ray', 'nickname': 'John'}
>>> name = HumanName("Dr. Juan Q. Xavier de la Vega III")
>>> name2 = HumanName("de la vega, dr. juan Q. xavier III")
>>> name == name2
Expand All @@ -71,7 +73,7 @@ Requires Python 3.10+.
5
>>> list(name)
['Dr.', 'Juan', 'Q. Xavier', 'de la Vega', 'III']
>>> name[1:-2]
>>> name[1:-3]
['Juan', 'Q. Xavier', 'de la Vega']


Expand Down Expand Up @@ -145,6 +147,7 @@ available from the nickname attribute.
last: 'Smith'
suffix: ''
nickname: 'John'
maiden: ''
]>

Exception: content that looks like a suffix (a member of
Expand All @@ -166,6 +169,7 @@ written in parenthesis.
last: 'Perkins'
suffix: 'MBA'
nickname: ''
maiden: ''
]>

A few suffix acronyms, listed in
Expand All @@ -186,6 +190,7 @@ reading in that ambiguous context:
last: 'BRICKEN'
suffix: ''
nickname: 'JD'
maiden: ''
]>

Leading Period-Abbreviation Titles
Expand All @@ -211,6 +216,7 @@ word appearing after the first name is left as a middle name.
last: 'Smith'
suffix: ''
nickname: ''
maiden: ''
]>
>>> name = HumanName("J. Smith")
>>> name.first
Expand Down
Loading