Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,7 +76,7 @@ Each module defines a plain Python set of known name pieces:
- `titles.py` — `TITLES` (prenominals) and `FIRST_NAME_TITLES` (e.g. "Sir", which treat the following name as first, not last)
- `suffixes.py` — `SUFFIX_ACRONYMS` (with periods, e.g. "M.D.") and `SUFFIX_NOT_ACRONYMS` (e.g. "Jr.")
- `prefixes.py` — `PREFIXES` (lastname particles, e.g. "de", "van")
- `first_name_prefixes.py` — `FIRST_NAME_PREFIXES` (bound given-name prefixes, e.g. "abdul", "abu"); `_join_first_name_prefix` joins the first non-title piece to its following piece before the main assignment loop
- `bound_first_names.py` — `BOUND_FIRST_NAMES` (bound given-name prefixes, e.g. "abdul", "abu"); `_join_bound_first_name` joins the first non-title piece to its following piece before the main assignment loop
- `conjunctions.py` — `CONJUNCTIONS` (e.g. "and", "of") used to chain multi-word titles
- `capitalization.py` — `CAPITALIZATION_EXCEPTIONS` mapping (e.g. `{'phd': 'Ph.D.'}`)
- `regexes.py` — compiled regular expressions wrapped in a `TupleManager`
Expand All @@ -97,7 +97,7 @@ Parse flow:
3. `parse_pieces()` — splits on spaces, detects dotted abbreviations like "Lt.Gov." and adds them to constants dynamically
4. `join_on_conjunctions()` — merges pieces adjacent to conjunctions into single tokens (e.g. `['Secretary', 'of', 'State']` → `['Secretary of State']`); also joins prefix particles to the following lastname token
— a piece merged with a title *or prefix* neighbor is re-registered into that constant set so later steps still recognize it, e.g. `von`+`und`+`zu` → registered as a prefix so the whole phrase joins to the last name (German "von und zu"; PR #191)
4a. `_join_first_name_prefix()` — called immediately after step 4 in both the no-comma and lastname-comma paths; merges bound given-name prefixes (e.g. "abdul") with the next piece before the assignment loop runs; suffixes are still in `pieces` at this point, so the `reserve_last` guard must count non-suffix pieces only
4a. `_join_bound_first_name()` — called immediately after step 4 in both the no-comma and lastname-comma paths; merges bound given-name prefixes (e.g. "abdul") with the next piece before the assignment loop runs; suffixes are still in `pieces` at this point, so the `reserve_last` guard must count non-suffix pieces only
5. Iterates pieces, assigning to `title_list`, `first_list`, `middle_list`, `last_list`, `suffix_list`
6. `post_process()` — `handle_firstnames()` swaps first/last when only a title + one name; then, gated on `patronymic_name_order`, `handle_east_slavic_patronymic_name_order()` and `handle_turkic_patronymic_name_order()` reorder Russian-formal-order and reversed Turkic patronymics; then, gated on `middle_name_as_last`, `handle_middle_name_as_last()` folds `middle_list` into `last_list`; finally `handle_capitalization()` applies optional auto-cap. Any new `self._attr` used by `post_process()` helpers must be initialized in `__init__` (with its default value) — the direct-kwargs path bypasses `parse_full_name()`, so the attribute won't exist otherwise.

Expand All @@ -116,7 +116,7 @@ Each named attribute (`title`, `first`, etc.) is a `@property` that joins its co

Add a dedicated `copy.deepcopy()` round-trip test for it too (see `test_regexes_deepcopy_roundtrip`/`test_nickname_delimiters_deepcopy_roundtrip` in `tests/test_constants.py`), not just reliance on conftest's autouse snapshot/restore exercising it incidentally. `TupleManager`/`RegexTupleManager.__getattr__` answer *any* unknown attribute lookup — including dunder probes like `__deepcopy__` — so a new manager subtype or a `__getattr__` tweak can silently break `copy.deepcopy` (this bit `RegexTupleManager` before the dunder-lookup guard was added). A direct test on the new attribute's own manager instance catches that where the conftest fixture, which never asserts on the copy, would not. As with the scalar-attribute pattern above, also check `AGENTS.md` itself for now-stale references when you touch this.

**Adding a word to a config set** — first check the *other* sets for the same word (grep `nameparser/config/` or intersect the sets in a `python3 -c`). Real overlaps exist: `do`/`st`/`mc` ∈ `PREFIXES` ∩ `TITLES`/`SUFFIX_ACRONYMS`; `abd` = "ABD" ∈ `SUFFIX_ACRONYMS`; `abu` ∈ `PREFIXES` ∩ `first_name_prefixes` (position-dependent: leading token → first-name join, mid-name → last-name join). Usually position-dependent and harmless, but can force a guard or an exclusion (the `last_base` all-particles guard; dropping `abd` from `first_name_prefixes`).
**Adding a word to a config set** — first check the *other* sets for the same word (grep `nameparser/config/` or intersect the sets in a `python3 -c`). Real overlaps exist: `do`/`st`/`mc` ∈ `PREFIXES` ∩ `TITLES`/`SUFFIX_ACRONYMS`; `abd` = "ABD" ∈ `SUFFIX_ACRONYMS`; `abu` ∈ `PREFIXES` ∩ `bound_first_names` (position-dependent: leading token → first-name join, mid-name → last-name join). Usually position-dependent and harmless, but can force a guard or an exclusion (the `last_base` all-particles guard; dropping `abd` from `bound_first_names`).

**Before adding a short/common word to `PREFIXES` globally**, test it mid-string against realistic 3-token names, not just check for English-word collisions: Korean/Vietnamese given names put a short syllable in the middle slot (`Park In Hwan`, `Nguyen To Nga`), and Western names put a bare initial there (`John V. Smith`). A word that looks safe ("nobody is named 'to'") can still swallow a real middle name/initial into the last name once it's a global prefix — confirmed regressions for `to`/`in`/`an`/`ten`/`then` and bare `v` this way (PR #191).

Expand Down Expand Up @@ -144,7 +144,7 @@ Add a dedicated `copy.deepcopy()` round-trip test for it too (see `test_regexes_

**`TupleManager.__setattr__`/`__delattr__` guard dunder names too, not just `__getattr__`** — constructing a subscripted generic, e.g. `TupleManager[re.Pattern[str] | str]({...})` (needed so mypy sees the right value type instead of inferring one from the dict literal), makes `typing`'s `GenericAlias.__call__` set `__orig_class__` on the new instance right after `__init__` returns. Before this guard existed, `__setattr__` was a bare `dict.__setitem__` alias, so that assignment silently inserted a bogus `'__orig_class__'` entry into the dict itself, corrupting `.values()`/iteration for *every* `TupleManager`/`RegexTupleManager` instance, not just the one being constructed — this bit `nickname_delimiters`'s construction (#22) before the guard was added. Same fix shape as the `__getattr__` dunder guard above: fall back to `object.__setattr__`/`object.__delattr__` for dunder names, dict-backed storage for everything else.

**`_join_first_name_prefix` guard must exclude trailing suffixes** — suffix tokens are still in `pieces` when the helper runs (suffix detection happens in the assignment loop, later). The `reserve_last` guard must count `if not self.is_suffix(p)` to avoid treating a trailing suffix like "Jr." as a last-name slot; otherwise `"abdul salam jr"` → `last='jr'`.
**`_join_bound_first_name` guard must exclude trailing suffixes** — suffix tokens are still in `pieces` when the helper runs (suffix detection happens in the assignment loop, later). The `reserve_last` guard must count `if not self.is_suffix(p)` to avoid treating a trailing suffix like "Jr." as a last-name slot; otherwise `"abdul salam jr"` → `last='jr'`.

**Doctests** — docstring examples in `nameparser/*.py` run under `uv run pytest` (`--doctest-modules`; `testpaths` is `tests` + `nameparser` only). The `.rst` doctests in `docs/` (`usage.rst`, `customize.rst`) are **not** run by pytest or CI (CI does `sphinx-build -b html`, not `-b doctest`), so verify `.rst` examples manually: `python3 -c "import doctest; print(doctest.testfile('docs/usage.rst', module_relative=False, optionflags=doctest.NORMALIZE_WHITESPACE))"`. Note `customize.rst` has pre-existing failures under `-b doctest` (CONSTANTS state leaks across examples — no per-example reset like `tests/conftest.py` provides — plus non-deterministic `SetManager` repr).

Expand Down
16 changes: 8 additions & 8 deletions docs/customize.rst
Original file line number Diff line number Diff line change
Expand Up @@ -260,10 +260,10 @@ a secondary key::

sorted_names = sorted(names, key=lambda n: (n.last_base.lower(), n.first.lower()))

First-Name Prefixes
-------------------
Bound First Names
------------------

``CONSTANTS.first_name_prefixes`` controls bound given-name prefixes that attach
``CONSTANTS.bound_first_names`` controls bound given-name prefixes that attach
to the following word to form one first name. By default it contains
``{'abdul', 'abdel', 'abdal', 'abu', 'abou', 'umm'}``.

Expand All @@ -277,20 +277,20 @@ Example::
To **disable** the feature entirely::

>>> from nameparser.config import CONSTANTS
>>> CONSTANTS.first_name_prefixes.clear()
>>> CONSTANTS.bound_first_names.clear()

To **add** a word (e.g. if your data uses ``mohamad`` as a bound prefix)::

>>> CONSTANTS.first_name_prefixes.add('mohamad')
>>> CONSTANTS.bound_first_names.add('mohamad')

To **remove** a single entry::

>>> CONSTANTS.first_name_prefixes.remove('umm')
>>> CONSTANTS.bound_first_names.remove('umm')

You can also pass a custom set per ``Constants`` instance::

>>> from nameparser.config import Constants
>>> c = Constants(first_name_prefixes={'abu', 'umm'})
>>> c = Constants(bound_first_names={'abu', 'umm'})
>>> hn2 = HumanName("abu bakr al saud", constants=c)
>>> hn2.first, hn2.last
('abu bakr', 'al saud')
Expand All @@ -310,7 +310,7 @@ Example::
('', 'de Mesnil')

A member must be a prefix that is never a given name in any culture, and the set
must stay **disjoint** from ``first_name_prefixes`` (a word cannot both join to
must stay **disjoint** from ``bound_first_names`` (a word cannot both join to
the first name and never be a first name). Ambiguous particles that *can* be
given names (``van``, ``von``, ``della``, ``di``, ``del``, ...) are intentionally
excluded; add them yourself if your data warrants it::
Expand Down
4 changes: 2 additions & 2 deletions docs/release_log.rst
Original file line number Diff line number Diff line change
Expand Up @@ -29,11 +29,11 @@ Release Log
- Fix suffix boundary lookup for prefixed last names with a title before and after (e.g. ``"dr Vincent van Gogh dr"`` producing a corrupted middle name) (closes #100)
- Add ``patronymic_name_order`` flag to ``Constants`` and ``HumanName`` for opt-in detection and reordering of Russian formal-order names (Surname GivenName Patronymic) (#85)
- Add Turkic (Azerbaijani/Central-Asian) patronymic detection to ``patronymic_name_order``, rotating the reversed 4-token formal shape (``Surname GivenName PatronymicRoot Marker``, e.g. ``oglu``/``qizi``) into Western order (#185)
- Add ``first_name_prefixes`` set to ``Constants``; bound Arabic given-name
- Add ``bound_first_names`` set to ``Constants``; bound Arabic given-name
prefixes (``abdul``, ``abu``, etc.) now join forward to form a single first
name (e.g. ``"abdul salam ahmed salem"`` → ``first="abdul salam"``,
``middle="ahmed"``, ``last="salem"``). Disable via
``CONSTANTS.first_name_prefixes.clear()``. **Default-on: changes parsing
``CONSTANTS.bound_first_names.clear()``. **Default-on: changes parsing
output for names with these prefixes.** (#150)
- Add ``middle_name_as_last`` flag to ``Constants`` and ``HumanName`` for opt-in folding of middle names into the last name, for naming systems with no middle-name concept (e.g. Arabic patronymic chaining) (#133)
- Treat an unrecognized, multi-letter token ending in a period in the leading title run (before the first name is set), e.g. ``"Major."``, as a ``title`` instead of a ``first`` name; internal-period abbreviations (``"E.T."``) and single-letter initials (``"J."``) are unaffected. **Default-on: changes parsing of names with a leading unknown period-abbreviation** (closes #109)
Expand Down
14 changes: 7 additions & 7 deletions nameparser/config/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@

from nameparser.util import lc
from nameparser.config.prefixes import PREFIXES, NON_FIRST_NAME_PREFIXES
from nameparser.config.first_name_prefixes import FIRST_NAME_PREFIXES
from nameparser.config.bound_first_names import BOUND_FIRST_NAMES
from nameparser.config.capitalization import CAPITALIZATION_EXCEPTIONS
from nameparser.config.conjunctions import CONJUNCTIONS
from nameparser.config.suffixes import SUFFIX_ACRONYMS
Expand Down Expand Up @@ -262,13 +262,13 @@ class Constants:
:py:attr:`~suffixes.SUFFIX_ACRONYMS_AMBIGUOUS` wrapped with :py:class:`SetManager`.
:param set conjunctions:
:py:attr:`conjunctions` wrapped with :py:class:`SetManager`.
:param set first_name_prefixes:
:py:attr:`~first_name_prefixes.FIRST_NAME_PREFIXES` wrapped with :py:class:`SetManager`.
:param set bound_first_names:
:py:attr:`~bound_first_names.BOUND_FIRST_NAMES` wrapped with :py:class:`SetManager`.
:param set non_first_name_prefixes:
:py:attr:`~prefixes.NON_FIRST_NAME_PREFIXES` wrapped with :py:class:`SetManager`.
The subset of prefixes that are never a first name, so a *leading* one
marks the whole name as a surname. Must stay disjoint from
``first_name_prefixes``.
``bound_first_names``.
:type capitalization_exceptions: tuple or dict
:param capitalization_exceptions:
:py:attr:`~capitalization.CAPITALIZATION_EXCEPTIONS` wrapped with :py:class:`TupleManager`.
Expand All @@ -293,7 +293,7 @@ class Constants:
titles = _CachedUnionMember()
first_name_titles: SetManager
conjunctions: SetManager
first_name_prefixes: SetManager
bound_first_names: SetManager
non_first_name_prefixes: SetManager
suffix_acronyms_ambiguous: SetManager
capitalization_exceptions: TupleManager[str]
Expand Down Expand Up @@ -468,7 +468,7 @@ def __init__(self,
titles: Iterable[str] = TITLES,
first_name_titles: Iterable[str] = FIRST_NAME_TITLES,
conjunctions: Iterable[str] = CONJUNCTIONS,
first_name_prefixes: Iterable[str] = FIRST_NAME_PREFIXES,
bound_first_names: Iterable[str] = BOUND_FIRST_NAMES,
non_first_name_prefixes: Iterable[str] = NON_FIRST_NAME_PREFIXES,
capitalization_exceptions: TupleManager[str] | Iterable[tuple[str, str]] = CAPITALIZATION_EXCEPTIONS,
regexes: RegexTupleManager | TupleManager[re.Pattern[str]] | Iterable[tuple[str, re.Pattern[str]]] = REGEXES,
Expand All @@ -484,7 +484,7 @@ def __init__(self,
self.titles = SetManager(titles)
self.first_name_titles = SetManager(first_name_titles)
self.conjunctions = SetManager(conjunctions)
self.first_name_prefixes = SetManager(first_name_prefixes)
self.bound_first_names = SetManager(bound_first_names)
self.non_first_name_prefixes = SetManager(non_first_name_prefixes)
self.suffix_acronyms_ambiguous = SetManager(suffix_acronyms_ambiguous)
self.capitalization_exceptions = TupleManager(capitalization_exceptions)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
#: one first name (e.g. "abdul salam" → first name "abdul salam"). They are
#: never standalone names. Join logic runs in the given-name region only,
#: mirroring :py:data:`~nameparser.config.prefixes.PREFIXES` for last names.
FIRST_NAME_PREFIXES: set[str] = {
BOUND_FIRST_NAMES: set[str] = {
'abdul',
'abdel',
'abdal',
Expand Down
8 changes: 4 additions & 4 deletions nameparser/config/prefixes.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
from nameparser.config.first_name_prefixes import FIRST_NAME_PREFIXES
from nameparser.config.bound_first_names import BOUND_FIRST_NAMES

#: The sub-set of :py:data:`PREFIXES` that are *never* a standalone first name.
#: A name that *starts* with one of these has no first name -- the whole thing
Expand All @@ -8,7 +8,7 @@
#: name prefix (`abu`). When unsure, leave a word out: a missing member just
#: means that name is not auto-fixed, whereas a wrong member misparses a real
#: person. Must stay a subset of :py:data:`PREFIXES` and disjoint from
#: :py:data:`~nameparser.config.first_name_prefixes.FIRST_NAME_PREFIXES`.
#: :py:data:`~nameparser.config.bound_first_names.BOUND_FIRST_NAMES`.
NON_FIRST_NAME_PREFIXES = set([
"'t",
'af',
Expand Down Expand Up @@ -93,5 +93,5 @@
# happens to catch it.
assert NON_FIRST_NAME_PREFIXES <= PREFIXES, \
"NON_FIRST_NAME_PREFIXES must stay a subset of PREFIXES"
assert not (NON_FIRST_NAME_PREFIXES & FIRST_NAME_PREFIXES), \
"NON_FIRST_NAME_PREFIXES must stay disjoint from FIRST_NAME_PREFIXES"
assert not (NON_FIRST_NAME_PREFIXES & BOUND_FIRST_NAMES), \
"NON_FIRST_NAME_PREFIXES must stay disjoint from BOUND_FIRST_NAMES"
16 changes: 8 additions & 8 deletions nameparser/parser.py
Original file line number Diff line number Diff line change
Expand Up @@ -595,26 +595,26 @@ def is_prefix(self, piece: str) -> bool:
else:
return lc(piece) in self.C.prefixes

def is_first_name_prefix(self, piece: str) -> bool:
"""Lowercased, leading/trailing-periods-stripped version of piece is in :py:attr:`~nameparser.config.Constants.first_name_prefixes`."""
return lc(piece) in self.C.first_name_prefixes
def is_bound_first_name(self, piece: str) -> bool:
"""Lowercased, leading/trailing-periods-stripped version of piece is in :py:attr:`~nameparser.config.Constants.bound_first_names`."""
return lc(piece) in self.C.bound_first_names

def is_non_first_name_prefix(self, piece: str) -> bool:
"""Lowercased, leading/trailing-periods-stripped version of piece is in
:py:attr:`~nameparser.config.Constants.non_first_name_prefixes`."""
return lc(piece) in self.C.non_first_name_prefixes

def _join_first_name_prefix(self, pieces: list[str], reserve_last: bool) -> list[str]:
def _join_bound_first_name(self, pieces: list[str], reserve_last: bool) -> list[str]:
"""Join a first-name prefix to its following piece.

Finds the first non-title piece; if it is in ``first_name_prefixes``,
Finds the first non-title piece; if it is in ``bound_first_names``,
merges it with the next piece — unless ``reserve_last`` is True and no
further piece would remain for the last name.
"""
fi = next((i for i, p in enumerate(pieces) if not self.is_title(p)), None)
if fi is None:
return pieces
if not self.is_first_name_prefix(pieces[fi]):
if not self.is_bound_first_name(pieces[fi]):
return pieces
next_i = fi + 1
if next_i >= len(pieces):
Expand Down Expand Up @@ -1038,7 +1038,7 @@ def parse_full_name(self) -> None:
# part[0]

pieces = self.parse_pieces(parts)
pieces = self._join_first_name_prefix(pieces, reserve_last=True)
pieces = self._join_bound_first_name(pieces, reserve_last=True)
p_len = len(pieces)
for i, piece in enumerate(pieces):
try:
Expand Down Expand Up @@ -1121,7 +1121,7 @@ def parse_full_name(self) -> None:
# parts[0], parts[1], parts[2:...]

log.debug("post-comma pieces: %s", str(post_comma_pieces))
post_comma_pieces = self._join_first_name_prefix(post_comma_pieces, reserve_last=False)
post_comma_pieces = self._join_bound_first_name(post_comma_pieces, reserve_last=False)

# lastname part may have suffixes in it
lastname_pieces = self.parse_pieces(parts[0].split(' '), 1)
Expand Down
2 changes: 1 addition & 1 deletion tests/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@
"titles",
"first_name_titles",
"conjunctions",
"first_name_prefixes",
"bound_first_names",
"non_first_name_prefixes",
"capitalization_exceptions",
"regexes",
Expand Down
Loading