Styxis
Company
Knowledge

Duplicate Polars Map keys: when lookup and dictionary conversion choose different values

Define key uniqueness within each parent row and an explicit duplicate policy before consumers choose different Map operations.

Define the unit of duplication before selecting a value

Establish duplicate-key and winner-selection policy within a Map row before converting it to a dictionary. Reusing a key across different parents is normal. Multiple values for the same key within one parent can instead make the selected operation determine the result. [1, 2, 3]

A report with different answers from the same input

Polars issue 30595 demonstrates repeated keys retained in an Arrow-imported Map, with get and Python-object conversion choosing different values. It also compares entry counts and reconstruction. This concerns the reported ingestion path, not a claim that a Python dictionary inherently preserves multiple values for one key. [1]

Hand over the parent and key together

Flatten Map entries while retaining parent identity and check uniqueness of the parent-key combination. Do not leave first-value or last-value selection to implicit conversion. If the application allows repetition, retain entry order and create a separate result under an approved merge policy. [1, 3]

Where an expected count of 1 applies

Define an illustrative rule that key x in parent P has one value. A P-x combination appearing 1 time passes; appearing 2 times fails uniqueness even if values match. Key x in another parent Q is not a duplicate of P's entry. This fixture defines which inputs need merge review; it is not an executed order-based merge. [1, 3]

Truthound checks the flattened entry table

Truthound Core's MultiColumnUniqueValidator can check parent ID and Map key together. Prepare the entry table first rather than assuming automatic interpretation of the original Map. The check does not change the winner selected by get or Python conversion. Keep intentional repetitions as entries and apply an explicit winner policy before delivering them to consumers. [1, 2, 3]

Application conditions

Flatten parent_id, map_key, entry_position and entry_value for each entry of the same source Map, defining duplicate policy within each parent first.

Alternatives

When repeated keys are allowed, retain the Map as an entry list and select the winner explicitly.

Limitations

  • Truthound row-uniqueness checks do not change Map get or Python-conversion semantics.

Evidence

Event checked
Subject checked
Modified