Reconciliation¶
Reconciliation .add files define aliases and groups for add.scan('@diff'). They let you normalize variant spellings before comparison and detect changes at different levels of a column hierarchy.
Why Reconciliation?¶
Real-world data is messy. The same value appears in different forms:
"Active","ACTIVE","active"— same meaning, different case"North America","NA","N. America"— same region, different abbreviations
Without reconciliation, the diff engine flags these as changes. With reconciliation, they're treated as equivalent.
Reconciliation .add File Format¶
A reconciliation .add file uses TOML format with two sections: [aliases] and [groups].
[aliases]
status = ["Active", "ACTIVE", "active", "Inactive", "INACTIVE", "inactive"]
region = ["NA", "North America", "N. America", "EU", "Europe", "EMEA"]
[groups]
location = ["country", "state", "city"]
product = ["category", "subcategory", "sku"]
Aliases¶
Aliases map variant spellings to a canonical form. During diff comparison, all variants are normalized before checking for changes.
Inline Aliases¶
Pass aliases directly in the strategy dict:
import additory as add
diff = add.scan('@diff', old=df_old, new=df_new,
strategy={
'aliases': {
'status': ['Active', 'ACTIVE', 'active'],
'region': ['NA', 'North America'],
},
},
)
The first value in each list becomes the canonical form. So "ACTIVE" and "active" are both normalized to "Active" before comparison.
Aliases from a File¶
Register a reconciliation file, then reference it by name:
# Load reconciliation definitions
add.scan('@set', './reconciliation/')
# Use by name
diff = add.scan('@diff', old=df_old, new=df_new,
strategy={'aliases': 'my_reconciliation'},
)
How Aliases Work¶
- Before comparing rows, the diff engine applies alias normalization
- Each column listed in
[aliases]has its values mapped to the canonical form - The comparison runs on the normalized values
- The output shows the original values (not the normalized ones)
This means a row where old.status = "ACTIVE" and new.status = "active" is reported as no_change rather than modified.
Groups¶
Groups define column hierarchies for structured change detection. They help answer questions like: "Did the country change, or just the city?"
Group Definition¶
This defines a hierarchy: country → state → city. When the diff engine detects a change in any of these columns, it reports which level of the hierarchy changed.
Using Groups¶
How Groups Work¶
- The diff engine checks each group's columns in hierarchy order
- If a higher-level column changed (e.g.,
country), lower-level changes are expected - If only a lower-level column changed (e.g.,
city), it's flagged as a minor change - This gives you structured visibility into what actually changed
Complete Example¶
The reconciliation file¶
[aliases]
diagnosis = [
"Hypertension", "HTN", "High Blood Pressure",
"Diabetes", "DM", "Diabetes Mellitus",
"Asthma", "Bronchial Asthma",
]
status = ["Active", "ACTIVE", "active", "Discharged", "DISCHARGED"]
[groups]
location = ["hospital", "ward", "bed"]
treatment = ["department", "procedure", "medication"]
Using it¶
import additory as add
import polars as pl
# Load reconciliation
add.scan('@set', './reconciliation/')
old_patients = pl.DataFrame({
'patient_id': [1, 2, 3],
'diagnosis': ['HTN', 'DM', 'Asthma'],
'status': ['ACTIVE', 'active', 'Active'],
})
new_patients = pl.DataFrame({
'patient_id': [1, 2, 3],
'diagnosis': ['Hypertension', 'Diabetes', 'Bronchial Asthma'],
'status': ['Active', 'Active', 'Discharged'],
})
# Without reconciliation: 3 rows show as modified
diff_raw = add.scan('@diff', old=old_patients, new=new_patients)
# With reconciliation: only patient 3 shows as modified (status changed)
diff_reconciled = add.scan('@diff', old=old_patients, new=new_patients,
strategy={'aliases': 'clinical_reconciliation'},
)
Next Steps¶
- add.scan('@diff') — full diff documentation
- Expression Files — the
.addfile format for expressions - Troubleshooting — common issues