Preprocessing
Text preprocessing prepares strings for comparison by handling differences in casing, whitespace, punctuation, and accents. MatchForge provides normalizeString() to apply these transformations with configurable options.
Basic usage
Import normalizeString() from the package and pass the input string. The default options convert text to lowercase, trim its edges, and normalize consecutive whitespace.
import { normalizeString } from "@matchforge/core";
const result = normalizeString(" NEW DELHI ");
console.log(result);
// "new delhi"Normalization options
Customize how input strings are normalized before comparison.
| Option | Default | Description |
|---|---|---|
| lowercase | true | Converts text to lowercase. |
| trim | true | Removes whitespace from the beginning and end. |
| normalizeWhitespace | true | Replaces consecutive whitespace with a single space. |
| removePunctuation | false | Removes punctuation characters. |
| removeAccents | false | Removes accents from characters, e.g. José → Jose. |
Configure normalization
Pass a configuration object to enable or disable individual transformations. The following example enables every available normalization option.
import { normalizeString } from "@matchforge/core";
const result = normalizeString(" José SHARMA! ", {
lowercase: true,
trim: true,
normalizeWhitespace: true,
removeAccents: true,
removePunctuation: true,
});
console.log(result);
// "jose sharma"Options can be disabled explicitly by setting them to false. Passing an empty object or omitting the options argument uses the default behavior.
Important considerations
Best practicesKeep these guidelines in mind when preparing strings for comparison.
Choose the right transformations
Removing punctuation or accents can change meaningful characters. Enable these options only when appropriate for your data.
Keep preprocessing consistent
Apply the same normalization settings to corresponding fields across your source and target datasets.
Separate normalization from comparison
Normalization prepares strings. Similarity algorithms calculate how closely the prepared strings match.