Preprocessing

Text preprocessing prepares strings for comparison by handling differences in casing, whitespace, punctuation, and accents. MatchForge provides normalizeString() to apply these transformations with configurable options.

Basic usage

Import normalizeString() from the package and pass the input string. The default options convert text to lowercase, trim its edges, and normalize consecutive whitespace.

typescript
import { normalizeString } from "@matchforge/core";

const result = normalizeString("  NEW   DELHI  ");

console.log(result);
// "new delhi"

Normalization options

Customize how input strings are normalized before comparison.

OptionDefaultDescription
lowercasetrueConverts text to lowercase.
trimtrueRemoves whitespace from the beginning and end.
normalizeWhitespacetrueReplaces consecutive whitespace with a single space.
removePunctuationfalseRemoves punctuation characters.
removeAccentsfalseRemoves accents from characters, e.g. José → Jose.

Configure normalization

Pass a configuration object to enable or disable individual transformations. The following example enables every available normalization option.

typescript
import { normalizeString } from "@matchforge/core";

const result = normalizeString("  José   SHARMA!  ", {
  lowercase: true,
  trim: true,
  normalizeWhitespace: true,
  removeAccents: true,
  removePunctuation: true,
});

console.log(result);
// "jose sharma"

Options can be disabled explicitly by setting them to false. Passing an empty object or omitting the options argument uses the default behavior.

Important considerations

Best practices

Keep these guidelines in mind when preparing strings for comparison.

Choose the right transformations

Removing punctuation or accents can change meaningful characters. Enable these options only when appropriate for your data.

01

Keep preprocessing consistent

Apply the same normalization settings to corresponding fields across your source and target datasets.

02

Separate normalization from comparison

Normalization prepares strings. Similarity algorithms calculate how closely the prepared strings match.

03