Remove Duplicate Lines, Sort and Clean a List

Remove duplicates and empty lines, trim spaces and sort a list in one pass.

How to use

What it checks and how it works

The steps run in a fixed order: trim and collapse spaces, drop empty lines, remove duplicates (keeping the earliest copy), sort, reverse, then number. Sorting uses Intl.Collator, the browser's language-aware comparison, so accented and Arabic letters sort by the rules of the language you pick. Natural sort treats digit runs as numbers, so item 2 comes before item 10. Two lines count as duplicates only when they match after the cleaning options you ticked, and sorting rules differ a little between browsers. The tool works on whole lines: it does not look for repeated words inside a line.

Which copy of a duplicate is kept?

The earliest one. If you also sort, the kept lines are sorted afterwards.

Can it ignore upper and lower case when finding duplicates?

Yes. Case-insensitive comparison is on by default, so Cairo and cairo count as one line, and the kept line keeps its own spelling.

Does the sort work for Arabic?

Yes. Set the sorting language to Arabic and the lines follow the Arabic alphabet through the browser's Intl.Collator.

Sources

Related tools

Privacy

Runs in your browser; nothing is uploaded. The interactive tool needs JavaScript.