Remove Duplicate Lines — Clean Up a List
Delete repeated lines and keep the first of each, in order
Runs in your browser · nothing is uploaded
What it is
This tool finds repeated lines in a list and keeps only the first occurrence of each. Paste email addresses, keywords, log lines or config values, one per line, and the cleaned list appears immediately with the number of lines before, kept and removed.
How to use
- Paste your lines into the input box, one item per line.
- Turn on options if you need them: Ignore case treats Apple and apple as the same, Ignore leading/trailing spaces compares lines without surrounding whitespace, and Remove empty lines deletes blank lines completely.
- The de-duplicated list shows in the result box.
- Press Copy to take the result.
How it works
- The text is split on line breaks (\n or \r\n) and read from top to bottom. A line already seen is skipped and a new line is added to the result.
- The comparison key is the line after Unicode NFC normalization. Depending on your options it is also trimmed or lowercased. The result always keeps the original line.
- Ignore case folds English letters; scripts without case such as Korean and Chinese are not affected.
- By default, blank lines are treated like any other line, so only one is kept. With Remove empty lines they all disappear, and lines containing only spaces or tabs count as empty too. The count of empty lines removed is shown separately from the duplicates.
- If the input uses Windows line endings (\r\n), the output does too.
- A single trailing line break is not counted as an extra empty line.
Examples
Input lines: apple, banana, apple, Apple, 사과, banana, 사과.
| Option | Result | Removed |
|---|---|---|
| Default | apple, banana, Apple, 사과 | 3 lines |
| Ignore case | apple, banana, 사과 | 4 lines |
FAQ
Which copy of a duplicate line is kept?
The first one. Later copies are removed, and the remaining lines keep their original order.
Does ignoring spaces also trim the output?
No. Spaces are ignored only when deciding whether two lines match. The kept line is left exactly as you typed it.
How are Korean lines compared?
Every line is Unicode-normalized (NFC) before comparing, so the same Hangul written with decomposed letters, which happens with some file names copied from macOS, matches the composed form.
Are my lines uploaded?
No. Everything runs in your browser.
