Duplicate Line Remover
Remove repeated lines while preserving the first occurrence. Your work stays on this device whenever possible.
How to use Duplicate Line Remover
- Paste one entry per line into "Your text".
- Set "Keep occurrence" to first (the default) or last; keeping the last copy retains the version that appeared latest.
- Choose what happens under "Empty lines": keep, remove, or collapse to a single empty line.
- Adjust "Case-sensitive comparison" (on by default), "Trim whitespace before comparison" and "Sort output alphabetically".
- Press "Remove duplicates" and read the bracketed count at the bottom of the result.
Worked example
Lines: apple, Banana, apple, banana. With the defaults the output is apple, Banana, banana plus [Duplicates removed: 1, Lines remaining: 3]. Unticking Case-sensitive leaves apple and Banana (2 removed). Unticked with Keep occurrence set to last, the result is apple then banana, because the lowercase banana is the final copy.
When to use this tool
- Merging two exported email lists and dropping repeated addresses.
- Shrinking a log excerpt where the same error line repeats dozens of times.
- Cleaning a keyword list where "Shoes" and "shoes" should count once.
Important limitations
Comparison evaluates complete lines between line breaks. Invisible trailing whitespace can make visually identical lines appear distinct unless whitespace removal is also applied.
Frequently asked questions
Does the order of my lines change?
Only if you tick "Sort output alphabetically". Otherwise surviving lines keep their relative order. The sort is natural, so item2 comes before item10. Turn the sort off when list order carries meaning, such as a ranking or a sequence of steps.
Is the bracketed line part of what I copy?
Yes. The result ends with a note such as [Duplicates removed: 1, Lines remaining: 3], or [No duplicate lines found]. Delete that note before pasting the list elsewhere.
Why did " apple" and "apple" both survive?
A leading space makes the lines different. Tick "Trim whitespace before comparison" to treat them as equal; the line that stays keeps its original spacing.
How are blank lines treated?
Blank lines are compared like any other line, so under "Keep all empty lines" only the first blank one survives. That label is misleading. Use "Remove empty lines" to drop them all, or accept the single survivor.
How does this differ from the sorter's duplicate option?
The Text Sorter's checkbox removes exact matches only. This tool can ignore case, ignore surrounding spaces and choose whether the first or last copy is kept.