Remove Duplicate Lines from Text
Strip repeated lines and keep the first of each.
Removed 0 duplicate lines · 0 of 0 kept
Use via API
Remove repeated lines from text, keeping the first occurrence of each and the original order. Send JSON, get JSON back.
POST https://pithlab.com/api/v1/tools/remove-duplicate-lines
curl https://pithlab.com/api/v1/tools/remove-duplicate-lines \
-H "Authorization: Bearer $PITHLAB_KEY" \
-H "Content-Type: application/json" \
-d '{"input":"apple\nbanana\nApple\napple\ncherry","caseInsensitive":true}'About the Remove Duplicate Lines
Delete repeated lines from any list or text while keeping the first occurrence and original order. Optional case-insensitive and whitespace-tolerant matching.
How to use the Remove Duplicate Lines
- Paste your list or text into the input box, one item per line.
- Choose whether matching should ignore letter case and surrounding spaces.
- Tick Keep blank lines if you want empty lines left where they are.
- The result updates instantly and shows how many duplicates were removed. Copy it or download it as a text file.
Examples
Input
Apple banana apple BANANA cherry
Output
Apple banana cherry
Frequently asked questions
Which copy of a duplicate is kept?
The first occurrence is kept in its original position and every later copy is removed, so the order of your list does not change.
What does trimming before comparing do?
With it on, lines that differ only by leading or trailing spaces or tabs count as duplicates. The kept line is output exactly as it was written, including its spaces.
Does it treat accented characters consistently?
Yes. Lines are compared after Unicode normalisation, so an é typed as one character and an e followed by a combining accent are treated as the same.
Is there a size limit?
Only your device's memory. Lists with hundreds of thousands of lines are processed in a fraction of a second, and nothing is uploaded.