Clean a list without changing its sequence
This tool removes repeated lines from text and keeps the first occurrence of each distinct entry. Paste one item per line, choose how entries should be compared and press Remove duplicates. The result stays in the order of the first appearances. It is not automatically sorted, and an entry does not have to sit next to its duplicate for the tool to find it.
That makes it useful for a reading list assembled from several notes, repeated labels copied from a document, or a set of reference codes. The unit of comparison is the whole line. Two lines match only when their complete comparison values match after the selected options are applied. A repeated word inside a sentence is not removed, and two sentences about the same subject are not considered equivalent.
Before starting, decide what counts as the same item in your list. Is Apple the same as apple? Does an extra space around a code matter? Should blank lines separate groups, or should they disappear? Those decisions are more important than the act of deleting repeats. An option that is helpful for ordinary labels may merge entries that need to remain distinct in a technical dataset.
Set the three comparison options deliberately
Ignore letter case
With Ignore letter case off, different capitalization can keep entries separate. Apple and apple are different comparison values. Turn the option on to compare their lowercase forms using the selected page language. The first retained entry keeps its own capitalization: ignoring case does not mean that every output line becomes lowercase.
For example, when Apple appears before apple, the retained spelling is Apple. If apple appears first, that is the spelling kept. This is useful when the source order already expresses which form you prefer. It is less suitable when the first entry is not trustworthy or when you need a particular standardized spelling, because the tool does not choose the best-written version.
Trim spaces at line edges
Trimming removes whitespace at the beginning and end of each line before comparison. It also removes those edge spaces from the retained output. This is more than an instruction to ignore spaces while matching. If a line begins with padding you need to preserve, leave the option off or keep a copy of that formatting elsewhere.
The option does not collapse spaces inside a line. New York and New York, with two spaces between the words in the second version, can remain distinct. It also does not remove punctuation or accents. Use it to clean accidental padding around entries, not as a general promise that all visually similar text will be treated as identical.
Remove blank lines
Remove blank lines is initially enabled. It excludes both empty lines and lines containing only whitespace, regardless of whether trimming is enabled. This keeps copied lists from carrying accidental gaps into the result. The summary counts these removals separately from duplicate entries, so a reduced line count does not necessarily mean every removed line contained repeated text.
When you disable this option, blank lines participate in comparison like other entries. Several exactly empty lines collapse to one. Lines containing different amounts of whitespace may remain distinct if trimming is off. If trimming is on, those whitespace-only lines become empty and compare equally. Keeping blank lines therefore does not mean preserving every original separator or group layout.
Three examples with clear comparison rules
Keep the first spelling and position
Paste the following list, leave trimming off and keep blank-line removal enabled:
Apple
Pear
apple
Apple
With Ignore letter case off, the output is Apple, Pear and apple on three separate lines. One duplicate is removed. With Ignore letter case on, the output is Apple and Pear. Two duplicates are removed. In both cases, Apple remains before Pear because their first appearances were in that order; the operation does not alphabetize the result.
This example also explains what “keep one” means. An entry that appears more than once is not removed entirely. Its first occurrence survives. If your goal is to retain only items that appeared exactly once, or to show only repeated items, this is not that filter. The tool returns one retained line for each comparison value it encounters.
Remove padding, but not internal spaces
Consider three lines: the word blue surrounded by a space on each side, blue without padding, and blue sky. With trimming off, all three remain distinct. With trimming on, the first two compare equally and the result contains blue followed by blue sky. The retained blue has no surrounding spaces because trimming changes the output too.
Now compare blue sky with blue sky, where the second line has two internal spaces. Trimming does not make those entries equal. This distinction is useful for diagnosing a list that still looks repetitive after cleaning. Inspect the characters within the line, not only its beginning and end. The same principle applies to punctuation, added labels and different reference numbers.
Understand matching accented text
Unicode allows some accented text to be represented in more than one way. A visible é may be stored as one composed character or as an e followed by a combining accent. Before comparison, this tool uses NFC normalization so these canonically equivalent representations can match. Two lines displaying café in those forms therefore collapse to the first occurrence.
That does not make café equal to cafe. NFC is not accent removal, spelling correction or a general similarity search. The retained text comes from the first line, after optional edge trimming; normalization is used for the comparison key. This matters when text arrives from different editors, but it does not justify merging names that merely look or sound alike.
Read the summary and check what disappeared
After processing, the summary reports lines kept and lines removed, with removed lines divided into duplicates and blank lines. Read those categories together. If ten lines disappear and nine were empty, the operation has not found ten repeated records. Conversely, a list with no empty lines can still shrink substantially when the same entry appears many times.
A final line break can introduce an empty line at the end of the input. With blank-line removal enabled, that line contributes to the blank count. This can explain a summary that differs by one from a quick visual count. The tool recognizes common newline forms, then joins retained entries with line breaks in the result.
Review several retained entries, including one near the beginning and one near the end. When case-insensitive comparison is enabled, check which spelling won by appearing first. When trimming is enabled, inspect an indented line to confirm that losing its padding is acceptable. For important lists, compare the original and cleaned versions before replacing your working copy.
Changing the input or an option clears the previous result. Press Remove duplicates again to generate a result for the new settings. This prevents an old output from being mistaken for a new comparison. Copy result copies the cleaned text; it does not include the summary counts as part of the list.
Prepare the right kind of input
Use one item per real line break. A comma-separated paragraph is a single line, not a list of separately comparable words. A long entry that wraps visually in the input box remains one item unless it contains an actual newline. Before processing a pasted list, check whether its separators are line breaks, commas, semicolons or tabs.
Rows copied from a spreadsheet can contain several tab-separated cells. Here, the entire pasted row is compared as one text entry. The tool does not deduplicate by a chosen column, match records by an ID field, or merge the remaining fields of two records. If two rows share a name but have different notes, they are different complete lines.
Do not assume repeated names represent the same person. Two people may share a name, and one person may appear under several spellings. For a participant list, retain an appropriate distinguishing code when you need to tell entries apart. The comparison is a text rule, not an identity check, and it cannot decide which business records should be merged.
Preserve the source before changing it. Removal is intentionally one-way: the result does not include a separate list of discarded entries or a full edit history. The original stays in the input box during normal processing, but Clear removes the current input and result. Keep a separate source copy when you need to review or explain the cleaning decisions later.
Finish the list without mixing separate tasks
The form supports up to 200,000 characters per operation and asks for text when the input is empty or contains only whitespace. If a large source exceeds that limit, take care with batching: cleaning each batch separately does not remove duplicates that occur in different batches. You would need a final combined comparison within the limit, or a tool designed for the larger dataset.
The result field is read-only. To adjust the rules, change the options and run again; to edit individual retained entries, copy the result into your editor. If the copy button cannot access the clipboard, select and copy the output manually. Check the pasted line breaks in the destination before treating the transfer as complete.
If the cleaned entries also need a sequence, continue with Alphabetical order. Sorting is deliberately separate from keeping first occurrences. For a randomized activity order, use Randomize a list after deciding whether repeated entries should exist at all. Removing duplicates first changes the set of entries; shuffling afterward only changes their positions.