VLOOKUP, Power Query and Remove Duplicates all work. They also all take longer than the job deserves. Here is the one-pass way.
You have several files with mostly the same columns โ exports from the same system, lists from different colleagues, monthly reports โ and you need one sheet with the duplicates gone. In Excel that is: open each file, copy, paste below the last row, hope the columns line up, then Data โ Remove Duplicates and hope you ticked the right columns. It works, and it is where mistakes happen.
A merger that reads each file's first row as its headers can stack Email under Email even if it is column B in one file and column D in another. Columns missing from some files are kept and left blank for those rows. The AIMerger CSV & Excel Merger does this in your browser โ no file is uploaded, which matters for customer lists.
Two rows are duplicates when every cell matches, comparing text case-insensitively. That is the safe default. It also means Asha Rao with a trailing space is not a duplicate of Asha Rao without one. If your data is messy, trim spaces first (Excel's TRIM, or find-and-replace double spaces) and then merge. Put the most trustworthy file first โ when duplicates are found, the first occurrence is kept.
Try it on your own files below.
Sales exports, signup lists, bank statements โ combined in your browser, never uploaded.
๐ Drop 2โ20 spreadsheet files here, or click to choose
CSV, TSV or Excel (.xlsx) ยท columns are matched by their header names
If the files are already merged into one sheet and you just want the duplicates gone, Excel has three tools and they do different things. Pick by what you want to happen to the rows.
| Method | Where | What it does | Use when |
|---|---|---|---|
| Remove Duplicates | Data tab โ Data Tools | Deletes duplicate rows permanently and reports how many went | You are sure and you have a backup |
| Conditional formatting | Home โ Conditional Formatting โ Highlight Cells Rules โ Duplicate Values | Colours duplicates, deletes nothing | You want to look before you cut |
| UNIQUE() | Any empty cell: =UNIQUE(A2:D5000) | Spills a clean copy, leaves the original untouched | Excel 365 or 2021, and you want a live list |
Remove Duplicates, step by step. Click any cell in your data, then Data โ Remove Duplicates. Tick My data has headers if the first row is a header. Then the important part: untick every column except the ones that define a duplicate. Leaving all columns ticked means a row only counts as a duplicate when every single cell matches โ which is why people run it and nothing happens. Matching on Email alone, or Name + Phone, is usually what you actually want.
The catch nobody mentions: Remove Duplicates keeps the first occurrence and deletes the rest, so if your newest data is at the bottom, you keep the oldest version of each record. Sort newest-first before you run it, or use the merger above, which keeps the most complete row instead of the first one.
All of the above assumes the files are already one sheet. Merging several workbooks by hand โ matching up columns that are in a different order, or named "Email" in one file and "E-mail" in another โ is where the time actually goes.
The CSV & Excel merger does both jobs at once: drop in as many .xlsx or .csv files as you like, columns are lined up by header name rather than position, and duplicate rows are removed as it merges. It runs inside your browser, so customer lists and price files are never uploaded anywhere. Download the result as .xlsx or .csv.
Only the first sheet of each file is used. Save other sheets as separate files first.
Values are kept as text as they appear in the file. Reformat in Excel after export if needed.
Everything on AIMerger brings things together โ here is what pairs with this one.