For people who work with large data sets in Excel, or databases for that matter, one of the most common error checking tasks is the removal of duplicates values. Some people refer to this process as de-duping. The main reason this is necessary in data sets is because any time you create a primary key field, you can only have unique values in that column. Any repeating value is therefore an error.

Duplicate values can pop up in your data set for several reasons. Thus it is always prudent to check for this error in any data that you receive from another person. Whenever you want to check for duplicate entries, make sure to note which columns should only have unique values. Then use one of the five methods described below to identify and remove the duplicates