Duplicates are ruining your results – here’s why

Did you know 80% of new records coming from integrations are duplicates? And across all sources, more than 45% of records are duplicates. You and your team members are reaching out to those leads to find out they’re already in conversation with a sales colleague. You’ve wasted precious sales time and made an unprofessional impression on your lead. This article will show you how duplicates come in, how they impact your results, and how to fix this.

How does duplicate data happen?

Duplicate data is entering your CRM through poor data migration and import processes, external data sources like contact forms, or your employees’ data entry. Most of all, duplicates are created by third-party integrations with your CRM. Think of meeting scheduler software, webinar software, lead gen forms on social media, and marketing automation solutions.

Why should we care about duplicate data?

If you work in sales, you know that dealing with duplicates gets you out of your flow. You either spend hours per week checking for duplicates and wasting precious contact time, or you skip checking for duplicates only to hear from your lead they are already in pricing discussions with your sales colleague. This leads to wasted time, angry sales teams, and a poor customer experience.

Lead ownership and sales commission conflicts occur in the best sales teams. But when you leave duplicates unchecked, you are asking for trouble. When assigning leads round-robin, it is very likely the duplicate lead or account will get assigned to a different owner. This sets the stage for commission conflicts.

Data-driven decision-making is essential for modern organizations. Be it when interpreting a report or as a basis for a decision tree in marketing automation. Duplicates will hugely inflate your forecast and pipeline, leading to faulty resource allocation and a guaranteed disappointment at the end of the quarter.

Stats that back this up?

Plauti, developers of Duplicate Check for Salesforce, aggregated 2021 client data and analyzed more than 12 billion Salesforce records. Imports return a relatively modest 19% of duplicates, but third-party integrations connecting to the Salesforce API return a massive 80% duplicate rate.

An 80% duplicate rate for third-party integrations is not necessarily something to be scared of (if you have something in place to merge them). It means that your leads and customers interact with you across various channels. With the right tools to merge these duplicates, you will get a completer customer view with each interaction.

Clean your CRM of duplicates

OK, so duplicates are bad. What now? Your CRM is probably filled with some 15-30% duplicate records if you have never done anything about it. Time to get cleaning!

1. Find a solution

Before you start with the built-in cleaning feature of your CRM, I recommend listing your requirements first. The built-in feature might not suffice, and you could be wasting a lot of time.

The first step is to list what objects you need to be deduplicated. Probably leads/contacts and accounts/organizations, but maybe you have issues with duplicate opportunities/deals or cases. You might need some cross-object comparison, too. In Salesforce, you need to compare Leads to Contacts (different objects). Add the cross-object deduplication to your list too.

The second step is to write down which fields you need to assess to identify duplicates. Consider obvious ones like name, email, phone number, and domain name.

For the third step, think of the rules you want to apply when merging one or more records. Which record should be the master (‘winning’) record? Should it be the oldest, the most complete, something else? Do you need any specific merge rules for fields?

You have put together a list of requirements for a deduplication solution in the steps above. Do some research on the internet and on the app store of your CRM to find possible solutions and check them against your requirements. You will probably be able to start a free trial.

2. Clean your current database and repeat

Now you have chosen a solution; it is time to start the actual deduplicating. Build a scenario (your definition of a duplicate record), set your merge rules (what data is saved to the master record), and run a job to deduplicate your database. We recommend not performing any automatic merges just yet. See how your merge rules turn out, and if you’re happy, it’s time to automatically merge duplicate records with a high confidence score.

Do not forget to perform this job regularly.

3. Prevent new duplicates

Good, your existing database is clean. Let’s keep it that way. New records coming from integrations are 80% duplicate on average, so we need to prevent them from messing up our results.

Depending on your solution, either block new duplicates from entering your CRM or merge them with the existing record. All data points are valuable so merging a record is infinitely better than blocking.

4. Alert on duplicates

As a last precaution against reaching out to a duplicate lead/contact, I advise implementing a duplicate warning on the record detail page. Such a warning will indicate to sales reps that (possible) duplicates exist, and they should consider merging the records.

In addition, some solutions offer the option of warning users of potential duplicates when manually creating a record in the CRM.

Tying it all together

Let’s recap everything we just learned. Duplicates are bad for your business because they waste valuable sales time, cause commission conflicts, negatively impact data-driven decision-making, and lead to a bad user experience. Duplicate records are already forming around 15-30% of your current contact records, and more and more are entering each day, with up to 80% duplicate rate for software integrations with your CRM.

The first step is to select a deduplication solution. Next up is to clean your existing CRM data of duplicates. The third step is to prevent new duplicates from doing any harm by merging them as soon as possible. Fourth is to put alerts in place to ensure your sales and support teams do not reach out to a duplicate lead by accident.

When you have everything in place, enjoy the rewards of a more focused sales team, a better experience, and in the end, more revenue.