There is something strange happening in the world of NPS. Practices and implementations that seem straight out of another world.
We have stood our ground and continue to fight for our customers to prevent these strange features from disrupting your results and the impact they have on your bottom line.
One of these “strange things” being the concept of an in-app widget or pop-up survey for measuring NPS within your product or app. We’ve already covered our reasoning in detail in an earlier post, so I won’t belabor our position and go into why we don’t support it yet again.
What I will say is that since publishing that post over 10 months ago, not a week has gone by when we haven’t been asked if we offer an in-app solution … so we continue to educate the market on our reasoning and the advantages of email (relationship) vs. in-app (transactional) NPS surveys.
To be honest, somedays it seems like it would just be easier to build the darn feature. But, that’s not the right thing to do. Not for our mission here at Promoter and especially not for our customers.
If we’re going to do something, we’re going to do it right while ensuring that real value is being provided to our customers. The fact that something may be slightly easier to set up or “seems logical” does not change the facts behind the data.
To help explain, a recent study stated, “Nearly three quarters (72%) of consumers said [on-site] surveys interfere with the experience of a website.”
This one simple stat alone is reason enough for customers to question the efficacy of in-app surveys and for us to never consider building out an option, at least not as a stand-alone solution.
But frankly, we want to fulfill our customers’ desires, so it’s likely that we will have an in-app solution some day. With that said, I can tell you with 100% confidence that it will not be until we can figure out how to address its issues and make it beneficial to our customers.
A spray and pray approach simply doesn’t work and when large portions of customers simply want to dismiss a pop-up survey they find intrusive, you end up collecting a lot of data that isn’t accurate. Risky to say the least, especially if you want to action as an organization on this very data.
The other most requested (strange) feature that we haven’t implemented is the ability to add additional questions to the NPS survey (beyond the two-primary questions).
Much like the request for an in-app solution, we get inquiries about this ability on a constant and consistent basis (multiple times per week in fact).
This topic has led to much debate both internally at Promoter and externally within the NPS community at large.
Just the other week there was an extensive conversation that started within an NPS community forum on LinkedIn to debate this exact topic.
As with any debate, there were arguments for both sides. Some believed that adding one or two additional questions could provide a deeper-level of customer insights, while others stated that doing so will decrease your overall results, making the whole survey less relevant.
While we didn’t participate in that particular discussion, we were (and are) on the side of the argument that believes adding additional questions to your NPS survey is not a good idea.
But that isn’t just a gut-feeling we have, it’s a statistically proven fact.
In the same study I mentioned previously, they also discovered that, “80% of customers have abandoned a survey halfway through.” And, “52% of customers said that they would not spend more than 3 minutes filling out a feedback form.”
Those are some major hurdles to overcome for traditional surveys (and even those that seem short at the start).
NPS was designed to eliminate both of those issues, but it only works when the integrity of the survey remains in tact.
Let me explain why that’s so important:
There is a reason why NPS is so popular.
Have you ever wondered why there are so many companies (including some of the biggest in the world) that use NPS so religiously?
I mean, what’s the big deal about a simple two question survey?
Well, that’s exactly the “it” … it’s simple. So simple that it takes just thirty to sixty seconds to complete for your customer. So simple that it generates the highest response rate of any survey in existence.
In a recent conversation we were having with a Promoter customer, they mentioned that before measuring NPS, they were asking 20+ questions every quarter and getting an average of a 4% response rate. Just four percent! That’s almost meaningless and likely driven by a very vocal minority.
Once they switched over to NPS, there average response rate improved to over 25%.
(To be honest, even that’s on the lower end of a typical average for a Promoter customer, it’s usually between 30 – 40%.)
The point is, an NPS survey is two questions for a reason. If you complicate the simplicity of it, you’ll pay the price in effectiveness.
Adding questions reduces your overall response rate
You might be inclined to think that there is a happy medium. If asking 20 questions gives you a 4% response rate and asking 2 questions gives you a 30% response rate, adding just one more question shouldn’t hurt too much. Maybe it decreases your response rate by a couple of points … no biggie.
Unfortunately, the reality is that it’s not a perfectly balanced scale, so adding questions doesn’t decrease your response rate proportionately. It’s very disproportionate in fact.
On average, for every additional question you add, you’ll decrease your overall response rate by 30 – 50%.For every additional question you add to NPS, you decrease your response rate by 30 - 50%. Click To Tweet
The decrease that you’ll see in response rate from adding just one question alone could mean the difference between receiving 1,000 responses instead of 500.
How valuable is that extra question now?
Additional questions don’t deliver additional insights
While it’s true that asking your customer why they gave you a 5 instead of a 10 on an NPS survey won’t reveal any personal details … these are types of questions that really shouldn’t be asked in any survey to begin with.
Time and again we see companies continue to ask their customers questions that they could have (and quite frankly should have) discovered on their own. There are plenty of tools and technologies that can provide you with endless amount of customer data without bothering your customer.
At the end of the day, what companies are looking for are key insights and actionable information from their customers — in other words … what can we do to improve and grow our business? Quite frankly, everything else is just fluff. It’s as straightforward as that.
The Ultimate Question was developed specifically to cut the fluff and get you the actionable information you need. It was tested and refined over time to elicit the greatest insights from your customers.
The biggest challenge we see is with companies not trusting in the process. Believing that they’re not going to get the insights they’re looking for from the two questions being asked.
Just remember that the NPS survey, in its true two-question form, is the only survey that has proven to be effective at measuring customer intent in a high-engagement manner.
The value of your efforts are directly proportional to the number of customers you are engaging at the end of the day. This is because NPS is not an exercise in statistical analysis. It’s a tool meant to understand the sentiment of the customer and their relationship with your brand. What drives one customers sentiment is not the same for others, even if scores are similar.
Understanding that difference and learning how to strategically apply what you learn is how you drive measurable value from your efforts.
When it comes to both an in-app widget and the ability to add additional questions, it’s not that we don’t have the technical capability to add these two features.
And, it’s certainly not because we don’t want to meet the needs of our customers. It’s actually the opposite.
The reason we haven’t yet succumb to these requests is really quite simple: They don’t help you – our customers. Oh, and because they lead to the other side … which is dark and scary place to be.