The fruits of feedback

by Giuseppe Lanzi on 07/11/2014

June has come and gone, and as promised, it’s time once again to talk about Instant Developer technical support. However, this time I want to go beyond just presenting performance indicators. I’d like to dig a little deeper and look at the real value underlying user feedback.

Since the start of the second quarter, the average grade rose from 4.24 to 4.28, and the percentage of sessions receiving a grade rose from 55% to 64%. Now, if we only looked at the numbers, it would be like saying “well, we’ve managed to complete the thankless task of technical support”. But that’s not the case. Technical support should not be viewed as a thankless task to be avoided if possible, but as an important resource for innovating and improving the product. I’d like to use today’s post to share what I’ve learned from your feedback.

To better understand where improvement is needed, I broke down the average grades by type of session:

  • 4.32 for support requests.
  • 3.93 for bug reports.
  • 4.78 for internal error reports.
  • 5 for consulting services.

It’s clear that bug reports have the lowest average grade, which is perhaps to be expected, but I was surprised when I broke the figure down by outcomes:

  • 2 for bug reports closed because the error could not be reproduced.
  • 3.30 for bug reports closed because the problem was due to improper use of InDe.
  • 4.35 for bug reports closed with a workaround provided.

This data shows that the weak point is related to non-reproducible bugs. To find out more, I contacted most users who gave a grade, and it turns out that the negative factor was the perception that the discussion was closed, not the inability to reproduce the problem.

And this is perhaps understandable. Some might view the response to such reports as “the error could not be reproduced, so we are closing the request; if you want, you can open another one (in other words, it’s your problem)”, but that is not what we mean to say! What we are trying to say in these cases is “as of now, we have not been able to reproduce the error; can you open another request with more information or with a different project?”. We always prefer a continuous dialogue, but the fact that these tickets were being placed in closed status led to misunderstanding.

In response to this business intelligence exercise, I’ve decided to experiment with a change in the support procedure over the rest of the year: when a bug report is closed due to a non-reproducible error, to re-open it and provide additional information, the user will only need to reply via email with more information on how to reproduce it. If the results of this trial period are positive, we will implement the ability to perform this operation with a simple click from the response email.

My analysis has confirmed that there are two factors involved in improving both the product and our relationship with customers: being receptive to feedback and not being afraid to acknowledge and address it.

If I could give any advice to readers, it would be to do likewise.

Image: Steven Shorrock.

Leave a Comment

Previous post:

Next post: