A Methodology for Crafting Awesome Experiences – Part 7

Part 1 – Introduction
Part 2 – Strategic Measurement Framework
Part 3 – Design
Part 4 – Validation
Part 5 – Implement
Part 6 – Measure

If you have not been following the series – or for a refresher – please read the previous entries.

I was always bothered by the shallow measurements in two critical areas of experience management: at the moment-of-truth (when customer and organization interact) and for the overall experience. Using this methodology we are measuring moment-of-truth by integrating feedback events into them as part of the experience design.  That feedback is collected when it happens, focused on effectiveness, and uses escalation if necessary.

The measurement for the overall  experience is something usually addressed poorly.  Most organizations simply deliver a satisfaction survey and ask a question about overall satisfaction with the entire experience.  However, it is very hard for a customer to place a value in overall satisfaction in an experience where they may have been good and bad elements (and we won’t get into the discussion of using customer satisfaction as a metric).  That question would also not measure the effectiveness of the experience.

I want to introduce you to the end-to-end efficiency and effectiveness index (EEX), which accomplishes two things:

  1. It provides a method to measure and monitor the overall experience.
  2. It consolidates all the moment-of-truth feedback measurements into an index that can be adapted to specific goals of measurement.

There are two stages to compose this index.  First, each moment-0f-truth is measured:

Feedback Event

The three vertical ovals in the chart above represent the three elements present in each interaction between customers and the organization.  Each moment-of-truth must have a feedback event and the smaller bubbles in the chart represent what is potentially a feedback event (which one is chosen depends greatly on the process, what we are trying to measure, and the purpose of the moment-of-truth)m represented with a numeric value (or a quality-value transformed to numeric).  If there is more than one feedback event in the moment-of-truth, those values are to be aggregated into a common index (the big bubble at the bottom of the chart) — or you have to choose (and document the choice) which one you will use to incorporate into the EEX.

Second, we measure all these moment-of-truth in the experience:

Experience Measurement

Of course, the individual events will still be reported and followed separately, and will likely have workflows associated with individual values and thresholds.  The index does not override the business’ need to focus on each element of the process to understand weak spots and potential improvement.

Finally, we insert all the values we collected from all those feedback events into a formula to calculate the EEX:

EEX Formula

There are two variables in this formula.  First, the feedback (F) refers to the numeric value of each feedback event created and deployed in the experience we designed.  Each will we assigned a weight (W) which specifies how important that specific part of the experience is to the overall value of it.  For example, if your strategy right now is to focus on the value of online interactions (web, email, chat, SMS, Social Media) then you can assign a higher weight to functions that happen online.  The score will then be higher (if you are doing a good job) or lower (if you are doing a poor job).

By using weights you can quickly compare any specific portion of the experience versus the overall benchmark (a benchmark is created by assigning each interaction measured equal weights).  Let’s use  an example.

  • You are measuring the experience for change of address.
  • There are six steps in your web site to change an address, and four of them are moment-of-truth and have a feedback event associated with them.
  • If you insert those four values (F1, F2, F3, F4) to the above formula and assign an equal weight (W1, W2, W3, W4) of .25 to each, you can get the benchmark value for the experience (the average effectiveness per customers’ comments).
  • If you want to highlight one of the steps over the others (let’s say that you have been having problems with the confirmation of the old address before allowing the customer to enter the new one), you can then increase the weight for that feedback event to .40, and decrease the value of each of the other three events to .20.
  • The resulting value can then be compared to the benchmark to determine the effectiveness in the overall experience
    • If the value from altering the weights is higher – then that portion is performing above the overall function.
    • If it is lower, it is performing below the overall function.

Finally, how to use this index — I know you will be tempted to use this index as a single metric to represent the effectiveness or efficiency of the experience.

Don’t do it.

I wrote before about how using single metrics for measurement is wrong, and it does not represent the true value of what it intends to measure.  Instead, use it as an internal metric to let you know when an experience is performing below par, or when a specific channel or portion of the same is lagging.

You can use it in dashboards, scorecards, or reports as you wish – but make sure it is part of a larger reporting structure.  At the very least, when you use the index (as you are supposed to do with any index) you have to make sure you include all the values that led to that specific index – in this case, the different feedback values you gathered in the different moment-of-truth.

We reached the end of this methodology, and I am anxious to hear your comments.  What do you think?  Is this something you could use? Can you see the value for using it to craft your awesome methodologies?  Am I missing something? Is there something I should change / add / delete?  Please leave me your comments here or send me an email with your questions, comments, or concerns.

Thanks for reading.

3 thoughts on “A Methodology for Crafting Awesome Experiences – Part 7”

  1. Hello Esteban,

    I have been waiting for this 7th part, as you know. And let me start by saying you made it worth the wait!. The EEX is a great approach and acknowledges the fact that the customer experience is multi-touchpoint driven and should be measured (as well as designed) and actioned as such. I will highly recommend the entire sequel to anyone interested in the field of Customer Experience Management (and design).

    Now, with regard to the measurement, I do have a few questions/additions:

    I’m all in favor of adding weights to different parts of the experience in the total measurement. I do think though it is best to weigh based on importance of one piece of the experience compared to the overall experience as per customer feedback. It cannot be the company deciding which (part of the) experience matters most: the Customer will decide. Therefor one should weigh highest that part of the experience that matters most to customers. When done correctly you will have a killer metric (oops 😉 for making decisions on where to invest/improve or where not (it really does not matter investing in parts of the experience that your customer doesn’t care about that much, considering you have limited resources).

    The starting value of the weights could come from collecting these in the design-phase where you involve Customers to collect initial feedback on your design. Since Customer expectations, desires and needs shift over time, it is important to keep track of movement in the weights you apply. One should adjust the weights accordingly to stay in tune with customer expectations/needs.

    To make it even better (and I believe/hope/think you agree but I am not sure because I only read it between the lines) it would be great if one could add different elements of Customer feedback (not just satisfaction with MOT) per MOT/feedback event in the experience, that have different weights among each other too. These feedback elements should be tied to internal metrics on the process or agent (effectiveness etc).

    To conclude: as you stated often, crafting awesome experiences is not an easy thing to do, crafting experience measurement is maybe the most difficult part in it. I believe it is probably the most important part, because once you “release/launch” the experience it is bound to be imperfect (regardless of the diligence and effort in crafting it). You need an excellent set of metrics to pin-point you in the right direction of adjusting it, based on the feedback you get from your customers and your internal metrics.

    Let me know what you think. Am I making sense? Did I miss anything? Looking forward to your response.


Comments are closed.