What are some best practices/best processes for connecting insights from CX survey projects to business decisions - which may happen months apart and be led by a different team - to track the ROI of the survey project?

When the CX insights lead to a business decision NOT to do something (e.g. add/change a product or service), how do you translate that into value realization? Is there anything to consider beyond the money not spent?

What do you see as best practice in defining ROI in non-financial terms? This is something that we struggle with in the public sector. Sometimes it can be framed as cost savings, but I wonder if there are other measures around user volume, cost per user, or specifically satisfaction or engagement measures?

We want our senior executives to make better decisions. We ensure that the customer data and the operational data is always present with the financial data. We worry that our customer insights will be over looked, What advice/tips can you give to facilitate a more rounded leadership conversation?  

​@Hillary Blevins I love your questions. Here’s my perspective on the first one, “Connecting insights to decisions.” Nothing groundbreaking, just a little elbow grease and simple tools:

1. Having clearly documented insights is a great start to my favorite 2-column spreadsheet; Column A: Insights, Column B: Resulting actions/decisions
2. Whenever you have visibility into the decisions or actions, document them next to the associated insight. When you don’t have visibility, consider periodically asking stakeholders, “We shared sx] insight with you, what did you do with it?” If you’re sharing insights with a lot of stakeholders, it might make sense to run a survey. Keep a running log of who you share which insights with and upload this as embedded data to send out monthly or quarterly. Here’s a link to a simple survey (.qsf file) for this: Insight tracking .qsf (known insights)
3. Because large decisions can be so messy, it isn’t always possible to untangle what role your insights played vs. other sources of information. It may make sense for the CX team to play the role of celebrating everyone’s contributions without needing to attribute specific impact to any one team or project.
4. It may also be worth exploring where you can get more involved in some decisions. While not realistic to always be involved, it can be useful (e.g. learning more about how the decisions are being made, helping decision-makers interpret data better, access to teams deploying actions, etc.). Maybe it makes sense to design a mix for each quarter; e.g. 1 downstream / hands-on project per team member, with remaining time spent on scale and upstream work. 

Are your survey projects purpose-built for specific decisions, or more generic? Do all insights pass through your team’s hands, or are others generating insights based on your team’s data? Do you always know who the insights are going to? What kinds of decisions are you generally trying to influence? Have you tried anything else that seems to work?

Thanks for sharing your question! Happy to keep the conversation going. 

​@Hillary Blevins to your second question about measuring the impact of avoiding an action:

The counterfactual question you are trying to answer is, “How would customers have behaved if we hadn’t stopped the business from taking XYZ action?” That would let you compare actual customer behavior under vnon-action scenario] against the imagined behavior under vaction scenario]. Usually very hard and often not possible to answer with as much precision as you would like. But, especially for large projects/changes, I think it’s worth seeing what you can do:

1. I might start by looking at the insight that led to the decision to avoid the action. Did it include a prediction of how customers would behave in response to the action? To the extent the business believes in your insight (if they stop a big action, that’s a good sign), you might be able to compare the predicted behavior under daction scenario] against actual behavior under non-action scenario].
2. You might also be able to construct the counterfactual based on similar initiatives within your company or other companies. Does the company continue to take similar actions in other parts of the business? Has it tried these before? Have other companies? 
3. In some scenarios, it may even make sense to have a holdout group (e.g. some customers continue to receive the action). The business may believe it’s better to not take the action, but they may not know by how much. The holdout group can help them answer this question to inform future decisions, and also help measure the impact of this specific intervention. 

As you go through this, your goal is to build as clear and defensible of a counterfactual as you can. Specifically, here’s how we imagine customers would be behaving if we hadn’t stopped XYZ action.

Do you have a specific example of one of these avoided actions that you can share? I might be able to suggest a more specific approach based on the scenario. 

​@AdamK12 You nailed it that cost savings can usually be identified regardless of the type of organization and there often isn’t a revenue side (or it isn’t the move valued outcome) for public sector organizations. 

I like starting with the question, “Who do we want to create value for?” And then we can ask, “What do they value?” For example:

1. Businesses usually value cash flow and ROI (monetary benefits less costs divided by costs)
2. Employees often value things like engagement, professional development, career advancement, etc.
3. Students (or their parents) might value how much they are learning, level of safety, job placement, etc.
4. National Library of Medicine might value increasing access to health information, public adoption of health-related tools, accelerating health innovation, efficiency (the cost savings you mentioned earlier), etc.
5. A high school might value student performance, impact on the community, teacher and student wellbeing, etc.
6. A citizen might value predictability, ease of access, clear communications, etc.

Who do you have in mind when you want to create value for someone? I might be able to suggest some more specific outcomes.

Also, there is a helpful Venn diagram in this article. You could replace the circles (customers, employees, business) with your own stakeholders (the Library, citizens, researchers, etc.). This is useful to show how some initiatives create value for multiple groups, while some may only benefit certain ones (sometimes at the expense of others).

​@Keith Fletcher That is such a useful question to ask. I think it’s a legitimate risk that you called out. I’ll try to do it justice:

1. Is your team providing fully developed insights to the executive team, or just data? There is certainly something to be said for scaling your team by providing data and enabling others to self serve. When it comes to your executive team, it may be worth going further. The odds that they go as far with the data as you would are low. Bring the most important insights to their attention.
2. How many insights do you usually share with this group over a given time period (like a quarter)? Quality over quantity. Help them focus on a small number of things that matter the most. Might be better to be a broken record, repeating yourself for a while, than to be white noise with too much information.
3. Are these insights in response to a specific request from the executives, or are they emergent themes that your team has found? If it’s in response to a request, much easier to get their attention. If it’s an emergent theme, you may have more work to do to call out why it is so important right now.
4. When you develop an insight, when do you consider it done? In other words, what are the essential ingredients for a fully developed insight? When I think of a fully developed insight, I look for 1. What is going on, 2. Who does it affect, 3. How does it affect their behavior, 4. What does that behavior change mean to the business (often in monetary terms).
5. How do these conversations usually go? For this group, getting straight to what they care about is often the right approach. I like packaging up insights into something like this: “We think the company is leaving $xM on the table this year due to XYZ broken experience… Here’s why… Here’s what we think we should do about it.” 
6. It may also be worth investing in decision quality training for your team. This can put them in a better place to guide the executives to a higher quality decision. For example, here are some extremely common decision making traps -- just being aware of these can often help you put language to the problem and point the executives in a better direction. 

Do any of those points resonate? Anything else you’ve learned works particularly well for this?

Thanks so much, ​@TopherM -- I really appreciate how you’ve added dimensions like accessibility and acceptance to the value equation. I’m going to share this on my team because it helps us relate our CX program to our strategic goals.

Michael Maslansky and his team have thought in interesting ways about what “return on responsibility” around sustainability and climate means--that it can be tied to long-term viability in terms of business results, but also tangible nonfinancial returns: The Return on Responsibility: the business benefits of communicating on climate - We Mean Business Coalition. It also speaks to how to relate lived experience to sustainability in sone really valuable ways.

​@AdamK12 glad it will be helpful! And thanks for sharing the report, it’s a great message. If we know what an organization values, we know what it is motivated to get more of.

Absolutely--and it really helps to encourage people who are less familiar with EX to embrace it as something that will help them do their jobs better.

We're wrapping up our first 'Ask the Expert' session with ​@TopherM, and we want to extend a huge thank you to everyone who participated!    

Topher provided some incredibly valuable insights on value realization, addressing important questions about connecting insights to business decisions, measuring the impact of avoided actions, defining ROI in non-financial terms, and facilitating more rounded leadership conversations.    

Thank you, Topher, for sharing your expertise and engaging so thoughtfully with the community!

And thank you to all who asked such insightful questions – your contributions made this a rich and rewarding discussion.    

We hope you found this session helpful.  Let us know if you have any further thoughts or how you plan to apply these insights!

Stay tuned for future 'Ask the Expert' sessions!