You’re a senior executive who has relied on your intuition and experience for years to come up with new innovative projects, and they've served you well.

You've drawn from your background, sought consensus among team members and stakeholders, and analyzed market trends.

However, with increasing stakes and complex challenges, you recognize that these approaches may no longer suffice.

Doubts linger: Are you overlooking critical evidence? Are biases distorting your judgment?

While personal experiences and anecdotal evidence can offer valuable insights, excessive reliance on them may narrow your perspective and lead to flawed decisions. This reliance often overlooks the need for a systematic evaluation of the quality and relevance of the evidence at hand.

This is especially critical in the social impact sector. Systemic issues demand a comprehensive and nuanced approach, particularly from those directly affected. Failing to challenge assumptions or implementing misguided solutions can exacerbate harm to vulnerable populations.

🪴 Joyful Ventures helps you win funding & contracts for lasting community impact through program discovery, positioning, and optimization, fusing Harvard PhD insight with Silicon Valley agility

Strategic Decision Making Framework: The Evidence Quality Ladder

Yet it’s not about eliminating intuition. Rather my suggestion is to harness it correctly by pairing it with a systematic approach that rapidly improves your decision-making.

I propose the Evidence Quality Ladder as a pragmatic approach to decision-making. Evidence quality refers to the fidelity and relevance of information to your specific problem or target market.

A common pitfall in improving evidence quality is the tendency to aim for the most rigorous tests immediately.

For instance, while prototyping or a large statistically-significant sample can provide robust evidence, these often demand significant upfront resources and investment.

In contrast, our approach suggests that you go up “one rung” in the evidence quality ladder, given the decision’s importance, risk, and current evidence quality.

Here’s a visual depiction of the evidence ladder, with the apex representing higher levels of quality.

Here’s a more detailed breakdown of the above.

(Lowest): No concrete data (e.g., gut feeling, intuition)

Secondary data (e.g., forums, broad surveys, other organizations' experiences, broader group consensus). At the highest level, this data consists of existing statistically significant tests, such as randomized control trials conducted with your target or similar groups.

Verbal interest from your target (e.g., interviews with target). For instance, this can include verbal expression of interest with your target or alignment between their unmet needs and your offering — without real meaningful action yet.

Small step or test with target segment (e.g., prototype testing, direct observation). The step with the highest weight is one that directly links to your outcome, but a variety of meaningful actions are possible. For instance, if your desired outcome is a sale, a sale, of course, has the most weight, but a target customer putting down a deposit or signing a letter of intent are also meaningful. In any case, all of these actions speak louder than words.

(Highest) Repeated action with target segment. For instance, high volume of sales with your target customer segment.

Note: I have a larger list of tests you can try that go up the evidence quality ladder. (If this is interesting, please reach out and I can go deeper into this idea).

By recognizing and applying this ladder in your strategic decision making framework, you can prioritize evidence-based conclusions and increase confidence in your approach.

How to Use the Ladder in Your Strategic Decision Making Framework

So how do you put this evidence quality ladder into practice?

Here are the key steps:

Prioritize: Score the importance and risk of the decision if incorrect.

Hypothesize: Generate a guess using your intuition and prior knowledge

Test & reprioritize: Actively seek out information that contradicts your initial hypotheses, going up one or two rungs on the evidence quality ladder. Based on the test, reprioritize and consider a stronger test.

This approach acknowledges intuition as a valuable starting point for hypothesis formation.

However, intuition alone—regardless of consensus within the company—is inadequate. When addressing issues affecting vulnerable populations, it is crucial to "get out of the building" and incorporate diverse perspectives.

Strategic Decision Making Framework with the Evidence Quality Ladder

1) Strategic Decision Making Framework: Prioritize

Not all decisions carry equal weight or risk. Many leaders fall into the trap of treating every decision with the same level of scrutiny, leading to decision fatigue and inefficient use of resources.

Our strategy enhances your approach by introducing a prioritization system into your strategic decision making framework.

Strategic Decision Making Framework: Prioritize

This method focuses on identifying the importance of each decision, with special attention to how risky the underlying assumptions are if proven wrong.

Try these steps:

List the key decisions or questions that are top of mind

Score

Importance: How critical is the project that this decision is tied to?
Risk: How extensive will the impact be if you get this wrong?

Prioritize:

If the decision or question scores high on risk and importance, move to the next step below.
If it is not a high impact or risky decision, make a quick, yet reasonable decision with lower quality evidence and/or delegate it to someone else.

2) Strategic Decision Making Framework: Hypothesize

Once you've prioritized your decisions, the next crucial step in improving your strategic decision making framework is hypothesis generation.

This step is where many leaders falter, either by making a risky decision using a hypothesis with low evidence quality or by becoming paralyzed by the need for perfect data before forming any ideas.

A typical approach to this challenge is to conduct extensive research before making any hypotheses. While thorough research is valuable, this method can lead to analysis paralysis or bias towards existing solutions, potentially stifling true innovation.

Our strategy takes a different approach. We advocate for generating initial hypotheses using your intuition and prior knowledge, then testing these hypotheses against evidence — that is, climbing the “evidence quality ladder.”

Try these steps:

Define success criteria: Clearly state your desired outcome. Example: Reduce respiratory illnesses in your client community.

Formulate a hypothesis: Create a specific statement about your proposed solution and its mechanism. Example: Air purifier #2, with UV-C and HEPA filter (mechanisms), will outperform purifier #1 in reducing respiratory illnesses in my client community.

Score evidence quality: Rate your hypothesis using the evidence quality ladder above. Example: If based on intuition about UV-C effectiveness, score 1-2.

Identify key assumptions: Break down your hypothesis into testable sub-questions. Example: How effective is UV-C in air purifiers at reducing respiratory illnesses in my specific community context?

3) Strategic Decision Making Framework: Testing & Reprioritization

With prioritized decisions and clear hypotheses in hand, the next critical step in enhancing your strategic decision making framework is rigorous testing and reprioritization.

This phase is where many well-intentioned initiatives falter, often due to confirmation bias – the tendency to seek out information that supports our existing beliefs while ignoring contradictory evidence.

A common approach to testing is to conduct a literature review or consult with a few trusted experts. While these methods can provide valuable insights, they often fall short in challenging our assumptions and may reinforce existing biases.

Our strategy takes a more robust approach. We advocate for actively seeking out information that contradicts your initial hypotheses, with a particular focus on gaining evidence that is next in the evidence quality ladder.

Often skipping rungs in the ladder may be infeasible because, for instance, testing through direct action requires way more resources and upfront investment then finding solid existing random control trials on the topic.

Here are steps you can take to test your assumptions:

Identify a test that would enable you to go up one or two steps on the ladder. If desired, mitigate the cost or impact of the test. Consider these levers:

Scope: Limit the scope of your test, for instance, reduce the frequency of collection, the sample size, the geographic area, or the number of items tested at once

Commitment: Find ways to change the extent of the commitment. As I alluded to above, if your outcome is a sale, there are gradations of this that are not as onerous you can start with. For instance, signing a letter of intent or memorandum of understanding or paid discovery session to develop a validated scope of work to develop the solution, rather than the solution itself.

Accessibility: Choose test sites or segments strategically to balance similarity and cost. For instance, choose proxy groups or situations that resemble your target but are easier to access or less costly to engage.

Actively seek out and collect this type of evidence, testing the sub-questions or other risky assumptions you identified

Rescore your hypothesis based on the evidence quality in a backlog (see below for example). If the hypothesis fails, then go back to the drawing board. Or if it succeeds, assuming importance and risk are still high, you can prioritize it for a higher-quality test on the evidence quality ladder.

To track your hypothesis, you can use a backlog like the example below, highlighting evidence quality, which is the last column.

👉 If you want a deeper dive on any of these topics, reach out

Strategic Decision Making Framework: Backlog

Your Turn: Apply This Strategic Decision Making Framework

Remember, every decision you make has the potential to change lives. By mastering your strategic decision making framework, you're not just improving your organization – you're amplifying your ability to create positive change in the world.

As you move forward, consider how you can apply these principles to your own work. Start by identifying one key decision you're currently facing and apply this framework.

You might be surprised by the insights you uncover and the clarity it brings to your decision-making process.