Emerging Issues in Data Storytelling
CHAPTER 2 | integrity in data storytelling
How to tell a defensible story, and how not to — integrity in data storytelling
The Tension Point – integrity in data storytelling
We live in an age when data are lauded as the most valuable resource, that a key to success in all walks of life is to harness this "new oil" to inform ourselves to make better decisions.
Yet, we also live in an age of anti-intellectualism, fake news, alternative factors and pseudo-science, where expert purveyors of data are undermined and data are used to deceive.
Integrity of the Person, the data, analysis & interpretation can all be the target of attack
Data visualizations have been key to COVID-19 public messaging—but “counter-visualizations” have been leveraged by COVID sceptics, anti-vaxxers and anti-maskers.
Integrity through trust
- In any organisation, the willingness to use data to make decisions is directly related to whether the people trust the data.
- But data ≠ truth
- Humility and scepticism from both storyteller and audience are critical to gain trust.
PROFESSOR OF DATA SCIENCE, NYU
"After millennia of relying on anecdotes, instincts, and old wives’ tales as evidence of our opinions... data are now perceived as being the Rosetta Stone for cracking the code of pretty much all of human existence. But in the frenzy, we’ve conflated data with truth.
The usefulness of data and science comes not from the fact that it’s perfect and complete, but from the fact that we recognize the limitations of our efforts.
We should not just ask "what does it say?" but ask, “who collected it, how did they do it, and how did those decisions affect the results?”
What might be missing from this picture? What’s another way to consider what happened? And what does this particular measure rule in, rule out, or incentivize?"
How to tell a defensible story
The Storyteller - who is doing this and why?
Self-interest -> promotion
There will always be questions of integrity when there’s a commercial or political agenda.
Self-interest -> suppression
When findings are inconvenient, organisations are willing to ignore, undermine or suppress.
The Audience - who am I trying to influence?
The Data - how were they collected?
Every data set has an origin story... and this story conveys crucial context that must be considered when crafting a defensible data-driven story.
Failure to understand the data generating process can lead to invalid conclusions and can reinforce existing biases baked into the data generating process.
To ensure your data story rests on a defensible foundation, consider the following features to drive the story:
Originating entity
- Were the data generated by internal business processes? An outside agency or public entity?
- What reputation does the entity have? What incentives might influence the data collection process?
Representativeness
- Sample size is often reported but can the data be shown to be truly representative of the larger context being explore?
- Is validating external data available? Can the data be reconciled with known fact patterns?
The Analysis
What methods were applied?
Interpretation
What is the story?
The Analysis
Common pitfalls to avoid
Interpretation: balancing scepticism with decision making
As data are often not definitive, we should seek to balance the level of confidence with the risk associated with the decisions we recommend.
So that we don’t suffer decision paralysis, but also don’t overstate what data may only infer.
Recommending hedged investments or experiments can be more defensible options than a high stakes decision on indefinite data.
This is how Avinash Kaushik, Digital Marketing Evangelist for Google, suggests balancing scepticism with making a decision:
If you are 100% sure about the data and analysis, you could recommend opening several branches in the city.
If you are 80% sure about the data and analysis, you could recommend trying to open a branch in the city.
If you are 60% sure about the data and analysis, you could request more time to dig down or increase the budget to collect more reliable data.
Share your thoughts on LinkedIn
Join the next Data Storytelling Council discussion!
Uncover insights with the leading Data Marketing professionals, and experience the value of Council Meetings firsthand! Potential Members can participate in up to 2 trial Council Meetings. Register now to explore your topics of interest.