What should I check when a chart gives no information about how data were collected?
Quick answer
When a chart leaves out how its data were collected, note what it actually shows, then ask what was measured, which people or observations were eligible and included, how they were selected, and what time period the data cover. OpenStax distinguishes qualitative from quantitative data and random from convenience sampling, making these useful questions to ask [S1]. Missing details alone do not show that the data are representative—or unrepresentative.
The question, the details, the next step.
I found a chart with a title and bars, but no explanation of its data collection. What should I do before drawing conclusions?
First, describe only what is visible: the chart’s categories or measurements, its labels, and any stated dates or groups. Then separate that description from unanswered questions. A chart may show a count, a percentage, a rating, or something else; without a definition, the same label can be interpreted in different ways.
Ask: “What exactly was measured, and how was it defined?” If the chart says “satisfaction,” for example, the missing description might be whether that means a rating on a scale, a yes/no response, or another measure. Treat those as possibilities to ask about, not facts about this chart. OpenStax distinguishes qualitative data, which describe categories or qualities, from quantitative data, which are numerical measurements or counts [S1]. That distinction can help you frame the question, but it does not reveal which kind of data the chart used.
How do I ask about who or what is represented without assuming the chart is biased?
Ask who or what was eligible to be included, and who or what actually contributed data. Depending on the chart, the observations might be people, events, objects, or measurements. Then ask how the included observations were chosen: for example, was selection random, based on availability, or described another way? OpenStax discusses random and convenience sampling as different approaches [S1].
Keep the conclusion modest. You can say, “The chart doesn’t state who was eligible or how observations were selected, so I can’t tell from this chart what group the results describe.” Don’t jump from missing information to “the sample is unrepresentative” or “the collection was poor.” The omission leaves those questions unanswered; it does not settle them.
What about dates, repeated observations, or values that may not have been counted?
Ask when collection began and ended, whether the chart combines multiple periods, and whether each person or item could contribute once or more than once. Also ask how incomplete responses, excluded observations, and missing values were handled. These details affect what a displayed total or comparison means, but the chart alone may not tell you the answers.
For example, imagine a hypothetical chart titled “Visitors by day.” You could ask whether “visitor” means a person or a visit, what dates were covered, whether repeat visits count, and whether all locations were included. Those are questions, not assumptions about the chart. If you can’t get an explanation, record the uncertainty alongside any summary you share rather than filling in the gaps.
Can you give me a short set of questions I can copy when I contact the chart’s author?
You can send this neutral template:
“Could you clarify what each chart value represents and how it was defined? What people, items, or events were eligible, and which were included? How were they selected? What dates does the data cover? Could an observation appear more than once? How were missing or excluded observations handled? Are there limits on which group or period the chart describes?”
For a quick note if no explanation is available, write: “The chart shows [visible result]. It does not state [missing collection detail], so I can’t determine [specific implication].” This keeps the visible evidence separate from what remains unknown. OpenStax supplies useful concepts for asking about data and sampling, but it does not prescribe a complete chart-evaluation checklist or establish how this particular chart was made [S1].
A practical checklist
Write down the chart’s exact title, labels, units, and any stated dates.
Ask what was measured and how the measure was defined.
Ask which people, items, events, or observations were eligible and included.
Ask how observations were selected and whether any could be counted more than once.
Ask how missing, incomplete, or excluded observations were handled.
State what remains unknown without treating missing details as proof of poor or unrepresentative data.
A few more useful answers
Does an unexplained collection method mean a chart is unreliable?
Not by itself. It means you lack information needed to judge how the data were collected; the omission alone does not establish that the data are reliable or unreliable.
What if I cannot contact the chart’s author?
Describe only what the chart shows and note the unanswered questions that matter to your interpretation. Avoid claims about the wider population or collection process that the chart does not support.
Sources
References used for this guide.
- [S1] OpenStax: Data, Sampling, and Variation in Data and Sampling openstax.org · Checked