As a former survey researcher, I know the importance of delivering quality data to a client.
As I mentioned in a previous post, titled “The Importance of Reliable Sources,” it is ultimately the consumer’s responsibility to verify that the information that he or she reads, hears or sees on the Internet (or any other communication channel) is factual before using that information to make a decision, particularly a business decision.
While it is often impossible to fact-check everything, what you can do is make sure that the information that you receive and share with others is coming from a reliable source.
In fact, it is often prudent to check with the original source when making major business decisions.
After reading this post, I hope that you agree that what I said in the last sentence is of the utmost importance.
The Denominator Makes a Difference
When market research companies do research, they will often publish their findings in a report or white paper that they will sell to customers, or possibly give away as a token of goodwill.
However, in an effort to get exposure for their company and drive sales of the research report, they usually give away a certain amount of the data, with the hope that it will be used in articles and blog posts.
In the actual report, the research findings are often reported in data tables or other graphical presentations that allow the user to better understand what the research is telling them.
In most cases, trusted research firms will list what population or subpopulation that the data tables, graphs or infographics are based on (e.g., Internet users in the United States, smartphone users worldwide, all consumers who purchased Brand X in the last six months, etc.) in a footnote.
This information is more important than you might realize.
Let me give you a quick hypothetical example.
Let’s say that a research firm conducts a survey of all Fortune 100 companies. And, let’s say that 40 of the 100 companies currently use a software package that is being offered by XYZ, Inc.
Now, say that they find out that 39 of the companies that use the software package are extremely satisfied with it.
With the new data in hand, the market research firm publishes a press release that reports that 98% of Fortune 100 companies that use XYZ’s software package are extremely satisfied with the product.
Along with the press release is a data table that reports the percentage of Fortune 100 companies that are extremely satisfied with XYZ’s software package. And, at the bottom of the data table is a footnote that says something like, “Note: Based on Fortune 100 companies that use XYZ’s software package.”
The press release with the data table is then passed from person to person over the Internet and somewhere down the line, someone makes a graph that says that 98% of Fortune 100 companies are extremely satisfied with XYZ’s software package. However, they forget to include the footnote.
Now, that graph is shared with others over the Internet.
And, those people who read it get the impression that 98% of all Fortune 100 companies use XYZ’s software package and are extremely satisfied with the product.
However, in reality, only 39% of all Fortune 100 companies use XYZ’s software package and are extremely satisfied with it. (In fact, it could be the case that a competitor’s product has a greater market share and has customers who are just as satisfied.)
Headlines Can Be Misleading
The example that I just gave is a little extreme.
However, I wanted to make a point.
And, I must point out, that I have noticed similar things happen many times.
In many cases, it is the headline or title of the article or blog post that is misleading.
Using the previous example, an article might be titled, “98% of Fortune 100 Companies Are Satisfied With XYZ’s Software Package.” However, later in the article, the author points out that the data is only based on the 40 Fortune 100 companies that use the product.
If a consumer does not take the time to read the whole article, and instead just reads the headline, they can be left with a completely incorrect impression about what is really going on in the marketplace.
And, with so many people sharing information on Twitter and other social networking sites, where people sometimes only read the headline, it is easy to see how incorrect information can spread very fast.
Quotes Taken Out of Context
Often, when people write articles or blog posts, they will list what population or subpopulation that the numbers are based on in the first paragraph.
In subsequent paragraphs, they will leave out the base, with the understanding that the reader already knows what population or subpopulation that they are referring to.
However, inevitably, someone will quote something from the article or blog post without mentioning the actual population or subpopulation associated with the statistics.
Now, when their readers read the new blog post or article, the information that they will be given will be incorrect, based on the fact that they don’t have the correct population or subpopulation in mind.
When you are dealing with percentages, knowing what population or subpopulation is actually being talking about is crucial.
It is strongly suggested that people who write articles and blog posts include the correct population or subpopulation in the title or headline of the article or blog post, to avoid confusion. This is particularly important in a world where people often glean information from the headline that is included in a tweet on Twitter or a post on any other social networking site.
And, when making tables, graphs, or infographics, be sure to include a footnote that lists the correct population or subpopulation. This will help avoid confusion when others share your work.
Furthermore, when quoting statistics from other sources, be sure to make it abundantly clear what population or subpopulation the data is actually referring to.
As mentioned in a previous blog post, it is ultimately the consumer’s responsibility to make sure that the information that he or she receives is factual, or in this case technically correct, before making any decision based on what he or she reads, hears or sees from any source.
After reading this post, I hope that you can now see that data can easily be misinterpreted as a result of negligent omission of information that explains to the reader what population or subpopulation that the data is actually based on.
Therefore, I hope that you can also see why it is so important to check with the original source before making any major decisions, particularly major business decisions.
As the title of this blog post states, it is important to mind your bases when analyzing data.
With that said, I can now climb down from my soapbox.
Photo credit: MonsieurLui on Flickr.