2 min read

Data-Driven Decision Making: Seven Tests for Data Quality

For the last 11 years, Forest2Market has been collecting, aggregating, reporting and analyzing data about market prices for forestry-related industries. Because our business is based on data, we’ve spent considerable time and resources understanding the role of quality data in making business decisions.

Just how important is data? For a recent study conducted by researchers at MIT, the University of Pennsylvania and the National Bureau of Economic Research looked specifically at the performance of 179 companies and found that the primary distinction between companies with higher performance rankings and the rest was whether those companies make decisions based on data and analysis or on the traditional method of experience and intuition.

The study found that “firms that adopt data-driven decisionmaking have output and productivity that is 5-6% higher than what would be expected” given other factors. Furthermore,” the study suggests, “the relationship between data-driven decisionmaking and performance also appears in other performance measures such as asset utilization, return on equity and market value.”

How much of a difference can a 5-6 percent increase in output and productivity have for the average forest-related business?The study’s lead  author, Erik Brynjolfsson, an economist at the Sloan School of Management at MIT, noted that a 5-6 percent increase “in output and productivity is significant enough to separate winners from losers in most industries. The companies that are guided by data analysis are ‘harbingers of a trend in how managers make decisions. And it has huge implications for competitiveness and growth’” (read the article by Steve Lohr).

The challenge for companies these days, however, is determining which data sets are high quality enough to produce better decisions and therefore results. As Lohr writes: Companies are swimming, if not drowning, in wave after wave of data. . . . Internet-era technologies, by one estimate, are doubling the quantity of business data every 1.2 years.” In forestry-related industries specifically, the challenge is often finding accurate and therefore reliable data.

Ensuring that the data your company uses to make decisions is the highest quality available isn’t easy. But the results of our research and experience in the data business provide insight into the questions that data providers should be able to answer. The following tests can be used to evaluate the quality of  the data sets you are considering.

1. How is the data collected? Data collected by survey can be manipulated. When data is collected on a transaction-by-transaction basis, directly from contracts, scale tickets, orders or invoices, however, gaming the system is difficult.

2. Is the sampling size broad enough to be representative of the market? The more data your provider collects, the better. A true reflection of what is happening in the market can only be gleaned from data that is statistically significant.

3. Is the database deep enough that it can be sliced and diced in ways that allow for more accurate readings of the market? A provider that gathers data gathers price per ton will not provide the same level of detail as the provider that gathers data on stumpage prices per ton, hauling costs, supplier costs and fuel costs will provide a much more granular insight into the market.

4. Is pricing data determined based on weighted averages? Does it recognize volume discounts? A data provide that collects price by volume is more representative of the market.

Example: 1000 mbf sold for $306 and 100,000 mbf sold for $247.

  • Many providers report average price: $247 + $306 = $553/2 = $277 per mbf
  • Others report a weighted average price, which is more reflective of the market: $247*100,000 + $306*1,000)/101,000 = $250 per mbf.

5. Is the data expertized? Data that is reviewed by experienced market analysts is more likely to be scrubbed to remove outliers, questionable and incomplete data. An additional layer of accuracy is guaranteed when all data is subject to third-party audit.

6. Are data submissions auditable? An additional layer of accuracy is guaranteed when all data is subject to audits.

7. Are reports customizable to specific business needs? Does the data provider work with clients to design just the reports and analysis that they need?


Comments

With Data from Market2Mill, Turn your Attention to

01-18-2012

[...] about the relationship between data-driven decision making, productivity and profitability in Data-Driven Decision Making.  Share this:PrintEmailTwitterLinkedInDiggRedditStumbleUponFacebookLike this:LikeBe the first to [...]