Business Insider Intelligence
The COVID-19 pandemic has forced retail to innovate for survival. BII presents its assessment of how brick and mortar retail can apply technologies to assist in its re-emergence and to ultimately utilize human associates to their greatest potential.
by William Barnett from the July 1988 Issue
Recent history is filled with stories of companies and sometimes even entire industries that have made grave strategic errors because of inaccurate industrywide demand forecasts. For example:
The inaccurate suppositions did not stem from a lack of forecasting techniques; regression analysis, historical trend smoothing, and others were available to all the players. Instead, they shared a mistaken fundamental assumption: that relationships driving demand in the past would continue unaltered. The companies didn’t foresee changes in end-user behavior or understand their market’s saturation point. None realized that history can be an unreliable guide as domestic economies become more international, new technologies emerge, and industries evolve.
As a result of changes like these, many managers have come to distrust traditional techniques. Some even throw up their hands and assume that business planning must proceed without good demand forecasts. I disagree. It is possible to develop valuable insights into future market conditions and demand levels based on a deep understanding of the forces behind total-market demand. These insights can sometimes make the difference between a winning strategy and one that flounders.
A forecast of total-market demand won’t guarantee a successful strategy. But without it, decisions on investment, marketing support, and other resource allocations will be based on hidden, unconscious assumptions about industrywide requirements, and they’ll often be wrong. By gauging total-market demand explicitly, you have a better chance of controlling your company’s destiny. Merely going through the process has merit for a management team. Instead of just coming out with pat answers, numbers, and targets, the team is forced to rethink the competitive environment. Total-market forecasting is only the first stage in creating a strategy. When you’ve finished your forecast, you’re not done with the planning process by any means.
There are four steps in any total-market forecast:
At the outset, it’s best to be overly inclusive in defining the total market. Define it broadly enough to include all potential end users so that you can both identify the appropriate drivers of demand and reduce the risk of surprise product substitutions.
The factors that drive forecasts of total-market size differ markedly from those that determine a particular product’s market share or product-category share. For example, total-market demand for office telecommunications products nationally depends in part on the number of people in offices and their needs and habits, while total demand for PBX systems depends on how they compare on price and benefits with substitute products like the local telephone company’s central office switching service. Beyond this, demand for a particular PBX is a function of price and benefit comparisons with other PBXs.
In defining the market, an understanding of product substitution is critical. Customers might behave differently if the price or performance of potential substitute products changes. One company studying total demand for industrial paper tubes had to consider closely related uses of metal and plastic tubes to prevent customer switching among tubes from biasing the results.
Understand, too, that a completely new product could displace one that hitherto had comprised the entire market—like the electronic calculator, which eliminated the slide rule. For a while after AT&T’s divestiture, the Bell telephone companies continued to forecast volume of long-distance calls by using historical trend lines of their revenues—as if they were still part of a monopoly. Naturally, these forecasts grew more inaccurate with time as end users were presented with new choices. The companies are now broadening their market definitions to take account of heightened competition from other long-distance carriers.
There are several ways you can make sure you include all important substitute products (both current and potential). From interviews with industrial customers you can learn about substitutes they are studying or about product usage patterns that imply future switching opportunities. Moreover, market research can lead to insights about consumer products. Speaking with experts in the relevant technologies or reviewing technological literature can help you identify potential developments that could threaten your industry.
Finally, careful quantification of the economic value of alternative products to different customers can yield deep insights into potential switching behavior—for example, how oil price movements would affect plastics prices, which in turn would affect plastic products’ ability to substitute for metal or paper.
Analyses like these can lead to the construction of industry demand curves—graphs representing the relationship between price and volume. With an appropriate definition, the total-industry demand curves will often be steeper than demand curves for individual products in the industry. Consumers, for example, are far more likely to switch from Maxwell House to Folgers coffee if Maxwell House’s prices increase than they are to stop buying coffee if all coffee prices rise.
In some cases, managers can make quick judgments about market definition. In other cases, they’ll have to give their market considerable thought and analysis. A total-market forecast may not be critical to business strategy if market definition is very difficult or the products under study have small market shares. Instead, your principal challenge may be to understand product substitution and competitiveness. One company analyzed the potential market for new consumer food cans, and it concluded that growth trends in food product markets were not critical to the strategy question. What was critical was knowing the value positions of the new packages relative to metal cans, glass jars, and composite cans. So the company spent time on that subject.
The second step in forecasting is to divide total demand into its main components for separate analysis.
There are two criteria to keep in mind when choosing segments: make each category small and homogeneous enough so that the drivers of demand will apply consistently across its various elements; make each large enough so that the analysis will be worth the effort. Of course, this is a matter of judgment.
You may find it useful in making this judgment to imagine alternative segmentations (based on end-use customer groups, for example, or type of purchase). Then hypothesize their key drivers of demand (discussed later) and decide how much detail is required to capture the true situation. As the assessment continues, managers can return to this stage and reexamine whether the initial decisions still stand up.
Managers may wish to use a “tree” diagram like the accompanying one constructed by a management team in 1985 to study demand for paper. In this disguised example, industry data permitted the division of demand into 12 end-use categories. Some categories, like business forms and reprographic paper, were big contributors to total consumption; others, such as labels, were not. One (other converting) was fairly large but too diverse for deep analysis. The team focused on the four segments that accounted for 80% of 1985 demand. It then developed secondary branches of the tree to further dissect these categories and to determine their drivers of demand. It analyzed the remaining segments less completely (that is, via a regression against broad macroeconomic trends).
Other companies have used similar methods to segment total demand. One company divided demand for maritime satellite terminals by type of ship (e.g., seismic ships, bulk/cargo/container ships). Another divided demand for long-distance telephone service into business and residential customers and then subdivided it by usage level. And a third segmented consumer appliances into three purchase types—appliances used in new home construction, replacement appliance sales in existing homes, and appliance penetration in existing homes.
In thinking about market divisions, managers need to decide whether
to use existing data on
segment sizes or to commission research to get an independent estimate. Reliable public information on historical
demand levels by segment is available for many big U.S. industries (like steel, automobiles, and natural gas) from industry associations, the federal
government, off-the-shelf studies by industry experts, or ongoing market
data services. For some foreign markets and less well-researched industries
in the United States, like the labels industry, you may have to get independent estimates. Even with good data sources, however, the readily available information may not be divided into the best
categories to support an insightful analysis. In these cases, managers must decide whether to develop their forecasts
based on the available historical
data or to undertake their own market research
programs, which can be time-consuming and expensive.
Note that while such segmentation is sufficient for forecasting total demand, it may not create categories useful for developing a marketing strategy. A single product may be driven by entirely different factors. One study of industrial components found that consumer industry categories provided a good basis for projecting total-market demand but gave only limited help in formulating a strategy based on customer preferences: distinguishing those who buy on price from those who buy on service, product quality, or other benefits. Such buying-factor categories generally do not correlate with the customer industry categories used for forecasting. A strong sales force, however, can identify customer preferences and develop appropriate account tactics for each one.
The third step is to understand and forecast the drivers of demand in each category. Here you can make good use of regressions and other statistical techniques to find some causes for changes in historical demand. But this is only a start. The tougher challenge is to look beyond the data on which regressions can easily be based to other factors where data are much harder to find. Then you need to develop a point of view on how those other factors may themselves change in the future. An end-use analysis from the commodity paper example, reprographic paper, is shown in the accompanying chart. The management team, using available data, divided reprographic paper into two categories: plain-paper copier paper and nonimpact page printer paper. Without this important differentiation, the drivers of demand would have been masked, making it hard to forecast effectively.
In most cases, managers can safely assume that demand is affected both by macroeconomic variables and by industry-specific developments. In looking at plain- paper copier paper, the team used simple and multiple regression analyses to test relationships with macroeconomic factors like white-collar workers, population, and economic performance. Most of the factors had a significant effect on demand. Intuitively, it also made sense to the team that the level of business activity would relate to paper consumption levels. (Economists sometimes refer to growth in demand due to factors like these as an “outward shift” in the demand curve—toward a greater quantity demanded at a given price.)
Demand growth for copy paper, however, had exceeded the real rate of economic growth and the challenge was to find what other factors had been causing this. The team hypothesized that declining copy costs had caused this increased usage. The relationship was proved by estimating the substantial cost reductions that had occurred, combining those with numbers of tons produced over time, and then fashioning an indicative demand curve for copy paper. (See the chart “Understanding Copy Paper Demand Drivers.”) The clear relationship between cost and volume meant that cost reductions had been an important cause of past demand growth. (Economists sometimes describe this as a downward-shifting supply curve leading to movement down the demand curve.)
Further major declines in cost per copy seemed unlikely because paper costs were expected to remain flat, and the data indicated little increase in price elasticity, even if cost per copy fell further. So the team concluded that usage growth (per level of economic performance) was likely to continue the flattening trend begun in 1983: growth in copy paper consumption would be largely a function of economic growth, not cost declines as in the past. The team then reviewed several econometric services forecasts to develop a base case economic forecast.
Similar studies have been performed in other industries. A simple one was the industrial components analysis mentioned before, a case where the total forecast was used as background but was not critical to the company’s strategy decision. Here the team divided demand into its consuming industries and then asked experts in each industry for production forecasts. Total demand for components was projected on the assumption that it would move parallel to a weight-averaged forecast of these customer industries. Actual demand three years later was 2% above the team’s prediction, probably because the industry experts underestimated the impact of the economic recovery of 1984 and 1985.
In another example, a team forecasting demand for maritime satellite terminals extrapolated past penetration curves for each of five categories of ships. These curves were then adjusted for major changes in the shipping industry (e.g., adding the depressing effect of the growing oil glut, taking out of these historical trends the unnatural demand growth that had been caused by the Falklands war). The actual figure three years later was within 1% of the forecast.
Knowing the drivers of demand is crucial to the success of any total-market demand forecast. In 1974, as I mentioned earlier, most electric utilities used an incomplete total- demand forecast to predict robust demand growth. In the early 1980s, one company’s management team, however, decided to study potential changes in end-user demand as well. The team divided electricity demand into the three traditional categories: residential, commercial, and industrial. It then profiled differences in residential demand because of more efficiency in home appliances and changes in home size and the ratio of multi-unit to single-family dwellings. Industrial demand was analyzed by evaluating the future of several key consuming industries, paying special attention to changes in their total production and electricity use. This end-use approach sharply reduced the utility’s initial forecasts and led to cancellation of two $700 million generating plants then in the planning stage.
In 1983, forecasters in the U.S. personal computer industry were saying that demand would continue to rise at a rapid rate because there were 50 million white-collar workers and only 8 million installed PCs. One company, however, did a more detailed demand forecast that showed that growth would soon flatten out. It found that more than two- thirds of white-collar workers either did not require PCs in their jobs—actors and elevator operators, for instance—or were supported mostly by inexpensive terminals linked to large computers, as in the case of many clerical workers. The potential market was not big enough to support the growth rate. Indeed, the market began to flatten the next year.
Forecasting total demand became crucial for another company that was thinking about acquiring a maker of video games. Many thought that low overall market penetration (10% of U.S. households) signified a lot of room for growth before the market became saturated, when about 50% of the households would have games. Using available data, however, the management team created categories based on family income and children’s ages. The analysis made clear that the main target market, upper-income families with children, was already well penetrated. Families with incomes exceeding $50,000 and children between the ages of 6 and 15 already were 75% penetrated. This finding convinced management that demand would fall and that the proposed acquisition did not make sense. The dramatic decline in video game sales shortly thereafter confirmed the wisdom of this judgment.
Managers who rely on single-point demand forecasts run dangerous risks. Some of the macroeconomic variables behind the forecasts could be wrong. Despite the best analysis, moreover, the assumptions behind the other demand drivers could also be wrong, especially if discontinuities loom on the horizon. Imaginative marketers who ask questions like “What things could cause this forecast to change dramatically?” produce the best estimates. They are more likely to identify potential risks and discontinuities— developments in competing technologies, in customer industry competitiveness, in supplier cost structures—than those who do not. So once a baseline forecast is complete, the challenge is to determine how far it could be off target.
At one level, such a sensitivity analysis can be done by simply varying assumptions and quantifying their impact on demand. But a more targeted approach usually provides better insight.
Begin such an analysis by thinking through and quantifying the areas of greatest strategic risk. One company’s strategy decision may be affected only if demand is well below the baseline forecast; in another case, big risks may result from small forecasting errors.
Next, gauge the likelihood of such a development. In the white paper example, the baseline forecast called for continued market growth, though below historical levels. In any particular year, demand could fluctuate with the economy, but the critical question was whether demand would at some point begin a long decline. If so, the companion supply-curve analysis indicated that prices would probably fall dramatically.
The team created two scenarios of a gradual decline, one based largely on changes in the economy and the other on changes in assumed end-use trends. These scenarios showed what would make demand fall (e.g., different rates of decline in copier prices) and thereby provided a basis for evaluating the likelihood of a downturn.
The forecasting framework outlined above can work for both comprehensive and simple assessments, but there are different ways to carry out these analyses. A big challenge in demand forecasting (just as with other types of market analysis) is to gauge the appropriate effort for the project’s purpose. It’s useful to ask: “How much do I need to know to make the decision at hand?”
Managers can invest a lot of time in such analyses—the paper example took about 8 man- weeks and the large-scale electricity forecast about 14 man-weeks. Some companies have forecasting departments who work year-round on these subjects. The more thorough, though time-consuming, approach generates greater confidence, and the effort will be appropriate where the demand projection can significantly influence corporate strategy (whether to make a several hundred million dollar capital investment, for example), or where there is great uncertainty about total demand.
Often, however, the issues are not complicated, time is limited, or the total demand forecast is not important enough to merit that commitment (for example, the company is looking to add a couple of points to its small market share). In such cases, managers should proceed quickly and inexpensively. They can, for example, rely on experts’ judgment or unsophisticated regressions to forecast drivers of demand. Even the limited approaches can yield insights. Furthermore, beginning the demand analysis process can help managers determine whether important demand issues exist that should be analyzed in greater depth.
Total-demand forecasting can be important to strategy decisions. Developing independent forecasts through the four-step framework I’ve outlined will not only lead to better recommendations but also help build conviction and consensus for action by creating understanding of the drivers of demand and the risks in forecasts.
Even when the work is sound, though, uncertainties will remain: discontinuities will still be difficult to predict, especially if they are rooted in momentous political, macroeconomic, or technological changes. But managers who push their thinking through the steps in this framework will have a better chance of finding these discontinuities than those who do not. And those who base their business strategies on a solid knowledge of demand will stand a much greater chance of making wise investments and competing effectively.
A version of this article appeared in the July 1988 issue of Harvard Business Review.
