The COVID-19 pandemic has forced retail to innovate for survival. BII presents its assessment of how brick and mortar retail can apply technologies to assist in its re-emergence and to ultimately utilize human associates to their greatest potential.
Recent history is filled with stories of companies and sometimes even entire industries that have made grave strategic errors because of inaccurate industrywide demand forecasts. For example:
In 1974, U.S. electric utilities made plans to double generating capacity by the mid- 1980s based on forecasts of a 7% annual growth in demand. Such forecasts are crucial since companies must begin building new generating plants five to ten years before they are to come on line. But during the 1975–1985 period, load actually grew at only a 2% rate. Despite the postponement or cancellation of many projects, the excess generating capacity has hurt the industry financial situation and led to higher customer rates.
The petroleum industry invested $500 billion worldwide in 1980 and 1981 because it expected oil prices to rise 50% by 1985. The estimate was based on forecasts that the market would grow from 52 million barrels of oil a day in 1979 to 60 million barrels in 1985. Instead, demand had fallen to 46 million barrels by 1985. Prices collapsed, creating huge losses in drilling, production, refining, and shipping investments.
In 1983 and 1984, 67 new types of business personal computers were introduced to the U.S. market, and most companies were expecting explosive growth. One industry forecasting service projected an installed base of 27 million units by 1988; another predicted 28 million units by 1987. In fact, only 15 million units had been shipped by 1986. By then, many manufacturers had abandoned the PC market or gone out of business altogether.
The inaccurate suppositions did not stem from a lack
of forecasting techniques; regression analysis, historical trend smoothing, and others
were available to all the players. Instead, they shared a mistaken
fundamental assumption: that relationships driving demand in the past would continue unaltered. The companies
didn’t foresee changes in end-user behavior or understand their market’s saturation point. None realized that history can be an unreliable
guide as domestic
economies become more international, new technologies emerge, and industries evolve.
As a result of
changes like these, many managers have come to distrust traditional techniques. Some even throw up their
hands and assume
that business planning must
proceed without good demand forecasts. I disagree. It is possible
to develop valuable insights into future market conditions and demand
levels based on a deep understanding
of the forces behind total-market demand. These insights can sometimes make the difference between a winning
strategy and one that flounders.
of total-market demand won’t guarantee a successful strategy. But without it, decisions on investment, marketing support, and other resource
allocations will be based on
hidden, unconscious assumptions about industrywide requirements, and they’ll often be wrong. By gauging total-market demand
explicitly, you have a better chance
of controlling your company’s destiny. Merely going through the process
has merit for a
management team. Instead of just coming
out with pat answers, numbers, and targets, the team is forced to rethink
the competitive environment.
Total-market forecasting is only the first
stage in creating
a strategy. When you’ve finished your forecast, you’re not done with the
planning process by any means.
There are four steps in any total-market forecast:
Define the market.
Divide total industry demand into its main components.
Forecast the drivers of demand in each segment and project how they are likely to change.
Conduct sensitivity analyses to understand the most critical assumptions and to gauge risks to the baseline forecast.
Deﬁning the Market
At the outset, it’s best to be overly
inclusive in defining
the total market. Define it broadly enough
to include all potential end users so that you can both identify
the appropriate drivers of demand and reduce the risk of surprise
The factors that drive forecasts of total-market size
differ markedly from those that determine a particular product’s market share or product-category share. For example, total-market demand for office telecommunications products nationally depends in part on the number
of people in offices and their needs
and habits, while total demand for PBX systems depends on how they compare on price and benefits with substitute products like the local telephone
company’s central office switching service. Beyond this, demand for a particular PBX is a function
of price and benefit comparisons with other
In defining the market, an understanding of product substitution is critical. Customers might behave differently if the price or performance of potential substitute products changes. One company studying total demand for industrial paper tubes had to consider closely related uses of metal and plastic tubes to prevent customer switching among tubes from biasing the results.
Understand, too, that a completely new product could displace one that hitherto had comprised the entire market—like the electronic calculator, which eliminated the slide rule. For a while after AT&T’s divestiture, the Bell telephone
companies continued to forecast volume of long-distance calls by using historical trend lines of their revenues—as if they were
still part of a monopoly. Naturally, these forecasts grew more inaccurate with time as end users were
presented with new
choices. The companies are now broadening their market definitions to take account of
heightened competition from other
There are several ways you can make sure you include
all important substitute products (both current
and potential). From interviews with industrial customers you can learn about substitutes they are studying
or about product usage
patterns that imply future switching
opportunities. Moreover, market research can lead to insights about consumer
products. Speaking with experts in the relevant technologies or reviewing
can help you identify
potential developments that could threaten your industry.
Finally, careful quantification of the economic value of alternative products to different customers can yield
deep insights into potential switching behavior—for example, how oil price movements would affect plastics prices, which in turn would
affect plastic products’ ability to substitute for metal or paper.
Analyses like these can lead to
the construction of industry demand curves—graphs representing the relationship between price and
volume. With an appropriate definition, the total-industry demand curves will often be steeper
than demand curves for individual
products in the industry. Consumers, for example, are far more likely to switch from Maxwell House to Folgers
coffee if Maxwell
House’s prices increase than they are to stop buying coffee if all coffee prices rise.
In some cases, managers can make quick judgments about market definition. In other cases, they’ll have to give their market considerable thought and analysis. A total-market forecast may not be critical to business strategy if market definition is very difficult or the products under study have small market shares. Instead, your principal challenge may be to understand product substitution and competitiveness. One company analyzed the potential market for new consumer food cans, and it concluded that growth trends in food product markets were not critical to the strategy question. What was critical was knowing the value positions of the new packages relative to metal cans, glass jars, and composite cans. So the company spent time on that subject.
Dividing Demand into Component Parts
The second step in forecasting is to divide total demand into its main components for separate
There are two criteria to keep in
mind when choosing segments: make each
category small and homogeneous enough so that the drivers
of demand will apply consistently across its various
elements; make each large enough so that the analysis will be worth the
effort. Of course, this is a matter of judgment.
You may find it useful in
making this judgment to imagine alternative
segmentations (based on end-use customer groups, for example, or type of purchase). Then hypothesize their key drivers of demand
(discussed later) and decide how much detail is required
to capture the true situation. As the assessment continues, managers can return to this
stage and reexamine whether the
initial decisions still stand up.
Managers may wish to use a “tree” diagram like the accompanying one constructed by a management
team in 1985 to study demand for paper. In this disguised
example, industry data permitted the division of
demand into 12 end-use categories. Some categories, like business forms and reprographic paper, were big contributors to total consumption; others, such as labels, were not. One (other converting) was fairly large but too diverse
for deep analysis. The team focused on the four segments that accounted for 80% of 1985 demand. It then developed secondary branches of the tree to further
dissect these categories and
to determine their drivers
of demand. It analyzed the remaining
segments less completely (that is, via a regression
against broad macroeconomic trends).
Other companies have used similar methods to segment
total demand. One company divided
demand for maritime satellite terminals by type of ship (e.g., seismic ships, bulk/cargo/container ships). Another divided demand for long-distance telephone service into business and residential customers and
then subdivided it by usage level. And a third segmented consumer appliances into three purchase types—appliances used in new home construction, replacement appliance sales in existing
homes, and appliance penetration in existing homes.
In thinking about market divisions, managers need to decide whether
to use existing data on
segment sizes or to commission research to get an independent estimate. Reliable public information on historical
demand levels by segment is available for many big U.S. industries (like steel, automobiles, and natural gas) from industry associations, the federal
government, off-the-shelf studies by industry experts, or ongoing market
data services. For some foreign markets and less well-researched industries
in the United States, like the labels industry, you may have to get independent estimates. Even with good data sources, however, the readily available information may not be divided into the best
categories to support an insightful analysis. In these cases, managers must decide whether to develop their forecasts
based on the available historical
data or to undertake their own market research
programs, which can be time-consuming and expensive.
Note that while such segmentation is sufficient for forecasting total demand, it may not create categories useful for developing a marketing
strategy. A single product may be driven
by entirely different factors. One study of industrial components found that consumer industry categories provided a good basis
for projecting total-market demand but gave only
limited help in formulating a
strategy based on customer preferences: distinguishing those who buy on price
from those who buy on service, product quality, or other benefits. Such buying-factor categories generally do not correlate
with the customer
industry categories used for forecasting. A strong sales force, however, can identify customer preferences and develop
appropriate account tactics for each one.
Forecasting the Drivers of Demand
The third step is to understand and forecast
the drivers of demand in each category. Here you can make good use of regressions and other
statistical techniques to find some
causes for changes
in historical demand. But this is only a start. The tougher challenge is to look beyond
the data on which regressions can
easily be based
to other factors
where data are much harder
to find. Then you need to develop
a point of view on how those other factors may themselves change in the future.
An end-use analysis from the commodity paper example, reprographic paper, is shown in
the accompanying chart. The management team, using available data, divided reprographic paper into two categories: plain-paper copier paper and nonimpact page printer
paper. Without this important differentiation, the drivers
of demand would have been masked, making it hard to forecast effectively.
In most cases, managers can safely assume that demand is affected both by macroeconomic variables and by industry-specific developments. In looking at plain- paper copier paper, the team used simple and multiple regression analyses to test relationships with macroeconomic factors like white-collar workers, population, and economic performance. Most of the factors had a significant effect on demand. Intuitively, it also made sense to the team that the level of business activity would relate to paper consumption levels. (Economists sometimes refer to growth in demand due to factors like these as an “outward shift” in the demand curve—toward a greater quantity demanded at a given price.)
Demand growth for copy paper, however, had exceeded the real rate of economic growth and the challenge was to find what other factors had been causing this. The team hypothesized that declining copy costs had caused this increased usage. The relationship was proved by estimating the substantial cost reductions that had occurred, combining those with numbers of tons produced over time, and then fashioning an indicative demand curve for copy paper. (See the chart “Understanding Copy Paper Demand Drivers.”) The clear relationship between cost and volume meant that cost reductions had been an important cause of past demand growth. (Economists sometimes describe this as a downward-shifting supply curve leading to movement down the demand curve.)
Further major declines in cost per
copy seemed unlikely because paper costs were expected
to remain flat, and the data indicated little increase in price elasticity, even if cost per copy
fell further. So the team
concluded that usage growth (per level of economic performance) was likely to continue the flattening trend begun in 1983: growth in copy paper
consumption would be largely a function
of economic growth, not cost declines as in the
past. The team then reviewed several econometric services forecasts to develop a base case economic
Similar studies have been performed in other industries. A simple one was the industrial components analysis mentioned before, a case where the total forecast was used as background but was not critical to the company’s strategy decision. Here the team divided demand into its consuming industries and then asked experts in each industry for production forecasts. Total demand for components was projected on the assumption that it would move parallel to a weight-averaged forecast of these customer industries. Actual demand three years later was 2% above the team’s prediction, probably because the industry experts underestimated the impact of the economic recovery of 1984 and 1985.
example, a team forecasting demand for maritime satellite terminals extrapolated past penetration curves for
each of five categories of ships. These curves were then adjusted for major
changes in the shipping industry (e.g., adding the
depressing effect of the growing oil
glut, taking out of these
historical trends the unnatural
demand growth that had been caused
by the Falklands
war). The actual figure three years later was within 1% of the forecast.
Knowing the drivers of demand is crucial
to the success of any total-market demand forecast. In 1974, as I mentioned
earlier, most electric utilities used an incomplete total- demand forecast to predict robust
demand growth. In the early 1980s, one company’s management team, however, decided to study potential changes in end-user demand as well. The team divided electricity demand into the three traditional categories: residential, commercial, and industrial. It then profiled differences in residential demand because of more efficiency in home appliances and changes
in home size and the ratio of
multi-unit to single-family dwellings. Industrial demand was analyzed by evaluating the future of several
key consuming industries, paying special attention to changes in their
total production and electricity use. This end-use approach sharply reduced the utility’s initial forecasts and led to cancellation of two $700 million generating plants then in the planning stage.
In 1983, forecasters in the U.S. personal computer industry were saying that demand would continue to rise at a rapid rate because there were 50 million white-collar workers and only 8 million installed PCs. One company, however, did a more detailed demand forecast that showed that growth would soon flatten out. It found that more than two- thirds of white-collar workers either did not require PCs in their jobs—actors and elevator operators, for instance—or were supported mostly by inexpensive terminals linked to large computers, as in the case of many clerical workers. The potential market was not big enough to support the growth rate. Indeed, the market began to flatten the next year.
Forecasting total demand became crucial for another company that was thinking about acquiring a maker of
video games. Many thought that low overall
market penetration (10% of U.S. households) signified a lot of room for growth before the market became saturated, when about 50%
of the households would have games. Using available data, however, the management team created categories based on family
income and children’s ages. The analysis made clear that the main target market, upper-income families with children, was already well penetrated. Families with incomes exceeding $50,000 and children between the ages of 6 and 15 already were 75% penetrated. This finding convinced management
that demand would fall and that the proposed acquisition did not
sense. The dramatic decline in video
game sales shortly thereafter confirmed the wisdom
of this judgment.
Managers who rely on single-point demand forecasts run dangerous
risks. Some of the macroeconomic variables behind the forecasts
could be wrong. Despite the best analysis, moreover, the assumptions behind the other demand drivers could also be wrong, especially
loom on the horizon. Imaginative marketers who ask questions like “What things could cause this forecast to change
dramatically?” produce the best estimates. They are more likely to identify
potential risks and
discontinuities— developments in competing technologies, in customer industry competitiveness, in supplier cost structures—than those who do not. So once a baseline forecast is complete, the challenge is to determine how far
it could be off
At one level, such a sensitivity analysis can be done by simply varying assumptions and quantifying their impact on demand. But a more targeted approach usually provides better insight.
Begin such an analysis
by thinking through and quantifying the areas of greatest strategic
risk. One company’s strategy decision may be affected
only if demand
is well below the
baseline forecast; in another case, big risks may result from small forecasting errors.
Next, gauge the likelihood
of such a development. In the
white paper example, the baseline
forecast called for continued market growth, though below historical levels. In any particular year, demand could fluctuate with the economy, but the critical question
was whether demand
would at some point
begin a long decline. If so, the companion supply-curve analysis indicated that prices would probably fall dramatically.
The team created two scenarios of a gradual decline, one based largely on changes
in the economy
and the other on changes
in assumed end-use trends. These scenarios showed what would make demand fall (e.g., different rates of decline in copier prices) and thereby provided a basis
for evaluating the
likelihood of a downturn.
The forecasting framework outlined above can work for both comprehensive and simple assessments, but there are different
ways to carry out these analyses. A big challenge in demand
forecasting (just as with
other types of market analysis) is to gauge the
appropriate effort for the
project’s purpose. It’s useful to ask: “How much do I need to know to make the decision at hand?”
Managers can invest a lot of time in such analyses—the paper example took about 8 man- weeks and the large-scale electricity forecast about 14 man-weeks. Some companies have forecasting departments who work year-round on these subjects. The more thorough, though time-consuming, approach generates greater confidence, and the effort will be appropriate where the demand projection can significantly influence corporate strategy (whether to make a several hundred million dollar capital investment, for example), or where there is great uncertainty about total demand.
Often, however, the issues are not complicated, time is limited, or the total demand forecast is not important enough to merit that commitment (for example, the company is looking to add a couple
of points to its small market share). In such cases, managers should proceed quickly and inexpensively. They can, for example, rely on experts’ judgment or unsophisticated regressions to forecast
drivers of demand. Even the limited approaches
can yield insights. Furthermore, beginning the demand analysis
process can help managers determine
whether important demand issues exist that should be analyzed
in greater depth.
can be important to
strategy decisions. Developing
independent forecasts through the four-step framework
will not only lead to
better recommendations but also help
build conviction and consensus for action by creating understanding of the drivers of
demand and the risks in forecasts.
Even when the work is sound, though, uncertainties will remain: discontinuities will still be difficult to predict, especially if they are
rooted in momentous political, macroeconomic, or technological
changes. But managers who push their
thinking through the steps in this
framework will have
a better chance of finding these discontinuities than those who do not. And those who base their
business strategies on a
solid knowledge of demand
will stand a much greater
chance of making
wise investments and competing
A version of this article
appeared in the July 1988 issue
of Harvard Business Review.