您还没有绑定微信,更多功能请点击绑定

第十五篇 Assessing Forecast Accuracy

本帖最后由 小编D 于 2011-12-16 16:12 编辑 _


本文翻译者:lindamsh 校稿者:chengguo0740

Assessing Forecast Accuracy: Be Prepared, Rain or Shine
评估预测的准确性:无论晴雨,都应做好准备
Practitioners can assess the accuracy of forecasts using control charting and analysis of variance (ANOVA). Screening a corporation's forecasts with these two tools will reveal the evolution of forecast bias and consistency over time.从业者可以利用控制制图, 方差变异分析来评估预报的准确性。为了展示一个公司的前景,用这两个工具将会揭示随着时间的变化,预测偏差和一致性的演化。
By http://www.isixsigma.com/index ... Ohler作者Michael Ohler
Just as people follow weather predictions to know if they should carry an umbrella, organizations use forecasting to predict and prepare for future events. Across industries, companies attempt to determine what will happen – they forecast for product or raw material prices, market demand, exchange rates, and numerous other key metrics. Based on these forecasts, production sites are built, new products are launched and business entities are re-scaled.就像人们根据天气预告来判断他们是否应该带伞,组织使用预测来预报和准备未来的事情。在整个行业中,公司会试图测定未来会发生什么——他们预测产品或原材料的价格、市场需求、汇率和其它诸多主要的绩效指标。在这些预测的基础上,建立生产基地,推出新产品,重新调整商业实体。
While such forecasts are crucial to making significant business decisions, they may not be as accurate as people think. Organizations often assess the accuracy of forecasts in an elementary way. They monitor, for example, the last available forecast in comparison to the current actual values. Or they may compare the current to the last forecast for a given business quarter.虽然这种预测对做出一个重大的商业决策是至关重要的,但是它们可能并不像人们想象的那样准确。组织经常用一种基本的方法来评估的这种预测的准确性。例如,他们来监控这些,用当前的实际数值和最新的可用的预报相比,或者他们比较当前的和最新的一个季度的业务预测。
In his book _The Black Swan: The Impact of the Highly Improbable_ (Random House, 2007), Nassim Nicholas Taleb points out the shortcomings of forecasting in light of the randomness involved in business processes. To address this randomness, practitioners can assess the accuracy of forecasts using control charting and analysis of variance (ANOVA). Screening a corporation’s forecasts with these two tools will reveal the evolution of forecast bias and consistency over time.在他的书《黑天鹅》中:“极不可能的影响 (随机的屋子,2007), Nassim Nicholas Taleb指出了在业务流程中随机性预测的缺陷。为了阐明这一问题的随机性、从业者可以利用控制制图和方差分析来评估预测的准确性。为了展示一个公司的前景,用这两个工具将会揭示随着时间的变化,预测偏差和一致性的演化。
Forecasting Basics预测的偏差In corporations, there are typically two methods for compiling a forecast. In one way, past performance is extrapolated into the future. This approach assumes that the underlying mechanisms remain the same. For example, hotels might forecast their occupancy rate across seasons and plan accordingly for hiring temporary staff. Or, an organization might aggregate multiple individual anticipations of what will happen into one system. For example, predictions from sales agents are consolidated in one database.在公司通常有两种方法来编译一个预测。在某种程度上,过去的表现可以推断未来。该方法假潜在的设机制是相同的。例如, 酒店根据季节可以预测入住率,根据这个来计划临时员工招聘的需求。或者,一个组织可以集合成倍的个体建议并汇成一个体系。例如,将销售代理预测合并在一个数据库。
To analyze properly the accuracy of forecasts provided by either method, practitioners must understand the following terms:为了用两种方法中的任一种方法来恰当地分析预测的准确性,从业人员必须了解以下术语:
_Forecast horizon_ describes how far into the future a metric is forecast. For example, weather forecasts typically have a horizon of up to 10 days.预测范围是描述预测未来多久的一个公制。例如,天气预报通常长达10天的范围。
_Forecast bias_ measures how much, on average, forecasts overestimate or underestimate future values. For example, a sales forecast may have a positive (optimistic) or a negative (pessimistic) bias. A positively biased sales forecast, on average, predicts higher sales than what is later achieved. Such a bias can occur when business units get allocated production capacity according to their forecasts and thus have an incentive to be optimistic.预测偏差指测量偏差有多少,一般来说,预测评估最高值或最低值相差有多少。例如,一个销售预测可能有一个正的(乐观的)或负的(悲观)的偏差。一般来说,一个正偏差的销售预测,销售预测比后来实际值要高。这样一种偏差可能发生在,业务单位根据他们的预报来分配生产能力,就会有一个乐观的动机。
_Forecast consistency_ quantifies the spread of forecasts. People expect a forecast with a short horizon (e.g., the high temperature provided in a one-day weather forecast) to be more consistent than a forecast with a long horizon (high temperature 10 days from now). The reason is that for longer horizons, unknown or non-understood influences typically play a more important role, and the high temperature 10 days ahead may deviate significantly in either direction from what is forecast today.预测一致性量化了预测传播。人们期望一个短范围的预测(如为期一天的天气预报高温)同长期范围的预测一致,(从现在起10天高温),原因是对于一个长期的预测,不知道的或不明确的影响因素很大,而10天高温的预测可能会明显偏离今天的预测。
Forecast bias and consistency are two important elements of forecast accuracy. Notice that bias and consistency are aggregated quantities based on multiple offsets between forecasted and actual values. In other words, they usually are determined using predictions and actual data over a period of time. The offset typically is measured in one of two ways:预测偏差和一致性对于预测准确性来说是两个重要的因素。我们注意到基于预测和真实值之间若干的偏移量,偏差和一致性的量聚集在一起。换句话说,他们通常取决于一段时间的预测和实际数据。偏移量通常用两种不同的方式来测量:_1. __Forecast – Actua_1.预测-实际When a time series, or a sequence of data points, can be considered stationary 当一段连续的时间,或一序列的数据点,偏移量可以考虑是稳定的。_2. __Forecast - Actual _2.预测-实际Actual for a time series with a long-term trend一段连续时间的真实数据和一个长期的趋势A prominent example of a long-term trend can be seen in computer memory capacity, which has, over the past 50 years or so, followed a trend first observed by Intel co-founder Gordon Moore. As Moore’s Law predicts, about every two years there is a doubling of the number of transistors that can be placed on an integrated circuit, allowing for a consistent increase in memory.有一个很突出的例子,长期的趋势可以在计算机内存容量中看出, 这已经持续了50年左右,是由英特尔的共同创始人Gordon Moore第一次观察出来的趋势。作为摩尔定律的预测,大约每两年会有两倍的晶体管数量可以放置到集成电路,使存储一致性地增长。
Figure 1: Factors Influencing Forecast Bias and Consistency图1:预测偏差的影响因素和一致性

The fishbone diagrams in Figure 1 introduce the two independent parameters, time and forecast horizon, that influence forecast bias and consistency. Also shown are the tools used to analyze them.图1中的鱼骨图介绍了影响预测偏差和一致性的两种独立参数、时间和预测范围,同样也展示了用来分析这些数据的工具
Case Study: Forecasting Price of a Subassembly Component案例研究:一组组件的价格预测
As an example, consider the price forecast data over 20 months of a component used for a subassembly (Figure 2). Price forecasts were made over a horizon of one to six months in advance. For instance, a three-month forecast (3mFC) for the price in Month 9 is actually made in Month 6. This data is derived from a supply-and-demand based model, which considers production capacity to determine future supply and demand, and derive sales prices.举例,考虑用于组装的一个组件超过20个月的价格预测数据,(图2 提前6个月范围的价格预测。)比如,一个为期3个月的价格预测(3 mFC),在第9个月价格实际上是由第6个月得出的,这些数据源于供求关系模式,由生产能力确定未来的供应和需求,并推导出销售价格。
Figure 2: Actual Component Price Data vs. Forecasts for Different Horizons图2:在不同的预测范围下,实际组件价格数据对预测部件价格



Using control charts, it is possible to confirm that the dip of the actual price from Months 8 through 10 is significant across the time frame explored here (Figure 3). Until Month 10, forecasts are for the most part positively biased, whereas later they display a significant negative bias (Figure 4). In Figure 4, the time section “Before” the dip displays a lower control limit of about zero, which means that the average of forecasts minus actuals is significantly larger than zero, and thus positively biased. For the time section “After” the dip, the upper control limit is about zero, which shows that forecasts are now negatively biased.利用控制图,就能确定从8mFC到10mFC个月的实际价格的点明显超过了这里展示图。尽管后来它们表现出了一个重要的负偏差(图4,在图4种,从时间段上显示了倾角以前,对于零点的控制限较低,这意味着预测的平均值减去实际值明显大于零,从而积极偏见。对于倾角以后,上控制限是零,结果表明预测有负偏差。
Figure 3: Individuals Control Chart of Actual Pricess图3: 实际价格的单值控制图
Figure 4: X-Bar Chart of Forecasts Minus the Actual Values of Horizons of One to Six Months in the Future图4: 在未来一到六个月的时间里,预测的X平均值减去范围内的实际值。


The relation of the price forecasts to the actual price of the component can be understood from a business context. The company’s purchasing and production departments celebrated the price reduction after Month 8 as a permanent breakthrough, and simply brushed off the idea of prices regaining their previous levels. Forecasters, who received most of their information from the purchasing department, were thus induced to adapt parameters in their models according to this general belief.从商业报道中可以我们可以看出一个组件的预测价格和实际价格的关系。公司的采购及生产部门在第8个月后会将降价后的庆祝作为一个持续性的突破,并轻易的放弃价格恢复到以前水平的想法。根据这个普遍的想法,从采购部门接收大量信息的预报员会因此在他们的模式中采用这些参数。
Unfortunately, such occurrences appear to be commonplace; because a “think positive” attitude is often rewarded in the corporate world, experts with an understanding of complex market mechanisms may be listened to too late. To avoid this, organizations should ask the experts to quantify the likelihood of the perceived risk, assess its potential impact and prepare a contingency plan accordingly.不幸的是,这样的事件似乎是司空见惯的事,因为在这个团体中我们经常会有“正向思考”的态度, 带着理解这个复杂的市场机制态度的专家会因此听力顿觉。为了避免此类情形,组织应该让专家量化认知风险的可能性,评估其可能产生的影响,从而准备一个应急计划。
Setting the Time Period设定时间期限
Is a one-month forecast any better than a six-month forecast? Translated into mathematical terms, this question is asking: Do one-month forecasts spread around the actual values significantly less than six-month forecasts? Using the forecast-minus-actual metric, practitioners can perform an ANOVA test for equal variances to answer this question. For the data above, when the time frame of the studied 20 months is considered, forecast bias does not significantly depend on the forecast horizon (Figure 5). The dots indicate the standard deviations and the lines their 95 percent confidence intervals.一个月的预测一定比六个月更好吗?把这个转化为一个数学术语,这个问题可以这样来问:围绕真实数据做一个月预报会远远低于六个月的预报吗?利用这个预报减去真实值的公式,从业者可以对分散的同一性进行方差分析测试来回答这个问题。根据以上数据,考虑20个月研究的时间轴时,预测偏差不会明显预测范围(图5,这些圆点显示了标准差和95%置信区间线。
Figure 5: Test for Equal Variances of the Forecast Minus Actual Data图5:预测值减去真实值的方差试验



Determining Confidence Limits确定置信区间
The analysis of the forecast-minus-actual price data can be used to estimate forecast confidence limits based on past forecast performance. Assuming that the forecast process delivers results with the same performance as before, consistency (Figure 6) and bias (found not to be significantly different from zero) from the previous ANOVA assessment can be used to display confidence limits around the forecasted values.预测值减去真实价格的数据分析在基于过去的预测记录上可以用来评估预测置信区间,假设预测结果和以前的预测一样。之前从方差评估中的一致性(图6)和偏差(与零点的偏离不是很明显)可以得出预测值的置信区间。
Figure 6: 1-sigma Confidence Limits Around Forecasted Values图6:预测价格1-sigma置信区间
There is a roughly 30 percent chance that the actual price values will land outside these 1-sigma confidence limits (15.9 percent chance for a value above the upper 1-sigma confidence limit and 15.9 percent for a value below the lower 1-sigma confidence limit). Typically, decisions are based on a 95 percent or higher confidence interval with far wider limits; here, only a 70 percent confidence interval is considered. The price three months in the future could either be at $3.90 or at $2.90, with a 30 percent chance it would be beyond these two values. The actions that would need to be taken for either of the two cases are of a completely different nature.大约有30%的机会真实价格会落在1-sigma的置信区间外(有15.9%的机会落在1-sigma以上的置信区间,有15.9%的机会落在1-sigma以下的置信区间) 决策取决于一个95% 或者更高的更宽泛的置信区间。这里仅考虑70%的置信区间。未来三个月价格在内将会是$ 3.90或2.90美元,有30%的机会价格会超出这两个值。用来解决任意这两个问题的措施属于完全不同的性质From the practical point of view for managing the subassembly business, forecasts with such accuracy at this confidence level are considered virtually worthless. A useful forecast is one in which the upper and lower confidence limits, at a reasonably high confidence level, lead to about the same practical consequence.为了管理组件的业务,从实用的观点上,在这个置信水平下,这样精确的预测是毫无价值的。一个有用的预测是在上下线的置信区间内,取一个合理的高置信水平,这会使预测得到相同的实际结果。
Considering Models考虑的模型While a supply-and-demand-based model of forecasting is applied in the case study example, there are many potential forecasting models. The more elaborate a model, the more the forecasts of such models tend to be taken for granted. The methods discussed in this article can gauge the validity of forecasts at any level of sophistication.当一种供求关系的模型应用到实际研究的例子中,有许多潜在的预测模型。模型越详细,这些模型的预测更容易被认可。本文讨论的模型就可以在任何复杂的情况下测量预报的有效性。
Addressing Inaccurate Forecasts解决不准确的预测
Faced with a forecast that does not provide useful data, a team is left with two alternatives:面对一个不能提供有用数据的预测,有两种方法可以选择:1. Significantly improve the forecasting process and its accuracy1.重点改进预测过程及其精度2. Improve the reaction time of the process such that less forecast accuracy is needed2.改进过程反应时间,这样会对预测的准确性的需求较少
The impossibility of the first in many business contexts led Taleb to write that "we simply cannot forecast." But not forecasting is not an option in today’s business world. Because of the resulting necessity of the second alternative, Six Sigma teams may find forecasting accuracy a tremendous source for improvement projects. For example, if a product mix forecast is inaccurate, a Six Sigma team could launch a project to reduce lead times in order to improve reaction time to changing demands. Or, if forecasts for product and raw material prices are inaccurate, a team could lead a project to decrease waste in the supply chain. 第一种(选择)在很多业务环境中的不可能性使得Taleb写道:“我们简直不能预测”。但是在当今的商业世界中不做预测不是一种选择。 由于第二种选择导致的必要性,六西格玛团队也许发现对于改进方案来说,预测准确性是一个巨大的源头。 比如,假设一种结合了预测的产品是不正确的,六西格玛团队可以启动一个方案来减少预定时间,目的是通过改进反应时间来变换需求。 或者,假设产品和原材料价格的预测是不准确的,团队可以启动一个方案来减少在供应链中的浪费。
As Taleb describes eloquently, the hardest part may not be to assess the accuracy of forecasts but rather to make the results generally accepted throughout the organization and to translate them into a consistent approach for improving and managing business processes。正如Taleb辩证阐述的一样,最难的部分不是评估预测的准确性,而是在整个组织得到普遍的接受的结果并且把它们转化成一种改进和管理业务流程的一贯性的方法。
_About the Author:__ Michael Ohler has worked on financial forecasting as cost controller, and quality and project manager. He is a certified Master Black Belt and holds a doctorate in experimental physics. He can be reached at http://www.ohlermichael.de/._
关于作者:_ Michael Ohler_曾作为成本控制员就职于财务预测领域,质量和项目经理。他是一个得到认证的黑带大师,有一个实验物理学博士学位的头衔。可以通过http://www.ohlermichael.de/联系到他。
6.jpg 5.jpg 4.jpg 3.jpg 2.jpg 1.jpg
对“好”的回答一定要点个"赞",回答者需要你的鼓励!
已邀请:

chengguo0740 (威望:2) (天津 北辰) 在校学生 品质部长

赞同来自:

本帖最后由 chengguo0740 于 2011-7-8 17:21 编辑

  1. Improve the reaction time of the process such that less forecast accuracy is needed
对以上文字,本人的翻译如下,仅供参考:
  1. 改进过程反应时间,这样会对预测的准确性的需求较少


The impossibility of the first in many business contexts led Taleb to write that "we simply cannot forecast." But not forecasting is not an option in today’s business world. Because of the resulting necessity of the second alternative, Six Sigma teams may find forecasting accuracy a tremendous source for improvement projects. For example, if a product mix forecast is inaccurate, a Six Sigma team could launch a project to reduce lead times in order to improve reaction time to changing demands. Or, if forecasts for product and raw material prices are inaccurate, a team could lead a project to decrease waste in the supply chain.

第一种(选择)在很多业务环境中的不可能性使得Taleb写道:“我们简直不能预测”。但是在当今的商业世界中不做预测不是一种选择。 由于第二种选择导致的必要性,六西格玛团队也许发现对于改进方案来说,预测准确性是一个巨大的源头。 比如,假设一种结合了预测的产品是不正确的,六西格玛团队可以启动一个方案来减少预定时间,目的是通过改进反应时间来变换需求。 或者,假设产品和原材料价格的预测是不准确的,团队可以启动一个方案来减少在供应链中的浪费。

5 个回复,游客无法查看回复,更多功能请登录注册

发起人

扫一扫微信订阅<6SQ每周精选>