Home » Behind the Headlines » Measuring Forecasting Methodology & Accuracy

Measuring Forecasting Methodology & Accuracy

7-1-21 Updated to 2020 actual spending revisions issued 7-1-21. Also updated initial 2021 forecasts to include May 2021 year-to-date actual spending.

1-30-21 How can we tell if the adjusted starts forecasting method produces reliable results?

This plot of predicted spending from the starts cash flow model compared to actual spending is a check on this analytical modeling method. It shows a comparison of the cash flows predicted from all construction starts vs actual spending. If the forecast plot is accurate, then actual spending should move in the same direction, at the same slope. While we sometimes see lag in the plots movement, over time, the cash flow model of new starts does a good job of predicting where spending is headed.

The Dec 2020 chart below incorporates changes to residential (only residential has been modified) from earlier 2020 forecasts: no delayed projects canceled; all delayed spending restarted by August; new construction starts beginning in August, for the final 5 months of 2020, fastest growth in 15 years. This shows the latest starts data as adding to the recovery forecast between May and December and moving the future forecast residential spending line up on the index.

Settings in the pandemic forecast model resulted in the residential divergence. First, projects delayed were predicted to take six to eight months to come fully back up to production. But residential project spending was fully back to prior levels by August, within 3 months from the May bottom. About 60% of the return to prior spending was supported by growth in residential renovations. The rapid growth in spending is represented by the steep recovery in the spending curve between May and August. Second, a small portion of jobs delayed were predicted to be canceled permanently. Based on the spending data, this likely did not occur at all, or the impact was very small. Finally, Dodge at that time was forecasting that residential new starts in 2020 would finish the year down slightly. With December starts data now in, residential starts for 2020 finished up 4%. In fact, over the final 5 months of 2020, new residential construction starts posted 4 of the 5 highest monthly totals since 2004-2006. Residential new starts finished 2020 at a 15-year high, with almost 50% of new activity for the year posting in the final 5 months, which will put a lot of that spending into 2021.

Here’s the most recent forecast with 2020 revisions updated 7-1-21

The nonresidential buildings plots (and the residential plot prior to 2020) are remarkably close, providing an indication the method of analysis employed, cash flow of all construction starts to get spending forecast, is reasonably accurate. Below is the nonresidential plot to a larger scale.

The table below shows the 2020 forecasts published at midyear by numerous analysts, the first opportunity to incorporate impacts from the pandemic recession. This table compares Construction Analytics Midyear July 2020 forecast to eight firms that reported nonresidential forecasts in the AIA Midyear July Outlook. Two of those firms, FMI and ConstructConnect, also published full forecast reports at midyear. The Actual totals for 2020 based on data through December (2020 data revised 7-1-21), are shown in the first column. Forecasts, all compared to Actual 2020, are marked best, 2nd best, worst. Where there’s limited comparison (Total, Residential and Non-building), only the closest is marked.

Construction Analytics midyear 2020 forecast garnered more best estimates than any other firm when comparing Midyear estimates to actual totals for the year. No one got residential correct, some reasons cited above, but Construction Analytics was the closest. I think it’s fair to say, Construction Analytics Midyear 2020 forecast was closest to 2020 Total Actual overall. Though, I do remember some other times with red in my column. Markstein Advisors had a close 2nd best nonres bldgs midyear 2020 forecast.

7-1-21 Actual YTD 2020 updated to include 2020 revisions for final comparison.

This next table shows the current forecast for full year 2021 forecast published as of January 2021. Of the eight nonresidential markets in the AIA Outlook report, the spread between hi and low forecast is 14%-17% for 4 markets, but 24%-25% for 3 markets and a spread of 45% for lodging. Spreads that wide are indicating some forecasts are all over the place. This will get compared next January when we know the Actual amounts for 2021. Watch closely nonresidential buildings.

Also, in July, along comes the Midyear Outlook, when usually forecasts improve a bit. That also will get compared next January.

7-1-21 table below includes May ytd actual spending for latest comparison. Forecasts are all at start of 2021.

1 Comment

  1. This is an excellent article Ed, and very useful. High marks for accuracy of Zarenski analytics. Thank you so much.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Follow Construction Analytics on WordPress.com

Enter your email address to follow this blog and receive notifications of new posts by email.

Join 647 other subscribers


%d bloggers like this: