409 B.D. Baker, C.E. Richards Economics of Education Review 18 1999 405–415
were applied.
15
For Group Method of Data Handling GMDH, default parameters were also applied.
16
For both the recurrent backpropagation models and Generalized Regression neural networks, the training
sets consisted of data from 1960 to 1980 and test sets consisted of data from 1981 through 1990.
17
Alternate models were trained with log ln transformed data. Pre-
dictors and lags were the same as those specified for the linear regression equation. Only the main equation for
the prediction of CUREXP was modeled with the neural networks. Forecasts were generated for the period from
1991 through 1995 by providing the trained neural net- works with the same set of forecast predictors univariate
results used for the linear regression model including regression predictions of SGRNT from the secondary
multiple linear regression equation.
4. Results and discussion
4.1. Univariate models All models selected for univariate forecasting were
either 1 Linear Holt Exponential Smoothing
18
PCI, PERTAX or 2 Damped Trend Exponential Smoothing
ADAPOP, BUSTAX, RCPIANN models. Parameters for the models and measures of fit are displayed in
Appendix B.
4.2. Multivariate model The estimated AR1 multivariate regression model for
forecasting SGRNT is: ln SGRNT 5 0.13 0.90
15
Including linear scaling 0,1 of the input layer and model selection based on test set error, with the stop criterion set to
20 generations without reduction of error. One alteration made was increasing the size of the gene pool to 300 from the default
of 20. Incrementally increased gene pool size, while requiring longer training times, produced more accurate results.
16
Including linear scaling of the input , 2 1, 1 1 and output layers [ 2 1, 1 1], “Smart” model selection and optimiz-
ation, applying FCPSE as the selection criteria, and the option to construct a model of “high” non-linearity, indicating the
option to include second- and third-order terms on all para- meters and two- and three-way interactions.
17
An arbitrary specification of 33, in excess of the rec- ommended 20 WSG, 1995, p. 101 test set that yielded equi-
valent results.
18
Which for prediction or forecasting purposes can be expressed as: Z
ˆ
n
l 5 a[Z
n
1 1 2 aZ
n 2 1
1 1 2
a
2
Z
n 2 2
1 … ] 5 S
n
, where the predicted future observation Z
ˆ
n
l is based on previous observations Z
n −
t
multiplied by a decaying smoothing constant a, 1 2 a
2
… and S
n
represents the smoothed statistic Liu Hudak, 1994, p. 9.3.
1 0.67 ln BUSTAX{1} 0.04
1 0.30 ln PERTAX{1} 0.12
3 1
0.31 ln ADAPOP 0.01 2
0.03 lnRCPIANNRCPIANN{1} 0.08 with Rho 5 0.19 0.37, R
2
5 0.992, p-values in parenth-
eses. Forecasts of SGRNT were generated from this model
using the univariate forecasts of predictors BUSTAX, PERTAX, ADAPOP, RCPIANN Appendix C.
The estimated AR1 multivariate regression model for forecasting CUREXP is:
ln CUREXP 5 2 1.30 0.29 1
0.55 ln PCI 0.02 1 0.63 ln SGRNT 0.00 4
2 0.32 ln ADAPOP 0.02
with Rho 5 0.41 0.04, R
2
5 0.997, p-values in parenth-
eses. Forecasts of CUREXP were generated from this
model using the univariate forecasts of PCI and ADAPOP,
and multivariate
forecast of
SGRNT Appendix C.
4.3. Neural models Table 1 displays the training sample fit and pro-
duction forecast fit results for each of the neural net- work models, including both fitness measures and each
network’s respective measure of input contribution to prediction. For the recurrent backpropagation models,
each obtained exceptionally strong fit to the training set R
2
5 0.993 to 0.994ln and produced reasonable fit to
the production set R
2
5 0.756 to 0.757ln. The aggre-
gate contribution factors
19
implicate state-level support SGRNT as the primary influence on spending.
Both GRNN models were slightly less fit R
2
5 0.910
to 0.956ln to the training set than recurrent backpro- pagation models and both produced looser fit to the pro-
duction set 0.643 to 0.688ln. Again, aggregate weighting factors, in this case smoothing parameters,
suggest relatively high importance of state-level support SGRNT for predicting spending CUREXP.
GMDH produces comparable fitness to both training and production sets to recurrent backpropagation. Note,
19
Like regression coefficients, these factors represent a meas- ure of the responsiveness of the outcome measure to changes
in inputs, yet, as aggregate contribution factors they express no direct mathematicalfunctional relationship between inputs and
outputs. Rather they indicate a rough measure of relative impor- tance of inputs derived from the weighting matrix. For a more
detailed discussion, see WSG 1995, p. 191.
410 B.D. Baker, C.E. Richards Economics of Education Review 18 1999 405–415
Table 1 Training and forecast fitness of neural models
Training R
2
Production R
2
PCI ADAPOP
SGRNT Backpropagation
0.993 0.756
0.289
a
0.078 0.525
Backpropagation ln 0.994 0.757
0.310 0.062
0.517 GRNN
0.910 0.643
2.318
b
0.024 2.694
GRNN ln 0.956
0.688 2.141
0.035 2.765
GMDH 0.983
0.793 0.99
c
– –
GMDH ln 0.997
0.740 –
0.21
d
0.84
e
PCI: per capita income; ADAPOP: average daily attendance as a percent of the population; SGRNT: state-level support to public education per pupil in ADA
a
Aggregate weighting or “Contribution” factor see WSG, 1995, p. 191.
b
Aggregate smoothing factor a see WSG, 1995, pp. 198–205.
c
Linear coefficient b, based on rescaled PCI ˆ
PCI 5 2 3 PCI 2 7504.3913932.61 2 1.
d
Linear coefficient b, based on rescaled SGRNT ˆ
SGRNT 5 2 3 ln SGRNT 2 4.791.28 2 1.
e
Linear coefficient b, based on rescaled ADAPOP ˆ
ADAPOP 5 2 3 ln ADAPOP 1 1.980.48 2 1.
however, that the GMDH model that produces greater fit to the training data 0.997ln compared to 0.983 does
not perform as well on the production data 0.740ln compared to 0.793. In addition, while measures of pre-
dictor importance were fairly consistent between back- propagation and GRNN models, GMDH models took
varied forms, with the first not log transformed includ- ing only per capita income and the second log
transformed including only state support and average daily attendance. Interestingly, though GMDH learning
was set for the possibility of including both second- and third-order terms as well as two- and three-way interac-
tion terms, resultant best predicting models included only first-order linear parameters.
4.4. Model forecasts and accuracy assessment Table 2 displays the AR1 and neural forecasts of edu-
cational spending alongside actual spending values 1982–1984 constant dollars for the period 1991–1995.
Note that beyond 1993 all models provide overestimates
Table 2 Comparison of forecast accuracy
Group method of data Recurrent backpropagation
Generalized regression handling
Year Actual US AR1 US
A US B
a
US A US
B
a
US A US
B
a
US 1991
3923 3856
3869 3853
3934 3917
3870 3870
1992 3920
3918 3901
3886 3949
3947 3906
3906 1993
3916 3973
3934 3925
3994 3970
3942 3942
1994 3942
4027 3970
3968 4026
4017 3977
3978 1995
3979 4081
4004 4010
4029 4061
4013 4013
Prices reported in constant school-year 1982–84 dollar using reported consumer price index CPI as the deflator.
a
Indicates model trained with log ln transformed series.
of educational spending, the most significant overesti- mates coming from the AR1 model.
Table 3 displays the comparisons of forecast accuracy for the AR1 multiple regression model and the three neu-
ral network models. While the NCES model performed quite well for the entire period MAPE 5 1.60, each
of the neural models outperformed the NCES model on average for the forecast period. Period-to-period predic-
tions, however, vary quite significantly, for example, with the AR1 model outperforming the others for the
second 1992 time period. Overall, recurrent backpro- pagation and GMDH models performed particularly
well, providing an approximate 2 3 increase in forecast accuracy over the NCES model.
Despite less accurate overall predictions of the level of spending, GRNN models yielded the only results sug-
gesting non-linear sensitivity, revealing more accurate changing rates of growth in spending from 1991 to 1993,
given the relatively constant univariate extrapolations of predictors. During this period, actual spending declined
2 0.1, AR1 forecasts increased 1.5, and on aver-
411 B.D. Baker, C.E. Richards Economics of Education Review 18 1999 405–415
Table 3 Comparison of forecast accuracy
Recurrent backpropagation Generalized regression
Group method of data handling Year
AR1 A
B
a
A B
a
A B
a
1991 1.71
1.38 1.78
0.27 0.18
1.37 1.36
1992 0.07
0.50 0.88
0.74 0.67
0.38 0.36
1993 1.47
0.47 0.23
2.00 1.38
0.66 0.68
1994 2.17
0.72 0.68
2.14 1.92
0.91 0.92
1995 2.57
0.64 0.80
1.26 2.06
0.87 0.87
MAPE
b
1.60 0.74
0.88 1.28
1.24 0.84
0.84
a
Indicates model trained with log ln transformed series.
b
Among others, Chase 1995 explains the interpretation and use of Mean Absolute Percentage Error MAPE in assessing forecast accuracy.
age backpropagation and GMDH forecasts increased 0.9. GRNN forecasts increased more slowly from
0.7 to 0.8, with a low of 0.4 for 1991–1992 GRNN and 0.6 for 1992–1993 ln GRNN. That
GRNNs revealed this pattern is consistent with Caudill’s Caudill, 1995, pp. 47–48 suggestion that GRNNs are
particularly sensitive to non-linear relationships in sparse andor noisy data.
5. Conclusions and recommendations
