This is the basic GMDH algorithm. It use an input data sample as
a matrix containing N levels (points) of observations over
a set of M variables. A data sample is divided into two parts.
If regularity criterion
AR(s) is used, then approximately twothirds of observations
make up the training subsample N_{A}, and
the remaining part of observations (e.g. every third point with
same variance) form the test subsample N_{B }. The
training subsample is used to derive estimates for the coefficients
of the polynomial, and the test subsample is used to choose the
structure of the optimal model, that is one for which the
regularity criterion AR(s) takes on a minimal value:
or better to use the crossvalidation criterion PRR(s)
(it take into account all information in data sample and it can
be computed without recalculating of system for each test point).
Each point successively is taken as test subsample and then averaged
value of criteria is used:
To test a model for compliance with the differential balance
criterion, the input data sample is divided into two equal parts.
This criterion requires to choose a model that would, as far as
possible, be the same on both subsamples. The balance criterion
will yield the only optimal physical model solely if the input data
are noisy.
To obtain a smooth curve
of criterion value, which would permit one to formulate the exhaustivesearch
termination rule, the full exhaustive search is performed on models
classed into groups of an equal complexity. The first layer uses
the information contained in every column of the sample; that is
the search is applied to partial descriptions of the form
Nonlinear members can be taken as new input variables in data
sampling. The output variable is specified in this algorithm in
advance by the experimenter. For each model system of Gauss normal
equations is solved. At the second layer all modelscandidates of
the following form are sorted:
The models are evaluated for compliance with the criterion, and
the procedure is carried on as long as the criterion minimum
will be find. To decrease calculation time we recommend now to select
at some (68) layer a set of the best F variables and use only them
in further full sortingout procedure. In this way number of input
variables can be significantly increased. For extended
definition of the only optimal model the discriminating criterion
is recommended.
Figure: Combinatorial GMDH algorithm
1  data sampling;
2  layers of partial
descriptions complexing;
3  form of partial
descriptions;
4  choice of best
models set for structure identification;
5  additional
optimal model definition by discriminating
criterion.
In this way it's not complex to realize this algorithm as program
 solving of system of linear equations in loop is needed to realize.
The sequence of the basic Combinatorial algorithm computation is
described in faq.
A salient feature of the GMDH algorithms
is that, when they are presented continuous or noisy input data,
they will yield as optimal some simplified nonphysical model.
In the case of discrete or exact data the exhaustive search for
compliance with the precision criterion will yield what is called
a (contentative) physical model, the simplest of all unbiased
models. For noisy or short continuous input data, simplified Shannon
nonphysical models [12,25], received by GMDH algorithms, prove
more precise in approximation and for forecasting tasks. GMDH is
the only way to get optimal nonphysical models. Usage of sortingout
procedure guarantees selection of the best optimal model from all
possible.
