<- file stat 93neural.html -> In this file: Warren Sarle, a glossary of "neural net jargon"; and a description of maximum redundancy analysis.

Neural Net jargon

=======================Warren Sarle, 08 Jun 1993==========ssc Subject: Neural Net Jargon While I'm on the subject... (Loosely) Corresponding Terms in Neural Network and Statistical Literature. Particularly loose correspondences are marked by ~. Terminology in both fields is often vague, so precise equivalences are not always possible. Neural Network Jargon Statistical Jargon ===================== ================== Statistical methods Linear regression and discriminant analysis Training, Learning Estimation Supervised learning Regression, Discriminant analysis Unsupervised learning Principal components, Cluster analysis, Data reduction Competitive learning (usually) Cluster analysis Backpropagation Computation of derivatives, Gradient descent algorithms, Stochastic approximation Training data Sample data Test data Crossvalidation data Pattern Observation, Case Binary, Bivalent Binary, Dichotomous Input Independent variables, predictors, regressors, explanatory variables Output Predicted values Training values Observed values, Dependent variables, Responses Errors Residuals Noise Error term Generalization Interpolation, Extrapolation, Prediction ~(Two-layer) perceptron ~Generalized linear model (GLIM) Activation function, (Inverse) link function as in GLIM Signal function, Transfer function Squashing function bounded function Linear three-layer Maximum redundancy analysis, Principal perceptron components of instrumental variables ~~Three-layer perceptron ~~Projection pursuit Weights (Regression) coefficients, Parameter estimates ~Bias ~Intercept ~Shortcuts, Jumpers ~Main effects (direct connections from input to output) Functional links Interaction terms Higher-order network Polynomial regression, Response-surface model Weight decay Shrinkage estimation Pruning setting some coefficients to zero, model selection Probabilistic neural network Kernel discriminant analysis ~Adaptive vector quantization ~K-means cluster analysis Encoding, Autoassociation Dimensionality reduction (Independent and dependent variables are the same) Heteroassociation Regression, Discriminant analysis (Independent and dependent variables are different) ~Epoch ~Iteration Continuous training, Stochastic approximation, iteratively Incremental training updating estimates one observation at a time via difference equations Batch training Iteratively updating estimates after each complete pass over the data as in most nonlinear regression algorithms *--------

Maximum redundancy

=======================Warren Sarle, 09 Jun 1993==========ssc Subject: Re: Neural Net Jargon Message-ID: <C8DBsE.H3s@unx.sas.com> In article <1v27q1INNiu0@bHARs12c.bnr.co.uk>, Peter Hamer <pgh@bnr.co.uk> writes: |> Could you give a brief explanation and/or a good reference for maximum |> redundancy analysis? I haven't heard the term before. Rao, C.R. (1964) "The use and interpretation of principal component analysis in applied research," Sankhya A, 26, 329-358. van den Wollenberg, A.L. (1977) "Redundancy analysis--an alternative to canonical correlation analysis," Psychometrika, 42, 207-219. The idea is to find a small number of linear combinations of a set of independent variables that optimally (least squares) predicts a set of dependent variables. It's what most people really want to do when they think they want to do canonical correlation. -- * * * * * * * * * * * * * * * * * * * * * * * * * * * * *

Document by Rich Ulrich. E-mail to wpilib+@pitt.edu

FAQ top.

Ulrich home page.

Ulrich FAQ. http://www.pitt.edu/~wpilib/stats99.html