by Nikolai V. Shokhirev
Error: Estimation | Amplification | Propagation
In the previous section we discussed a basic error estimation. Here we consider the propagation and amplification of errors during data processing. The simplest procedure is the solution of a system of linear equations.
Let us start with the following example:
The inverse system is
Obviously this is also a linear system.
Experiment 1. Compare the solution for b1 = 1 and b2 = -1 with the solution for the right hand side with 1% noise.
|1.01||-1.01||40.4||1 %||-20.2||1 %|
Here ei are the relative errors for the r. h. s. The accuracy of the solutions is also 1% or less.
Experiment 2. Compare the solution for b1 = 1 and b2 = 1 with the solution for the r. h. s. with 1% noise.
|1.01||1.01||0.202||1 %||0.404||1 %|
The result speaks for itself: a simple linear transformation can cause a catastrophic increase of an error.
Assuming that the errors for b1and b2 are uncorrelated with the standard deviation σ, we get from (2) (see the appendix for details)
|var(x1) = 800.02 σ2|
|var(x2) = 200.08 σ2||(3)|
|cov(x1, x2) = -399.96 σ2|
The errors for x are not only much larger but also are strongly correlated. This can be represented graphically. Two independent errors of the magnitude 0.01 form a circle in the b1, b2 plane
The corresponding solution errors form the following elongated ellipse in the x1, x2 plane
Equations (1a) and (2a) can be rewritten as
Using the singular value decomposition (SVD) the matrices can be presented in the following form
Any vector in the x-space can be presented as a linear combination of the vectors v1 and v2:
Similarly any vector in the b-space can be expressed in terms of u1 and u2:
Eq. (5) can be interpreted in the following way. The first term transforms the u1 component of b into the v1 component of x, and is the transformation factor. The second term transforms the u2 component of b into the v2 component of x. However, the transformation factor in this case is 100 times larger. This explains noise amplification.
The components corresponding to large noise amplification cause instability in the solution of linear systems. The filters based on the SVD can improve stability without essential loss of accuracy.
Extended variant of Eq. (3).
|var(x1) = 404.01 var(b1) - 799.98 cov(b1, b2) + 396.01 var(b2)|
|var(x2) = 96.04 var(b1) - 199.92 cov(b1, b2) + 104.04 var(b2)||(1)|
|cov(x1, x2) = -196.98 var(b1) + 400.04 cov(b1, b2) + 202.98 var(b2)|
©Nikolai V. Shokhirev, 2001-2004