Error amplification

by Nikolai V. Shokhirev

Error: Estimation | Amplification | Propagation


In the previous section we discussed a basic error estimation. Here we consider the propagation and amplification of errors during data processing. The simplest procedure is the solution of a system of linear equations.

Let us start with the following example:


The inverse system is


Obviously this is also a linear system.

Numerical experiments

Experiment 1. Compare the solution for  b1 = 1 and b2 = -1 with the solution for the right hand side with 1% noise.

  b1   b2 x1  e1   x2  e2 
 1.0  -1.0  40.0     -20.0   
 1.01  -1.01  40.4  1 %  -20.2  1 %
 1.01   -0.99   40.002   0.005%  -19.996  0.02% 

Here ei are the relative errors for the r. h. s. The accuracy of the solutions is also 1% or less.

Experiment 2. Compare the solution for  b1 = 1 and b2 = 1 with the solution for the r. h. s. with 1% noise.

  b1   b2 x1  err1   x2  err2 
 1.0  1.0  0.2     0.4   
 1.01  1.01  0.202  1 %  0.404  1 %
 1.01   0.99   0.6   300 %  0.2  50% 

The result speaks for itself: a simple linear transformation can cause a catastrophic increase of an error.


Assuming that the errors for b1and b2 are uncorrelated with the standard deviation σ, we get from (2) (see the appendix for details)

 var(x1) = 800.02 σ2                   
 var(x2) =  200.08 σ2            (3)
 cov(x1, x2) = -399.96 σ2           

The errors for x are not only much larger but also are strongly correlated. This can be represented graphically. Two independent errors of the magnitude 0.01 form a circle in the  b1, b2 plane 

The corresponding solution errors form the following elongated ellipse in the  x1, x2 plane

Algebraic approach

Equations (1a) and (2a) can be rewritten as


Using the singular value decomposition (SVD) the matrices can be presented in the following form








Any vector in the x-space can be presented as a linear combination of the vectors v1 and v2:


Similarly any vector in the b-space can be expressed in terms of u1 and u2:


Eq. (5) can be interpreted in the following way. The first term transforms the u1 component of b into the v1 component of x, and is the transformation factor. The second term transforms the u2 component of b into the v2 component of x. However, the transformation factor in this case is 100 times larger. This explains noise amplification. 


The components corresponding to large noise amplification cause instability in the solution of linear systems. The filters based on the SVD can improve stability without essential loss of accuracy.


Extended variant of Eq. (3).

 var(x1) = 404.01 var(b1) - 799.98 cov(b1, b2) + 396.01 var(b2)                    
 var(x2) =  96.04 var(b1) - 199.92 cov(b1, b2) + 104.04 var(b2)         (1)
 cov(x1, x2) = -196.98 var(b1) + 400.04 cov(b1, b2) + 202.98 var(b2)         


  1. Singular value decomposition (SVD).
  2. Linear algebra: a summary of vector and matrix operations is available here.
Error Estimation | ABC Tutorials | Data Processing | Indirect Measurements

Home | Resumé | |  Computing |  Links Publications

©Nikolai V. Shokhirev, 2001-2004