Chapter 5. Systems of Linear Equations

  • 5.1 Introduction

  • 5.2 Existence of Solutions

  • 5.3 Gaussian Elimination Techniques

  • 5.4 LU Factorization Methods

  • 5.5 Iterative Techniques

  • 5.6 Visualizing the Solution: Code5

  • 5.7 Summary

    • Numerical Exercises

    • Programming Challenges

INTRODUCTION

A linear system consists of a set of equations with some governing parameters or variables that control the system. The variables that make up the system can be stated in the form of differential equations or autonomous variables. The system works in a dependent manner where changes in one or more variables affect the performance of the whole system in general.

Definition 5.1. An equation in the following form with the unknowns xi and constants ai, for i = 1, 2, ..., n, is said to be a linear equation:

Equation 5.1. 

INTRODUCTION

In this definition, each unknown xi must have an index power of 1. Also, a linear equation cannot have a variable inside sine, cosine, tangent, exponent, logarithm, and other operators. It follows that any equation in the form other than Equation (5.1) is called a nonlinear equation.

The following equations are examples of linear equations:

Equation 5.2. 

INTRODUCTION

A system of linear equations consists of a set of linear equations and a set of unknowns. In general, a system of linear equations with m equations and n unknowns is expressed as follows:

Equation 5.2. 

INTRODUCTION

The above system is represented in matrix form as Ax = b, as follows:

Equation 5.3. 

INTRODUCTION

where A = [aij] is the coefficient matrix of the system, for i = 1, 2, ..., m and j = 1, 2, ..., n. In this system, m is the number of rows in the coefficient matrix, whereas n is the number of columns (unknowns). The vector b = [bi] is the given constants, whereas x = [xi] is the vector of the unknowns. If bi = 0 for all i, then the linear system is called a homogeneous system; otherwise, if at least one of bi ≠ 0, then the system is nonhomogeneous.

In designing computer programs for solving the problem involving a system of linear equations, it is important to incorporate good programming habits. These habits include making the program easy to modify in order to have the problem scalable and flexible. A program is said to be scalable if the code for a program with small data applies to one with large data as well. A flexible program is one that adapts well to changes in the parameters governing the problem with minimum modification in the program. It is our duty to discuss these approaches in this chapter.

EXISTENCE OF SOLUTIONS

The solution to a system of linear equations may or may not exist, depending on the given equations and several other factors. To complicate matters, a solution that exists may not be unique, and there can be an infinite number of solutions for the system. We study all cases that lead to the existence of the solution for a system of linear equations.

Definition 5.2. A system of linear equations is said to have consistent solutions if all its solutions satisfy each equation in the system. If one or more of the solutions do not satisfy at least one equation in the system, then the system is said to be inconsistent.

The above definition describes the state of the solutions in a system of linear equations. Referring to Equation (5.3), if the solution to the first equation also satisfies all other equations in the system, then this solution is consistent. It follows that the system normally produces consistent solutions if mn and inconsistent solutions if m > n. A consistent system may have either unique or many solutions. A set of inconsistent solutions consists of solutions that are valid in some but not in all equations in the system.

Ideally, good and consistent solutions are obtained if the coefficient matrix is square; that is, the number of equations in the system equals the number of unknowns, or m = n. With this assumption, Equation (5.3) can now be rewritten as follows:

Equation 5.4. 

EXISTENCE OF SOLUTIONS

Equation (5.4) is also expressed compactly in matrix form as A x = b, where

Equation 5.5. 

EXISTENCE OF SOLUTIONS

In this representation, the square matrix A is the coefficient matrix of the system, whereas x is the solution vector.

Theorem 5.1. If the coefficient matrix A of a system of linear equations is square and if |A| ≠ 0, then the system has consistent and unique solutions.

The above theorem states that a system whose coefficient matrix is not singular has consistent and unique solutions. Unique solutions imply each unknown in the system has only one value as its solution, and this value satisfies all equations in the system.

Example 5.1. The following system is consistent with unique solutions:

Equation 5.7. 

EXISTENCE OF SOLUTIONS

Solution. It can be shown that

EXISTENCE OF SOLUTIONS

Theorem 5.2. If the augmented matrix A | b reduces to U | v after row operations with one or more zero rows in U and the corresponding row(s) is v is also zero, then the system is consistent with an infinite number of solutions. If one or more corresponding row(s) in v is not zero, then the solution does not exist.

Theorem 5.2 says an infinite solution exists if a row (or more) in the augmented matrix has zero contents in all its (their) elements. The theorem is best illustrated using two examples, one with an infinite number of solutions and another with no solution.

Example 5.2. The following system is consistent with many solutions:

Equation 5.8. 

EXISTENCE OF SOLUTIONS

Solution. Reduce the augmented matrix, as follows:

Equation 5.9. 

EXISTENCE OF SOLUTIONS

Since all the elements in the second row of the augmented matrix have zero values, the system has infinite solutions. (x, y) = (1,0) and (x, y) = (−1, 1) are some of the possible solutions to the system.

Example 5.3. The following system is inconsistent:

Equation 5.10. 

EXISTENCE OF SOLUTIONS

Solution. Reduce the augmented matrix, as follows:

Equation 5.11. 

EXISTENCE OF SOLUTIONS

The last row of the final reduced augmented matrix has zero elements on the left-hand side and nonzero on the right-hand side. This contradicts the equation, and by Theorem 5.2, no solution exists for this system.

Table 5.1. Common methods for solving a system of linear equations

Method

Type

Condition

Gaussian elimination

Elimination

Nonzero pivot elements and nonsingular coefficient matrix.

Gaussian elimination with partial pivoting

Elimination

Nonsingular rearranged matrix.

Gauss-Jordan

Elimination

Nonzero pivot elements and nonsingular coefficient matrix.

Crout

LU factorization

Nonzero diagonal elements in the L matrix.

Doolittle

LU factorization

Nonzero diagonal elements in the U matrix.

Cholesky

LU factorization

Positive-definite coefficient matrix.

Thomas

LU factorization

Tridiagonal coefficient matrix.

Jacobi

Iterative

Diagonally dominant coefficient matrix.

Gauss-Seidel

Iterative

Diagonally dominant coefficient matrix.

In this chapter, we will concentrate on discussing the system of linear equations in the form of Equation (5.4) that is consistent with a unique solution. In general, there are many techniques for solving this system, and they can be categorized as elimination, LU factorization, and iterative methods. An elimination technique reduces the augmented matrix in the system to its triangular form before applying backward substitutions for the solution. The LU factorization technique factorizes the coefficient matrix as a product of its upper and lower triangular matrices before generating the solution through substitutions. Lastly, an iterative technique performs iterations on all unknowns until their values converge to within a tolerable range.

Table 5.1 shows the common methods for solving a system of linear equations that can be categorized under the elimination, LU factorization, and iterative techniques. Note that some methods are very sensitive to the conditions imposed on the coefficient matrix, and we will address them later in the chapter.

GAUSSIAN ELIMINATION TECHNIQUES

The elimination technique reduces the rows of a matrix into a simpler form through a series of row operations, as discussed earlier in Chapter 2. We discuss three methods commonly used in solving a system of linear equations, namely, the Gaussian elimination method, Gaussian elimination method with partial pivoting, and Gauss-Jordan method.

Gaussian Elimination Method

The Gaussian elimination method is an elimination technique for solving a system of linear equations through two series of steps. The method works best when the coefficient matrix is not singular; that is, |A| ≠ 0 so that the solution obtained is unique according to Theorem 5.1. The Gaussian elimination method consists of two main steps, as depicted in Figure 5.1. The first step is the row operations for eliminating the elements in the coefficient matrix so as to reduce the matrix into its upper triangular form. If all elements in this triangular matrix have nonzero values, then the matrix definitely has a unique solution. The second step, which is backward substitutions, follows directly from the triangular matrix relationship for producing the solution.

Main steps in the Gaussian elimination method.

Figure 5.1. Main steps in the Gaussian elimination method.

We discuss a system of linear equations Ax = b having n linear equations and n unknowns. The row operations technique in the Gaussian elimination method is similar to the technique for matrix reduction discussed in Section 2.4. This time row operations are applied to the augmented matrix A | b from the system of linear equations according to the following relationships:

Equation 5.12. 

Main steps in the Gaussian elimination method.

for k = 1, 2, ..., n − 1, followed by i = k + 1, k + 2, ..., n and j = 1, 2, ..., n.

The above step reduces A | b to U | v. Backward substitutions are performed on U | v to produce the solution x according to the following:

Equation 5.13. 

Main steps in the Gaussian elimination method.

Example 5.4. Solve the following system of linear equations using the Gaussian elimination method:

Equation 5.14. 

Main steps in the Gaussian elimination method.

Solution. The above system is represented as the following augmented matrix:

Equation 5.15. 

Main steps in the Gaussian elimination method.

Operations with Respect to the First Row. The row operation is RiRipR1 for the rows i = 2, 3,4. The pivot element of the first row is a11 = 2, and, therefore, p = ai1/a11. All elements in the second, third, and fourth rows are reduced to their corresponding values using the relationships aijaijpa1j and bibipb1 for the columns j = 1, 2, 3, 4.

Equation 5.16. 

Main steps in the Gaussian elimination method.

Operations with Respect to the Second Row. The row operation is RiRipR2 for the rows i = 3, 4. The pivot element of the second row is a22 = 4, and, therefore, p = ai2/a22. All elements in the third and fourth rows are reduced to their corresponding values using the relationships aijaijpa2j and bibipb2 for the columns j = 1, 2, 3, 4.

Equation 5.17. 

Main steps in the Gaussian elimination method.

Operations with Respect to the Third Row. The row operation is RiRipR3 for the rows i = 4. The pivot element of the third row is a33 = −35, and, therefore, p = ai3/a33. All elements in the fourth row are reduced to their corresponding values using the relationships aijaijpa3j and bibipb3 for the columns j = 1, 2, 3, 4.

Equation 5.18. 

Main steps in the Gaussian elimination method.

Row operations on A and b produce U and v, respectively, where

Equation 5.19. 

Main steps in the Gaussian elimination method.

Backward substitutions. The reduced system oflinear equations after row operations is

Equation 5.20. 

Main steps in the Gaussian elimination method.

This system is solved by applying backward substitutions to get

Equation 5.21. 

Main steps in the Gaussian elimination method.

The C++ code for this method follows from the same coding for reducing a square matrix to its upper triangular form and for finding the inverse of a matrix in Sections 3.3 and 3.4, respectively. Row operations reduce A | b to U | v, as shown below:

double Sum,m;
for (k=1; k<=n-1; k++)
      for (i=k+1; i<=n; i++)
      {
            p=a[i][k]/a[k][k];
            for (j=1; j<=n; j++)
                  a[i][j] -= p*a[k][j];
            b[i] -= p*b[k];
      }

This is followed by backward substitutions:

for (i=n; i>=1; i--)
{
      Sum=0;
      x[i]=0;
      for (j=i;j<=n; j++)
            Sum += a[i][j]*x[j];
      x[i]=(b[i]-Sum)/a[i][i];
}

Code5A.cpp solves a system of linear equations using the Gaussian elimination method. The data for the system is obtained from an external file called Code5A.in.

Code5A.cpp: Gaussian elimination method

#include <fstream.h>
#include <iostream.h>
#define n 4
void main()
{
      int i,j,k;
      double *x,*b,**a;
      for (i=0;i<=n;i++)
      {
            x=new double [n+1];
            b=new double [n+1];
            a=new double *[n+1];
            for (j=0;j<=n;j++)
                  a[j]=new double [n+1];
      }
      ifstream InFile("Code5A.in");
      cout.setf(ios::fixed);
      cout.precision(5);
      cout << "Our input data: " << endl;
      for (i=1;i<=n;i++)
      {
            cout << "a: ";
            for (j=1;j<=n;j++)
            {
                  InFile >> a[i][j];
                  cout << a[i][j] << " ";
            }
            InFile >> b[i];
            cout << "b=" << b[i] << " " << endl;
      }
      InFile.close();

      // row operations
      double Sum,p;
      for (k=1;k<=n-1;k++)
            for (i=k+1;i<=n;i++)
            {
                  p=a[i][k]/a[k][k];
                  for (j=1;j<=n;j++)
                        a[i][j]-=p*a[k][j];
                  b[i] -= p*b[k];
            }

      // backward substitutions
      for (i=n;i>=1;i--)
      {
            Sum=0;
            x[i]=0;
for (j=i;j<=n;j++)
                  Sum += a[i][j]*x[j];
             x[i]=(b[i]-Sum)/a[i][i];
      }
             cout << endl;

      // display results
      cout << "results after row operations:" << endl;
      for (i=1; i<=n; i++)
      {
            cout << "u: ";
            for (j=1;j<=n;j++)
                  cout << a[i][j] << " ";
            cout << "v=" << b[i] << " " << endl;
      }
      cout <<endl<<"results after backward substitutions: x=";
      for (i=1; i<=n; i++)
            cout << x[i] << " ";
      cout << endl;
      for (i=0;i<=n;I++)
            delete a[i];
      delete a,x,b;
}

Gaussian Elimination Method with Partial Pivoting

Very often, the diagonal elements of a matrix have values equal or close to zero. A matrix with this feature is called an ill-conditioned matrix. Division on a zero value produces an undefined number. Division on a small number whose value is very close to zero is the inverse of the division process that produces a very large number. Such a number does not fit to be a pivot element in the row operations as the result affects the accuracy of the division significantly.

An ill-conditioned matrix is treated differently when the Gaussian elimination method is applied. To avoid division by zero or a number close to this value, partial pivoting on the diagonal element is performed. Partial pivoting is a technique of interchanging a row whose pivot element has a value equal to or close to zero with another row whose element in the same column as the pivot element has the largest modulus value. Therefore, partial pivoting requires an additional step of interchanging two rows before row operations are performed.

Consider the following system:

Equation 5.22. 

Gaussian Elimination Method with Partial Pivoting

The augmented matrix A | b is

Equation 5.23. 

Gaussian Elimination Method with Partial Pivoting

With the Gaussian elimination method, this augmented matrix reduces to the following form with R2R2R1:

Equation 5.24. 

Gaussian Elimination Method with Partial Pivoting

Continuing the next row operation with a22 as the pivot element will be disastrous as this element is zero, and

Gaussian Elimination Method with Partial Pivoting

Equation 5.25. 

Gaussian Elimination Method with Partial Pivoting

Reduction with R2R2 + 2R1 produces

Equation 5.26. 

Gaussian Elimination Method with Partial Pivoting

which rectifies the above problem. We discuss an example here.

Example 5.5. Solve the following system of linear equations using the Gaussian elimination method with partial pivoting:

Equation 5.27. 

Gaussian Elimination Method with Partial Pivoting

Solution. The above matrix is ill-conditioned as some pivot elements have zero values, namely a11 and a22. Applying the Gaussian elimination method directly to solve the system will be disastrous as m is undefined due to division by these zero values. Let's try on the partial pivoting technique. The augmented matrix for this system is shown as follows:

Equation 5.28. 

Gaussian Elimination Method with Partial Pivoting

Test for the Dominance of the Pivot Element in Row 1. The modulus of the pivot element in the first row is not the largest as |a11| < |a31|. Therefore, an interchange between rows 1 and 3 is performed.

Equation 5.29. 

Gaussian Elimination Method with Partial Pivoting

Operations with Respect to the First Row. The row operation is RiRipR1, where a11 = 8 is the pivot element of the first row and m = ai1/a11 for the rows i = 2,3, 4. All elements in the second, third, and fourth rows are reduced to their corresponding values using the relationships aijaijpa1j and bibipb1 for the columns j = 1, 2, 3, 4.

Equation 5.30. 

Gaussian Elimination Method with Partial Pivoting

Test for the Dominance of the Pivot Element in Row 2. The modulus of the pivot element in the second row is not the largest as |a22| < |a42|. Therefore, an interchange between rows 2 and 4 is performed.

Equation 5.31. 

Gaussian Elimination Method with Partial Pivoting

Operations with Respect to the Second Row. The row operation is RiRipR2, where a22 = − 8.250 is the pivot element of the second row, and therefore, p = ai2/a22 for the rows i = 3, 4. All elements in the third and fourth rows are reduced to their corresponding values using the relationships aijaijpa2j and bibipb2 for the columns j = 1, 2, 3, 4.

Equation 5.32. 

Gaussian Elimination Method with Partial Pivoting

Test for the Dominance of the Pivot Element in Row 3. No interchange of rows is performed as the modulus of the pivot element in the third row is larger than the elements below it.

Operations with Respect to the Third Row. The row operation is RiRipR3, where a33 = 6.864 is the pivot element of the third row. Therefore, p = ai3/a33 for the row i = 4. All elements in the fourth row are reduced to their corresponding values using the relationships aijaijpa3j and bibipb3 for the columns j = 1, 2, 3, 4.

Equation 5.33. 

Gaussian Elimination Method with Partial Pivoting

Row operations with partial pivoting reduce A | b. to U | v. with the following values:

Equation 5.34. 

Gaussian Elimination Method with Partial Pivoting

Backward substitutions. The same backward substitution technique as in the Gaussian elimination method is applied to produce the solutions, x4 = −0.258, x3 = −0.248, x2 = −0.479, and x1 = −0.097.

Gauss-Jordan Method

The Gauss-Jordan method is another elimination technique for solving the Ax = b problem involving row operations. Unlike the Gaussian elimination method, the Gauss-Jordan method is wholly based on row operations and does not involve backward substitutions. The method assumes the coefficient matrix to be nonsingular and fails to produce the desired result if the matrix is singular.

The Gauss-Jordan method is implemented as follows: Reduce the coefficient matrix A into an identity matrix I by eliminating the elements in the rows and columns of the other diagonals. The augmented matrix A | b. reduces to I | v. through row operations where v is the final solution. This is described in Figure 5.2.

The single main step in the Gauss-Jordan method.

Figure 5.2. The single main step in the Gauss-Jordan method.

The Gauss-Jordan method is illustrated through Example 5.6.

Example 5.6. Solve the problem from Example 5.4 using the Gauss-Jordan method.

Solution. Form the augmented matrix A | b:

Equation 5.35. 

The single main step in the Gauss-Jordan method.

Reduce a11 value to 1 through R1R1/a11.

Equation 5.36. 

The single main step in the Gauss-Jordan method.

Eliminate a21, a31, and a41 through R2R2a21R1, R3R3a31 R1, and R4R4a41R1, respectively.

Equation 5.37. 

The single main step in the Gauss-Jordan method.

Reduce a22 to 1 through R2R2/a22.

Equation 5.38. 

The single main step in the Gauss-Jordan method.

Eliminate a12, a32, and a42 through R1R1a12 R2, R3R3a32R2, and R4R4a42 R2, respectively.

Equation 5.39. 

The single main step in the Gauss-Jordan method.

Reduce a33 to 1 through R3R3/a33.

Equation 5.40. 

The single main step in the Gauss-Jordan method.

Eliminate a13, a23, and a43 through R1R1a13 R3, R2R2a23R3, and R4R4a43 R3, respectively.

Equation 5.41. 

The single main step in the Gauss-Jordan method.

Reduce a43 to 1 through R4R4/a44.

Equation 5.42. 

The single main step in the Gauss-Jordan method.

Eliminate a14,a24, and a34 through R1R1a14 R4, R2R2a24R4, and R4R4a34 R4, respectively.

Equation 5.43. 

The single main step in the Gauss-Jordan method.

The final step gives the solutions as x1 = −0.050, x2 = −0.409, x3 = −0.261, and x4 = −0.520, which are the same as in Example 5.4.

LU FACTORIZATION METHODS

LU Factorization Concepts

An important feature of a nonsingular square matrix A is it can be factorized into a pair of upper and lower triangular matrices, U and L, as follows:

Equation 5.6. 

LU Factorization Concepts

The factorization into triangular matrices helps in solving the linear equations as the unknowns can be directly determined through forward or backward substitutions. It can be generalized that for An × n, the number of unknowns in the matrices L and U produced from the factorization of A is n2.

This is illustrated using an example on a 4 × 4 matrix A, as follows:

Equation 5.45. 

LU Factorization Concepts

The values uij and lij for i, j = 1, 2, 3, 4 can be obtained by comparing the two matrices element by element. Comparing the terms one by one, we have a11 = l11u11, a12 = l11u12, and so on. From these relationships, the values of all elements in L and U can be determined.

We now discuss the solution to the LU factorization technique. The objective is to solve the system of linear equations given by Ax = b. Factorize A = LU to get

Equation 5.46. 

LU Factorization Concepts

This simplifies into

Equation 5.47. 

LU Factorization Concepts

Let Ux = w, where w = [wi] for i = 1, 2, ..., n is a temporary vector. This gives Lw = b, where the values in w can be determined through forward substitutions. The values of x are then obtained from backward substitutions in Ux = w.

The general factorization technique in A = LU is simplified through four main methods for solving several different cases of the system of linear equations. Basically, these four methods differ in their representation in the triangular matrices, as follows:

  1. If uii = 1 for i = 1, 2, ..., n in matrix U, then the method is called the Crout method.

  2. If lii = 1 for i = 1, 2, ..., n in matrix L, then the method is called the Doolittle method.

  3. If U = LT for the positive-definite matrix A, then the method is called the Cholesky method.

  4. The Thomas algorithm is a special adaptation of the Crout method when A is a tridiagonal matrix.

We discuss each method and its C++ implementation.

Crout Method

The Crout method simplifies the whole factorization process by setting the values of all diagonal elements of matrix U equal to 1; that is, uii = 1 for i = 1, 2, ..., n. This technique had the advantage over the Gaussian elimination method where the number of unknowns in L and U is n2n, which is lower.

Figure 5.3 shows a schematic flowchart of the Crout method. The values of the elements in L and U are determined by comparing the two matrices, A and LU, element by element starting from the top row downward from left to right. These values can be formulated as follows:

Equation 5.7a. 

Crout Method

Equation 5.7b. 

Crout Method

Example 5.7. Solve the system from Example 5.4 using the Crout method.

Schematic flowchart of the Crout and Doolittle methods.

Figure 5.3. Schematic flowchart of the Crout and Doolittle methods.

Solution. Let A = LU, or

Equation 5.50. 

Schematic flowchart of the Crout and Doolittle methods.

By matching the resultant matrix from the multiplication with the original A matrix element by element, we get the following L and U matrices:

Equation 5.51. 

Schematic flowchart of the Crout and Doolittle methods.

The vector w is obtained from Lw = b through forward substitutions:

Equation 5.52. 

Schematic flowchart of the Crout and Doolittle methods.

which is the same as

Equation 5.53. 

Schematic flowchart of the Crout and Doolittle methods.

to produce w1 = −0.5,w2 = 0.5, w 3 = 0.029, and w4 = −0.52. Finally, x is obtained from Ux = w through backward substitutions, as follows:

Equation 5.54. 

Schematic flowchart of the Crout and Doolittle methods.

This produces the solutions, x4 = −0.520, x3 = −0.261, x2 = −0.409, and x1 = −0.050.

We discuss the C++ implementation of the Crout method. The main routine in this method is in factorizing A = LU. The code is obtained by comparing the left-hand and right-hand sides of the equations, element by element, to produce:

for (i=1;i<=n;i++)
      l[i][1]=a[i][1];
for (j=2;j<=n;j++)
      u[1][j]=a[1][j]/l[1][1];
for (j=2; j<=n-1;j++)
{
      for (i=j; i<=n; i++)
      {
            z=0;
            for (k=1;k<=j-1;k++)
                  z +=
                  l[i][k]*u[k][j]; l[i][j]=a[i][j]- z;
      }
      for (k=j+1;k<=n;k++)
      {
            z=0;
            for (r=1;r<=j-1;r++)
                  z += l[j][r]*u[r][k];
u[j][k]=(a[j][k]-z)/l[j][j];
      }
}
z=0;
for (k=1;k<=n-1;k++)
      z += l[n][k]*u[k][n];
l[n][n]=a[n][n]-z;

The next step is to apply forward substitutions into the equation Lw = b using the given values in b and the computed values in L. The code is shown as follows:

w[1]=b[1]/l[1][1];
for (j=1; j<=n; j++)
{
      z=0;
      for (k=1; k<=j-1;k++)
            z += l[j][k]*w[k];
      w[j] =(b[j]-z)/l[j][j];
}

Finally, we obtain the solution x through backward substitutions from Ux = w, as follows:

for (i=n; i>=1; i--)
{
      z=0;
      for (k=i+1; k<=n; k++)
            z += u[i][k]*x[k];
      x[i]=w[i]-z;
      cout << "x[" << i << "]=" << x[i] << endl;
}

Code5B.cpp shows a complete C++ code for the Crout method, which reads input from Code5B.in.

Code5B.cpp: Crout
#include <iostream.h>
#include <fstream.h>
#define n 4

void main()
{
      int i,j,k,r;
      double z,*x,*w,*b,**a,**l,**u;
      for (i=0;i<=n;i++)
      {
            x=new double [n+1];
            w=new double [n+1];
b=new double [n+1];
            a=new double *[n+1];
            l=new double *[n+1];
            u=new double *[n+1];
            for (j=0;j<=n;j++)
            {
                  a[j]=new double [n+1];
                  l[j]=new double [n+1];
                  u[j]=new double [n+1];
            }
      }
      cout.setf(ios::fixed);
      cout.precision(5);

      // read the input data
      ifstream InFile("Code5B.in");
      for (i=1;i<=n;i++)
      {
            for (j=1;j<=n;j++)
            {
                  InFile >> a[i][j];
                  l[i][j]=u[i][j]=0;
                  if (i==j)
                        u[i][j]=1;
            }
            InFile >> b[i];
      }
      InFile.close();

      // compute L and U
      for (i=1;i<=n;i++)
            l[i][1]=a[i][1];
      for (j=2;j<=n;j++)
            u[1][j]=a[1][j]/l[1][1];

      for (j=2; j<=n-1;j++)
      {
            for (i=j; i<=n; i++)
            {
                  z=0;
                  for (k=1;k<=j-1;k++)
                        z += l[i][k]*u[k][j];
                  l[i][j]=a[i][j]- z;
            }
            for (k=j+1;k<=n;k++)
            {
z=0;
                  for (r=1;r<=j-1;r++)
                        z += l[j][r]*u[r][k];
                  u[j][k]=(a[j][k]-z)/l[j][j];
            }
      }
      z=0;
      for (k=1;k<=n-1;k++)
            z += l[n][k]*u[k][n];
      l[n][n]=a[n][n]-z;

      cout << endl << "L matrix:" << endl;
      for (i=1;i<=n;i++)
      {
            for (j=1;j<=n;j++)
                  cout << l[i][j] << " ";
            cout << endl;
      }
      cout << endl;

      cout << endl << "U matrix:" << endl;
      for (i=1;i<=n;i++)
      {
            for (j=1;j<=n;j++)
                  cout << u[i][j] << " ";
            cout << endl;
      }
      cout << endl;

      // forward substitutions for finding w
      w[1]=b[1]/l[1][1];
      for (j=1;j<=n;j++)
      {
            z=0;
            for (k=1; k<=j-1;k++)
                  z += l[j][k]*w[k];
            w[j] =(b[j]-z)/l[j][j];
      }
      cout << endl << "w vector:" << endl;
      for (i=1;i<=n;i++)
      {
            cout << w[i] << endl;
      }
      cout << endl;

      // find x through backward substitutions
for (i=n;i>=1;i--)
      {
            z=0;
            for (k=i+1; k<=n; k++)
                  z += u[i][k]*x[k];
            x[i]=w[i]-z;
            cout << "x[" << i << "]=" << x[i] << endl;
      }

      // deallocate all used arrays
      for (i=0;i<=n;i++)
            delete a[i],l[i],u[i];
      delete x,w,b,a,l,u;
}

Doolittle Method

The Doolittle method is the other side of Crout, where the values of the diagonal elements in the lower triangular matrix are set to 0, or lii = 1, for i = 1, 2, ..., n in factorizing A = LU. Doolittle shares the same flowchart as Crout in terms of execution, as shown in Figure 5.3.

As in Crout, the values of the elements in L and U are evaluated by comparing matrix A with the product LU, element by element starting from the top row, from left to right. The rest of the steps in Doolittle are exactly similar to that of Crout: find the values in w through forward substitutions from the equation Lw = b, and then get the solution x from Ux = w through backward substitutions.

Example 5.8. Solve the system from Example 5.4 using the Doolittle method.

Solution.

Equation 5.55. 

Doolittle Method

Direct comparison between the two matrices produces L and U, as follows:

Equation 5.56. 

Doolittle Method

From Lw = b, the temporary matrix w = [−1 2 −1 8.107] T is obtained through forward substitutions. Finally, applying backward substitutions into Ux = w. we get the solution x = [−0.050 0.409 0.261 0.520]T.

Cholesky Method

The Cholesky method is a method that can be applied to a system of linear equations whose coefficient matrix is a special type of matrix called positive-definite. In most cases, the method will not work if this requirement is not fulfilled. However, the method may work in some exceptional cases where the coefficient matrix is symmetric but not positive definite.

A square matrix A = [aij] is said to be symmetric if A = AT, or aij = aji. A symmetric matrix is one where the elements in its ith column have equal entries as the ith row in the matrix. This type of matrix is common in many applications.

Definition 5.3. A square symmetric matrix A of size n × n is said to be positive definite if it is symmetric, and the matrix satisfies the following requirement:

xT Ax > 0, where x = [x1 x2 ... xn] T is any n × 1 vector.

Definition 5.3 is an analytical method for determining whether a given square matrix is positive definite. The expression xT Ax in the definition produces a characteristic polynomial, P(x) > 0.

Example 5.9. Determine whether the following matrix is positive definite:

Equation 5.57. 

Cholesky Method

Solution. Let x = [x1 x2 x3] T, and then

Equation 5.58. 

Cholesky Method

Therefore, A is positive definite.

In most cases, the characteristic polynomial method of determining the positive-definiteness of a matrix may not be easy to implement especially when the size of the matrix is large. The difficulty arises in converting the given equation into the form of sum of squares. Alternative computational approaches using the properties of a positive-definite matrix can also be applied, as follows:

Method 1. A square matrix A is positive definite if all conditions as stated below are satisfied:

  1. A is symmetric; that is, aij = aji for i, j = 1, 2, ..., n.

  2. All the diagonal elements in A are positive, or aii > 0 for i = 1, 2, ..., n.

  3. The largest absolute value in each row of A is the diagonal element.

  4. aiiajj > ai2j for i, j = 1, 2,..., n.

It can be shown that, from Example 5.10, all four conditions above are satisfied to prove that A is positive definite.

Method 2. All diagonal elements of the matrix U reduced from the symmetric matrix A through row operations are positive.

A reduces to U through row operations, as follows:

Equation 5.59. 

Cholesky Method

A is positive definite since uii > 0 for i = 1, 2, 3.

Method 3. All eigenvalues of the square matrix A are real and positive.

Figure 5.4 shows the schematic flowchart of the method. The method starts by assuming U = LT in the equation A = LU. Hence, the Cholesky method requires finding the unknowns in L only. This assumption reduces the number of unknowns to 1 + 2 + ... + n, which is lower than that in Crout and Doolittle. For example, the number of variables when A is a 4 × 4 matrix is only 10, against 16 in the previous two methods.

As in Crout and Doolittle, the elements in L and U are found by directly comparing A with LU. In summary, these values are obtained using the following equations:

Equation 5.8a. 

Cholesky Method
Schematic flowchart of the Cholesky method.

Figure 5.4. Schematic flowchart of the Cholesky method.

and

Equation 5.8b. 

Schematic flowchart of the Cholesky method.

Once L and U have been found, the same forward and backward substitutions steps as in Crout and Doolittle are applied before arriving at the solution.

Example 5.10. Solve the following system of linear equations using the Cholesky method:

Equation 5.62. 

Schematic flowchart of the Cholesky method.

Solution. Set A = LU, and let U = LT. This produces

Equation 5.63. 

Schematic flowchart of the Cholesky method.

All the elements in the left and right matrices are compared to produce

Equation 5.64. 

Schematic flowchart of the Cholesky method.

Applying forward substitutions on Lw = b:

Equation 5.65. 

Schematic flowchart of the Cholesky method.

we get w = [0.816 −2.008 0.728 0.025]T. Finally, the last step is to perform backward substitutions on Ux = w:

Equation 5.66. 

Schematic flowchart of the Cholesky method.

and this produces the solution w = [0.665 −1.178 0.382 0.016] T.

We now discuss the program design for the Cholesky method. The difficulty in this method lies on finding the values of matrix L according to Equation (5.8a) and (5.8b). The code is given as follows:

for (k=1;k<=n;k++)
{
      Sum=0;
      for (j=1;j<=k-1;j++)
            Sum += pow(l[k][j],2);
      l[k][k]=sqrt(a[k][k]-Sum);
      for (i=1;i<=k-1;i++)
      {
            Sum=0;
            for (j=1;j<=i-1;j++)
                  Sum += l[i][j]*l[k][j];
            l[k][i]=(a[k][i]-Sum)/l[i][i];
      }
}

Once L is found, the other triangular matrix U is simply its transpose, or U = [uij] = [lji]. This is achieved by setting u[i][j]=l[j][i] in the program for all i and j. The code for the rest of the steps involving forward and backward substitutions is similar to those in Crout and Doolittle, and it will not be discussed here. Code5C.cpp shows the complete program of the Cholesky method, which reads A and b values from an external file called Code5C.in.

Code5C.cpp: Cholesky method.
#include <iostream.h>
#include <fstream.h>
#include <math.h>
#define n 4

void main()
{
      int i,j,k;
      double *x,*w,*b,**a,**l,**u;
      double Sum;
      for (i=0;i<=n;i++)
      {
            x=new double [n+1];
            w=new double [n+1];
            b=new double [n+1];
            a=new double *[n+1];
            l=new double *[n+1];
            u=new double *[n+1];
            for (j=0;j<=n;j++)
            {
a[j]=new double [n+1];
                  l[j]=new double [n+1];
                  u[j]=new double [n+1];
            }
      }
      cout.setf(ios::fixed);
      cout.precision(5);

      // read the input data
      ifstream InFile("Code5C.in");
      for (i=1;i<=n;i++)
      {
            for (j=1;j<=n;j++)
            {
                  InFile >> a[i][j];
                  l[i][j]=u[i][j]=0;
            }
            InFile >> b[i];
      }
      InFile.close();

      for (k=1;k<=n;k++)
      {
            Sum=0;
            for (j=1;j<=k-1;j++)
                  Sum += pow(l[k][j],2);
            l[k][k]=sqrt(a[k][k]-Sum);
            for (i=1;i<=k-1;i++)
            {
                  Sum=0;
                  for (j=1;j<=i-1;j++)
                        Sum += l[i][j]*l[k][j];
                  l[k][i]=(a[k][i]-Sum)/l[i][i];
            }
   }
   for (i=1;i<=n;i++)
         for (j=1;j<=n;j++)
               u[j][i]=l[i][j];
   cout << endl << "L matrix:" << endl;
   for (i=1;i<=n;i++)
   {
         for (j=1;j<=n;j++)
               cout << l[i][j] << " ";
         cout << endl;
   }
   cout << endl;
cout << endl << "U matrix:" << endl;
   for (i=1;i<=n;i++)
   {
         for (j=1;j<=n;j++)
               cout << u[i][j] << " ";
         cout << endl;
   }
   cout << endl;

   // forward subscriptstitution for finding w
   for (j=1;j<=n;j++)
   {
         Sum=0;
         for (k=1; k<=j-1;k++)
               Sum += l[j][k]*w[k];
         w[j] =(b[j]-Sum)/l[j][j];
   }
   cout << endl << "w vector:" << endl;
   for (i=1;i<=n;i++)
   {
         cout << w[i] << endl;
   }
   cout << endl;

   // find x through backward substitutions
  for (i=n;i>=1;i--)
  {
        Sum=0;
        for (k=i+1; k<=n; k++)
              Sum += u[i][k]*x[k];
        x[i]=w[i]-Sum;
        cout << "x[" << i << "]=" << x[i] << endl;
  }

  for (i=0;i<=n;i++)
        delete a[i],l[i],u[i];
  delete x,w,b,a,l,u;
}

Thomas Algorithm

Another LU factorization method is the Thomas algorithm, which is a derivative of the Crout factorization method. The algorithm can only be applied in solving a system of linear equation whose coefficient matrix is a special type of matrix called a tridiagonal matrix. An advantage in applying the Thomas algorithm is its computational time at O(n) is lower than the Gaussian elimination approach, which stands at O(n3).

Definition 5.4. A tridiagonal matrix is a three-band matrix whose elements other than in the main diagonal and upper and lower diagonals have the value of zero, or

Equation 5.67. 

Thomas Algorithm

In the above definition, * stands for any value including zero. An example of a tridiagonal matrix of size 5 × 5 is

Equation 5.68. 

Thomas Algorithm

The tridiagonal matrix is often encountered in engineering problems. A typical application is the computational fluid dynamics (CFD) problem, which involves breaking down the big problem into smaller ones by converting the continuous differential equations in the system into ones that are discrete. The discrete equations are then reduced further into one or more systems of linear equations, including some that may have the triangular form. Therefore, the task of computing a triadiagonal system of linear equations has become a challenging but rewarding exercise in numerical computing.

The Thomas algorithm starts with a three-band coefficient matrix A. This matrix is factorized and reduced into L and U, both of which are two-band. The next step is to get w from the equation Lw = b through forward substitutions. Finally, the solution x is obtained from Ux = w through backward substitutions.

In implementing the Thomas algorithm approach, the process is very much simplified by setting the values of all diagonal elements in U to 1; that is, u11 = 1. This same approach is deployed in the Crout method. We discuss this approach through an example.

Example 5.11. Solve the following system using the Thomas Algorithm:

Equation 5.69. 

Thomas Algorithm

Solution. The coefficient matrix A is tridiagonal. Let A and its factors, L and U, be the following unknowns with αi and βi:

Equation 5.70. 

Thomas Algorithm

In U, all diagonal elements have values equal to 1. Next, factorize A = LU:

Equation 5.71. 

Thomas Algorithm

Comparing the two matrices element by element produces the following matrices:

Equation 5.72. 

Thomas Algorithm

The next step is to compute w from Lw = b through forward substitutions:

Equation 5.73. 

Thomas Algorithm

We get w = [0.25 −0.389 −0.05 −1.063]T. Finally, applying backward substitutions to Ux = w:

Equation 5.74. 

Thomas Algorithm

produces the solution x = [0.077 −0.347 0.189 −1.063]T.

Schematic flowchart of the Thomas algorithm method.

Figure 5.5. Schematic flowchart of the Thomas algorithm method.

The implementation of Thomas algorithm follows the same path as the previous LU factorization methods. This is shown in the schematic flowchart of Figure 5.5. The task of finding L and U, and solving for x through forward and backward substitutions, is shown below:

for (i=1;i<=n;i++)
{
      d[i]=a[i][i];
      if (i<=n-1)
            e[i]=a[i][i+1];
      if (i>=2)
            c[i]=a[i][i-1];
}

for (i=2;i<=n;i++)
{
      c[i] /= d[i-1];
      d[i] -= c[i]*e[i-1];
}
for (i=2;i<=n;i++)
      b[i] -= c[i]*b[i-1];

cout << "x values: " << endl;
x[n]=b[n]/d[n];
cout << x[i] << " ";
for (i=n-1;i>=1;i--)
{
      x[i]=(b[i]-e[i]*x[i+1])/d[i];
      cout << x[i] << " ";
}

Code5D.cpp shows the full source code for Thomas algorithm, which reads A and b values from an input file called Code5D.in.

Code5D.cpp: Thomas algorithm

#include <iostream.h>
#include <fstream.h>
#define n 5

void main()
{
      int i,j,k;
      double *x,*w,*b,*c,*d,*e,**a;
      for (i=0;i<=n;i++)
      {
            x=new double [n+1];
            w=new double [n+1];
            b=new double [n+1];
            c=new double [n+1];
            d=new double [n+1];
            e=new double [n+1];
            a=new double *[n+1];
            for (j=0;j<=n;j++)
                  a[j]=new double [n+1];
      }
      cout.setf(ios::fixed);
      cout.precision(5);

      // read the input data
      ifstream InFile("Code5D.in");
      for (i=1;i<=n;i++)
      {
            for (j=1;j<=n;j++)
                  InFile >> a[i][j];
InFile >> b[i];
      }
      InFile.close();

      for (i=1;i<=n;i++)
      {
            d[i]=a[i][i];
            if (i<=n-1)
                  e[i]=a[i][i+1];
            if (i>=2)
                  c[i]=a[i][i-1];
      }

      for (i=2;i<=n;i++)
      {
            c[i] /= d[i-1];
            d[i] -= c[i]*e[i-1];
      }
      for (i=2;i<=n;i++)
            b[i] -= c[i]*b[i-1];

      cout << "x values: " << endl;
      x[n]=b[n]/d[n];
      cout << x[i] << " ";
      for (i=n-1;i>=1;i--)
      {
            x[i]=(b[i]-e[i]*x[i+1])/d[i];
            cout << x[i] << " ";
      }
      cout << endl;
      for (i=0;i<=n;i++)
            delete a[i];
      delete x,w,b,c,d,e,a;
}

ITERATIVE TECHNIQUES

An iterative method is easy to implement as it is a repetition of the same set of variables that does not require too much computer memory. An iteration starts with some given initial values to the variables. These values are constantly updated at each iteration, and eventually they converge to the desired values after some successful iterations. The iterations only stop once the values have converged to some numbers where stopping criteria have been met.

The stopping point for the iterations is normally based on a comparison between an error in the iteration with some small value. A common error for an iterative process is the maximum from the difference in vector magnitude norm based on Equation (3.8), or

Equation 5.9. 

ITERATIVE TECHNIQUES

In the above equation, iteration on x = [x1 x2 ... xn] T is indicated by the superscript k. Equation (5.9) specifies the error as the difference between the computed values of x at iteration k + 1 with the ones at iteration k. The difference is compared with some small number ε. If

ITERATIVE TECHNIQUES

We discuss two iterative methods that are commonly applied in solving a system of linear equations. The first system is the Jacobi method, which updates the values of the unknowns when an iteration has completed. The second system is the Gauss-Seidel method, which performs an update on the values immediately without waiting for an iteration to complete. These two methods are basically similar, having the same formula for updating the values. Both methods require the coefficient matrix of the system to be a special type of matrix called a diagonally-dominant matrix. The difference between them lies in the way updates are made: Gauss-Seidel uses the latest values of the variables, whereas Jacobi is not using the latest values.

Definition 5.5. A square matrix A is said to be diagonally-dominant if the absolute value of its diagonal element in each row is greater than the sum of the absolute value of all other elements in the same row. This condition is expressed as follows:

Equation 5.10. 

ITERATIVE TECHNIQUES

In the above definition, a diagonally-dominant matrix must always have the largest absolute values at its diagonal elements. As an illustration, the following matrix is diagonally-dominant:

Equation 5.77. 

ITERATIVE TECHNIQUES

The following matrix is not diagonally-dominant but becomes diagonally-dominant by exchanging rows 2 and 3:

Equation 5.78. 

ITERATIVE TECHNIQUES

The diagonally-dominance criteria test on a coefficient matrix is implemented in C++, as follows:

for (i=1;i<=n;i++)
{
      Sum=0;
      for (j=1;j<=n;j++)
            if (i!=j)
                  Sum += fabs(a[i][j]);
      if (fabs(a[i][i])<=Sum)
      {
            cout <<"Unsuccessful, the matrix is not
                    diagonally-dominant" << endl;
            return;
      }
}

Jacobi Method

The Jacobi method is an iterative method that performs updates based on the values of its variables from the previous iteration. The method requires the coefficient matrix A of the linear system to be positive definite to guarantee convergence to its solutions. Otherwise, the computed values of the variables will not converge and will cause a very significant error to the solution.

The Jacobi method starts with a formulation of the variables from the given equations in the system. The rth equation in the n × n system of linear equations is given as follows:

Equation 5.79. 

Jacobi Method

From this equation, the variable xr is formulated as a subject of the rest of the variables:

Equation 5.80. 

Jacobi Method

At iteration k, an update is made on xr based on the current values of the variables

Jacobi Method

Equation 5.11. 

Jacobi Method

The complete formulation of all variables in the system of linear equations based on Equation (5.11) is shown below:

Equation 5.82. 

Jacobi Method

Figure 5.6 shows the schematic flowchart of the Jacobi method. The iterations start by assigning initial values to the variables at iteration 0, which is denoted as

Jacobi Method

The stopping criteria for the iterations are normally set according to the maximum magnitude of the vector difference of Equation (5.9) or

Jacobi Method

The same process as above is repeated for the subsequent iterations until the stopping criteria of Equation (5.9) is fulfilled. The final solution is x, which consists of the values in the last iteration where the stopping criteria are met.

Example 5.12. Solve the following system using the Jacobi method with the initial value of x(0) = [0 0 0 0]T until

Jacobi Method

Equation 5.83. 

Jacobi Method
Schematic flowchart of the Jacobi and Gauss-Seidel methods.

Figure 5.6. Schematic flowchart of the Jacobi and Gauss-Seidel methods.

Solution. It can be shown that the coefficient matrix in the above system is not positive definite. Therefore, the rows are interchanged in such a way that the dominant coefficient of each row forms the diagonal in the new coefficient matrix. This results in the following arrangement where the coefficient matrix is positive definite:

Equation 5.84. 

Schematic flowchart of the Jacobi and Gauss-Seidel methods.

We get expressions for x1, x2, x3, and x4, as follows:

Equation 5.85. 

Schematic flowchart of the Jacobi and Gauss-Seidel methods.

From Equation (5.11), the values of x1, x2, x3, and x4 are updated at each iteration k as follows:

Equation 5.86. 

Schematic flowchart of the Jacobi and Gauss-Seidel methods.

With k = 0:

Equation 5.87. 

Schematic flowchart of the Jacobi and Gauss-Seidel methods.

Table 5.2. Results from Jacobi iterations in Example 5.12

k

x(k)1

x(k)2

x(k)3

x(k)4

Results from Jacobi iterations in Example 5.12

0

0.000

0.000

0.000

0.000

0.600

1

−0.167

−0.600

0.400

−0.250

0.150

2

−0.317

−0.503

0.297

−0.133

0.112

3

−0.305

−0.577

0.399

−0.245

0.045

4

−0.314

−0.532

0.357

−0.208

0.021

5

−0.305

−0.550

0.378

−0.229

0.011

6

−0.308

−0.540

0.366

−0.218

0.005

7

−0.306

−0.545

0.372

−0.223

 

The error is greater than the tolerance. Therefore, the iterations continue with k = 1:

Equation 5.88. 

Results from Jacobi iterations in Example 5.12

The error is reduced significantly in this iteration, which means convergence to the solution is beginning to take its shape. Continuing the process gives the results as shown in Table 5.2.

The final solution is x(7) = [−0.306 −0.545 0.372 −0.223]T, which is achieved at k = 6 when

Results from Jacobi iterations in Example 5.12

Gauss-Seidel Method

The Gauss-Seidel method is an iterative method that updates the values of a variable based on the latest values of other variables in the system. Because of these latest values, convergence to the solution takes place faster than the Jacobi method.

The implementation of the Gauss-Seidel method is very similar to the Jacobi method, as shown in the flowchart in Figure 5.6. For an equation given by

Gauss-Seidel Method

Equation 5.89. 

Gauss-Seidel Method

The update on xr at iteration k is the one-step ahead value given as follows:

Equation 5.12. 

Gauss-Seidel Method

In the above equation, the update on xr involves the latest values of xi for i = 1, 2, ...,r − 1 at iteration k + 1 and the values of xi for i = r + 1, r + 2, ..., n at iteration k. The complete update on the variables is shown as follows:

Equation 5.91. 

Gauss-Seidel Method

The standard error in the iterations is the maximum magnitude norm as expressed in Equation (5.9). This error is compared with the preset threshold value ε, where the iterations stop immediately if the error is smaller than or equal to this value.

Example 5.13. Solve the system of linear equations in Example 5.12 using the Gauss-Seidel method with the initial value of x(0) = [0 0 0 0]T until

Gauss-Seidel Method

Solution. The coefficient matrix of the above system is not diagonally-dominant. Rearrange the rows so that the diagonal elements become dominant, as follows:

Equation 5.92. 

Gauss-Seidel Method

From Equation (5.15), updates are performed according to

Equation 5.93. 

Gauss-Seidel Method

Starting with k = 0:

Equation 5.94. 

Gauss-Seidel Method

The iteration continues with k = 1 since the error is greater than ε:

Equation 5.95. 

Gauss-Seidel Method

Additional iterations produce results as shown in Table 5.3.

Table 5.3. Results from Example 5.12

k

x(k)1

x(k)2

x(k)3

x(k)4

Results from Example 5.12

0

0.000

0.000

0.000

0.000

0.633

1

−0.167

−0.633

0.340

−0.133

0.175

2

−0.341

−0.574

0.395

−0.228

0.035

3

−0.318

−0.539

0.374

−0.228

0.013

4

−0.305

−0.541

0.368

−0.221

0.002

5

−0.306

−0.543

0.369

−0.221

 

The iterations stop at k = 4 to produce x(5) as the solution where the error given by

Results from Example 5.12

Code5E. Gauss-Seidel method.

#include <iostream.h>
#include <fstream.h>
#include <math.h>
#define n 4            // array size
#define MAX 10         // maximum number of iterations

void main()
{
      int i,j,k;
      double *x,*xOld,*b,**a;
      double Sum,error;

      // allocate memory
      x=new double [n+1];
      xOld=new double [n+1];
      b=new double [n+1];
      a=new double *[n+1];
      for (i=0;i<=n;i++)
            a[i]=new double [n+1];
      cout.setf(ios::fixed);
      cout.precision(3);

      // read the input data
      ifstream InFile("Code5E.in");
      for (i=1;i<=n;i++)
      {
            x[i]=xOld[i]=0;
            for (j=1;j<=n;j++)
                  InFile >> a[i][j];
            InFile >> b[i];
      }
InFile.close();

      // test for diagonally-dominance
      for (i=1;i<=n;i++)
      {
            Sum=0;
            for (j=1;j<=n;j++)
                  if (i!=j)
                       Sum += fabs(a[i][j]);
            if (fabs(a[i][i])<=Sum)
            {
                  cout <<"Unsuccessful, the matrix is not
                          diagonally-dominant" << endl;
                  return;
            }
      }

      // perform iterations & update x
      for (k=0;k<=MAX;k++)
      {
            error=0;
            cout << "k=" << k << ": ";
            for (i=1;i<=n;i++)
            {
                  Sum=0;
                  for (j=1;j<=n;j++)
                        if (i!=j)
                              Sum += a[i][j]*x[j];
                  xOld[i]=x[i];
                  x[i]=(b[i]-Sum)/a[i][i];
                  cout << x[i] << " ";
                  error=((error>fabs(x[i]-xOld[i]))?error:
                          fabs(x[i]-xOld[i]));
            }
            cout << error << endl;
      }

      // delete memory
      for (i=0;i<=n;i++)
            delete a[i];
      delete x,xOld,b,a;
}

VISUALIZING THE SOLUTION: CODE5

We now discuss the design and development of several modules for solving a system of linear equations. The modules are integrated into a single system based on Windows using the resources in the Microsoft Foundation Classes library. The interface is user-friendly with dialog boxes for input, and buttons for updates, which serves as a black box for solving the system of linear equations problem to its user. A user-friendly interface must have features that allow easy input of data, modification of the input data, a choice of several methods for the same data, and an error-proof mechanism to detect problems in data entry. The last factor is important since problems like singular matrix as input can cause the system to crash instead of producing a good solution.

Figure 5.7 shows the Windows interface for solving a system of up to five linear equations problem. The interface consists of a modal window with input in the form of edit boxes (white), the output in static boxes (gray), buttons for activating the methods, and text messages. The buttons are labeled Gauss, Crout, Cholesky, Thomas, and Gauss-Seidel, which activate the named methods for solving the problem. The Reset button clears all entries and refreshes the display.

Windows interface for solving a system of up to five linear equations.

Figure 5.7. Windows interface for solving a system of up to five linear equations.

The figure also shows a sample run of the Gauss-Seidel method with A and b as input in the edit boxes, to produce the output x in the static boxes once the Gauss — Seidel button is clicked. The program is intelligent enough to recognize the coefficient matrix A as diagonally dominant. Otherwise, if A is not diagonally dominant, then a click on the Gauss-Seidel button will not produce the desired results. The same input produces the same results when the Gauss and Crout buttons are clicked, but it will respond with some error messages if the Cholesky and Thomas buttons are clicked. In this case, the program can verify correctly that A is not positive definite and tridiagonal, respectively.

Any size of matrix, from two to five, can be entered. Entries can be made beginning from the top left-hand corner. The size of the matrix is then automatically determined from the first unfilled entry in the diagonal elements.

We discuss the steps for producing this interface. They consist of creating a new Win32 project codenamed Code5 using the MFC static library, creating the resource file Code5.rc, and inserting codes for the Gauss, Crout, Cholesky, Thomas, and Gauss-Seidel methods. Only three files need to be created, namely Code5.cpp, Code5.h, and Code5.rc. A single class called CCode5 is used, and this class is derived from MFC's CDialog.

Figure 5.8 shows a schematic drawing of the computational steps in Code5. The process starts with the resource file Code5.rc for creating the dialog window. The dialog window becomes the main window that hosts several child windows consisting of edit boxes for collecting the input, static boxes for displaying the output, and buttons for selecting a method for solving the problem.

Schematic drawing of the computational steps in Code5.

Figure 5.8. Schematic drawing of the computational steps in Code5.

Global objects and variables are initialized in the constructor, CCode5(). They include the matrices and vectors for the system of linear equations. ReadInput() reads the input data in the edit boxes, and the data are then passed for processing to a function when a push button that points to the function is clicked. The activated function returns either fStatus=1 or fStatus=0, which indicates whether the supplied data fit into the named method or not, respectively. With fStatus=1, the results from the function are displayed in the static boxes through ShowResults(), and the method is flagged as successful. A return value of fStatus=0 indicates the method fails because of irrelevant data, and the error message is shown through ShowError(). Another function called OnReset() refreshes the edit and static boxes for a new data entry.

There are six major steps in the development of Code5. This project differs slightly from the skeleton program in Code3A project as the main window is a dialog window derived from MFC's CDialog. A dialog window is a container class that can host several objects from other classes.

Step 1. Create a new Win32 project.

From the menu, choose File, followed by New and Project. Name the project Code5, and press the OK button. A new class called CCode5 is created. Declare the project as an empty Windows application project through the check boxes.

From the Solution Explorer, highlight CCode5 and right-click Properties. Choose Use MFC in a Static Library from the item Use of MFC in the combo box. These options allow the application to use the resources in the MFC library.

Step 2. Add the resource file Code5.rc.

This step creates all the friendly tools in the modal window that are the resources in Windows. Right-click Resource Files in the Solution Explorer, as shown in Figure 5.9. Choose Add from the menu and then Add Resource.

Step 3. Add a dialog window.

A dialog window is host to several friendly tools such as buttons, edit boxes, combo boxes, check boxes, and static boxes. To create a dialog window, choose Dialog from the Add Resource window, as shown in Figure 5.10.

Step 5. Create edit boxes, static boxes, and buttons.

A dialog window as shown in Figure 5.11 appears. This window is automatically assigned with the id IDD_DIALOG1. From the Toolbox, choose Edit Control, and draw an edit box, as shown in the figure. Click Properties Windows, and assign this edit box with the id IDC_A11 for representing the element a11 in the coefficient matrix A.

Continue with the rest of the elements in matrix A (IDC_A12 up to IDC_A55) and vector b (IDC_b1 to IDC_b5) to complete the input interface. Next, click Toolbox and choose Static Text. Create static boxes for housing the vector x with ids from IDC_x1 to IDC_x5. Figure 5.12 shows the complete interface for this problem with ids on the items in the Windows.

Step 6. Create the files Code5.cpp and Code5.h, and insert their codes.

The main file is Code5.cpp, which drives the application by calling both Code5.h and Code5.rc.

Adding a resource file called Code5.rc.

Figure 5.9. Adding a resource file called Code5.rc.

Add Resource items.

Figure 5.10. Add Resource items.

The contents of the header file Code5.h differ from the skeleton program in Code3A, as its host window is a dialog window. It is necessary to add #include "resource.h" in the preprocessing area of Code5.h as this file contains all the necessary declarations for the resources created in Code5.rc.

Dialog window with the id IDD_DIALOG1.

Figure 5.11. Dialog window with the id IDD_DIALOG1.

The ids of the resources in the dialog window.

Figure 5.12. The ids of the resources in the dialog window.

#include <afxdisp.h>
#include <math.h>
#include "resource.h"
#define N 5
class CCode5 : public CDialog
{
public:
      CCode5(CWnd* pParent = NULL);
      enum { IDD = IDD DIALOG1 };
      virtual ~CCode5();
      virtual void OnOK();
      void ReadInput(),DisplayResults(),DisplayError(CString);
      afx msg void OnGauss();
      afx msg void OnCrout();
      afx msg void OnCholesky();
      afx msg void OnThomas();
      afx msg void OnGaussSeidel();
      afx msg void OnReset();
      int n;
      double **A,**L,**U,*x,*b;
      CString **sA,*sx,*sb,method;
      DECLARE_MESSAGE_MAP()
};

class CMyWinApp : public CWinApp
{
public:
      virtual BOOL InitInstance();
};

The dialog window is created in Code5.cpp. This window is called the modal window, as it does not allow background editing if the window is active. Its opposite is the nonmodal window, which allows background editing when the window is active. The following code segment in Code5.cpp creates the dialog window:

BOOL CMyWinApp::InitInstance()
{
   AfxEnableControlContainer();
   CCode5 dlg;
   m pMainWnd = &dlg;
   dlg.DoModal();
   return FALSE;
}

A method for solving the system of linear equations based on the input data becomes active when one of the push buttons that correspond to the method is left-clicked. Each push button has an id, and it calls the corresponding function through an event mapping given by

Table 5.4. Summary of functions, their ids, and their purposes in CCode5

Function

Control Id

Purpose

CCode5()

 

Constructor of the class CCode5.

~CCode5()

 

Destructor of the class CCode5.

ReadInput()

 

Reads input data in matrix A and vector b.

DisplayResults()

 

Displays the output.

DisplayError()

 

Displays the errors in the input.

Reset()

 

Clears all input and output information.

OnOK()

IDOK

Closes the dialog window.

OnGauss()

IDC_GAUSS

Activates the Gaussian elimination method.

OnCrout()

IDC_CROUT

Activates the Crout method.

OnCholesky()

IDC_CHOLESKY

Activates the Cholesky method.

OnThomas()

IDC_THOMAS

Activates the Thomas algorithm.

OnGaussSeidel()

IDC_GAUSSSEIDEL

Activates the Gauss-Seidel method.

BEGIN_MESSAGE_MAP(CCode5,CDialog)
    ON_BN_CLICKED(IDC_GAUSS, OnGauss)
    ON_BN_CLICKED(IDC_CROUT, OnCrout)
    ON_BN_CLICKED(IDC_CHOLESKY, OnCholesky)
    ON_BN_CLICKED(IDC_THOMAS, OnThomas)
    ON_BN_CLICKED(IDC_GAUSSSEIDEL, OnGaussSeidel)
    ON_BN_CLICKED(IDC_RESET, OnReset)
END_MESSAGE_MAP()

It is obvious in the message map that a click on the Gauss button, whose id is IDC_GAUSS, triggers a call on the function OnGauss(). This function reads the data provided by the user in the edit boxes and then solves the problem using the Gaussian elimination method.

Table 5.4 shows the member functions of the class CCode5. Included are eight main member functions representing the buttons in the class. Each function in the class is named according to its duty; for example, OnGauss() responds to the Gauss button click, and this function solves the system of linear equations based on the data input from the user. The ids of these buttons are shown in Figure 5.12.

The edit boxes provide input spaces for matrix A and vector b. A system of up to 5 × 5 linear equations is supported, and this is indicated as the macro N in the header file Code5.h. The program has been designed flexible so as to support other sizes simply filling in the values of A starting from the top left-hand corner. The actual size of the matrix is represented by a variable called n. The size is determined from the filled elements in the diagonal. For example, to have a 3 × 3 system, or n=3, entries must be made in the first three rows and three columns of the edit boxes beginning from the top left-hand corner.

Beside the button functions, CCode5 has four other application functions, ReadInput(), DisplayResults(), DisplayError(), and OnOK(). These functions are called from the selected method for reading the input data, displaying the output, displaying error messages, and closing the application, respectively.

The constructor in the class is CCode5(). This function activates the dialog box with the id IDD_DIALOG1, which is the one shown in Figure 5.11. This window is derived from the MFC class CDialog. Besides creating the class, CCode5() also initializes several arrays and allocates memory dynamically. This is shown as follows:

CCode5::CCode5(CWnd* pParentWnd)
: CDialog (IDD DIALOG1, pParentWnd)
{
      x=new double [N+1];
      b=new double [N+1];
      sx=new CString [N+1];
      sb=new CString [N+1];
      A=new double *[N+1];
      L=new double *[N+1];
      U=new double *[N+1];
      sA=new CString *[N+1];
      for (int i=0;i<=N;i++)
      {
            A[i]=new double [N+1];
            L[i]=new double [N+1];
            U[i]=new double [N+1];
            sA[i]=new CString [N+1];
      }
}

The destructor in the program is ~CCode5(). This function destroys the arrays allocated in the constructor and the class to mark the end of the application runtime. The code is shown as follows:

CCode5::~CCode5()
{
      for (int i=0;i<=N;i++)
            delete A[i],L[i],U[i],sA[i];
      delete x,b,sx,sb,A,L,U,sA;
}

ReadInput() is a function for reading all input from the user in the form of edit boxes, as shown in Figure 5.11. Input from the user is read through the function GetDlgItemText(). Each edit box has an idd that reflects its matrix element. For example, IDC_A24 represents the edit box for a24 in matrix A, whereas IDC_b3 represents b3 in b. However, the representation is not straightforward. An edit box takes only string (text) input, and this string needs to be converted to a double value before a computation can be performed. Hence, the value input by the user inside the box with the id IDC_A24 is stored as the string sA[2][4], whereas that of IDC_b3 is stored as sb[3]. These values are converted to the double variables called A[2][4] and b[3], respectively, using the C function atof(). The code is shown, as follows:

void CCode5::Read
{
      GetDlgItemText(IDC_A11,sA[1][1]);
      GetDlgItemText(IDC_A12,sA[1][2]);
      GetDlgItemText(IDC_A13,sA[1][3]);
      GetDlgItemText(IDC_A14,sA[1][4]);
      GetDlgItemText(IDC_A15,sA[1][5]);
      GetDlgItemText(IDC_A21,sA[2][1]);
      GetDlgItemText(IDC_A22,sA[2][2]);
      GetDlgItemText(IDC_A23,sA[2][3]);
      GetDlgItemText(IDC_A24,sA[2][4]);
      GetDlgItemText(IDC_A25,sA[2][5]);
      GetDlgItemText(IDC_A31,sA[3][1]);
      GetDlgItemText(IDC_A32,sA[3][2]);
      GetDlgItemText(IDC_A33,sA[3][3]);
      GetDlgItemText(IDC_A34,sA[3][4]);
      GetDlgItemText(IDC_A35,sA[3][5]);
      GetDlgItemText(IDC_A41,sA[4][1]);
      GetDlgItemText(IDC_A42,sA[4][2]);
      GetDlgItemText(IDC_A43,sA[4][3]);
      GetDlgItemText(IDC_A44,sA[4][4]);
      GetDlgItemText(IDC_A45,sA[4][5]);
      GetDlgItemText(IDC_A51,sA[5][1]);
      GetDlgItemText(IDC_A52,sA[5][2]);
      GetDlgItemText(IDC_A53,sA[5][3]);
      GetDlgItemText(IDC_A54,sA[5][4]);
      GetDlgItemText(IDC_A55,sA[5][5]);
      GetDlgItemText(IDC_b1,sb[1]);
      GetDlgItemText(IDC_b2,sb[2]);
      GetDlgItemText(IDC_b3,sb[3]);
      GetDlgItemText(IDC_b4,sb[4]);
      GetDlgItemText(IDC_b5,sb[5]);

      for (int i=1;i<=N;i++)
            if (sA[i][i]=="")
            {
                  n=i-1; break;
            }
     // convert the string input to double
     for (i=1;i<=n;i++)
     {
           for (int j=1;j<=n;j++)
A[i][j]=atof(sA[i][j]);
           b[i]=atof(sb[i]);
     }
}

The output in the program is x = [x1 x2 x3 x4 x5]T, which is displayed by the function DisplayResults(). The vector is represented by the double array x, and it is displayed on the static boxes using SetDlgItemText() through the string sx. Conversion from x to the string sx is achieved through the function Format(). The static boxes are represented byids that reflect their variables; for example,IDC_x1 represents the string sx[1]. The last statement in this function displays the successful message to acknowledge that the given method is successful. The following code shows the contents of DisplayResults():

void CCode5::DisplayResults()
{
      for (int i=1;i<=N;i++)
            sx[i].Format("%lf",x[i]);
      SetDlgItemText(IDC_x1,((1<=n)?sx[1]:""));
      SetDlgItemText(IDC_x2,((2<=n)?sx[2]:""));
      SetDlgItemText(IDC_x3,((3<=n)?sx[3]:""));
      SetDlgItemText(IDC_x4,((4<=n)?sx[4]:""));
      SetDlgItemText(IDC_x5,((5<=n)?sx[5]:""));
      SetDlgItemText(IDC_MESSAGE,method+" is successful.");
}

OnGauss() is the function that responds to the Gauss button click identified through the id IDC_GAUSS. This function is based on Code4A.cpp. A Boolean variable called fStatus is introduced to check the value of the pivot element A[k][k], where 0 (FALSE) indicates the element is 0. With this value, the Gaussian elimination method is considered a failure, and an error message is displayed through the function DisplayError(). Through this process, a division by zero in m=A[i][k]/A[k][k] during the row operations with respect to row k can be avoided. The function is given by

void CCode5::OnGauss()
{
      int i,j,k;
      double Sum,p;
      bool fStatus=1;
      CString condition="invertible";
      method="Gauss";
      ReadInput();
// perform row operations
      for (k=1;k<=n-1;k++)
            for (i=k+1;i<=n;i++)
            {
                  if (A[k][k]==0)
                  {
                        fStatus=0;
                        break;
                  }
                  else
                        p=A[i][k]/A[k][k];
                  for (j=1;j<=n;j++)
                        A[i][j] -= p*A[k][j];
                  b[i] -= p*b[k];
           }

     // perform backward substitutions
     for (i=n;i>=1;i--)
     {
           Sum=0;
          x[i]=0;
          for (j=i;j<=n;j++)
          Sum += A[i][j]*x[j];
          x[i]=(b[i]-Sum)/A[i][i];
     }
     if (fStatus)
           DisplayResults();
     else
           DisplayError(condition);
}

A click on the Crout button whose id is IDC_CROUT activates the function OnCrout(), which computes the problem using the Crout() method. This function also has a Boolean variable called fStatus to check for the zero values in the diagonal elements of matrix L. Successful runtime is achieved through fStatus=1, and the output is displayed through the function DisplayResults(). The code in this function is given, as follows:

void CCode5::OnCrout()
{
      int i,j,k,r;
      double Sum,*w;
      bool fStatus=1;
      CString condition="invertible";
      method="Crout";
w=new double [N+1];
      ReadInput();

      // initialize L and U
      for (i=1;i<=n;i++)
            for (j=1;j<=n;j++)
            {
                  L[i][j]=U[i][j]=0;
                  if (i==j)
                  U[i][j]=1;
            }

      // compute L and U
      for (i=1;i<=n;i++)
            L[i][1]=A[i][1];
      for (j=2;j<=n;j++)
            if (L[1][1]==0)
            {
                  fStatus=0;
                  break;
            }
            else
                  U[1][j]=A[1][j]/L[1][1];
      for (j=2; j<=n-1;j++)
      {
            for (i=j; i<=n; i++)
            {
                  Sum=0;
                  for (k=1;k<=j-1;k++)
                        Sum += L[i][k]*U[k][j];
                  L[i][j]=A[i][j]- Sum;
            }
            for (k=j+1;k<=n;k++)
            {
            Sum=0;
            for (r=1;r<=j-1;r++)
                  Sum += L[j][r]*U[r][k];
            if (L[j][j]==0)
            {
                  fStatus=0;
                  break;
            }
            else
                  U[j][k]=(A[j][k]-Sum)/L[j][j];
            }
       }
Sum=0;
       for (k=1;k<=n-1;k++)
             Sum += L[n][k]*U[k][n];
       L[n][n]=A[n][n]-Sum;

       // forward substitutions for finding w
       w[1]=b[1]/L[1][1];
       for (j=1;j<=n;j++)
       {
             Sum=0;
             for (k=1; k<=j-1;k++)
                   Sum += L[j][k]*w[k];
             if (L[j][j]==0)
             {
                   fStatus=0;
                   break;
             }
             else
                   w[j] =(b[j]-Sum)/L[j][j];
       }

       // backward substitutions for finding x
       for (i=n;i>=1;i--)
       {
             Sum=0;
             for (k=i+1; k<=n; k++)
                   Sum += U[i][k]*x[k];
             x[i]=w[i]-Sum;
       }
       delete w;
       if (fStatus)
             DisplayResults();
       else
             DisplayError(condition);
}

OnCholesky() responds to the Cholesky button click. This function first performs a test on the positive-definiteness of the coefficient matrix, with fStatus=1 indicating positive and fStatus=0 negative. The code is given, as follows:

void CCode5::OnCholesky()
{
      int i,j,k;
      double Sum,*w;
      bool fStatus=1;
CString condition="positive-definite";
      method="Cholesky";
      w=new double [N+1];
      ReadInput();

      // check for positive-definite
      for (i=1;i<=n;i++)
            for (j=i;j<=n;j++)
                  if ((A[i][j]!=A[j][i])
                       | | A[i][i]<=0
                       | | (i!=j && A[i][i]<=fabs(A[i][j])))
                  {
                       fStatus=0;
                       break;
                  }

      // initialize L and U
      for (i=1;i<=n;i++)
            for (j=1;j<=n;j++)
                  L[i][j]=U[i][j]=0;

     // compute L and U
     for (k=1;k<=n;k++)
     {
           Sum=0;
           for (j=1;j<=k-1;j++)
                 Sum += pow(L[k][j],2);
           L[k][k]=sqrt(A[k][k]-Sum);
           for (i=1;i<=k-1;i++)
           {
                 Sum=0;
                 for (j=1;j<=i-1;j++)
                       Sum += L[i][j]*L[k][j];
                 L[k][i]=(A[k][i]-Sum)/L[i][i];
           }
      }

      // forward substitutions for finding w
      for (j=1;j<=n;j++)
      {
            Sum=0;
            for (k=1; k<=j-1;k++)
                  Sum += L[j][k]*w[k];
            w[j] =(b[j]-Sum)/L[j][j];
      }
// backward substitutions for finding x
      for (i=n;i>=1;i--)
      {
            Sum=0;
            for (k=i+1; k<=n; k++)
                  Sum += U[i][k]*x[k];
            x[i]=w[i]-Sum;
      }
      delete w;
      if (fStatus)
           DisplayResults();
      else
           DisplayError(condition);
}

The case of the coefficient matrix in the system is tridiagonal, which leads to the deployment of the Thomas algorithm through the function OnThomas(). Just like other methods, this function uses the Boolean variable fStatus; a value of 1 indicates the matrix is tridiagonal, whereas 0 means the matrix is not tridiagonal. With fStatus=1, the Thomas algorithm is applied to the system, and the output is displayed in DisplayResults(). The code is shown below:

void CCode5::OnThomas()
{
      int i,j;
      bool fStatus=1;
      double *e,*f,*g;
      CString condition="tridiagonal";
      method="Thomas";
      e=new double [N+1];
      f=new double [N+1];
      g=new double [N+1];
      ReadInput();

      // check for tridiagonality
      for (i=1;i<=n;i++)
      {
            if (i==1)
                  for (j=3;j<=n;j++)
                        if (A[i][j]!=0)
                              fStatus=0;
            if (i==n)
                  for (j=1;j<=n-2;j++)
                        if (A[i][j]!=0)
                              fStatus=0;
            if (i>1 && i<n)
                  if (A[i][i+2]!=0 && A[i][i-2]!=0)
fStatus=0;
      }

      // compute e,f,g
      for (i=1;i<=n;i++)
      {
            f[i]=A[i][i];
            if (i<=n-1)
                  g[i]=A[i][i+1];
            if (i>=2)
                  e[i]=A[i][i-1];
      }
      for (i=2;i<=n;i++)
      {
            e[i]/=f[i-1];
            f[i] -= e[i]*g[i-1];
      }
      for (i=2;i<=n;i++)
            b[i] -= e[i]*b[i-1];
      x[n]=b[n]/f[n];
      for (i=n-1;i>=1;i--)
            x[i]=(b[i]-g[i]*x[i+1])/f[i];
      delete e,f,g;
      if (fStatus)
            DisplayResults();
      else
            DisplayError(condition);
}

The iterative technique in the program using Gauss-Seidel is illustrated through the OnGaussSeidel() function. This function checks to make sure the coefficient matrix is diagonally dominant through the Boolean variable fStatus, just like in the previous methods. The code is shown, as follows:

void CCode5::OnGaussSeidel()
{
      const int MAX=10;
      double Sum,error;
      double *xOld;
      int i,j,k;
      bool fStatus=1;
      CString condition="diagonally-dominant";
      method="Gauss-Seidel";
      xOld=new double [N+1];
      ReadInput();

      for (i=1;i<=n;i++)
{
            Sum=0;
            for (j=1;j<=n;j++)
                  if (i!=j)
                        Sum += fabs(A[i][j]);
            if (fabs(A[i][i])<=Sum)
                  fStatus=0;
      }

      for (i=1;i<=n;i++)
      x[i]=0;

      // perform iterations & update x
      for (k=0;k<=MAX;k++)
      {
            error=0;
            for (i=1;i<=n;i++)
            {
                  Sum=0;
                  for (j=1;j<=n;j++)
                        if (i!=j)
                              Sum += A[i][j]*x[j];
                  xOld[i]=x[i];
                  x[i]=(b[i]-Sum)/A[i][i];
                  error=((error>fabs(x[i]-xOld[i]))?error:
                          fabs(x[i]-xOld[i]));
            }
      }
      delete xOld;
      if (fStatus)
            DisplayResults();
      else
            DisplayError(condition);
}

The last application function is OnReset(), which responds to a mouse click on the Reset button whose id is IDC_RESET. The function resets the data in the edit boxes and prepares for a fresh input. This result is achieved by setting the string arrays sA, sb, and sx to null (""). The code is shown below:

Void CCode5::OnReset()
{
      int i,j;
      for (i=1;i<=N;i++)
      {
            for (j=1;j<=N;j++)
                  sA[i][j]="";
sb[i]="";
            sx[i]="";
      }
      SetDlgItemText(IDC_A11,sA[1][1]);
      SetDlgItemText(IDC_A12,sA[1][2]);
      SetDlgItemText(IDC_A13,sA[1][3]);
      SetDlgItemText(IDC_A14,sA[1][4]);
      SetDlgItemText(IDC_A15,sA[1][5]);
      SetDlgItemText(IDC_A21,sA[2][1]);
      SetDlgItemText(IDC_A22,sA[2][2]);
      SetDlgItemText(IDC_A23,sA[2][3]);
      SetDlgItemText(IDC_A24,sA[2][4]);
      SetDlgItemText(IDC_A25,sA[2][5]);
      SetDlgItemText(IDC_A31,sA[3][1]);
      SetDlgItemText(IDC_A32,sA[3][2]);
      SetDlgItemText(IDC_A33,sA[3][3]);
      SetDlgItemText(IDC_A34,sA[3][4]);
      SetDlgItemText(IDC_A35,sA[3][5]);
      SetDlgItemText(IDC_A41,sA[4][1]);
      SetDlgItemText(IDC_A42,sA[4][2]);
      SetDlgItemText(IDC_A43,sA[4][3]);
      SetDlgItemText(IDC_A44,sA[4][4]);
      SetDlgItemText(IDC_A45,sA[4][5]);
      SetDlgItemText(IDC_A51,sA[5][1]);
      SetDlgItemText(IDC_A52,sA[5][2]);
      SetDlgItemText(IDC_A53,sA[5][3]);
      SetDlgItemText(IDC_A54,sA[5][4]);
      SetDlgItemText(IDC_A55,sA[5][5]);
      SetDlgItemText(IDC_b1,sb[1]);
      SetDlgItemText(IDC_b2,sb[2]);
      SetDlgItemText(IDC_b3,sb[3]);
      SetDlgItemText(IDC_b4,sb[4]);
      SetDlgItemText(IDC_b5,sb[5]);
      SetDlgItemText(IDC_x1,sx[1]);
      SetDlgItemText(IDC_x2,sx[2]);
      SetDlgItemText(IDC_x3,sx[3]);
      SetDlgItemText(IDC_x4,sx[4]);
      SetDlgItemText(IDC_x5,sx[5]);
}

SUMMARY

We have discussed various systems of linear equations, their representation, methods for solving them, and program designs in C++ as their solution. We also developed a Windows-based version of the program using the Microsoft Foundation Class library, which provides a user-friendly interface for solving the problem.

In solving a system of linear equations, it is important to consider a user-friendly interface that allows interaction between the user and the program. On the problem side, the user must understand the factors affecting the existence and uniqueness of the solution. The solution to a mathematical problem can be drawn based on the solid understanding of the concepts that govern the problem.

Providing a friendly solution to the problem can be summarized to fall into three steps. First, the conceptual and analytic solution based on the mathematical theories must be fully understood. Second, when the first step has been cleared, the next step is to derive the manual solution to the problem using a calculator as a tool. This second approach should be applicable to a small problem, like a system of 3 × 3 linear equations. Once this step is completed, the third and last step involving programming can now be embarked on comfortably. This step is a realization to the problem where all the tedious work in step one and two is simplified with the click of some buttons. A computer is the ultimate tool that helps to digest the massive calculations involved in solving a problem involving systems of linear equations.

All three steps have been discussed in this chapter, with a special attention on step three. The methods for solving the systems of linear equations in this chapter are the fundamental solution to many problems in engineering and the sciences, as will be discussed later in the book.

NUMERICAL EXERCISES

  1. Determine whether each matrix below possesses each of the properties: tridiagonal, positive definite, or diagonally dominant.

    Equation 5.96. 

    NUMERICAL EXERCISES
  2. Determine whether a system of linear equations whose coefficient matrix is each of the matrices in Problem 1 can be solved using the the Gauss, Gauss with partial pivoting, Gauss-Jordan, Crout, Doolittle, Cholesky, Thomas, Jacobi, and Gauss-Seidel methods.

  3. Solve the following systems of linear equations using the indicated method(s):

    1. Gauss, Gauss-Jordan, and Crout.

      Equation 5.97. 

      NUMERICAL EXERCISES
    2. Gauss, Gauss with partial pivoting, and Doolittle.

      Equation 5.98. 

      NUMERICAL EXERCISES
    3. Cholesky and Gauss-Seidel.

      Equation 5.99. 

      NUMERICAL EXERCISES
    4. Cholesky and Gauss—Seidel.

      Equation 5.100. 

      NUMERICAL EXERCISES
    5. Gauss, Thomas algorithm, and Gauss-Seidel.

      Equation 5.101. 

      NUMERICAL EXERCISES

Check the results by running Code5.

PROGRAMMING CHALLENGES

  1. Design standard C++ programs for solving a system of linear equations using the following methods:

    1. Gaussian elimination method with partial pivoting.

    2. Doolittle method.

  2. Design a standard C++ program to solve the system of linear equations in Example 5.11 using the Jacobi method.

  3. Improve on the Code5 project by adding a few user-friendly features, as follows:

    1. Add the flexibility and scalability of the system. In the Code5 project, only up to a 5 × 5 system is supported. A flexible system allows scalability where any size of linear systems up to n × n can be solved. Design an interface to allow any choice of system up to 10 × 10 to be solved.

    2. Show the staggered results from the calculations in each method applied. For example, it is necessary to display the results from each iteration in the Gauss-Seidel method instead of just the final results. Design a new interface with the parent window from the MFC class CFrameWnd to achieve this objective.

  4. Add new buttons and their corresponding functions to the Code5 project using the Gauss-Jordan, Gauss with partial pivoting, Doolittle, and Jacobi methods. In each function, add features to check the runtime errors that may result from properties such as diagonally dominance and singularity of the coefficient matrix.

  5. Improve on Code5 by including the file read and open options, specifically to read and store the matrix and vector values in the problem.

..................Content has been hidden....................

You can't read the all page of ebook, please click here login for view all page.
Reset