Writing a small program is easy. You just type your program in an editor and shove your source code through a compiler. If you find any problems, dig through the source code, fix the problem, and recompile the whole thing all over again. No matter how many bugs pop up in your program, chances are good your program is small enough so you can hunt any bugs and wipe them out without much trouble.
What happens if you need to write a massive program to control the flight of a satellite in orbit or an automatic safety system for monitoring a nuclear plant? Unlike simple programs that can consist of a few hundred lines of code, massive programming projects can contain millions of lines of code. How can you create such a large program, let alone debug, test, and maintain it?
That's where software engineering comes into play. Just as it's much easier to pitch a pup tent by yourself than it is to construct a skyscraper on your own, it's also easier to write a simple program by yourself, but nearly impossible to write a million-line program all by yourself.
When you have to write a huge program, you probably can't do it alone, so you need to work with a team of programmers and coordinate your efforts. Even before you write a single line of code, you need a plan for creating the program from the start. Just as no one starts welding I-beams together to build a skyscraper without making plans first, no one tackles a massive software project without a lot of planning ahead of time.
Warning
Software engineering is about creating guidelines to help people consistently write reliable programs within a reasonable amount of time. The fact that governments and Fortune 500 companies routinely start massive software projects with no plans (and often abandon them after never getting them to work despite spending millions of dollars) is proof that software engineering is a nice term that often gets completely ignored in the real world.
In the early days of computer programming, a single programming genius might create a program. However, relying on the inspiration and creativity of a single genius isn't a way to run a business. If the programming genius dies or quits, the company selling that program won't know how to fix or update the program later.
So the idea behind software engineering is to remove a company's dependence on a single programming genius and establish a systematic method for creating software that can be done with an army of competent programmers rather than a single inspired programming wizard.
Over the years, computer scientists have created a variety of software engineering methods, which all have advantages and disadvantages. No matter which software engineering method is popular today or tomorrow, the goals are always the same:
Make it easy to write large computer programs within a reasonable period of time.
Create software that works reliably.
Writing large programs is hard enough. Being able to create large programs within a strict deadline is much harder and making sure the software can be trusted to work reliably is even tougher. Software engineering methods try to solve these problems in various ways with different results, ranging from complete success (rare) to abysmal failures (more common than you might like to think).
The waterfall model of software engineering divides the task of creating a large software project into distinct parts with each part being fully completed before the other part can even begin.
Typically, the waterfall method divides a software project into four parts, as shown in Figure 5-1:
Analysis
Design
Implementation
Testing
The idea behind the waterfall method is to complete tasks one at a time so that you're always focusing on a single task.
Figure I.5-1. The waterfall method divides a project into four distinct and mutually exclusive steps.
Analysis
The first step is to analyze the project to determine exactly what the program's supposed to do. (Surprisingly, many people, companies, and government agencies skip this first step all the time.)
During this analysis step, the customer (the person or group that wants the program) defines exactly what they want the program to do, which are the requirements. The programmers read these requirements and ask the customer for clarification until they "think" they understand exactly what the customer wants. At this point, the analysis step is done, and the customer can no longer ask for any more changes.
Note
When a customer's requirements are written down, they're often called the program specifications or just the specifications.
One common problem is that the customer may ask for a new feature that he didn't ask for initially. By shutting down the analysis step, the waterfall method prevents the customer from coming back two days before the project is done and asking, "Can't you add just one more feature? It'll only set the project back another 3 months and require rewriting 90 percent of the program from scratch. And by the way, we aren't going to pay you for the delay either."
Warning
The most common problem during the analysis phase is that the programmers never clearly understand what the customer wants:
Sometimes the customer clearly defines what he wants, and the programmers misinterpret it.
Sometimes the customer doesn't really know what he wants, so he keeps changing his mind, making it impossible for the programmers to define everything the program's supposed to do.
After the programmers complete the analysis step, they should know exactly what the program is supposed to do. Then they can start the design phase and decide how they'll create this program.
Design
The design phase is when the programmers write a plan for creating a program that meets all the customer's requirements. This can involve deciding how to break the overall program into smaller parts and then assigning each part to a different programming team.
At the design phase, the programmers also need to decide on
The programming language to use
A specific compiler and other tools to use
Procedures for keeping all programmers in constant contact with each other so one programmer doesn't accidentally duplicate the efforts of another programmer
Warning
The biggest problem with the waterfall model is that if one step isn't completed correctly, the rest of the steps won't work correctly either. For example, if the analysis part of the waterfall model is done poorly, then it will be impossible for the design part to be accurate.
Many times the design doesn't accurately reflect what the customer wants. Because the programmers talk only to the customer during the analysis phase, they may cheerfully design a program that almost does what the customer wants, but not quite. That means the programmers often create a program that the customer can't even use without moderate-to-extensive modifications.
During the design phase, every programmer is supposed to know what the customer wants, but sometimes programmers leave and new programmers get added. These new programmers may not have participated in the analysis phase, so now they have to take time to understand what the customer wants, which delays the design phase for the entire project.
When everyone finishes deciding how to create the project within a specific deadline, the design phase is considered over. By forcing programmers to design a program on paper and create a plan for writing that program, the waterfall method makes sure that nobody starts writing the program until they first have a clear idea what they're doing.
Implementation
After the design phase is done, the implementation phase begins with programmers actually writing their portion of the program. If the programmers developed a clear plan during the design phase, they can monitor their progress during the implementation phase and see, at any given time, what needs to be done next.
Warning
If the programmers don't fully understand the design, the program that they write reflects this misinterpretation. Sometimes after writing part of the program, the programmers suddenly realize that their initial design was flawed, but now it's too late to change, so the project winds up being harder to create than expected.
After everyone finishes writing their part of the program, they put it all together, like a giant jigsaw puzzle. At this point, all further programming stops, and the testing phase begins.
Testing
The testing phase insures that the entire program works. If the programmers find any problems, they fix the bugs and then test to make sure their fixes don't accidentally wreck another part of the program.
When the program finally works, the testing phase is over and the program's ready for delivery.
In theory, the waterfall method makes sense because everyone has a clear idea what needs to be done at any given time. Unfortunately in the real world, life rarely works according to theory.
After the testing phase, the program should be complete, but if the customer's idea of what he wanted and the programmers' analysis, design, and implementation of those ideas aren't completely accurate, the customer may wind up with a program that almost does what it's supposed to do. To add new features or features that should've been in the program to begin with, the waterfall method starts with a new analysis phase all over again, with the possibility that the revised program could wind up just as flawed as before.
Warning
Although the waterfall method makes sense and can work, it assumes that
A project can progress through a series of distinct steps that are done once and are never needed again for the remainder of the project.
The time needed to complete each step can be predicted.
Each step can accurately translate the customer's needs from the preceding step.
Warning
Despite the rationale behind the waterfall method, it hasn't consistently produced large, reliable programs on schedule. The biggest flaw with the waterfall method is its rigidity. If one phase gets stalled for any reason, the entire project grinds to a halt. If programmers misinterpret the customer's requirements between two different steps (such as the analysis and design phase), the error can continue down through each succeeding phase, resulting in a final program that's flawed (at best) or completely unusable (at worst).
In response to the over-structured waterfall method, programmers have gone to the opposite side with extreme programming (often abbreviated as XP).
The whole idea behind extreme programming (or agile programming) is to recognize that all four phases (analysis, design, implementation, and testing) are not distinct, isolated steps but integrated steps that flow back and forth throughout the life of a project.
Phases
Instead of the analysis, design, implementation, and testing phases defined by the waterfall method, extreme programming defines four different, mutually overlapping phases:
Coding
Testing
Listening
Designing
Advocates of extreme programming argue that the only truly important product of the system development process is code. Without a working program, you have nothing.
Coding is more than just typing in program commands — it also includes figuring out the best way to write that program in the first place. For instance, in extreme programming, when faced with several alternatives for a programming problem, one should simply code all solutions and determine with automated tests (which I discuss in the following section) which solution is most suitable.
Coding also means communicating on a consistent basis with
The customer. Constant communication with the customer means that the customer can always see the program's progress and offer feedback on what the program's doing right and what it's doing wrong. By doing this, extreme programming lets the customer play an active role in shaping the final program.
Under the waterfall method, the customer can give feedback to the programming team only during the analysis phase. After the analysis phase is complete, the customer never sees the program until it's completed the testing phase. By that time, the program may not look or act anything like the customer wanted.
Other programmers. Constant communication with other programmers on the project means not only talking with them, but writing clear, concise program designs that can't be misinterpreted. By constantly communicating with other programmers, extreme programming encourages programmers to know exactly what other programmers are doing so putting together the various parts of the program can be much simpler.
Under the waterfall method, it's perfectly possible for teams of programmers to work independently only to find later that their different program parts don't work together like they're supposed to.
Sequence
Unlike the waterfall method, which is predictive and process-oriented, extreme programming is adaptive and people-oriented. Extreme programming adapts to the customer's needs (rather than shutting the customer out after the analysis phase is complete). By keeping the customer involved, the program's directed by the customer's needs rather than a fixed time schedule.
Extreme programming follows this general sequence:
The customer defines the program requirements.
The customer may need more features later, but initially, the customer knows the main features he needs in a program.
A small team of programmers designs a simple program (a prototype) that acts like a model for the customer to study and approve.
Because this prototype is a simple program, it's easy to make and even easier to modify based on the customer's feedback and desire.
The programmers implement their idea as an actual program, with the customer giving them feedback along the way.
By keeping the customer in close contact with the programmers, extreme programming makes sure the customer can make changes to the program while the changes are still easy to make, rather than when the program's finally done.
After the programmers create a simple program that the customer likes, the programmers can slowly start adding additional features that the customer may suddenly want.
In this way, a program evolves slowly with the customer guiding its development at all times.
Warning
Extreme programming isn't without its critics, who argue the following points:
The chaotic nature of extreme programming means that it's nearly impossible to determine when a project gets done.
The close relationship between the customer and the programmers can be crucial.
If a programmer suddenly leaves, the entire project can slow down while the customer gets used to working with a new programmer (and vice versa).
Close communication is essential between
The programmers and the customers
The programmers themselves
Without constant communication, an extreme programming project can fall apart.
Software engineering consists of generally accepted practices for writing reliable software on time. Avoiding spaghetti programming (see Book 1, Chapter 2) is one example of an accepted software engineering practice.
To make software engineering practices easier to follow, computer scientists have developed Computer-Aided Software Engineering(CASE) tools. CASE tools are meant to simplify the practices of software engineering. The easier it is for programmers to follow software engineering practices, the more likely they'll do it and create reliable software. Some common CASE tools include
Project modelers
Code generators
Source code formatters
Revision control
Project management
Before programmers rush off to write a program, they need to design it first. The problem with designing a program is making sure everyone understands the design, and part of understanding a program design is using a consistent method.
For example, one programmer might design a program by scribbling a few notes on a napkin, whereas a second programmer might design a program by typing a description of that program in a word processor document.
Neither approach is good or bad, but the problem lies in understanding everyone's design. The programmer who scribbles notes on a napkin may understand the design of his program perfectly, but nobody else may understand the design of that program.
To make sure everyone can understand the design of a program, everyone needs to use the same design method.
Flowcharts
One of the earliest modeling methods are flowcharts, as shown in Figure 5-2. Flowcharts must specify every action and decision that the program goes through.
Warning
Flowcharts work for designing small programs, but when designing a large program, flowcharts have a couple of problems:
The flowchart can soon become so large and confusing that creating it can take way too much time to complete.
A large flowchart can be so complicated that nobody bothers using it, which makes creating the flowchart a waste of time in the first place.
Unified Modeling Language (UML)
Rather than describe every possible action that the program might do, other modeling methods just describe the parts of the program. One of the more popular methods for describing a program is the Unified Modeling Language (UML).
Like flowcharts, UML provides a way to visually design a program with universally understood symbols and diagrams. Unlike flowcharts, which model only a program's actions, UML offers different types of models for defining a program:
Functional models
Object models
Dynamic models
You don't have to use every type of diagram to define your program; use as many as you need.
Of course, UML and other modeling methods are never perfect:
Just because you model a program perfectly doesn't always mean that design is actually feasible. Many designs may look perfect on paper but fail dramatically when put into actual use. (Think of the watertight doors on the Titanic.)
Modeling methods, such as UML, offer so many different diagrams that it's possible that two diagrams of the same program can describe mutually exclusive features. Because each type of a diagram focuses on a different view of the program, you could create conflicting designs.
Despite these problems, design methods like UML are still an improvement over no design plans whatsoever. Just remember that UML diagrams are meant to offer a standard way to model programs so other people can see and understand your design before you start writing any actual code in a specific programming language.
Functional model
A UML functional model (or a Use Case diagram) describes how a user interacts with the program, as shown in Figure 5-3. By displaying users as stick figures and program functions as an oval, Use Case diagrams make it easy to see what the program is supposed to do without confusing you with how the program will do it.
Object model
A UML object model (or a Class diagram) defines how to divide a program into parts or objects. Each Class diagram defines an object's name, data (properties), and operations (methods), as shown in Figure 5-4.
Figure I.5-3. UML Use Case diagrams define how users need to interact with a program, such as this diagram describing an online shopping program.
Figure I.5-4. UML Class diagrams divide a large program into multiple objects and define the properties and methods of each object.
An object's properties define the type of data it can hold whereas an object's methods define what type of actions the object can perform. In Figure 5-4, the three classes are Students, Teachers, and Courses. Students have properties, such as their names and student ID numbers; Teachers have properties, such as their names and salaries; and Courses have properties, such as its name and days offered.
The Student class also has an Enroll method to model a student's actions in choosing a course to take. The Course class has three methods — Add Student, Drop Student, and Assign Teacher. By using UML Class diagrams, programmers can define objects and how they interact with each other without worrying about the specifics of any object-oriented programming language.
Dynamic model
A UML dynamic model (or a Sequence diagram) describes the sequences a program follows, as shown in Figure 5-5. Sequence diagrams are similar to flowcharts because they describe what a program does at any given time, but not necessarily how a program works, which is unlike a flowchart.
Sequence diagrams show you the order that events occur, the messages passed from one object to another, and the actual names of all interacting objects.
The boxes at the top of the Sequence diagram in Figure 5-5 shows object names and the classes they're based on (such as a sophomore object based on the Student class). The vertical dotted lines (lifelines) show how long an object exists. The arrows show the messages objects send to each other. A Sequence diagram helps explain the interaction between different objects.
Designing a program by using UML or some other modeling method can be great, but there's a big gap between designing a program and actually writing code that follows that design. To eliminate this problem of transferring a program design into working code, computer scientists have created special programs dubbed code generators. The basic steps for using a code generator are as follows:
Design a program in a specific format, such as a UML diagram.
Feed that model (UML diagram) in to the code generator.
The code generator cranks out working code in a specific programming language.
Modify or compile this source code and have a working program that accurately reflects your original design.
If you change your design, the code generator can crank out new code based on your revised design. Code generators keep your design and code synchronized.
Note
In the early days of computers when people had to write programs in assembly language, compilers were considered code generators because you could give them source code written in a language, like FORTRAN or COBOL, and the compilers would generate the proper machine language commands from that source code.
If you write a program by yourself, the source code for your entire program likely looks consistent. However, if you're working on a large program with multiple programmers, every programmer has his or her own unique style. Ultimately, the program may work, but it resembles a patchwork collection of code where each chunk of code can look completely different even if the entire program is written in the same programming language, as shown in Figure 5-6.
To solve this problem, computer scientists have created source code formatters. These programs rearrange the appearance of source code so that it appears uniform. Now instead of looking at a patchwork collection of code written by different programmers, an entire large program can look like it was written in a single style.
By looking at source code that's formatted the same, programmers can easily read and understand the entire program.
Modifying a program by yourself is easy, but what happens if you're working with a team of multiple programmers? It's possible that one programmer modifies the program while a second programmer modifies a copy of the program. Now you wind up with two slightly different versions of the same program. Which one should you use?
The answer is to avoid this problem in the first place by using a revision control program.
A revision control program acts like a librarian and stores a program's source code in a single location — a repository. If a programmer needs to modify the source code, he or she needs to check out that source code, much like checking out a book from a library.
If another programmer wants to modify that same source code, the version control program may offer four choices:
It may simply lock anyone else from modifying any source code that's been checked out. This prevents multiple programmers from working on different copies of the same source code.
It may let another programmer check out a second copy of the source code and make any changes to it. At this point, you wind up with two modified copies of the same source code.
When both programmers return their modified versions of the same source code back to the repository, the version control program merges all changes back into a single copy. This has the advantage of letting multiple programmers work on the same source code but also has the disadvantage that merging changes might wind up scrambling the original source code beyond recognition.
For additional protection, version control programs also archive old versions of source code so if someone totally messes up the program, he or she can always retrieve an earlier version of the program and start all over again.
Version control programs are essential to coordinate the activities of multiple programmers, although they can also be handy for individual programmers. The main points are to
Always keep a copy of your old source code (so you can retrieve it again if necessary).
Always keep your changes to the source code organized.
Programming is more than just writing code and seeing if it works. The larger the program, the more complicated it gets, and the more tools you likely need to keep everything organized. Can software engineering guarantee that you'll write better and more reliable programs? Nope.
All that software engineering techniques and practices can do is encourage you to write programs by using proven techniques. However, it's still possible to follow every software engineering principle and wind up writing a confusing program that doesn't work.
Software engineering has improved from the early days of letting programmers loose and hoping they wrote something useful, to today's programming projects with software engineering tools available to guide a project at every step of the way.
Think of software engineering as a way to funnel your thinking and actions toward writing well-designed and easy-to-understand programs. Programming geniuses will likely find software engineering practices too restrictive and confining, but they exist for your own protection, like seat belts in a car.
Without software engineering, only a handful of programming geniuses could ever write a large program successfully. With software engineering, teams of average programmers can write that same large program. The program may lack the elegance of one written by a programming genius, but it will likely work well enough to be useful.
Warning
In the computer industry, being good enough is all that most people want and need.