Ed Yourdon - Just Enough Structured Analysis

“It is impossible to dissociate language from science or science from language, because every natural science always involves three things: the sequence of phenomena on which the science is based; the abstract concepts which call these phenomena to mind; and the words in which the concepts are expressed. To call forth a concept a word is needed; to portray a phenomenon, a concept is needed. All three mirror one and the same reality.”

— Antoine Laurent Lavoisier
Traite Elementair de Chimie, 1789

“What we have to do is to be forever curiously testing new opinions and courting new impressions.”

— Walter Pater
The Renaissance, 1873

IN THIS CHAPTER, YOU WILL LEARN:

The role of the systems analyst in programming and testing;
Why prototyping and iterative development is advantageous during programming and testing;
What the systems analyst should look for when examining a program; and
The major forms of testing that should be carried out.

Programming and testing normally commence, as you might expect, when the design activity is finished. The programming, or implementation phase of a typical project involves the writing of statements in COBOL, Java, Visual Basic, or in some other programming language to implement what the systems analyst has specified and what the designer has organized into modules. And testing, as the name implies, involves exercising the system to ensure that it produces the proper outputs and exhibits the proper behavior for a wide range of inputs.

Why should this be of any interest to you as a systems analyst? Isn’t it true that you will be uninvolved in the project at this point? No, not necessarily. For various reasons, the work that the programmers and testers do may influence your work, and the way you organize your work may influence the way they do theirs. This interrelationship between systems analysis and programming/testing is the subject of this chapter.

23.1 THE ROLE OF THE ANALYST IN PROGRAMMING AND TESTING

In the extreme case, the systems analyst will finish his or her job of specifying the system, and then spend some time with the design team as the user implementation model is developed and as the early stages of design take place. But by the time programming begins, the systems analyst may have moved on to another project. But here are some reasons why you, as a systems analyst, may need to remain involved in the project as the programming activity commences:

A simple reason. You’re the project leader, and you’re in charge of the programmers. Obviously, you can’t abandon them. You’ll be involved with the project until the final testing, acceptance, and delivery to the end user. Thus, it will be important for you to know if the programmers have written high-quality code, and it will be equally important to know whether they have tested their programs adequately.
Another simple reason. You are a junior systems analyst, and your title is programmer/analyst or analyst/programmer. So, in addition to developing the specifications for the system, you are also involved in writing the computer programs.
A more interesting reason. You are part of the group writing test cases that will be used to exercise the programs being written by the programmers. In many projects, you may be joined by one or more users in this activity, on the theory that the users are the people best able to think of unusual and exceptional test cases. Development of test data can begin as soon as the specification is finished (indeed, even before it is completely finished); as Tom DeMarco says, “The specification is the test of the system.” Since you only know the logical content of the inputs and outputs at this point, you will probably have to wait until the user implementation model is finished before you can determine the physical format of the inputs and outputs. You will also need the user implementation model to know what operational constraints (response time, volumes, etc.) need to be tested. But this activity can easily keep you occupied right through the end of the project; after all, if the programs fail your test cases, you will need to work with the programmers to see whether your test case is wrong or whether their code is wrong
A less obvious reason. You may be involved in developing user manuals, training the users, planning for installation of the new system and conversion of data from the old system. In most cases, this can take place in parallel with the programming and testing of the new system. As the systems analyst involved in the new project from the beginning, you will often be seen as the ideal candidate for doing this work.
A discouraging reason. The programmers may not understand your specification. Or your specification may be incomplete, inconsistent, or contradictory. Tsk! Tsk! But these things do happen, and you may find that the programmers need to come back to you periodically to review and clarify the specification as they translate it into a programming language. Another variation on this theme: the programmers may ask you to change the specification because it is difficult to implement. In this case, of course, you will have to be the mediator (as well as interpreter) between the programmers and the other systems analysts.
Another discouraging reason. The users may have begun to change their mind about their requirements, even as the programmers are implementing the requirements they said they wanted. Aside from the fact that some users are ornery and do this just for the fun of it, there are very good reasons for this phenomenon; users live in a dynamic world and they must often react to changing policies imposed upon them by government legislation, customer requirements, or general marketplace conditions. So you may find that you are changing the specification even as the programmers are implementing the specification, something that won’t make anyone happy, but may have to be done anyway. This is discussed further in Section 23.4.

23.2 THE IMPACT OF ANALYSIS, PROGRAMMING, AND TESTING ON ORGANIZATIONAL STRUCTURE

Throughout this book, it has been evident that structured analysis involves a steady progression from high-level modeling aspects (e.g., top-level dataflow diagrams) to low-level modeling issues, such as the development of process specifications and a complete, detailed data dictionary. Similarly, the process of design involves the development of design models ranging from high-level structure charts to such low-level forms as pseudocode or flowcharts. Programming must follow this same pattern: programs have to be written for high-level executive modules, and will eventually be developed for the bottom-level modules that carry out detailed calculations, validate input data elements, and so on.

What we have not talked about is the relationship between levels of the system and the levels of the organization that builds the system. But you probably got the impression after reading through most of this book that people having the title of systems analyst would be responsible for doing all the work of systems analysis, people with the title of systems designer would be responsible for the job of design, and people with the title of programmer would be responsible for the job of writing computer programs.

But there is a problem with this approach — actually, two related problems. First, people with the title of systems analyst are usually relatively senior people with several years of experience. While such senior people generally enjoy the work of drawing dataflow diagrams and entity-relationship diagrams, it is hard for them to get excited at the prospect of writing hundreds of process specifications and defining thousands of data elements.

And then there is the other side of this problem: if the senior people actually do carry out all this detailed work, what is there for the programmers to do? Their work becomes almost mechanical in nature, translating carefully thought out process specifications into COBOL or Visual Basic. Whatever creativity they thought they had in their job seems to have disappeared. [1]

One solution to this apparent dilemma is to let senior people do all the high-level activities in a project and let the junior people do all the low-level detailed activities. This would mean, for instance, that the senior-level people (the ones with the title of senior systems analyst or something else equally impressive) would not only carry out the high-level activities of systems analysis (drawing dataflow diagrams and the like) but would also do the high-level activities of systems design, and would even (gasp! shudder!) write high-level code. The junior people, meanwhile, would be involved in the project from the beginning (or as soon as the senior people had completed the high-level aspects of analysis) and would be involved in the job of writing process and module specifications, developing data dictionary entries, and writing the code for low-level modules.

The advantage of this scheme for the programmers is that they get to do the creative work of writing process specifications and then have the pleasure of turning their own process specifications into code. It gets them involved in the process of systems analysis at an earlier stage in their career than might otherwise be possible. And it also has the advantage of keeping the senior people somewhat in touch with technology by forcing them to continue to do some design and programming work.

Not all senior systems analyst think this is a good idea, though they will admit that they don’t enjoy the drudgery of writing all the detailed process specifications as part of their job. In any case, there is growing agreement that the job of programming, if preceded by careful, detailed systems analysis of the sort described in this book, is becoming a menial, clerical job and may disappear entirely as we develop intelligent code generators that can compile process specifications directly. Thus, we can expect that organizations will gradually change their work assignments over the next 5 to 10 years to conform with the ideas discussed above.

23.3 PROTOTYPING AND ITERATIVE DEVELOPMENT

There is another impression you may have gotten from reading the material in this book: that the activities of systems analysis must be carried out and completed before the activities of design and programming can begin. While many projects do indeed work this way, it is not strictly necessary. Systems analysis, design, and programming can proceed in parallel with one another.

The concept of parallel development of the specification, design, and code for a system was historically referred to as “fast-tracking” or “top-down implementation” (see, for example, [Yourdon, 1988]); in more recent times, it has typically been described as prototyping, iterative development, or “adaptive” development (see, for example, [Highsmith, 2000] and [Beck, 2000]). We discussed the idea briefly in Chapter 5; you may wish to review the concept of top-down implementation as part of the overall systems development life cycle that was discussed in that chapter.

The construction industry and many engineering disciplines follow this approach on many projects. As many construction project managers have been heard to exclaim, “You don’t need to know the number of doorknobs in a building before you lay the foundation.” In the case of developing an information system, this means that the high-level products of systems analysis, the framework documents of dataflow diagrams, entity-relationship diagrams, and state-transition diagrams, can be used as the basis for high-level design. And the high-level design can be used as the foundation for writing high-level code even before the details of systems analysis have been finished.

There is a great deal of flexibility in this approach; one can finish 80% of the systems analysis work before beginning the design and programming, or one can finish only 10%. A project plan that calls for nearly complete systems analysis before the beginning of systems design is usually referred to as a conservative approach; a project plan that calls for the almost immediate overlap of systems analysis, design, and programming is known as a radical approach, or “extreme prototyping.” Every project manager can decide just how radical or conservative he wants his project to be, and he can change his mind dynamically during the project.

Why would the project manager consider a radical approach? Why would anyone want to begin the work of systems design and programming before the systems analysis is complete? There are many reasons, of which the following are the most important:

Because the work of systems analysis, design, and programming is being done concurrently, there is usually an opportunity to dramatically shorten the elapsed time of a project. In many of today’s competitive business environments, this is crucially important, for example, if a system absolutely, positively must be finished by a certain date.
The concurrent development work can be used as a form of prototyping: it allows the project team to show a skeleton version of the system to the user before all the detailed work of systems analysis has been finished. This can help avoid major misunderstandings between user and systems analyst that might otherwise take place even with the most carefully developed structured specification.
Beginning the programming work at an earlier point in the project often smooths out various resource demands, such as computer time, that would otherwise become a bottleneck. For example, the conservative approach often required enormous amounts of computer test time during the final stages of testing, and this can be a major problem.

Whether or not the project decides to follow a conservative or radical approach is beyond the scope of this book; some project environments may favor a conservative approach, and others will call for a highly radical approach. The main thing to be aware of is that the structured analysis approach described in this book does not preclude either approach, nor does it insist on either approach.

23.4 PROGRAMMING AND PROGRAMMING LANGUAGES

If you are still involved in the project during the implementation stage, you should have at least a general understanding of the issues and techniques of programming. In this section, we will discuss:

Five generations of programming languages
Important issues in programming
Things you should look for if you examine the programmers’ coding

23.4.1 The Five Generations of Programming Languages

People have been writing computer programs since general-purpose computers were first developed in the late 1940s. Programs are written in programming languages, of which Visual Basic, COBOL, and Java are common examples. It is convenient to group all the different programming languages (there are several hundred different languages in use around the world) into five distinct generations:

First generation languages: First generation programming languages were the machine languages used in the 1950s; programmers wishing to make a computer do something useful would code their instructions in binary ones and zeroes. There are still times today when a faulty computer system disgorges pages of mind-numbing digits; and there are still a few misguided youths who think that machine language is the best way of playing with personal computers. But the rest of the world stopped thinking about machine language long, long ago.
Second generation languages: Second-generation languages are the successor to machine language; they are generally known as assembly languages or assembler. Second generation languages are low-level languages in the sense that the programmer has to write one statement for each machine instruction. Thus, while she might conceptually think in terms of the statement X = Y + Z, she would have to translate the following statements into assembly language:
CLEAR ACCUMULATOR
LOAD Y INTO ACCUMULATOR
ADD Z TO CONTENTS OF ACCUMULATOR
STORE ACCUMULATOR IN X
Even this tiny example shows the major disadvantage of assembly language. Rather than being able to think in terms of the problem she wants to solve, the programmer must think in terms of the machine. Starting around 1960, more powerful languages began to be introduced; most sane programmers have long since abandoned assembly language. However, there are still a few situations where such languages are needed. Most of them involve very small, low-powered microprocessors (which can be manufactured very cheaply and which are small enough to fit, say, into a digital watch), for which the memory and CPU constraints are so severe that the programmer cannot afford the overhead usually associated with third generation languages.
Third generation languages: Third generation languages include COBOL, C, FORTRAN, Ada, and many others; we may also include some of the familiar object-oriented programming languages (OOPLs) such as C++ and Smalltalk in this category. They are high-level in the sense that a single statement (such as “MOVE A TO B” in COBOL) usually represents five or ten assembly language statements (and sometimes as many as a hundred statements); they are high-level in the more important sense that they allow the programmer to express thoughts in a form that is somewhat more compatible with the problem area in which she is working. However, they are low-level in some important respects. They require the programmer to become intimately involved in the tedious business of formatting the layout of computer reports, as well as editing and validating the input to the program. Often the programmer will think to herself, “This report should have the standard heading at the top of each page, with a page number on the right and the date on the left, just like all the other reports around here,” but she may have to write 20 or 30 COBOL statements to accomplish it.
Many of the third generation languages are also characterized as procedural languages. They require the programmer to think carefully about the sequence of computations, or the procedure, necessary to accomplish some action. In a scientific application, for example, the programmer may know that she wants to add array A to array B; however, she may be forced to write the detailed procedural steps to add each of the elements of the two arrays, rather than simply saying, “Add these two arrays” without having to worry about the procedural steps.
The procedural aspect of the programming language is typically much less significant with object-oriented programming languages, because each “object” is typically programmed to respond to certain “events,” without knowing the order or sequence in which those events occur. Indeed, many purists would object strongly to the suggestion that OOPLs are third-generation programming languages; we have included them here primarily because, during the early and mid-1980s, they still tended to be used in a fashion that required a great deal of attention to low-level, tedious issues of formatting, editing, and “house-keeping” details.
Fourth generation languages: Fourth generation languages, or 4GLs, became extremely popular in the late 1980s and early 1990s. Examples of 4GLs are FOCUS, IDEAL, MARK IV, RAMIS, MANTIS, MAPPER, dBASE-III Plus, and Rbase-5000. Most of these languages introduced structured programming features that the older third generation languages lacked, but they have other features, too. In particular, most of the tedious programming details associated with getting data into the computer (via a terminal) are hidden from the programmer; with one simple command, the programmer can specify that the computer should accept a specified kind of data from the keyboard, validate it, and store it in a designated data element. The same job might require 10 or 20 statements in a third generation programming language or 100 to 200 statements in a second generation programming language.
Similarly, many of the tedious programming details associated with producing an output report (e.g., an inventory listing, paychecks, invoices, or a summary of the day’s orders) are handled automatically by the fourth generation languages. If the precise placement of information on the computer printout is relatively unimportant (as is often the case), the programmer does not even have to specify it; otherwise (as in the case of a computer-produced paycheck, where dollar amounts must be printed in a precise location), the details are easily specified in a few 4GL instructions.
In this same context, many of the popular OOPLs have evolved into legitimate 4GLs, because they have developed class libraries that take care of these same tedious details of input, editing, formatting, and output. What makes a programming language like Smalltalk so powerful in the hands of an experienced programmer is not so much the object-oriented syntax of the language, but rather the existence of nearly a thousand well-designed components within the standard class library, which can be invoked to take care of all the housekeeping details.
Fifth generation languages — the visual development environments: Fifth generation languages began appearing in the early 1990s, when powerful PCs and user-friendly graphical user interfaces (GUIs) started becoming widely available. The first such language was Visual Basic, introduced by Microsoft; but other proprietary languages, such as PowerBuilder and Delphi, quickly became popular. By the mid-1990s, with GUIs and powerful workstations, almost universally available, the “visual development environment” paradigm was extended to almost all of the traditional programming languages as well. Thus, there are versions of Visual COBOL, and Visual C++; and most of the vendor-supplied versions of Java, Smalltalk, and other programming languages are also visual in nature. In addition to providing easy access to the high-level components that made 4GLs so popular, the visual development environments provide the programmer with a “window” broken into several “panes” that can be used to create different parts of the program at the same time. A point-and-click approach eliminates the need for most of the tedious typing that was required with earlier generations of languages; instant syntax-checking and incremental compilation gives the programmer quick feedback for eliminating simple errors, and also being able to run quick tests.

23.4.2 Important Issues in Programming

Regardless of the programming language used, there are common issues that face all programmers. As a systems analyst, you should be familiar with these issues; the most common ones are itemized next:

Productivity: Probably the most important issue in programming today is that of productivity: getting more software written more quickly than ever before. The major reason for this is the enormous backlog of unwritten systems and applications in large organizations: the typical large organization has a backlog of between 2-4 years of new work to be done. [2] Thus, programming languages and programming techniques that promote productivity are to be highly encouraged; except in rare cases, productivity is considered more important today than efficiency.
Efficiency: In some applications, efficiency is still important. This is still true for mainframe computer systems that process large volumes of data (e.g., most of the systems running at the Social Security Agency, the IRS, and equally enormous systems running in banks, airline reservation operations, stock brokerage companies, and insurance companies). More recently, we have begun to see concern about efficiency in high-volume Web/Internet systems that may have to handle millions of “hits” per hour. For these applications, it is usually important to minimize the amount the amount of CPU time required by the program; it may also be important to minimize memory utilization as well as the utilization of other resources such as network bandwidth and disk accesses. Note that the goal of efficiency is usually in conflict with the other goals discussed in this section: if one spends more time developing an efficient program, it is likely to be less maintainable and less portable, it is more likely to have subtle residual errors, and it is likely to decrease the productivity of the person who wrote the program.
Correctness: Ultimately, one could argue that this is the most important issue. After all, if a program does not work correctly, it does not matter how efficient it is. Programming languages like Ada, Pascal, and Java are considered preferable if correctness is a critical concern (as it would be, for example, if one was building an avionics system for a new generation of commercial aircraft, or a control system for a nuclear reactor) because they are “strongly typed”: the programmer is required to declare the nature of his or her variables (i.e., whether they are integer, character, floating point, etc.) and the language checks carefully to prevent illegal data references, and the like.
Portability: In some environments, portability is important; the user may want to run the same system on several different types of computers; indeed, someone developing a Web-based application has little or no control over the hardware/software environment that will be used for the “client” portion of the application. Some programming languages are more portable than others; ironically, this is more true of third generation programming languages (C, Pascal, FORTRAN, COBOL, etc.) than some of the higher-level languages (e.g., Microsoft Visual Basic does not operate on a Macintosh). While there have been extensive efforts to ensure portability with languages like Java, there is no such thing as a universally portable language; there are always ways for the programmer to take advantage of the special features of a specific computer or operating system. Thus, in addition to the programming language, we must also be concerned with the style of programming if portability is an important factor.
Maintainability: Finally, we must remember that systems live for a long time, and the computer software must be maintained. Maintenance is discussed in more detail in Chapter 24.

23.4.3 Things to Watch For

As a systems analyst, you may have occasion to watch the work done by the programmers on the project; indeed, you may be their supervisor. As indicated above, you should be aware that productivity, efficiency, correctness, portability, and maintainability are likely to be important issues. But how are these goals to be achieved? The following are key programming issues:

Structured programming, or object-oriented programming: Assuming that the programs are being written in a third or fourth generation language, a disciplined style of structured programming or object-oriented programming should be followed. Almost all modern programming textbooks strongly recommend either a formal structured- or object-oriented approach.
Small modules or objects: It is essential that programs be organized into small modules or objects so that the programming logic will fit onto one page of a program listing. It is important to remember that the complexity of a program does not increase linearly with the size of the program: a 100-statement program will almost always be more than twice as complex as a 50-statement program. As we saw in Chapter 22, this is primarily under the control of the designer; but the designer might not be able to tell how large a module/object will be, especially if he is not familiar with the programming language that will be used on the project. Thus, the programmer may have to continue the design activity, breaking a module into lower-level sub-modules so that each one represents no more than about 50 programming statements.
Simplicity of style: For many years, programming textbooks have provided detailed guidelines for writing simple programs — i.e., programs that the average programmer can understand and that can be passed on to the maintenance programmer; among the more current programming “style” textbooks are [Kernighan and Pike, 1999], [Maguire, 1993], [McConnell, 1993), and [Heller, 1996]. Style guidelines include such ideas as avoiding programming statements with compound Boolean expressions, for example,
IF A AND B OR NOT C AND D THEN ADD 3 TO X.

It is interesting to note that several mathematical models of program complexity have been developed during the past ten years; one of the more popular is McCabe’s cyclomatic complexity model (see [McCabe, 1976]), which provides a quantitative measure of the intrinsic complexity of a program. [3] Some organizations are now insisting that all new programs be run through an automated complexity checker to ensure that they are not too complex.

23.5 TESTING

The process of testing is likely to take as much as half of the development schedule for your system, depending on how carefully the initial activities of analysis, design, and programming has been done. Even if a perfect job of systems analysis, design, and programming have been done, some effort must be done to verify that there are no errors. If, on the other hand, an imperfect job was done (as is almost always the case!), then the testing becomes iterative: the first round of testing exposes the presence of errors, and subsequent rounds of testing check to see if the corrected programs are now operating correctly.

What do you need to know about testing as a systems analyst? This will depend, of course, on how deeply involved you are in the process. In many cases, the systems analyst works closely with the user to develop a thorough, comprehensive set of test cases based on the essential model and user implementation model of the system. This process of developing acceptance test cases can be carried out in parallel with the implementation activities of designing and programming, so that, when the programmers have finished writing their programs and carrying out their own local testing, the user/analyst team will be ready with their own test cases.

In addition to this basic concept (that the description of user requirements forms the basis of the final test cases), you should be familiar with different types of testing, as well as some concepts closely related to testing. These are discussed next.

23.5.1 Types of Testing

The first thing to realize is that there are different strategies of testing; the two most common strategies are known as bottom-up testing and top-down testing. The bottom-up approach begins by testing small, individual modules in a stand-alone fashion; this is often called unit testing, module testing, or program testing. Then individual modules are combined together into larger and larger units to be tested en masse; this is often referred to as subsystem testing. Finally, all the components of the system are combined together for testing; this is known as system testing, and it is often followed by acceptance testing, where the user is allowed to submit his own test cases to verify that the system is working properly.

The top-down testing approach begins with a skeleton of the system; that is, the testing strategy assumes that the top-level executive modules of the system have been developed, but that the lower-level modules exist only as dummy modules or stubs. [4] Because most of the detailed system functions have not been implemented, the initial tests are very limited; the purpose is simply to begin exercising the interfaces between major subsystems. Subsequent tests then become more and more comprehensive, exercising more and more detailed aspects of the system. The top-down approach to testing is generally considered a preferable approach for most systems today; for more details, see [Yourdon, 1986].

In addition to these basic concepts, you should be familiar with the following types of testing:

Functional testing: This is the most common form of testing; its purpose is to ensure that the system performs its normal functions properly. Thus, test cases will be developed and entered into the system; the outputs (and the results of updated files) will be examined for correctness.
Recovery testing: The purpose of this kind of testing is to ensure that the system can recover properly from various types of failures. This is particularly important on large on-line systems, as well as various types of real-time systems that may be controlling physical devices and/or manufacturing processes. Recovery testing may require the project team to simulate (or bring about) hardware failures, power failures, failures in the vendor operating system, and so on.
Performance testing: The purpose of this kind of testing is to ensure that the system can handle the volume of data and incoming transactions specified in the user implementation model, as well as ensuring that it provides the response time required. This may require the project team to simulate a large network of on-line terminals, so that the system will be fooled into thinking that it is operating under a heavy load.

There is one last concept that you should be aware of: the notion of exhaustive testing. In the ideal project, we would generate test cases to cover every possible input and every possible combination of situations the system could ever face; we would then exhaustively test the system to ensure that its behavior would be perfect. There is only one problem with this: it doesn’t work! The number of test cases for a typical large, complex system is so staggeringly large, often on the order of 10¹⁰⁰ distinct test cases or more, that even if we could conduct one test every millisecond, it would take longer than the age of the universe to finish all the tests! Consequently, nobody carries out truly exhaustive tests on anything other than a trivial system; at best, the system developers can hope to create test cases that will exercise (or cover) a large percentage of the different decision paths that the system can take. [5] This makes it all the more important to ensure that the model of user requirements and the various implementation models are as correct as possible.

Suppose, for example, that one wanted to develop test cases for a portion of a system that calculates an employee’s net pay, as shown in Figure 23.1. Assume that gross pay is defined in the data dictionary as an integer (i.e., a salary expressed in whole dollars) ranging from 0 to 10,000. Then it would appear that a truly exhaustive test would consist of specifying what the proper net pay should be for each of the 10,000 possible gross pay amounts. Presumably, if our implementation team then carried out those 10,000 test cases and verified that the proper net pay was, in fact produced, we would be confident that the process was operating correctly.

Figure 23.1: A small portion of a system

But wait! What about potentially incorrect values of gross pay? What if the user provides a negative gross pay? What if he provides a gross pay of 100,000? Since there are potentially an infinite number of potential values of gross pay [6], and since we have no knowledge of the internal behavior of the program that implements COMPUTE NET PAY, we are faced with an apparently infinite number of test cases. If the test cases are developed at the end of the analysis phase of the project, using the data dictionary and the process specification, then there is no way that we can know how the program will eventually work when the programmer writes the code; thus, we are forced to carry out a black box test.

If we do know the internal logic and structure of the program (i.e., after the programmer has written the code), then we can base the test cases on the existing program logic and carry out what is often called “glass box” or “white box” testing. We can generally demonstrate, for example, that if the program correctly identifies one value of gross pay less than zero, it will correctly identify all such negative gross pays. In general, we should be able to demonstrate that the program will exhibit consistent behavior for various ranges of gross pay, and thus reduce the number of required test cases to a manageable number. While this does not constitute exhaustive testing, we could presumably achieve a fairly high level of confidence that we had developed test cases for all of the significant paths that the program could take.

But wait! COMPUTE NET PAY is only one process, one bubble out of hundreds, or even thousands, in a large system. If it requires, say, 1000 test cases to verify that COMPUTE NET PAY is operating correctly (in terms of functional correctness), then it might well require 1000 tests for each of the other, say, 1000 processes in the system. The total number of distinct test cases, then, could be 1,000 * 1,000 = 1,000,000. And even this is conservative! (And it doesn’t take into account the added dimension of complexity caused by throughput testing, recovery testing, etc.).

Thus, we must admit from the outset that exhaustive testing is impossible. But it is possible, as noted above, to choose test cases judiciously, in order to exercise as many logic paths as possible in the system. Even so, we must be prepared for a large, if not enormous, volume of testing. To carry out such testing effectively, the systems development team needs three things: test plans, test descriptions, and test procedures. A test plan is exactly what it sounds like, an organized document describing some testing activity. A typical test plan document will contain the following information:

Purpose of the test: what the objective of the test is, and what part of the system is being tested.
Location and schedule of the test: where and when it will be done.
Test descriptions: a description of the inputs that will be provided to the system, and the anticipated outputs and results. Descriptions of test inputs will usually be given in the data dictionary format discussed in Chapter 10.
Test procedures: a description of how the test data are to be prepared and submitted to the system, how the output results are to be captured, how the test results will be analyzed, and any other operational procedures that must be observed.

23.5.2 Related Concepts

Though most organizations carry out testing in the manner discussed above, there are some related concepts that can be used to augment the standard process of testing. These include the following:

Walkthroughs
Inspections
Proofs of correctness

Walkthroughs, which are discussed in Appendix D, are a form of peer-group review of technical products; they are widely used in the data processing industry to review dataflow diagrams (and other products of systems analysis), structure charts (and other products of design), as well as computer programs. While different from testing, their objective is the same: to discover possible errors in the system.

Inspections are similar to walkthroughs, but with a more formalized agenda of items that must be examined in the program (or in the specification or design, depending on the type of inspection) before it can be approved. To draw an analogy, consider what happens each year when your automobile is inspected: the auto mechanic has a specific checklist of features — brakes, headlights, exhaust emissions, and so on — that must be examined before he can put the appropriate sticker on your car. See [Gilb, 1993] for more details on the practice and procedures associated with inspections.

Finally, there are a limited number of cases where formal proofs of correctness will be developed for a computer program; the process here is somewhat analogous to the process of developing geometry proofs that you once studied in high school. Unfortunately, it is extremely difficult and time consuming to develop a rigorous proof of correctness for computer programs, and it has rarely been done for anything larger than a few hundred program statements. However, at least one U.S. government project has developed a computer-assisted proof of correctness for a system involving approximately 10,000 program statements; while it cost nearly $500,000 and took 6 months of work, it could be justified for certain high-risk or high-security systems. For a discussion of proofs of correctness, see Chapter 6 of [Dunn, 1984] or the surveys in [Elspas et al, 1972], and [Dunlop and Basili, 1982].

23.6 MAINTAINING THE SPECIFICATION WHILE PROGRAMMING: A PRELUDE TO CHAPTER 24

As mentioned above, it is possible for the structured specification to change during the programming process. This may happen as a result of the fast-tracking strategy described above, or because the original specifications were wrong, or simply because the users change their mind about their requirement. In any case, it is a reality, and it highlights one important point: the system specification cannot be considered frozen after the systems analysis phase has been declared over. It must be considered a living document that will require ongoing maintenance even before the system itself has entered the maintenance phase. Chapter 24 discusses this issue in more detail.

23.7 WHAT HAPPENS AFTER TESTING?

You might think that your work is completely done when you have finished testing the system. Unfortunately, there is more to do, though you may not be involved in your role as systems analyst. However, someone (and often a large group of “someones”) must carry out the final activities in a systems development project:

Conversion
Installation
Training

Conversion is the task of translating the user’s current files, forms, and databases to the format required by the new system. In some rare cases, this may not be a relevant activity, for there may not be any existing data. However, if the user is replacing a current system with a new system, this is likely to be a delicate and difficult task. A conversion plan needs to be developed, preferably as soon as the user implementation model is complete, to cover the following issues:

If the user already has existing data associated with an existing system, he will probably want to use it until the last possible moment before “cutting over” to the new system. Thus, it is difficult to consider the existing data as static.
There may be such a large volume of existing data that it will be impractical to considering converting it all at once. Files and records may have to be converted on an incremental, as needed basis. This will obviously require careful coordination and planning.
The conversion should be carried out in an automated fashion; this can only be done if the current files and data exist in some automated form. If so, it should be relatively straightforward to write a computer program (or to use an existing vendor-supplied package) to translate the current files into the format required for the new system. However, it sometimes turns out to be rather difficult to convert the data in an automated form, especially if the existing files are located on several different computers, in different formats, and so on. Indeed, developing the conversion software can turn out to be a major systems development project of its own!
The existing data may contain errors; indeed, if the existing data were created manually and have been maintained manually, you can be virtually certain that there will be errors. Thus, part of the process of conversion is that of error detection and error correction, which can make the process even more difficult and time consuming. Some existing files and records may turn out to be illegible or incomprehensible; in other cases, it may be obvious that the existing data are wrong, but it may not be clear what the correct values are.
In addition to converting existing files, it may be necessary to convert existing programs and procedures. In some cases, existing programs and procedures can be used in their present form; in other cases, they will have to be thrown away and completely replaced.

Installation of the new system may be an instantaneous affair, but it is often a major task. Usually, the following things must be done:

Computer site preparation must precede the installation of the new system, usually by several months. This involves building or leasing a computer facility with appropriate power, space, lighting, and environmental controls (temperature, humidity, dust, static electricity, etc.). This is often done in conjunction with the computer hardware vendor or with the organization’s computer operations department.
User site preparation may also be required, especially in the case of on-line systems that have terminals and printers in the user’s work area. In the simple case, terminals can be distributed to the user’s work area just before the system is installed; in some cases, though, an entirely new workspace environment may have to be constructed (consider, for example, an airline reservation terminal at an airport).
Hardware installation, assuming that the new system requires its own computer hardware, is usually carried out by the hardware vendor; multiple vendors are sometimes involved, especially in the case of on-line and real-time systems. In the case of a simple system developed for a personal computer, installation may be as simple as taking the computer out of a box and plugging it in.
Software installation, which involves loading all the computer programs that were written for the new system onto the appropriate computer(s) and making them ready for operation.

Keep in mind that the scenario described above presumes that there is only one installation at a single site. But this is often not the case; for a large, distributed system, there may be a single, centralized computer site and dozens or even hundreds of user sites. Thus, it may be necessary to install the system in stages, with specially trained installation teams visiting each user site according to a prearranged schedule. In this case, installation and “cutover” to the new system cannot take place in one fell swoop, but must instead be phased in over a period of days, weeks, or even months.

Training is the final task of the systems development team: training of the users (obviously!), as well as training of the operations personnel, maintenance programmers, and various levels of management. A training plan should be developed early in the project, for there is a great deal of work to be done, and it needs to be ready at the same time (if not earlier than) the system is ready to go into operation. The training plan should deal with the following issues:

How will the training be carried out? Most systems development projects rely on user manuals and reference guides to provide written documentation for the users. However, live classes and seminars may be appropriate, as well as orientation briefings for managers and others who need to be aware of the system even though they will not interact with it every day. And, with current technology, there is a wide range of choices for training media: videotape and videodisk training, computer-based training, and even mock-up versions of the actual system so that users can try entering transactions and learn how to interact with the system.

In the extreme case, the training may consist of extremely sophisticated help facilities built into the system itself. This is becoming more and more popular with the proliferation of personal computers, but it is not very practical for large systems that interact with a large, diverse user community; on the other hand, it can be used to augment and reinforce other forms of training.

Who will do the training? In some cases, members of the systems development team participate in the training process, especially since they are presumed to be the best experts on how the system works. However, keep in mind that the best programmer (or systems analyst) is not always the best trainer; indeed, the developers often behave in a very defensive fashion if the users begin asking what they consider to be hostile questions. Also, the developers are (almost by definition) terribly busy with the designing, coding, and testing of the system until the very last minute; the systems analysts may have more time available after the essential model and user implementation model have been finished.
Who will be trained and on what schedule? Obviously, the users need training before they can begin using the system; on the other hand, it is not effective to train them six months before they will see the new system. So the training must be done in a fairly condensed period of time; but this, in turn, will often interfere with the normal day-to-day work that the users are trying to accomplish. A careful schedule of training activities must therefore be negotiated with the users.

23.8 SUMMARY

This chapter has covered a wide range of topics: programming, testing, conversion, installation, and training. Space does not provide the opportunity for a detailed coverage of these topics, but the brief coverage provided in this chapter should give the systems analyst an overview of these final activities in the systems development project. Additional details can be found in the many references at the end of the chapter.

REFERENCES

James A. Highsmith, Adaptive Software Development: A Collaborative Approach to Managing Complex Systems. Dorset House, 2000.
Kent Beck, eXtreme Programming eXplained. Addison Wesley, 2000.
Brian W. Kernighan, and Rob Pike. The Practice of Programming. Reading, MA: Addison-Wesley, 1999.
Boris Beizer, Black-Box Testing: Techniques for Functional Testing of Software and Systems. New York: John Wiley & Sons, 1995.
Tom Gilb and Dorothy Graham. Software Inspection. Reading, MA: Addison-Wesley, 1993.
Steve Heller. Who's Afraid of C++. New York: Academic Press, 1996.
Steve Maguire, Writing Solid Code. Redmond, WA: Microsoft Press,1993.
Brian Marick. The Craft of Software Testing. Englewood Cliffs, NJ: Prentice-Hall, 1995.
Steve McConnell. Code Complete. Redmond, WA: Microsoft Press, 1993.
Edward Yourdon, Managing the Systems Life Cycle, 2nd ed. Englewood Cliffs, N.J.: Prentice-Hall, 1988.
Edward Yourdon, Nations at Risk. New York: YOURDON Press, 1986.
Tom DeMarco, Controlling Software Projects. New York: YOURDON Press, 1982.
Glenford Myers, The Art of Software Testing. New York: Wiley, 1979.
Tom McCabe, “A Complexity Measure,” IEEE Transactions on Software Engineering, Volume SE-2, Number 12 (December 1976), pp. 308-320.
Edward Yourdon, Managing the Structured Techniques, 3rd ed. New York: YOURDON Press, 1986.
Robert Dunn, Software Defect Removal. New York: McGraw-Hill, 1984.
B. Elspas and others, “An Assessment of Techniques for Proving Program Correctness,” ACM Computing Surveys, Vol. 4 (June 1972), pp. 97-147.
D. Dunlop and V. Basili, “A Comparative Analysis of Functional Correctness,” ACM Computing Surveys, Vol. 14 (June 1982), pp. 229-244.

QUESTIONS AND EXERCISES

What activities in a systems development project commence when design has finished?
What are the six reasons why the systems analyst may need to remain involved with a project during the programming and testing activities?
If the systems analyst is also the project leader, do you think it is important for him or her to be familiar with programming techniques and testing strategies? Why or why not?
In your organization, are systems analysts also expected to participate in the design and programming activities? Do you think this is a good idea? Why or why not?
Why is the systems analyst likely to be involved in the development of test data for a system? Who else is likely to be involved?
What should the systems analyst do if the programmers ask to change the system specification during the programming phase of the project?
What should the systems analyst do if the users ask to change the requirements of the system after the programmers have begun implementing the system? How likely a situation do you think this is?
Why is it possible that a user will want to change the system requirements after the analysis phase of the project has ended?
What difficulties can be expected if a senior systems analyst is expected to carry out all the work of systems analysis in a project?
What kind of negative reaction might be expected from programmers in an organization if systems analysts carry out all of the detailed specification activities discussed throughout this book?
What kind of organizational structure could be set up to accommodate the combination of senior people/junior people and high-level/low-level technical activities in a project?
If systems analysis and systems design activities have been carried out completely and in detail, can the programming aspects be automated? Why or why not? Do you think this situation is likely to change in the next 5 to 10 years?
Is it necessary to carry out all the systems analysis activities to completion before the work of programming begins? Why or why not?
What does fast tracking mean?
In what other industries besides the systems development industry is fast-tracking practiced?
What is a conservative approach to implementing a system? What is a radical approach?
What are the three major reasons why a project manager might adopt a radical approach to systems implementation?
Why can’t the system specification be considered frozen at the end of the systems analysis phase of the project?

FOONOTES

[1] Actually, things could be even worse, and they might be a little better. The worst situation (from the programmer’s point of view) is that we might not need any programmers at all! If the process specification is written in a sufficiently formal language, it can be compiled without any human intervention at all! This is already happening in isolated cases (e.g., process specifications being written in the Ada language and then compiled directly, for high-reliability system in the defense or aerospace industries). On the other hand, there is the possibility that the programmer will still have a lot of creative work to do if the systems analyst wrote the process specification using the precondition/postcondition approach discussed in Chapter 11. In this case, the analyst will have specified the inputs and outputs for each module, but will have left the designer and, ultimately, the programmer the job of determining the best algorithm.
[2] This does not mean 2-4 years of work for a single person, but rather 2-4 years of work for the entire MIS organization. As discussed in [Yourdon, 1986], the backlog was typically 4-7 years throughout the 1970s and 1980s. By the 1990s, business departments in the typical organization began to realize that if they had to wait 7 years for an application to be developed, it would no longer be relevant. Indeed, even 4 years is intolerable in many of the fast-moving competitive industries today; but other areas, such as government agencies, still plod along at a slower pace.
[3] For third generation languages like COBOL, the cyclomatic complexity is approximately equal to the number of IF statements in the program. For more on this, see [DeMarco, 1982].
[4] An example of a stub is a module that does no processing at all, but simply exits as soon as it is called; another example is a module that returns the same output parameters, regardless of the input parameters passed to it when it is invoked. Thus, the initial top-down testing of a payroll system might involve stub modules that pay everyone $100 per week, regardless of their job classification; the stub module for tax calculations might always deduct $10 in taxes from everyone’s paycheck. The objective of the initial top-down test would be to simply determine whether the system can run at all and whether it is indeed capable of generating a series of $100 paychecks.
[5] Detailed discussions of test coverage is provided by [Dunn, 1984] and [Myers, 1979].
[6] Actually, the number of test cases is not infinite, because of the limited precision of numbers stored in the computer’s memory. If a number is stored as an integer, a typical computer will allow a number as large as 232, or possibly 264, to be stored. If the number is stored as a floating point number, we may be able to represent numbers as large as 10100 or larger, but we will usually have only eight or nine significant digits of precision. So, this does not represent infinity, but it is a very, very large number nonetheless.