What are the fundamental activities that are common to all software processes?

At least in principle, the result (artifact) of each phase is one or more documents that should be approved and the subsequent phase should not be started until the previous phase has completely been finished. In practice, however, these stages overlap and feed information to each other. During design, problems with requirements can be identified, during coding some of the design problems can be found, etc. The software process therefore is not a simple linear model but involves feedback from one phase to another as it is depicted on the figure. Documents produced in each phase may then have to be modified to reflect the changes made.

In principle, the waterfall model should only be applied when requirements are well understood and unlikely to change radically during development as this model has a relatively rigid structure which makes it relatively hard to accomodate change when the process in underway.

An important variant of the waterfall model is formal system development, where the specification is described using a mathematical model which will later be refined into executable code (possibly through a series of other models) using consistency-preserving mathematical transformations. If the mathematical transformations applied are correct, we have a strong argument that a program generated in this way is consistent with its specification.

Incremental development

Incremental development is based on the idea of developing an initial implementation, exposing this to user feedback and evolving it through several versions until an acceptable system has been developed. Activities of specification, development and validation are not separated but interleaved. There are also lots of feedback across those activities.

Figure 1.2. Icremental development

Incremental software development, which is a fundamental part of agile approaches, is better than a waterfall approach for most business, e-commerce, and personal systems. Incremental development reflects the way that we solve problems. We rarely work out a complete problem solution in advance but move toward a solution in a series of steps, backtracking when we realize that we have made a mistake. By developing the software incrementally, it is cheaper and easier to make changes in the software as it is being developed.

Each system increment captures a piece of the functionality that is needed by the customer. Generally, the early increments of the system should include the most important or most urgently required functionality. If this is the case then customers can evaluate the system at a relatively early stage in the development to see if it delivers what is required. If not, then only the current increment has to be changed and, possibly, new functionality defined for later increments. Compared to the waterfall model, incremental development has three important benefits:

1. There is a reduced cost of dealing with changing requirements since the amount of analysis and documentation activities that has to be redone is much less than is required when using the waterfall model.

2. It is much easier to get customer feedback on the work done. In waterfall-like processes (at least, theoretically) customers first meet the system when it is fully developed and tested and is deployed. If something is not acceptable than it is a real problem (that is the reason why waterfall should only be applied when requirements are well-understood). In iterative development, customers regularly receive a (not fully functional) version of the system that can be commented which provides valueable feedback.

3. More rapid delivery and deployment is possible so customers can get value out of the software earlier than is possible with the waterfall process. In incremental development it is allowed (and advised) to deliver useful functionalities even in early stages of the development.

Incremental development in some form is now the most common approach for the development of application systems. This approach can be either plan-driven, agile, or, more usually, a mixture of these approaches. In a plan-driven approach, the system increments are identified in advance; if an agile approach is adopted, only the early increments are identified but the development of later increments depends on progress and customer priorities.

Reuse-oriented software engineering

Reuse is one of the most important concepts of today's software engineering since it can not only save a given amount of work when existing components providing a given funtionality are reused but existing components might have lots of testing received so far so we can possibly build more reliable systems based on them. Nowadays more and more software projects apply reuse to some extent, however, some of them relies more on reused components than others.

Reuse often happens in an informal way when people working on the project know of designs or code that are similar to what is required. They look for these, modify them as needed, and incorporate them into their system. This is basically the application of patterns in the development process.

Figure 1.3. Reuse-oriented development

A general process model for reuse-oriented software engineering is shown in the above figure. Although the initial requirements specification stage and the validation stage are comparable with other software processes, the intermediate stages in a reuse-oriented process are different. These stages are:

1. Component analysis. Based on the requirements specification, components that implement (some part of) the specification are looked for. In the most of the cases there is no exact match and the components that may be used only provide some of the functionality required.

2. Requirements modification. During this stage, the requirements are analyzed using information about the components that have been discovered. They are then modified to reflect the available components. Where modifications are impossible, the component analysis activity may be re-entered to search for alternative solutions.

3. System design with reuse. During this phase, the framework of the system is designed or an existing framework is reused. The architects will perform the design by taking into account the components that are reused and they will organize the framework accordingly. New pieces of software may have to be designed if reusable components are not available.

4. Development and integration. Software that cannot be externally procured is developed, and the components and commercial-off-the-shelf (COTS) systems are integrated to create the new system. System integration, in this model, may be part of the development process rather than a separate activity.

There are basically three types of software components that can be used in a reuse-oriented process:

1. Web services that are developed according to well-known service standards and which will become available for remote invocation.

2. Collections of objects that are developed as a package to be integrated with a component framework such like .NET or Java EE.

3. Standalone software systems that are configured for use in a particular environment.

One of the most important advabtage of reuse-oriented software engineering is the reduced amount of software to be developed and therefore reduced cost and risks. As a consequence, it can also lead to faster delivery. However, compromises must be achieved on requirements which might lead to a system that does not meet the real needs of users. Some control over the system evolution might also be lost as new versions of the reusable components are not under the control of the organization using them (no influence on what functionalities to include/exclude, etc.).

Process activities

In this section some of the most important activities and sub-activities of the four general process activities will be discussed very briefly. For a more detailed description of activities, the reader should read some general software engineering textbooks like.

The four basic process activities of specification, development, validation, and evolution are organized differently in different development processes. In the waterfall model, they are organized in sequence while in incremental development they are interleaved. How these activities are performed might depend on the type of software, people involved in development, etc.

Specification

Software specification or requirements engineering is the process of understanding and defining what services are required from the system and identifying the constraints on the system’s operation and development.

Figure 1.4. Requirements engineering process

There are four main sub-activities in requirements engineering processes:

1. Feasibility study when an estimation of whether the identified requirements may be satisfied using current software and hardware technologies is made. A feasibility study should be relatively cheap and quick; it should inform the decision of whether or not to go ahead with a more detailed analysis.

2. Requirements elicitation and analysis, which is the process of deriving the system requirements through observation of existing systems, discussions with stakeholders, etc. This may involve the development of one or more system models and prototypes that can help us understanding the system to be specified.

3. Requirements specification, which is the activity of translating the information gathered during the analysis activity into a (formal or informal, depending on the underlying process used) document that defines a set of requirements. Two types of requirements may be included in this document:

   1. User requirements are abstract statements of the system requirements for the customer and end-user of the system;

   2. System requirements are a more detailed description of the functionality to be provided.

4. Requirements validation is the process of checking the requirements for realism, consistency and completeness. During this process our goal is to reveal errors in the requirements document. When an error is found, the requirements specification document needs to be modified to correct the problems.

Of course, the activities in the requirements process are not necessarily executed in a strict sequence but the activities of analysis, definition, and specification can be interleaved. In agile methods, such as extreme programming, requirements are developed incrementally according to user priorities and the elicitation of requirements comes from users who are part of the development team.

Design and implementation

The implementation phase is the process of converting a system specification into an executable system. It always involves processes of software design and programming but, if an incremental approach to development is used, may also involve refinement of the software specification.

Figure 1.5. Design process

The activities needs to be carried out during the design process vary depending on the type of the system needs to be developed. However, for the development of information systems, the main four activities are the following:

1. Architectural design, where the overall system structure is identified. It involves finding principal components (sometimes called sub-systems or modules) along with their relationships. Distribuution of components (to hardware nodes and to development teams) should also be discussed.

2. Interface design, where the interfaces among various system components are defined. This interface specification must be unambiguous. With a precise interface, a component can be used without other components having to know how it is implemented. Once interface specifications are agreed, the components can be designed and developed concurrently.

3. Component design, where we take each system component and design how it will operate. This may be a simple statement of the expected functionality to be implemented, with the specific design left to the programmer.

4. Database design, where the data structures used by the system are designed and their representation in a database is dealt with. The sub-activities that need to be performed here depends on whether an existing database is to be reused or a new database is to be created.

Validation

Software validation or, more generally, verification and validation (V&V) is intended to show that a system both conforms to its specification and that it meets the real needs of the customer or user of the application. Testing, where the system is executed using simulated test data, is an important validation technique but some checking processes, such like inspections or reviews should also be applied during V&V processes.

Testing has three main stages:

1. Development testing. The components that compose the system are tested by the people developing the system. Each component is tested independently, without other system components. Components may be simple entities such as functions or object classes, or may be coherent groupings of these entities. Test automation tools, such as JUnit are commonly used.

2. System testing. System components are integrated to create a complete system. This process is concerned with finding errors that result from unanticipated interactions between components and component interface problems. It is also concerned with showing that the system meets its functional and non-functional requirements, and testing the emergent system properties.

3. Acceptance testing. This is the final stage in the testing process before the system is accepted for operational use. The system is tested with data supplied by the system customer rather than with simulated test data. Acceptance testing may reveal errors and omissions in the system requirements definition, because the real data exercise the system in different ways from the test data.

Evolution

Requirements are always changing, even after the system has been put into its operating environment. Emerging systems with new interfaces that our software needs to communicate with, or business environment changes might result in changing needs.

Historically, there has always been a split between the process of software development and the process of software evolution (software maintenance). However, this distinction between development and maintenance is increasingly irrelevant. Hardly any software systems are completely new systems and it makes much more sense to see development and maintenance as a continuum. Rather than two separate processes, it is more realistic to think of software engineering as an evolutionary process where software is continually changed over its lifetime in response to changing requirements and customer needs.

What are the fundamental process activities?

Which is not a fundamental activity that are common to all software processes?

What are the fundamental activities of the software evolution process?

What are the fundamentals of software?