Using Partial Designs to Elicit Requirements in Web Development – a Survey of Commercial Practice

Dr John Eklund, Usability Analyst and Instructional Technologist, Access Testing Centre, Sydney Australia. Email: john@eklund.com

Associate Professor David Lowe, Associate Dean (Teaching and Learning) Faculty of Engineering, University of Technology, Sydney Email: david.lowe@uts.edu.au

Abstract

In this paper we describe our research that examines the activities in the commercial web development lifecycle, particularly in gathering and defining requirements. Often requirements are inadequately documented, or only emerge during development, or change as development proceeds. We offer an iterative model for Web systems development that incorporates the use of partial design prototypes as a crucial tool in resolving requirements. This is derived from an analysis of the results of a survey of commercial Web practice. The results support the increasingly accepted premise that within commercial Web development, design artifacts become a crucial element in supporting client understanding and driving the formulation of requirements.

Introduction

The growing importance of Web-based systems to organisations is becoming increasingly evident. They are becoming a crucial element for small to medium enterprises to conduct business. The rapid and successful deployment of these systems is often critical to the business strategy of many organisations – particularly with respect to the way in which they interact with customers, clients, or business partners. These projects typically have tighter timeframes, increased visibility to customers, business partners and other third-parties, much finer-grained ongoing evolutionary maintenance, and generally serve a less specific user group. They are often developed very quickly from templated solutions, using coarse-grained authoring tools, and by the efforts of a multi-disciplinary team. Client understanding of their specific needs evolve as a system emerges and is utilised (Dart, 2000). We believe that this is, at least in part, a consequence of the fact that the systems extend beyond the organisational boundaries, to be utilised in a broader context. This complicates the ability to clearly determine the system requirements.

In traditional software development the project moves more clearly from a requirements/specification phase, through successive designs that are evaluated and refined, until the system is built. Indications were obtained in our earlier work (Lowe, 2001; Lowe & Eklund, 2001; Lowe & Henderson-Sellers, 2001) that design artifacts actually play a crucial role in the development of the clients’ understanding. In Web development, there is far less clarity in these phases, with significant overlap. Designs are part of the build process, and lead through evaluation to a modification of specifications. Designs become successively deeper, moving from flat screens to functional prototypes, and there is an unclear distinction between the design process and the specification process, as in the figure below.

These iterative models are believed to be appropriate at a general level for the commercial development of Web systems, however what appears to be particularly misunderstood is what features of the system should be specified and in what manner during the various stages of the development process, and how development can facilitate an understanding of client requirements. This paper investigates current commercial practice in specifying Web projects at various stages of development, and outlines how this research has sought to gain insights into current commercial practice.

Research Evidence

There is a small but growing body of research literature regarding the differences between Web systems and more conventional software systems. In general, this literature identifies unique characteristics of these systems that reflect technical, usability and organisational issues (Burdman, 1999; England & Finney, 1999). These include aspects such as: a tighter linkage between the business and the system architecture; a complex information architecture (Russell, 2000); increased importance of quality attributes since applications are typically more visible externally; open modularised architectures; and rapidly changing technologies. Usability considerations reflect an increased emphasis on user interfaces and the requirement of the system to meet the needs of end users, who are more often a broader and more general demographic than for larger software systems. These considerations include both user acceptance of the system as well as making them usable - developed according to interface standards and matching user preferences and workflow.

There is significant anecdotal evidence (though as of yet little hard research evidence) that clients’ understanding of not only the technology’s capabilities but also their own needs can change dramatically during the course of a project, as they learn more about the capabilities of the technology and how it will be likely to impact on their business processes and models. Many web projects are vision-driven rather than needs-driven leading to an initial lack of clarity. This is also coupled with business models that are non-existent, immature, or evolving rapidly as organisations migrate to an increased reliance on Web technologies (Lowe & Eklund, 2001). These issues are exacerbated by the fact that Web projects tend to be driven by time-to-market pressures, and as a consequence shortcuts in process are often used.

Existing software processes for eliciting, analysing and understanding requirements (Lowe, 2001) assume that clients either understand their requirements, or at the very least understand the problem that is being addressed. Even when the client is not able to articulate their requirements precisely, they are at least able to understand whether a given design will address their needs. In the context of this lack of a research focus on handling client uncertainty, and informed by our earlier work (Lowe & Eklund, 2001; Lowe & Henderson-Sellers, 2001) we can speculate that design artifacts can play a crucial role in the development of the clients’ understanding and consequently the system specification. We formalise this as the hypothesis that systems that result in greater client satisfaction can be developed by adopting an iterative approach that incorporates client-developer joint exploration of partial designs as an integral part of the development of the system specification.

Research Approach

In order to explore issues around Web specification processes and what characteristics are typically included in these specifications, we undertook an extensive set of industry interviews and surveys, followed by a detailed analysis of a number of commercial specifications. The interviews were primarily intended to identify general perceptions and qualitative trends, while the surveys captured more quantitative information. The specification analysis was intended to extract detailed information on the structure and contents of commercial Web system specifications. The overall goal was to understand current best-practice in the specification of Web systems. The companies that indicated a high project success rate, and effective client interactions, were approached regarding obtaining samples of specifications for analysis. These companies covered multimedia development, Internet development, and intranet development. Their core businesses varied considerably from consulting and contract development to in-house development. The industries that they serviced also covered a broad spectrum: financial institutions, medical agencies, travel and tourism, legal, manufacturing, government, e-commerce retail, and consumer advisory services.

The interviews were conducted over a period of 6 weeks were typically 20-40 minutes and involved a series of questions focusing on interactions with clients and the processes for understanding client needs. A total of 23 interviews were carried out. Of the interviewees, 18 also completed the online survey. An additional 38 participants completed only the online survey. This gave a total of 56 responses. All respondents were from within Australian industry and so care would need to be taken in extrapolating to an international context.

Results

There was a very strong response (average rating 4.4 on a 0-5 scale) that clients did not fully understand the capabilities of the technologies. Similarly it was felt (average rating 4.2) that clients did not understand their own needs as they related to the technology. Perhaps surprisingly, anecdotal evidence indicated that respondents felt that clients had a low understanding of their own organisations and existing processes (most of the time undocumented) that need to be changed to allow for the effective integration of the new system. There was a majority consensus (i.e. 83% responding as Strongly Agree or Agree) that there needed to be a process at the beginning of the project focused on educating clients.

Almost all respondents (96%) stated that they interviewed clients as part of the process for identifying requirements. A much smaller number (58%) felt that interviewing potential users was important. The majority of respondents (84%) found that the look and feel and content issues were secondary to the business case. Interestingly, critical success factors (i.e. acceptance criteria or essential requirements) were brought up only by 5% of respondents as a vehicle for capturing the business case. The majority of respondents (78%) indicated the importance of getting the requirements and specification correct – though there was considerable divergence of opinion as to when this should occur. Most of the respondents attempt to capture requirements before they sign final contracts. It was also recognised (by 72%) that initial tendering often occurs before this point – leading to a two-step contract negotiation. Essentially all (94%) respondents recognised that client needs will change and evolve over the life of a project, despite a well-written requirements specification. Most respondents felt that this was a consequence of evolving client understanding.

Several respondents indicated that the client might dictate the specific process – though this was uncommon. Some clients were seen to be happy to commit to a budget (during the tender process) based on broad business objectives, and then finalise the contract at a later stage based on specific analysis of the detailed requirements. The scope of the contracted requirements is typically constrained by retaining a focus on the business case and establishing that there is good basis for specific detailed requirements.

One very important observation that needs to be made, and which might potentially impact on the approach taken to Web development is that the survey data indicates that clients have an initially poor understanding of their needs and this evolves during the project. It does not indicate that clients are aware of this problem! Indeed, the comments and data related to the need to educate clients would support the conclusion that clients are often not initially aware of these problems.

Development Practices

The quantitative data from the survey provided some interesting results. Firstly there was a relatively strong negative correlation (-0.69) between agreement with the statement It is important to understand the user requirements totally before starting development and agreement with the statement The client is generally satisfied with the resultant system. This supports the conclusion that attempting to understand all requirements prior to commencing design actually creates a solution that does not ultimately satisfy client needs. We also found a strong negative correlation (-0.73) between the level of agreement with the statement We usually undertake early stage prototyping and the level of agreement with the statement Our clients have difficulty understanding the implications or details our project bids. Both these results are consistent with the view that early prototyping of designs can assist in client understanding.

From the transcribed interviews there was a very strong consensus (83%) that clients typically need to see concrete designs in order to gain an adequate understanding of the detailed needs for a system and how it will impact them. The interviewees were also asked whether they felt that clients typically have a poor understanding of their own needs and what the technology is capable of providing them? and whether this changed during the project. Anecdotally, the responses indicated that clients understood their needs poorly at the beginning of the project but that it developed during the project. This also came through in the response to the question: Do you find that client requirements, needs or expectations evolve significantly during the course of a project? What brings this about? How are these changes handled? The interviewees responses indicated that this was a consequence of an evolving client understanding, which in turn was a result of their ability to see the emerging system and how it related to their business.

For the question: Are most of the client needs captured before or after signing a contract for the project? a significant number (38%) of the developers indicated that they often adopted a two-step negotiation specifically because of client uncertainty. The first step involved a contract (usually based on an hourly rate or equivalent) for the development of a specification. The second step would then involve another contract (usually fixed price) for the system build.

Conclusions and Implications for HCI practictioners

In this paper we have analysed current commercial practice with regard to the management of requirements for Web projects. Specifically, we have focused on how and when requirements are identified. By far the most significant observation from the interviews and surveys is that many aspects that would conventionally be regarded as “requirements” (in the sense that they express characteristics of the system that are desired by the client) emerge in commercial specifications after design artifacts have appeared. Indeed many of the requirements are actually expressed as part of a design. This is consistent with the hypothesis that the design process plays a critical role in reducing the domain uncertainty inherent in most Web projects. It also confirms what is generally understood by experienced developers - that clients might understand their existing systems, but not necessary the nature of the problem that appears as a result of the emerging systems, and that therefore requirements cannot be elicited independently of any designs being presented.

HCI practitioners are frequently called upon to act as a third party between the client and the development team to gather requirements and interpret them into acceptance criteria or early design specifications. What emerges as clear implications for HCI practice from this work is an emphasis on gathering requirements as an integrated activity rather than an isolated phase, and to use partial designs such as paper prototypes to express requirements. A Web project at risk of failure is one where requirements analysis has been done in an isolated phase, where requirements have been expressed simply as text rather than text and partial designs, and one where the fundamental principle of iterative design reflecting successively refined requirements has not been followed.

References

Burdman, J (1999) Collaborative Web Development. Addison-Wesley.

Dart, S (2000) Configuration Management: The Missing Link in Web Engineering. Artech House,

England, E. and Finney, A. (1999) Managing Multimedia: Project Management for Interactive Media. Addison-Wesley.

Lowe, D. (2001) A Framework for Defining Acceptance Criteria for Web Development Projects. in Murugesan, S. and Deshpande, Y. eds. Web Engineering: Managing Diversity and Complexity of Web Application Development, Springer-Verlag. 279-294.

Lowe, D. & Eklund, J. (2001) Commercial Issues in the Specification of Web Systems. Proceedings of The Sixth Australian Workshop on Requirements Engineering. (AWRE’2001) p.4-13.

Lowe, D. and Henderson-Sellers, B (2001), Impacts on the development process of differences between web systems and conventional software systems. in SSGRR 2001: International Conference on Advances in Infrastructure for Electronic Business, Science, and Education on the Internet, (L'Aquila, Italy, 2001), Scuola Superiore Guglielmo Reiss Romoli, 21.

Russell, P (2000), Infrastructure - Make or Break your E-Business. In TOOLS-Pacific 2000: Technology of Object-Oriented Languages and Systems (Sydney, Australia, 2000).

Sommerville, I. and Kotonya, G. (1998) Requirements Engineering: Processes and Techniques. John Wiley and Sons.

Stein, L.D. Profit, the Prime Directive. Web Techniques, 5 (11). 14-17.

Copyright

John Eklund and David Lowe, © 2002. The authors assign to Southern Cross University and other educational and non-profit institutions a non-exclusive licence to use this document for personal use and in courses of instruction provided that the article is used in full and this copyright statement is reproduced. The authors also grant a non-exclusive licence to Southern Cross University to publish this document in full on the World Wide Web and on CD-ROM and in printed form with the conference papers and for the document to be published on mirrors on the World Wide Web.