Department of
Information Systems
The University of Melbourne
Parkville VIC 3010 - Australia
ABSTRACT
In this paper we present a survey that was conducted in Melbourne (Australia), along with our analysis of the results. The aim of the survey is to find out, from expert practitioner’s opinions, what kind of web development fits the domain of agile and heavyweight methodologies. The survey’s results concede that agile methodologies can provide palpable benefits for both low and medium-level web development. In case of high-level web development, more embellishments are needed in shape of process and ceremony to effectively cordon off complexity.
Keywords
Web development methodologies, Agile, Heavyweight, practitioner survey.
In this paper we present a survey that was conducted in Melbourne (Australia), to help us find out whether agile methods (such as SCRUM [9] or XP [13]), or heavy methods (such as UP [7] or the Spiral development life cycle [2]) are suitable for web development.
This is an uphill task due to the enormous size of the World Wide Web. To unravel our quandary, we divide web into three different categories according to the Capability Maturity Model: Low, Medium and High [12]. For each category we then determine the extent of application of each methodology. In doing so, we base our analysis on the inference drawn from few empirical studies that have been done in regards to agile and heavyweight methodologies, in the context of web development, and simultaneously, we solicit feedback from software industry practitioners on their disposition.
Low maturity level websites consist of information and data providing websites, whereas, medium maturity level websites include e-commerce, e-learning, intranets, B2C, and B2B websites. While on the other hand, high maturity level websites include strategic alliances between organizations in the value chain and deliverance of new products to new markets via web technologies.
This categorization is necessary because for each complexity level a different set of techniques and methods might be applicable. Our assumptions are that low-level web development requires small team, and project size and complexity is fairly capped. Conversely, high-level web development encompasses larger scope requiring greater efforts and more resources. Various studies manifest that agile methodologies could provide a workable solution only if they are modified and adapted [5], [6]. Pure agile methodology, they state, is unsuitable for large projects. Agile methodology has to be disciplined and regimented with mechanical and methodical moldings if it is to be applied for high-level web development. Agile methodology has to be extended with some sort of formal and ceremonial approach to iron out the irregularities and peculiarities of large software development.
Our aim is to find out, from expert practitioner’s opinions, what kind of web development fits the domain of agile and heavyweight methodologies.
To collect the opinions from software industry practitioners about these findings, we conducted our own survey. For that purpose, we formulated a questionnaire and polled various people having significant experience in web development using both agile and heavy methods. Next, a summary of the results and analysis of the survey is presented.
The questionnaire was divided in four different parts. All the questions on the questionnaire were close-ended questions making it easier and quicker for respondents to answer. The different parts are listed below:
Organization Related Questions – These questions dealt with the respondent’s organization size and the readiness to accept new technologies and methods.
Methodology Questions – These questions rated the respondent’s knowledge of both heavyweight and agile methodologies.
Web Development Questions – These questions were divided into low-level web development, medium-level web development, and high-level web development. For each category of the web development, we aimed at finding out the different heavyweight and agile methodologies used. Furthermore, we wanted to draw feedback from the respondent as to what were the impediments experienced while practicing agile methods in the web domain. In addition, respondents were supplicated to respond if adoption of agile methodologies had any effect on project costs, software quality, and delivery schedule. Moreover, the questionnaire also quizzed respondent as to whether heavyweight and agile methodologies considered usability analysis and user-centered approaches when developing web applications. The most important question in this section was to determine if agile methodologies can scale up to the sophistication and maturity of web development.
General Questions – These questions were there to confirm the validity of web development and software development as either science or art.
15 practitioners were selected and asked to fill the questionnaire. The average time spent to complete the questionnaire was about 20 minutes. If there was any ambiguity in the respondent’s answer, follow-up questions were asked to eliminate any misconception.
We now present the key observations from our survey results below.
All the respondents questioned rated their knowledge of both agile and heavyweight methods as average or more. 14 out of 15 respondents acknowledged that their knowledge of agile methods was either extensive or highly extensive. 12 out of 15 respondents acknowledged that their knowledge of heavy methods was either extensive or highly extensive. 53% of respondents worked in small-sized organizations. A small sized organization was defined as an organization having revenues less than 50 million. 27% of respondents belonged to medium-sized organizations. A medium-sized organization was defined as an organization having revenues between 50 and 200 million. The remaining 20% of respondents were employed by large organizations. A large organization was defined as an organization having revenues in excess of 500 million.
A whopping 93% of respondents characterized their organization as either a market leader or a market follower, when it comes to accepting cutting edge technologies and methods. This observation relates to the fact that the majority of the Agile methodologies are only being accepted by Innovators and Early adopters according to Geoffrey Moore’s Law of Technology Adoption Curve [11]. Agile methods still have a long way to go before they are openly embraced by organizations with conservative approaches.
Among all different agile methods, XP was the most heavily used. Feature Driven Development and Scrum followed XP as the other two hot favorites. An interesting corollary from the results indicates that organizations followed the same methodology irrespective of web development project size. If XP was followed for low-level web development, the same approach was reoccurring in high web development as well.
In case of software costs, 67% of respondents were of the opinion that adopting an agile process in comparison to heavy process for low-level web development will result in decrease in costs. The remaining respondents strongly opposed that adopting an agile process over a heavyweight process will yield no benefits. However, when it came to medium-level web development, 87% of respondents were confident that adopting an agile process will have a positive bearing on the project in the shape of diminishing costs. The same pattern was extended for high-level web development as significant number of respondents cast their votes in favor of decreased costs. It might come as a consolation to agile methods that none of the respondents accredited higher costs to practicing agile methods.
When the issue of software quality was presented to the respondents, the majority of the respondents were heavily inclined to accede that quality improvements follow agile methods. In all three categories of web development, responses equivalent to increase in quality prevailed.
Figure 1 and 2 below elaborate our results.
Figure 1. Effect of Agile Methodologies on Project Costs.
Figure 2. Effect of Agile Methodologies on Software Quality.
Moreover, survey participants were questioned whether following a particular methodology has any effect on delivery schedule or not. Does delivery schedule expand or contract when following agile or heavyweight methods?
Delivery schedule contracts when agile methods are adopted in place of heavyweight methods for low-level web development. A majority of the respondents conceded that a low-level web development project would require less than 2 months to be completed, while a similar project operated under heavy methods would require 2 to 6 months for its completion.
Similarly, a medium-level web development project under agile techniques can be easily accomplished within 2 to 6 months. On the contrary, a heavyweight process will require 6 to 12 months to meet the requirements of a medium-level web development project. The same analogy can be extended for high-level web development as well. An agile process will attempt to finish within 12 to 18 months. A heavyweight process will generally require somewhere in the neighborhood of 18 to 24 months.
The syllogism behind the above pattern can be explained by a single word – Ceremony. Too much is done in the name of bureaucracy, formality, hierarchy, and autocracy, making the heavyweight process retard. In contrast, agile method marches on as it discards the excess baggage in the name of agility, swiftness, and nimbleness. Figure 3a and 3b below elucidates our results.
Figure 3a. Delivery Schedule in Agile Methods.
Figure 3b. Delivery Schedule in Heavyweight Methods.
Web applications are “ephemeral applications” and the time spent browsing each web page is of microscopic quantity. Therefore, usability becomes more paramount and prominent. Keeping that thought in our mind, we probed our respondents to compare and contrast the extent of usability analysis undertaken in both agile and heavyweight methods.
The respondents were almost evenly divided when asked – “Do agile methods consider usability analysis and user-centered approaches?” 47% were of the notion that usability analysis is present in agile methods, while 40% affirmed the nihility of usability analysis in agile methods. The remaining 13% were indifferent. However, in heavyweight methods, there was a clear mandate as 60% of respondents sided with the fact that usability analysis is a nonentity. Figure 4a and 4b below illustrates our results.
Agile Methodologies overlook user-interface design. In the words of Alistair Cockburn, “It (user-interface) is not a weak point – it is an absence.”[3], yet a sizeable number of respondents totally negate this assertion. When asked to clarify their stance, many respondents counterattacked by stating that usability analysis happens each time customer approves/disapproves any particular feature. In addition, since agile development is customer driven development, anything beyond the likings of the customer is ignored. Moreover, since all the development efforts take place in front of customer’s eyes, the modalities of usability analysis and user-centered approaches are already taken care of. Unlike heavyweight process, where the customer gets to see the final product when all the development efforts have been done, agile process is more user-centric.
Figure 4a. Usability Analysis in Agile Methods.
Figure 4b. Usability Analysis in Heavy Methods.
To further bolster the above stand of the respondents, one of the usability gurus, Larry Constantine also agrees with the same notion. Directly quoting Constantine – “In its most agile incarnation, user-centered design works some of the same card magic that powers XP. We model task cases on index cards, thus keeping them short and sweet to facilitate easy collection and distribution. We shuffle the cards to prioritize them by business and end-user importance. We sort them into related groups that help us construct complete usage scenarios and that guide the collection of user interface components into screens, pages or dialog boxes. The same task cases that steer the user interface design are grist for the programming mill that grinds out the objects and methods that power the software. As always, we focus on user intentions and system responsibilities to help distinguish real user needs from mere wants and wishes. In short, usage-centered design probably qualifies as an agile process, even if it wasn’t conceived with that in mind” [4].
The onus of any software development methodology should not be restricted to the development effort alone. It has to be lengthened to reduce the hardships of long-term maintenance.
Astonishingly, majority of the respondents favor agile methods over heavy methods for long-term maintenance of web development. This particular affinity is not short-lived as it does extend to all kinds of web development regardless of the size and complexity. Even though, agile methods are low on documentation there is an increased fondness for it. This perception offshoots from the fact that agile methods are test-driven. Before implementing any new feature, a developer/customer writes the acceptance tests and all these acceptance tests form a suite of tests, which continuously run to validate any new addition of code. A Majority of the respondents believe that it is the test suite, not voluminous and loquacious documentation that matters.
There was some reservation as to how agile methods would tackle the complexity of maintaining high-level web development as more respondents switched their sympathies to heavyweight methodologies. All in all, agile methods come very strong when it comes to long-term maintenance. The results are shown below in Figure 5.
Figure 5. Long-Term
Maintenance and
Methodology Selection.
The raison d'être of this composition is to cross-examine whether agile methodologies fare better or heavyweight methodologies cope well when it comes to different kinds of web development. What do the panel of software experts think? Agile methods are more compatible for low-level and medium-level web development. 87% of respondents acknowledged that using agile methods for low-level web development is the best approach. The same ratio also holds true for medium-level web development. For high-level web development, the respondents favored heavyweight methods. Why? Simply because heavyweight methods can easily maneuver the added complexity of running a large software team outstretched over multiple domains, functions, and continents in a more cohesive fashion.
Even though our questionnaire did not provide the respondents to select both agile and heavyweight methodology as a suitable candidate for web development, many respondents favored that amalgamation of both agile and heavy methods could be a feasible solution. In the words of one respondent, “In order to properly tap creativity, you need some formalized process. Throwing agile in a heavyweight process will be a welcome change and it may take off some of the bureaucratic drudgery.”
Furthermore, we solicited feedback from respondents in terms of liking and disliking particular aspects of agile methods for different kinds of web development. Working Code over Documentation was the biggest incentive in selecting agile methods for low-level web development. For medium and high-level web development, Customer Collaboration over Contract Negotiation stood out as the most appealing virtue of agile methods, followed by People over Processes and Respond to change over following a plan.
Figure 6a. Agile Aspects Liked by Respondents.
Figure 6b. Agile Aspect Disliked by Respondents.
On the flip side, low planning was attributed as a downside for low-level agile web development. Similarly lack of project structure factored in as major deterrent for all kinds of web development. In fact, 67% of respondents upheld that one of the key inhibitions in the acceptance of agile methods for high-level web development is the kinship of agile methods with project structure looseness. In opposition, one of the respondents inclined towards agile methods retorts, “Agile methodologies are very rigorous in terms of process and project structure. Just to follow the 12 practices of XP in true essence is strict indeed.” The results are graphically depicted above in Figure 6a and 6b.
In the same instance, we also collected information from respondents as to what were the major hurdles in practicing agile methods for different kinds of web development. Lack of Customer Collaboration topped the list in low and medium-level web development. As one respondent reports, “One thing, I think, we really suffer from is the lack of customer and other stakeholder involvement except for at the beginning and end of most projects. It would be nice to have the XP ideal of continuous customer involvement!” Likewise, project size/complexity contributed heavily towards making an agile process nonfunctional in high-level web development. 60% of the respondents concurred that the most important factor in high-level web agile activity is complexity. In addition, some respondents also held lack of skilled people as detrimental to the smooth operation of agile activity. Figure 7 below represents our findings.
Figure 7. Major Hurdles in practicing Agile Methods.
Moreover, in our survey we targeted the extent of agile methodology used for different kinds of web projects. The results indicate that none of the respondents are following all agile techniques.
Figure 8. Extent of Agile Methodologies Used.
For low-level web development, the weighted average of using agile techniques came to 60%. So, if using XP on low-level web project, 7 out of 12 XP prescribed techniques or rules will be followed in true essence. The rest 5 are either totally ignored or modified to fit the project constraints and needs. Comparably, 56% agile techniques are followed for medium-level web development. Likewise, 48% agile techniques are implemented for high-level web development. As we can analyze that as the maturity level of web increases, there is a corresponding decrease in the number of agile techniques used. The rationale behind this phenomenon can be demystified with the fact that as the project size, complexity, team, architecture, scope, and risks increase so does the use of heavyweight methods. There is an inverse relationship between agile techniques and project complexity. Figure 8 below sums up our results.
Earlier, we stated that agile methods are compatible for low and medium-level web development, whilst heavyweight methods are more applicable for high-level web development. The quintessence is - “it all depends.” Methodologies are not a silver bullet. Almost any methodology can be made to work on any project. In the same manner, any methodology can manage to fail on any project. One methodology can’t fit all projects and all projects can’t fit one methodology. The punch line is that having multiple methodologies is appropriate and equally indispensable. No methodology can be an antidote to all the evils of software development; no methodology can pledge success for its every undertaking.
Alistair Cockburn, in his article, “Selecting a Project’s Methodology”, states that a process methodology should be selected on a three dimensional criteria - people, criticality, and priority [3]. Expounding on the selection criteria, Cockburn asserts that a larger group needs a larger methodology. Furthermore, a critical system needs more correctness in its construction. A methodology geared towards building atomic power plant software should be more laborious and expensive as compared to a methodology for building bowling-match tracking software. In addition, project priorities also play a huge role in methodology selection. Do sponsors want to have the software soon, want it defect-free, want it prioritized for productivity and tolerance, or want it prioritized for legal liability?
The “one size fits all” approach to applying software methodologies is no longer appropriate [8]. Each methodology is only effective under certain conditions. Each methodology has its own comfort zone. Elongating the comfort zone is trespassing in restricted areas. As we saw earlier, heavyweight methodologies are the disciplined approach to analysis and design of software applications, making them an ideal choice for high-end and large-scale projects. Agile methods are a compromise between no process and too much process, making them a suitable choice for projects subjected to short timeframes and excessive uncertainty and change. Applying a heavy process in low and medium-level web development is counter-intuitive. Similarly, an agile process for high-level web development is precarious and risky.
For a moment, let’s step back from these methodological clichés and start musing on whether software development, specifically web development, is a science or an art. This distinction is very important because it sets the tone as to how we treat software and web development.
Our respondents with a thumping majority declared that both web and software development is a dilution of scientific and artful ingredients. As one respondent proclaims, “You need discipline and creativity to mastermind a web development project.”
However, Rob Austin et al. oppose the above notion and portray software making as an Art [1]. They support their premise by extracting the commonalities between a software process and a theater. In their own words, “Artful making, whether in software development or theater, has a recognizable structure. It’s iterative and thus involves doing and doing again… It de-emphasizes planning in favor of learning from experience. It delegates control to makers themselves, aspire to enhance individual and idiosyncratic talents, and uses the environmental uncertainty of ongoing change to fuel innovation.” They concur that it is actors, designers, singers, and directors, who make the majority of the choices, as script is not detailed enough to be taken as a specification. They ascribe that web development is creation not conformance. Web development is imagination not compliance. Web development is ingenuity not rigidity. As we can see, all these arguments have an agile flavor to them, and why not? Agile paints web development as an artistic rendering. It is the fusion of art and creativity coupled with improvisation and dexterity that makes agile so artsy.
However, the story does not end here. There is another twist to this. Some believe web development has its roots in Chaos Theory - a branch of scientific system theory with a strong mathematical foundation. Chaos Theory can be defined as “ the qualitative study of unstable, aperiodic behavior in deterministic, nonlinear dynamic systems” [10]. What this essentially means in laymen terms is that we need to be aware that sometimes the same input results in different output. In other words, small variations in initial conditions might result in huge transformations in the concluding events.
As we all concur that it is humans that grind the software mill. Web development without human involvement is not called software development; it’s called a compiler. Web development is the personification of human interactions. It is the representation of how human actors work, process, and describe their interest. Yet at the same time, let’s not condone that humans are highly unpredictable. No two human beings can produce the same result because individuals differ in mental, physical and intellectual capacity. A team working smoothly under a particular project manager might totally crumble under the leadership of another. Addition and subtraction of team members can result both positively and negatively. It is this unpredictability of human nature, which makes it so unique, that it can be classified as Chaos Theory. It is this haphazard, random, and unpremeditated behavior exhibited by humans in their work settings that give web development a new dimension.
So where does the buck stop? Is web development an art or a science? We think the argument continues. There is no single right answer. We feel web development starts off as an art and as web complexity progresses, in the shape of more team members, distributed development, and larger domains, it transforms itself into a science, thus requiring more sophisticated scientific paraphernalia with added cognizance of human behavior subtleties.
Most methodologies settle for simple linear descriptions of software processes. They write processes as if they were writing for robots. They write for rule-followers rather than problem-solvers. However, in actuality, we humans do not live and work in even remotely linear world; in fact our acts should be characterized as non-linear, where dicey human behavior is rampant compounded with ambiguity and uncertainty prevailing in large doses.
It would be easier to take humans out of this equation but software development without human intervention and resilience is a spiritless activity. Humans are the one who breathe life into software development. The ability and propensity of humans to both invent and adapt processes is prodigious. It is this prodigy, which fuels the software engine to rack up all these miles effortlessly.
In a wake up to all these arguments to circumvent chaos and exalt human stature, Agilist are right by putting “individuals and interactions over processes and tools” and “responding to change over following a plan” and “working software over comprehensive documentation” and “customer collaboration over contract negotiation” in the limelight. Improvisation and creativity, conformance to actual, and balancing flexibility and structure form the crux of the matter. Agilists believe the yardstick of measuring success is not man-hours; it is results. According to Agilists, expecting to solve people issues through technological or process solutions is totally flawed. On the other hand, proponents of heavyweight methodologies advocate that it is not improvisation, rather it is carefully articulated processes, which are the key to success. It is detailed planning, extensive documentation, and thorough design, which produce quality products. They maintain that we need the same structure, vigor, and repeatability, in software development, that paved the way for Industrial Revolution. Heavyweight processes use scientific approaches to narrow complexity by thriving to provide certainty in face of uncertainty, objectivity in face of subjectivity, uniformity in the face of multiplicity, designed process in the face of evolving process.
When these conflicting postulates are put to test to evaluate applicability in web development, we find that agile can scale up from low to medium level web development. High-level web development requires more precision, analysis and accuracy, which can only be delegated via heavyweight methodologies. Again, we have to be cautious that we cannot assign domains to each methodology. “One size fits all” maxim has no rhyme and reason in software development.
[1] Austin, R. and Devin, L. Beyond Requirements: Software Making as Art. IEEE Software, pp.93-95, January/February 2003.
[2] Boehm, B.W. A Spiral Model of Software Development and Enhancement. IEEE Computer, May 1988
[3] Cockburn, A. Selecting a Project’s Methodology. IEEE Software, pp. 64-71, July/August 2000.
[4] Constantine, L. Methodological Agility. Software Development, pp. 67-69, June 2001.
[5] Elssamadisy, A. XP On A Large Project – A Developer’s View. http://www.agilealliance.org/articles/. Accessed on 4 Nov 2004.
[6] Harrison, N. B. A Study of Extreme Programming in a Large Company. http://www.agilealliance.org/articles/. Accessed on 4 Nov 2004.
[7] Larman, C. Agile & Iterative Development: A Manager’s Guide. Addison-Wesley, 2004.
[8] Lindvall, M. and Rus, I. Process Diversity in Software Development. IEEE Software, pp.14-18, July/August 2000.
[9] Mar, K. and Schwaber, K. Experiences of using Scrum with XP. http://www.controlchaos.com/old-site/XPKane.htm. Accessed on 4 Nov 2004.
[10] McBride, N. Chaos Theory and Information Systems. http://www.cse.dmu.ac.uk/~nkm/CHAOS.html. Accessed on 4 Nov 2004.
[11] Moore, G. Crossing the Chasm. HarperBusiness, Revised Edition 2002.
[12] Vidgen, R. Using the Multiview 2 Framework for Internet-Based Information System Development. In Proceedings of the Sixth International Conference on Information Systems Methodology, Salford University, Manchester, UK, August, 1998.
[13] Williams, L. The XE programmer: The Few-Minutes Programmer. IEEE Software, pp. 16-20, May/June 2003.