---

Download PDF Version 5.9mb

---

Abstract A content management system in its simplest form allows a website owner to maintain their own website through a web browser interface. Much research has been conducted into producing usable content management systems, however there has been no research investigating how to achieve user satisfaction in such systems. This is particularly important given the daily use such systems can receive by the same users, as well as the pressures imposed on such systems used in a commercial environment. This thesis seeks to readdress the balance in content management system design. Research is conducted to determine which factors can affect user satisfaction in content management systems. These factors are then investigated and validated by means of redesigning and evaluating the THE SYSTEM content management system. The results produce an importance ranking of these factors, finds some particularly interesting correlations between the affective and functional notions of user satisfaction, and recommend areas for further study. Anonymity Company and system names have been anonymised to protect the interests of the parties involved in this project. The company offering a CMS as a service to their clients for website maintenance is referred to as THE PROVIDER. The company utilising this CMS within their organisation is referred to as THE CLIENT. The content management system itself is referred to as THE SYSTEM. Screenshots have been altered as necessary. License This dissertation is protected under the Creative Commons Attribution-No Derivative Works 2.0 UK: England & Wales license and remains the copyright of Nick Day. For full details visit http://creativecommons.org/licenses/by-nd/2.0/uk/.

---

Contents

---

Chapter 1
Introduction

A content management system (CMS) in its simplest form allows a website owner to maintain their own website through a web browser interface. Typical features include the ability to create new pages, delete or edit existing pages and publish changes to a web server.

Usability applies to all aspects of a system with which a user might interact (Nielsen, 1993) and is broadly defined under the ISO 9241-11 standard as consisting of three distinct aspects: effectiveness, efficiency and satisfaction (Userfocus, 2007). That is, the user can accomplish the tasks they require, as timely, accurately and completely as possible, and the interaction experience of doing so is as pleasurable as possible with minimal user frustration.

There has been considerable research into the usability of CMSs (Byrne, 2005a,b; Kowalski, 2002a,b; Robertson, 2003, 2007) but this has focussed prominently on the development of usability guidelines and heuristics. As a result, this research centres very much around the more functionality-related effectiveness and efficiency aspects of usability. No research has been carried out into the importance or methodology of achieving user satisfaction in CMSs. This study is the first to date to apply measures of user satisfaction specifically to CMSs.

Although research has been conducted into user satisfaction in traditional IT systems, many of the significant studies were carried out at least ten to fifteen years ago (Doll & Torkzadeh, 1988; Hiltz & Johnson, 1990; Etezadi-Amoli & Farhoomand, 1991), with little subsequent “realignment” since with systems of today. This was obviously a time when computer usage was much less commonplace and computers did not have the “everyday object” status they have attained today. It is not clear a) how relevant these user satisfaction measures are to users of today; and b) how relevant these measures are in promoting user satisfaction in CMS design. A contemporary investigation of these measures is required, specifically targeted at influencing future CMS development.

1.1 Motivation

The management of web content using a CMS is often something on which a business depends (Bisson, 2003). Many large businesses have dedicated teams of content authors—usually within the marketing department—whose sole job it is to maintain the organisation’s website. However, the inability or unwillingness of content authors to use a CMS for user satisfaction reasons is likely to have a detrimental effect both inside and outside a business. Indeed “the success of a site often depends on content authors being able (and willing) to use the CMS” (Robertson, 2003, 2007).

Internally, a system which is frustrating or tedious to operate is likely to affect many aspects of a content author’s daily working life—particularly job satisfaction, productivity levels and general morale and attitude. The effects of these aspects are likely to worsen over time too, with content authors becoming increasingly frustrated with the same issues day-in day-out.

As a consequence of these internal issues, the external-facing website is likely to suffer from a range of problems. In particular, content is likely to suffer from a lack of updates, become outdated quickly or not be updated at all—thereby defeating the purpose of actually having a CMS. Furthermore, the site (as seen by its audience) will become “stale” and other businesses will have the opportunity to take a competitive advantage.

The sheer lack of literature and research into user satisfaction pertaining to CMSs is therefore particularly worrying and suggests a major imbalance in design priorities. This thesis is motivated by the desire to readdress the balance in CMS design, to promote user satisfaction and therefore better support sustained usage.

1.2 Aims, Collaboration & Intended Outcomes

The aims of this thesis are to a) show that user satisfaction is an important yet often overlooked factor in CMS design; and b) investigate how user satisfaction can be achieved in CMSs. This thesis will therefore focus on addressing the following three objectives:

• What factors determine user satisfaction in content management systems?
• How important is the overall notion of user satisfaction in content management systems?
• Are there any issues or constraints in achieving content management system user satisfaction in a commercial environment?

A case study will be carried out in collaboration with THE PROVIDER and THE CLIENT as a means of validating these objectives.

THE PROVIDER [...] have developed their own in-house CMS, THE SYSTEM, which is deployed on clients websites to allow them to maintain their sites themselves.

THE CLIENT delivers data, editorial, pictures and sounds in real-time to clients worldwide. They are one of THE PROVIDER’s largest clients and use THE SYSTEM CMS to maintain their website.

The secondary motivation for this thesis comes from THE PROVIDER who are looking to develop a “super-usable” interface for THE SYSTEM, with the aim of increasing their clients’ satisfaction and reducing technical support demands. Empirical evidence will be gathered from evaluation sessions with THE CLIENT to determine the factors of user satisfaction found to be particularly relevant in CMS design.

The intended outcome of this thesis is a clear definition of measures that have proven (by means of the case study) to elicit user satisfaction in CMSs. It is hoped that these measures will influence future CMS design.

1.3 Structure

Chapter 2 expands further on the notion of user satisfaction and investigates relevant literature. Contemporary trends and design methodologies are also discussed.

Chapter 3 documents the development of a case study which will be used to investigate the objectives of this thesis. The current THE SYSTEM system is also analysed.

Chapter 4 describes the evaluation carried out, analyses the results and raises issues and points for future development.

Finally, chapter 5 revisits the objectives of this thesis for explanation in light of the evaluation findings.

---

Chapter 2
Background

2.1 Usability & User Satisfaction

As outlined in the introduction, usability consists of three aspects: effectiveness, efficiency and satisfaction (Userfocus, 2007). All research, however, carried out so far into CMS usability has accounted for only one or two aspects of usability, either effectiveness or efficiency. These aspects focus on the functional criteria and quantitative metrics that a system must meet—that is, the functions provided by the system and a quantitative measure of how efficient these functions are to carry out. Studies such as these make “risky assumptions about correlations between usability aspects” (Frøkjær et al., 2000) and assume that by meeting the effectiveness and efficiency criteria that satisfaction will be assumed. However, this is not the case—Frøkjær et al. (2000) found that the three aspects of usability are in fact weakly correlated.

Therefore, the three aspects of usability must be considered individually—one cannot base conclusions on the measure of one aspect based on the results of the other two measures (Frøkjær et al., 2000). For example, Frøkjær et al. (2000) found that some users preferred using a less efficient system—even though it had the same functionality as other systems—because it was subjectively more satisfying to use.

There is therefore much motivation for looking wider than pure usability to the subjective sum of an interaction, including user experience, satisfaction and subconscious affective reactions relating to moods, feelings or attitudes (Zajonc, 1980)—to ensure that a system is not only considered usable but also satisfying to use. As a result the user values their interaction experience.

2.1.1 Creating “Value”

The importance of looking wider than usability is backed by Cockton (2004b) who argues that modern HCI is currently facing its final challenge: the quest for value (Cockton, 2004a). HCI must become value-centred to support the design of useworthy systems (Cockton, 2005a). No longer is it sufficient just to focus on usability—true notions of satisfaction are derived from the value that users can derive from their interaction experience. Usability cannot reside in generic definitions, it is always relative to the specific context a system is operated under (Cockton, 2004a).

Cockton therefore argues that the use of usability guidelines is very much based on “luck and magic” (Cockton, 2004b). It is essentially “luck” as to whether a usability guideline is still applicable once the context and purpose of a system is taken into account. Too many developers are ready to believe in “guideline magic”, based on statistical evidence when usability guidelines do strike it lucky—“they work when the right users interact with a system in the right way” (Cockton, 2004b). However, usability guidelines do not always work, and a usability problem in one usage context may not be one in another (Cockton, 2006).

Cockton’s concern about usability guidelines is echoed by Bevan & Macleod (1994). In addition, there is often no guarantee that a set of guidelines is exhaustive for all aspects of a system—particularly if a system has been tailored to its users through highly user-centred design methods. There is also often a conflict between guidelines—“changing an interface feature to be compatible with one guideline often makes it incompatible with another” (Bevan & Macleod, 1994)—and it is then difficult to reach a compromise between the benefits of each guideline. Furthermore, Bevan & Macleod (1994) argue that the effectiveness with which guidelines are applied depends very much on the skill of designers in interpreting and applying them.

However, one should be careful not to disregard usability guidelines completely—they do still offer a valuable source of information based on empirical psychological evidence and validated research. It is important that HCI practitioners do not base key design decisions solely on their presence—a mutual balance should be developed between guidelines and the psychology of creating value-based systems (Cockton, 2004b).

The two defining principles of value—quality in use and fit to context—are qualities of user satisfaction experienced during interaction. There are however outcomes and lasting impacts that endure beyond interaction (Cockton, 2006). This suggests that while the determinants of user satisfaction can occur during an interaction—through concepts such as ease of use and system responsiveness—there is much possibility and scope of influencing the long lasting effect of user satisfaction beyond the “moment” of experience. In an environment where daily CMS usage is required this is thought to be particularly important (Section 2.4).

2.1.2 Summary

The notion of user satisfaction is particularly evasive (Lindagaard & Dudek, 2003). There is no “magic pill” to achieve user satisfaction—it comes from the subjective sum of the entire interaction experience—and centres around the inherently subjective and emotive issue of delivering value in design (Cockton, 2004b). It is therefore especially important that designers attempting to maximise user satisfaction have a deep understanding of their users and the context the system will be used under. To this end this chapter focusses strongly on how CMSs are used in the workplace and the motivations and importance of achieving user satisfaction in their usage.

2.2 Content Management Systems

Many businesses recognise the need to manage and publish growing amounts of information on their websites and generally invest in a CMS to assist them in doing so. The large majority of CMSs are highly complex (Byrne, 2005b) and difficult to operate (Byrne, 2005a) commercial products. They are often “obtuse and complex, packed with many gratuitous features at the expense of usability and user experience” and written “by geeks, for geeks” (Veen, 2004). Examples of CMSs with particularly unwieldy, complex and intimidating interfaces are TYPO3 (Figure 2.1(a)) and Vignette (Figure 2.1(b)).

---

---

Such commercial “off the shelf” products are often developed for mass-market appeal and designed for an ideal, general world—a “one fits all” solution to suit all users—and are not tailored to the users who will actually be using them on a regular basis. As one CMS researcher puts it: “It’s like trying to sell everyone an average-sized shoe” (Byrne, 2005a).

In the same way that the majority of people will never find satisfaction in an “average-sized shoe”, CMS users will never reach real levels of user satisfaction if they are forced to use a “one fits all” system. “Not every business runs [in a standardised way] like a factory floor” (Byrne, 2005a), and it is therefore common sense that any “one fits all” CMS will experience friction when trying to “fit in” with the goals, environment and culture of the majority of businesses.

There is therefore considerable motivation behind employing tailored design methods when designing a CMS, to ensure that the resultant system fits in with the needs, requirements, business processes and workflow relevant for users who will be using it on a daily basis. It is accepted that a good system design is one that “fits” its context of use thereby helping to achieve value in the system (Cockton, 2005b).

2.2.1 Business Motivation & Constraints

There is a lot of motivation for increasing user satisfaction in a CMS, primarily to foster a good working relationship between the provider and the client. CMS providers are often offering their clients a CMS as a service and the relationship between the provider and the client is paramount. If a CMS is technically successful and is delivered to the client on time and on budget then it can be considered a success—however it can still be a failure if users are unhappy with the result (Zviran et al., 2006).

A CMS very much affects the “bottom line” of a business, and in heavily web-oriented businesses can mean the difference between success or failure. This is illustrated by Angeles (2006) with the following five-step chain of effects:

• Ease of use leads to satisfaction with CMS and sustained use;
• Sustainability leads to a richer information repository (content on website);
• Rich information repositories lead to more meaningful information mining;
• Rich information mining leads to more informed decision making (by site visitors);
• Better decision making leads to less money lost and more business opportunities.

In the end, it’s about money (Angeles, 2006)—user experience and satisfaction in a CMS can lead to distinguishable business results and increased sales.

CMS providers are often working on a tight budget assigned by their client—a set cost of implementing a CMS. Clients often do not appreciate the benefits that CMS user satisfaction can bring, and generally “make do with what they have” (Angeles, 2006) unaware that relatively simple changes to achieve user satisfaction could have positive effects on, for example, motivation and productivity.

As clients often do not appreciate these benefits they are unlikely to assign specific portions of their budget to the quest of achieving user satisfaction, when they feel it could be better spent on system functionality.

Providers can try to “evangelise simplicity in the workplace”—possibly assisted by success stories such as 37signals (Section 2.5)—however it is thought that if user satisfaction is taken into account as part of the traditional development process then providers have more chance of implementing user satisfaction as standard with the system. Therefore, there is much incentive for providers to be aware of factors which have proven to elicit user satisfaction in CMSs, and strive to meet these factors during functional development work, with user experience and satisfaction as a “keystone” in their development strategy (Angeles, 2006).

2.2.2 Usability Principles

The usability of a CMS is now considered equally as important as its functionality, as if content authors cannot easily use a CMS it will never be a success—regardless of how powerful it may be (Robertson, 2007). Based on research of the field Robertson (2007) has developed eleven usability principles (Appendix A) which will be used to inform case study design decisions. However, these decisions will not be made solely on the suggestions of Robertson’s usability principles.

2.3 Determinants of User Satisfaction in IT Systems

The first major paper into end-user computing satisfaction was conducted by Doll & Torkzadeh (1988). Based on usability research at the time, the authors generated a 40-item questionnaire to measure end-user perception on a variety of measures. It is important to note that this study was conducted before substantial usability research as conducted by Nielsen (1993) was commonplace. The questionnaire was administered to 618 end users from a variety of industries and positions, and participants were asked to complete the questionnaire for an application of their choosing. The results found five determinants of user satisfaction: content, accuracy, format, ease of use and timeliness. The authors developed a 12-item model (Figure 2.2) to provide a common framework for analysis of user satisfaction.

---

---

For clarification, these criteria have been defined by Zviran et al. (2006) as follows:

Content:

User trust in site-provided information

Accuracy:

Precision of site-provided information

Format:

Clarity of information presentation

Ease of Use:

Subjective impression of user

Timeliness:

Temporal relevance of information

The model proposed by Doll & Torkzadeh (1988) has since been widely accepted as a standardised instrument for measuring user satisfaction (Doll et al., 1994; Doll & Torkzadeh, 1991)—particularly as it was the first model to incorporate the ease of use factor (Xiao & Dasgupta, 2002). In a later study, Doll et al. (1994) carried out further confirmatory factor analysis to estimate the reliability of individual factors and of the overall instrument. The results further supported their initial study, suggesting that the factors of content, accuracy, format, ease of use and timeliness can be used with confidence as a measure of user satisfaction. “The large number of organisations and variety of applications surveyed [also] support the generality of the findings to any computer-mediated system” (Doll et al., 1994).

Since the development and validation of the user satisfaction model by Doll & Torkzadeh (1988), there has been little research on determinants of user satisfaction in web-based systems. Since Doll & Torkzadeh’s original study in 1988 there have been significant changes in IT and exponential growth of the Internet—it is therefore unwise to apply their model to contemporary studies of web-based systems and assume it is still reliable and relevant. The only other study of user satisfaction in web-based systems was conducted by Xiao & Dasgupta (2002) who found that all aspects of the model other than question C4 (Figure 2.2) are still applicable. This study was by no means extensive and did not look into any other measures which may exist—its aim was distinctly to validate the pre-existing model.

There has been little further research on “realigning” these five factors of user satisfaction with modern day web-based systems. A preliminary study (Zviran et al., 2006) has attempted to expand the user satisfaction instrument (Doll & Torkzadeh, 1988) by incorporating web usability principles: personalisation, structure, navigation, layout, search and performance—however there has been no work to validate the effectiveness of these factors in achieving user satisfaction, and it is not clear how—if at all—these will be relevant for CMS design.

Achieving user satisfaction in a CMS is therefore a rather unknown area of research. There is a lack of extensive studies refining Doll & Torkzadeh’s five-factor model in relation to web-based systems. Furthermore, given that CMSs are a somewhat specialised area of web-based systems, it is necessary to investigate each of these five factors and only apply them to the case study development if they are actually relevant to CMSs. Each of the five factors are discussed below with respect to the definitions derived by Zviran et al. (2006). It is important to clarify at this stage that the following discussion is based on the Doll & Torkzadeh’s validated user satisfaction model and their validated definitions by Zviran et al. (2006).

2.3.1 Content

The information provided in a CMS will come directly from the site that the CMS is managing. There is no scope for this information changing between the end-user site and the CMS interface. The introduction of trust issues is therefore highly unlikely and this factor is thus thought to be irrelevant for CMS design.

2.3.2 Accuracy

The definition validated by Zviran et al. (2006) centres around the precision of information provided on the site. Similar to the content factor above, there is no scope for the precision and accuracy of information changing between the end-user site and the CMS interface. Therefore, this factor is also thought to be irrelevant for CMS design.

2.3.3 Format

Format is thought to be relevant for CMS design. Users should be able to use the system, browse the site and find and edit content easily, assisted by the clear, logical formatting of information. This itself draws many parallels with ease of use. Furthermore, the definition derived by Zviran et al. (2006) centres around clarity—I personally believe this implies that language should also be considered. Users should be able to understand clearly and consistently the information and terminology used in a CMS. This is also one of the usability principles suggested by Robertson (2007) (Appendix A). I therefore propose the redefinition of format to format and language.

2.3.4 Ease of Use

Ease of use is also thought to be especially relevant in CMSs, as corroborated by literature of the field: “If staff, particularly [content] authors, cannot easily make use of the CMS, then the system will never be a success, regardless of how powerful it may be” (Robertson, 2007). Furthermore, ease of use can help to encourage sustained usage in a workplace environment where a systems’ usage is mandatory—“if users are confronted with an easy-to-use technology, they will be more likely to continue using it” (Angeles, 2006). Taking considered advice (Section 2.1.1) from CMS usability principles (Appendix A) and web usability guidelines (Nielsen & Loranger, 2006; Krug, 2005) will help to satisfy this factor.

2.3.5 Timeliness

A study by Abdinnour-Helm et al. (2005) showed that timeliness is not particularly relevant when positioned in a web-based context. Furthermore, the temporal relevance of a piece of information is thought to be simply irrelevant to the scope of this thesis and therefore not applicable to a CMS. However, timeliness is still relevant in the context of the speed at which the user can operate the CMS. Working under the demands of updating content to strict deadlines, users will wish to accomplish tasks using the CMS as efficiently and as quickly as possible. Therefore, I propose the redefinition of timeliness into two components: efficiency* (the number of steps taken to complete a task); and speed (the length of time taken to complete a task).

* An important distinction needs to be made between efficiency as one of the three general aspects of usability and efficiency as defined here. The definition of efficiency defined here refers to a short sequence of steps to complete a task, with minimal mouse clicks and the interface of the system not “getting in the way”. Speed (a component of the efficiency aspect of usability) is now considered separately under its own user satisfaction factor.

2.3.6 Summary

This section has considered the five identified and verified factors of user satisfaction generalised for IT and web-based systems. Format (redefined as format and language), ease of use and timeliness (redefined as efficiency and speed) were considered relevant enough to CMSs to warrant their further inclusion in this study. There are however likely to be more measures of user satisfaction particularly relevant to CMSs, which are investigated in the following section.

2.4 Possible Determinants of User Satisfaction in CMSs

User satisfaction is particularly relevant to CMSs because of the day-to-day use the system will receive by the same people. Allowing the user to “enjoy” using the system will alter their behaviour towards it which, in terms of content authors, may encourage them to produce better content. Indeed “getting authors to produce the content [a business] needs means changing their motivations about the goals of their content, and the value they are trying to deliver [to the site audience]” (Boiko, 2005). Novel, interesting and enjoyable systems are proven to help improve users motivation (Hassenzahl, 2004) and can “donate unexpected value” by delighting users’ with their capabilities and user experience (Cockton, 2004b).

The following sections discuss research carried out into four concepts, which are felt to be particularly relevant to users of a CMS who have to use the system in the workplace as part of their daily job.

2.4.1 Funology

Funology reflects the move in HCI studies from focussing on pure usability to a wider set of concerns to do with fun, enjoyment, aesthetics and the overall experience of use (Blythe et al., 2004).

“The idea of designing for enjoyment rather than simply designing to reduce frustration is a relatively new one for HCI” (Nielsen, 2004) and extends beyond creating a system that simply allows users to get their tasks accomplished quickly and easily. In a recently study of 2000 people (Allison Van Dusen, 2007) 59% said that their job was a leading source of stress, showing reasonable impetus for reducing frustration by creating more “harmonious” interactions with systems that users will enjoy using. As previously mentioned, this could encourage content authors to produce better content, but it is also likely to make users more tolerant of system errors and annoyances and increase morale and productivity. Indeed, it is thought that enjoyment predicts efficiency, and people are generally at their happiest when they are at their most productive (Taylor, 1999).

The enjoyability factor of a system stems from how the system resonates with the user, and different users resonate to different things because they have “certain needs, desires and intentions, a social and cultural history and position” (Hummels et al., 2004). There are several “ingredients” that result in a resonant interaction including: usability, human skills, context of use, aesthetics of interaction and engagement. Due to the personal character of resonant interaction, “developers should involve people for whom they are developing [for] right from the start” (Hummels et al., 2004), thereby suggesting that user-centred design approaches are most suited for ensuring resonant interaction and therefore enjoyment. Aesthetics is discussed in depth in Section 2.4.3.

Enjoyment also helps to encourage cognitive absorption—a state of deep involvement and engagement with software—which helps to shape beliefs and perceptions influencing usage behaviour (Agarwal & Karahanna, 2000). CMSs “represent a substantial investment” for most businesses, “however their value is only realised when utilised by their intended users in a manner that contributes to the stratetic and operational goals of the business” (Agarwal & Karahanna, 2000). Allowing users to become involved and engaged with a system is therefore very important to ensure the system is used in a constructive manner—that is, content authors produce good content. Understanding these user reactions and relationships with technology is currently of particular interest in HCI.

In a workplace environment where CMS use is mandatory, the notion of achieving enjoyment is especially important to counteract any effects of frustration with the CMS. Causes of user frustration include software crashes, unclear error messages and confusing over-complicated interfaces (Preece et al., 2002). Research by Lazar et al. (2006) shows that users lose as much as a half of their working lives through frustration with such issues. Frustration at work can harm interpersonal relationships and strongly influence user moods—in relation to all aspects of work, not just interaction with the system causing frustration (Lazar et al., 2006).

The main cause of frustration is the interruption of the user from performing their task. Users have to respond to something “unexpected and unclear that interferes with their task goals” (Lazar et al., 2006). Preventing the user from completing their task goals wastes time—which is often limited for workplace users (Lazar et al., 2006)—and can produce the aforementioned personal issues.

Furthermore, it is not the case that the user can just “dismiss” the frustration they have encountered—Lazar et al. (2006) suggests that frustration has a very much ongoing effect in the workplace, particularly encouraging lower levels of job satisfaction (Murrell & Sprinkle, 1993). It is thought that prolonged job dissatisfaction can also lead to an unfavourable attitude towards the employer (Murrell & Sprinkle, 1993), which can raise obvious issues in terms of continuing employment.

2.4.2 Motivation & Job Satisfaction

There are many organisations who “employ people who view computers as a necessary evil: temporary fixes until something better comes along” (Brown, 2005). Research has shown that these types of mandatory users gain satisfaction from the “subjective sum of the interaction experience”, not just the degree to which the system enhances productivity (the effectiveness and efficiency aspects of usability) (Lindagaard & Dudek, 2003). It is therefore important that these users feel motivated to use a computer-based system, and it appeals to them beyond simply being quick and easy to use. In a workplace environment this is likely to have far reaching effects into the area of job satisfaction. Indeed “usable software increases employee satisfaction; difficult to use software reduces motivation and may increase staff turnover” (Bevan & Macleod, 1994).

The most conclusive study into the link between user satisfaction and job satisfaction was carried out by Ang & Soh (1997). Their study of people who use a computer system as an integral part of their jobs found that user satisfaction with such systems is a “very sound indication” of overall job satisfaction. One of the main determinants of job satisfaction found in the study was task variety. Users performing more administrative tasks—orderly tasks, with little variety and much repetition—were found to have lower levels of job satisfaction than those performing more research-related tasks with a higher level of variety. The range of tasks carried out in a CMS are quite limited for content authors and would fall under the bracket of administrative tasks. Given this constraint in achieving job satisfaction—and thereby user satisfaction—it is important that CMSs make these repetitious tasks as pleasant and motivational as possible. Furthermore, Ang & Soh (1997) found a positive relationship between users productivity levels and job satisfaction.

As well as this link between user and job satisfaction, user satisfaction also increases a user’s motivation to use that system. Even if the use of a CMS is mandatory, users will be more tolerant and accepting of it if they feel motivated to use it.

A study by F. Davis et al. (1992) reports an impetus for designing systems that are both more useful and more enjoyable, in order to increase their acceptability among potential users and increase motivation to use the system. Furthermore, the perceived usefulness and ease of use of a system will help to instill a positive attitude between the user and the system, thereby producing a behavioural intention to use the system with which they associate a positive affect (S. Davis & Wiedenbeck, 2001). This forms the basis of the technology acceptance model (Figure 2.3) (F. Davis, 1989)

---

---

It is also thought that determinants such as the first impression of a system and its aesthetic qualities will assist in motivating user interaction and creating general system satisfaction. These are discussed in the following section.

2.4.3 Aesthetics

The aesthetic/usability effect describes a phenomenon in which people perceive more-aesthetic designs as easier to use than less-aesthetic designs—whether they in fact are or not (Butler et al., 2007).

A major component of the aesthetic/usability effect is the first impression that a system gives to its users, which builds on the psychological theory that users “emotional responses are immediate and precede intellectual ones” (Lindagaard & Dudek, 2003). In the same way that your first impressions count when meeting other people, your first impressions of a user interface often determines your attitude and interaction behaviour with that system, and aesthetically pleasing interfaces can help to instill a positive relationship between the system and the user. Zajonc (1980) found that such emotional responses can be made in a mere five milliseconds and occur in the vast majority of users.

A further study by Hiltz & Johnson (1990) found that these first impressions of a system are in fact quite lasting—“if computers were perceived initially as difficult to use, users were more likely to express dissatisfaction with the interface of the system after four months of use”. An attractive user interface is therefore not just a visual element of design, but has psychological influences on a user’s entire interaction experience and “may influence long term attitudes towards [a] system” (Tractinsky, 1997). This is an area touched on by Cockton (2006) to create value in using a system which lasts beyond the “moment” of interaction (Section 2.1.1).

The aesthetic/usability effect is particularly relevant for everyday users such as content authors, who use a CMS to perform rudimentary content management tasks as part of their everyday working life, using the same system day-in day-out. Cooper (2004) argues that everyday users such as these “should not have to acquire computer literacy to use computers [as part of their job]”. It is therefore considered important that a CMS “lowers the barrier to entry” (Brown, 2005) for such users by looking easy to use, even if it is not inherently usable. Furthermore, the barrier to entry can be lowered be presenting a simple non-intimidating “first look” interface to the user. Much research has been carried out into the apparent usability afforded by aesthetically-pleasing interfaces (Kurosu & Kashimura, 1995; Tractinsky, 1997; Angeli et al., 2006; Norman, 2003). In particular, a study by Kurosu & Kashimura (1995) found that the apparent usability of a system was more strongly affected by an aesthetic user interface than the actual inherent usability of the system. It is also thought that aesthetically pleasing interfaces help to promote creative thinking and problem solving (Butler et al., 2007), something that a frequent user, such as a content author, may be lacking through repetitious use of the same system on a daily basis.

Another study was carried out by Tractinsky (1997) which validated the findings of Kurosu & Kashimura (1995). This study also compared how the aesthetic perceptions differ between cultures (Japanese and Israeli) and the preliminary findings found that the Japanese—known for their modern aesthetic tradition particularly in high-tech products—found aesthetically pleasing interfaces more apparently usable than Israelis—probably better known for their “action” orientation (Tractinsky, 1997) and not high-tech products. A further study (Desmet, 2004) actually found that Japanese people were more susceptible to admiration, satisfaction and fascination than other cultures. This raises an important point that the aesthetic/usability effect is open to interpretation between different users, and care should therefore be taken that the aesthetics of an interface do not favour one user group or alienate another.

However, designers sometimes have a “tendency to neglect usability in favo[u]r of aesthetics” (Nielsen, 1993) or overemphasise the aesthetic elements of an interface to the point of degrading usability (Tractinsky, 1997). It is therefore also important that a balance is struck between aesthetics and usability—“products should be apparently usable as well as inherently usable” (Kurosu & Kashimura, 1995).

2.4.4 Learnability & Documentation

It is generally considered that documentation is advantageous in a CMS to assist new users, minimise training required and support self-sufficiency (Appendix A). Gemoets & Mahmood (1990) found that user satisfaction was “considerably enhanced” and “strongly influenced” by the documentation provided in a system, providing it is “well organised and provides formal instructions”.

Documentation can also help to reduce frustration. It empowers users to “help themselves” and become self-sufficient, which Bessière et al. (2006) demonstrated as helping to reduce frustration. Empowering the user to tackle obstacles they face with a system themselves—rather than relying on their peers in the workplace—is also likely to bring a sense of achievement and encourage the user to become more tolerant with issues they encounter with the system in the future.

Furthermore, good quality user documentation may have a dramatic and continuing impact on user satisfaction (Doll & Ahmed, 1985)—especially important for a CMS which must support sustained usage and maintain user satisfaction over this time.

2.4.5 Summary

After consideration of some more contemporary literature, often related to mandatory computer usage in the workplace, the following four additional factors have been selected which are likely to influence user satisfaction in CMSs: funology, job satisfaction and motivation, aesthetics and learnability and documentation. There may be some overlap between these four factors and the four factors previously derived from user satisfaction studies of IT systems—however they are considered distinctive enough to be considered as eight separate factors. It will become clear if they are in fact correlated during the evaluation and results analysis.

2.5 Contemporary Trends

This section discusses contemporary web design trends and methodologies which are particularly relevant for enhancing user experience and satisfaction.

Many of the trends presented in this section are from the book Getting Real—the business, design, programming and marketing philosophies of 37signals (37signals, 2006). The Getting Real book contains details of their sometimes unconventional look at building web applications and has come to define a new era of web applications, “Web 2.0” (Section 2.5.3). The book gives a first hand look at how working methods are changing in the web industry. 37signals’ philosophies are also proven—their range of web-based applications achieve notably high levels of user satisfaction through their “small, simple and efficient” systems (Angeles, 2006) and are subject to much critical acclaim. Following aspects of the Getting Real philosophy—and being aware of other contemporary trends—during case study development is an excellent way of ensuring this thesis takes a fresh, contemporary look at CMS development.

2.5.1 Priorities & Feature Selection

Getting Real focusses on building less and leaving out anything that is not absolutely essential, delivering just what customers need in less time and with less complexity. This is the particularly key advantage of web applications over desktop applications—desktop application manufacturers need to sell a new version of their software every year and have to justify the expense of selling a new version by adding new features. This is inflating for the sake of inflating—driven by business needs and not user needs—leaving the application bloated and full of surplus, often unwanted features. This spiral of complexity is known as “feature creep” and is a result of developers generally “not noticing when more options make a [system] less usable” (Surowiecki, 2007). Indeed “more power means more complexity, so each feature inherently reduces the usability of a [system]” (Byrne, 2005b). A perfect example of this is Microsoft Word, which now has 31 toolbars and more than 1500 commands (Surowiecki, 2007)—many of which go unused by the large majority of users.

However, it is not only developers who are to blame for “feature creep”—users often unwillingly induce and indulge in this spiral of complexity themselves. Although users find systems overloaded with features unmanageable, they also find them attractive (Surowiecki, 2007). “Consumers give more weight to a product’s capability benefits and less weight to a product’s usability [...] despite the fact that a product’s usability strongly influences their satisfaction with the product” (Angeles, 2006).

Marketeers are very much aware of this, and so CMSs are often sold under the promise of, for example, “New Improved Functionality!” and “101 NEW Features!” to lure and tempt customers with features they think they will use and sound useful, but they most probably won’t touch at all—users are being “wowed by features over usability” (Angeles, 2006) and a technical economy is very much driving the design of the system (Cooper, 2004). “It is only once [the user uses] the system that [they] realise the virtues of simplicity” and “feature fatigue” sets in (Surowiecki, 2007). The user becomes frustrated by the plethora of features available to them, therefore finding the system difficult to use. “Feature fatigue” is difficult for users to avoid since people are not generally good at predicting what will make them happy in the future (Surowiecki, 2007). It is therefore down to the developer to ensure that all features of a system are essential to the user. The issue of selling the system and luring in customers must be left as purely a marketing exercise, not an implementation issue for developers to “fix” by introducing unnecessary features. Unfortunately such practices are commonplace among CMS vendors, who compete primarily on features and functionality in quite an immature marketplace (Byrne, 2005a).

37signals (2006) summarise the need for fewer features with the argument that it’s better to “build half a product, not a half-assed product”. It is often the case that if every feature idea is implemented, the resulting system has many average features. A better approach is to include half of the features, but spend your time implementing these features to an excellent quality. The notion of “It Just Doesn’t Matter” has been coined by 37signals (2006) to encourage developers to stick to only essential features. “This embodies what makes a product great—figuring out what matters and leaving out the rest”. This leaves the user free to concentrate on the task at hand and “they’ll achieve productivity they’ve never imagined” (37signals, 2006).

2.5.2 Interface Design & Simplicity

The “less-is-more” philosophy naturally follows through from feature selection to interface design, and as Cooper (2004) suggests “no matter how cool your interface is, less of it would be better”. The Getting Real approach to this issue involves designing the interface first—the real screens that people are going to use—and works “backwards” from this actual user experience (37signals, 2006). Many systems are developed with a program-first mentality, where the programming and logic is completed first with user interface then layered on top. This is a bad idea since programming is the most heavyweight and expensive component of the entire development (37signals, 2006). Designing the interface first brings three advantages.

Firstly, design is flexible—interface designs are much easier to change or scrap completely than programming logic—and programming first can severely restrict the functionality of the user interface.

Secondly, it allows for easy rapid prototyping to “flush out ideas that don’t work from the outset” (37signals, 2006). Interface designs produce “real, tangible prototypes” that when presented to the real users of a system will lead to real reactions and the truth about how satisfied users are with the system. This “rinse and repeat” approach ensures that prototypes are developed based strictly on user needs and feedback (37signals, 2006).

Thirdly, “the interface is your product” (37signals, 2006)—it is a users only point of direct interaction with the system and the whole experience of using a system is thought to be “the only thing users care about” (Merholz, 2007). Designing the interface first ensures that the original goals of the system are always at hand—does it make sense?; is it easy to use?; does it solve the problem at hand? (37signals, 2006)—ensuring you get the answers to those questions sooner rather than later on, at a stage where the system could be heavily entrenched in programming logic and constraints, to the detriment of user needs.

Radiant (Figure 2.4) and a custom-made CMS for AIGA.org (Figure 2.5) are good examples of CMSs with a strong focus on simple, aesthetic and effective interface design.

---

---

---

---

Keeping these goals in the forefront of the design process also helps to ensure effective information architecture (the arrangement and organisation of information) and interface/navigation design (the presentation and navigation through that information in a way that facilitates understanding) (Garrett, 2002). For example, frequently used features will be easy to find (Marick, 2007) and users won’t need to “shuffle off to various places in the interface to accomplish simple and common tasks” (Cooper et al., 2007).

2.5.3 Web 2.0 & AJAX

“Web 2.0” is a phrase coined by O’Reilly Media in 2003 to refer to the perceived “second generation” of web services, which embodies the transition of websites from information sources to sources of content and functionality, introducing the web as a platform to run web applications. The exact definition is open to much debate—and it is often considered a “buzzword”—but it centres around concepts of usability, user-centred design, joy of use and simplicity.

A key part of introducing the web as an application platform is to bring the user experience of a website closer to the immediacy of working within a desktop metaphor environment, where applications respond quickly and users can interact in a very fluid manner (Dürsteler, 2005). Web applications no longer operate strictly on a click-and-refresh basis on a page-by-page model, but are more interactive applications similar to the way a user may interact with a desktop-based application such as Microsoft Word. The technology that provides this functionality is AJAX (Asynchronous JavaScript and XML).

An area where AJAX could be particularly useful in a CMS is to cut the number of page refreshes required, making it more efficient to both frequent and infrequent users - in fact this is one of the recommended CMS usability principles (Appendix A). CMS vendors are only recently beginning to turn to AJAX-type technologies to overcome the constraints and trade-offs of traditional web applications—such as confusion about the use of the “back” button, having to refresh every screen and the inability to drag-and-drop items. It is thought that “[traditional click-and-refresh web applications] just feel slower” (Byrne, 2005b) and AJAX can generally improve usability leading to an increase in productivity (Stafford, 2006).

---

---

Widespread use of AJAX within a web-based system does however introduce a desktop-on-website metaphor, which can break users expectations of how they should interact with a website and how it should look and behave (Higgins, 2007). This can lead to users treating a web application as if it were a desktop application—for example, closing a window and expecting the contents to be saved automatically, or expecting to be prompted to save—and this may have a negative impact on learnability, pleasantness of use and adoption (Higgins, 2007). It is for this very reason that the usability and appropriateness of AJAX is currently a hot topic of debate (for example, 9rules (2005); Mickiewicz (2006); MacManus (2007)), but it is generally thought that a focus on AJAX to increase usability is “a very good thing” (Byrne, 2007) when implemented carefully and not used for purely aesthetic reasons. It is also thought that AJAX can help in achieving the notion of funology and enjoyment, which were discussed in (Section 2.4.1).

2.6 Summary

This section has stressed the importance of looking more broadly than the traditional view of usability to notions of experience and value in interaction design. Four factors of user satisfaction relevant for CMS design have been suggested from derivation of traditional measures—format and language, ease of use, efficiency and speed. A further four factors have been proposed which are thought to be relevant to the context of CMS usage—funology, motivation and job satisfaction, aesthetics and documentation. Contemporary trends relevant to this thesis have also been discussed which are thought to be important in the implementation of a CMS aiming to achieve user satisfaction.

Based on these findings the next section documents the design, prototyping and development of a new interface for the THE SYSTEM CMS. This will be used as a case study to empirically validate the importance of the eight factors arrived at through this background research.

---

Chapter 3
Case Study Development

3.1 Current System Analysis

An informal interview was carried out with the Technical Director of THE PROVIDER, to analyse the existing THE SYSTEM CMS. During this interview I asked a range of questions concerning the design, development, user interface, user feedback and evaluation of the current system.

[The exact interview findings cannot be released publicly. However, the main issues during development were: restricted access to stakeholders and users during system development; no formal requirements capture process; and no prototypes were given to users until the late stages of development.]

To summarise, while THE SYSTEM was designed specifically for THE CLIENT its main web interface was not designed for a specific user type. As a result, content authors—the most common type of user—find the system difficult and complicated to use.

3.2 Design

3.2.1 Methodology

Based on the findings of background research, it has been decided to employ user-centred design methods for the case study development. This will ensure that real-world users are taken into account at every stage of system development (Garrett, 2002) which will help to minimise error and promote productivity and performance gains (Noyes & Barber, 1999). Furthermore, it will ensure that the system fits its real-world context of use thereby allowing users to derive maximum value from using the system. Users who are consulted at early stages are also thought to have less antagonism towards the introduction of a new system, particularly in a workplace environment (Gould & Lewis, 1985; Greenbaum & King, 1991).

The appropriateness of user-centred design methods for achieving user satisfaction is also corroborated by several studies. Zviran et al. (2006) argues that “the better a web interface fits the user’s preferences, the higher would be the value and satisfaction attributed to the [system]” and Ang & Soh (1997) found that for “good” systems to be developed for commercial usage users must be “actively involved in the analysis, design and implementation of the system”. Baroudi et al. (1986) found a strong relation between user involvement in system design work and the user satisfaction derived from using the finished system, and (Keil & Carmel, 1995) found that employing user-centred methods generally increases the probability of a project’s success.

User-centred design methods can however be difficult to put into practice in a commercial environment (Trenner & Bawa, 1998) for a range of practical and political reasons, mostly related to gaining access to real users. Specifically, “people and companies don’t always appreciate the benefit it will bring” (Simpson, 1998) and therefore often take exception to providing ready access to users within their business. This issue is particularly prevalent amongst technically-minded developers who often feel as though the designer is encroaching on their “turf”—as was experienced during the original THE SYSTEM development—and dismiss the importance of usability. Many people think “usability is just common sense” (Kaderbhai, 1998).

A framework for user-centred design has been developed by Garrett (2002) consisting of five “planes”: strategy, scope, structure, skeleton and surface (Figure 3.1). Garrett’s user-centred design framework also shares many parallels with Cockton’s notion of a value-centred design framework (Cockton, 2005a)—in particular, involving users at preliminary stages to identify usage contexts, and following an iterative design and prototyping process. Garrett’s framework will be followed throughout case study design and prototyping activities to ensure that real-world users are considered throughout.

---

---

Strategy and scope are discussed in the following two sections; structure, skeleton and surface are taken into consideration throughout the prototyping section.

3.2.2 Strategy

The first stage of user-centred design is the most abstract and concerns defining objectives, goals and user needs for the system. A clear explicit understanding of what THE PROVIDER and THE CLIENT require is essential—and the more clearly this can be articulated, the more precisely design choices can be adjusted to meet the goals of the system (Garrett, 2002). Defining the strategy involves formulating objectives, success metrics and ascertaining user needs*. This section will also analyse and evaluate the existing system to help define the strategy for development.

* User-centred design also involves formalising this strategy into a strategy document to circulate to all project participants—however this is unnecessary as all user-centred work is being carried out by myself.

3.2.2.1 Objectives & Success Metrics

THE CLIENT’s primary objective is to have a system which is easier and quicker for content authors to use, thereby ensuring that THE CLIENT’s website is kept up-to-date. The CMS is also used both in the UK and US to send bulletins to high-profile paying subscribers worldwide, and it is therefore vital that the CMS allows these bulletins to be sent correctly and on time. Through discussion of strategic objectives with the Marketing Director and Marketing Manager at THE CLIENT it was found that they were both keen to “roll out the CMS to more junior members of staff without worrying”, and the easier the system is to understand and learn then the better this objective can be met. This project will be considered a success if THE CLIENT can accomplish their tasks with the CMS in less time and with less frustration, and feel happy about giving access to the CMS to members of staff with little or no prior training.

THE PROVIER’s primary objective is to develop a super-usable interface for their THE SYSTEM CMS which is tailored to the needs and requirements of THE CLIENT. This should dramatically reduce the amount of support THE PROVIDER need to provide to THE CLIENT in using the CMS, and will define the success of this project from THE PROVIDER’s point of view.

3.2.2.2 User Needs

Before gathering user needs it is important to define exactly who the user or users are through user segmentation and personas. For this case study I am focussing on the primary users being content authors at THE CLIENT, and secondary users being staff at THE PROVIDER. The needs of both parties are important, however priority will be given to THE CLIENT’s needs where they are not in conflict with constraints or technical requirements set by THE PROVIDER.

Based on discussions with THE CLIENT two personas (Figure 3.2) have been developed to represent the collective needs of content authors. Discussions were held with several content authors and the findings were distilled into two distinct types of user. These personas will help ensure that “users are kept in mind during the design process” (Garrett, 2002) and is it generally more manageable to relate design thoughts to two personas rather than a greater number of real people.

---

---

3.2.2.3 Current System Analysis

To help ensure that any issues with the current THE SYSTEM system do not progress into the new version, it is important that the current system is evaluated. Issues known to content authors may be assumed tacit knowledge during an interview and not verbalised, or the user may not be aware of a more usable way to accomplish a difficult task so may not feel qualified to bring it forward for consideration. Content authors have also been using the current CMS for approximately 18 months, and during this time they will have learned how to work “in harmony” with the system’s idiosyncrasies. Users are therefore unlikely to volunteer all issues they encounter with the system during an interview, as they will be used to working in this manner on a regular basis, when a quicker, easier, more usable way to accomplish the same task could be introduced. It is also a good opportunity to look at the current system from an outsiders view based on contemporary trends and both CMS- and web-based usability principles.

Figures 3.3–3.7 contain annotated analysis of the user interface and workflow of the current THE SYSTEM CMS from the point of view of the two content author personas and myself as an outsider to the system. Positive points have a green arrow, negative points have a red arrow and references to CMS usability principles (Appendix A) are given in square brackets.

---

---

---

---

---

---

---

---

---

---

Analysis of the current THE SYSTEM system has identified a number of usability issues which, once fixed—in conjunction with user-centred design—it is hoped will help reduce frustration and increase user satisfaction. These issues are not surprising, given that the Technical Director at THE PROVIDER agreed that “five minutes thought” went into the user interface. The specific issues can be summarised as:

• Users are presented with too much technical implementation-level information which is not human-readable, user-friendly or appropriate for content authors;
• The CMS does not afford a consistent interaction metaphor—“pages” are referred to interchangeably as “files”, “items” or “pages”;
• The interface has layout/colour issues which make certain features difficult to use;
• The navigation is not consistently task-based, vital icons and buttons are often very small, and links and icons/buttons are used inconsistently;
• Many common tasks require a large number of mouse clicks to accomplish;
• Error and notification messages are often not human-readable;
• Javascript alert boxes are used throughout for confirmation and notification, despite generally signifying error conditions and forcing the user to make more mouse clicks; and
• Technical language is often used without explanation for first-time or novice users.

3.2.3 Scope

The second stage involves taking what THE PROVIDER and THE CLIENT want and working out how to satisfy these objectives in terms of functionality.

3.2.3.1 Defining the Scope

Before gathering requirements it is important to define the scope for a project—that is, what will and will not be built. Similar to the concept of “feature creep”, “scope creep” encompasses a developer taking on additional requirements, which individually may not seem like much extra work, but when put together produce “a project rolling away out of control, blowing past deadlines and budget estimates on its way toward an inevitable final crash” (Garrett, 2002).

The first stage of scoping—“establishing the boundary for investigation” (Sutcliffe, 2002)—has been defined by THE PROVIDER: this case study just focusses on the web interface of THE SYSTEM as used for THE CLIENT’s content management. Other interfaces previously mentioned, and the Firefox toolbar, are outside of the scope of this case study development.

In practice scoping a user-centred design project is “rarely an easy process since users often don’t know what they want” (Sutcliffe, 2002). However, as this case study seeks to redesign an existing problematic system “scoping is less critical as the observed problem defines the initial scope for investigation”—therefore requirements gathering should be largely “problem-initiated” (Sutcliffe, 2002). However, in view of the fact that requirements gathering for the current system was non-existent, it is important that a combination of both problem-initiated and traditional requirements capture techniques are used.

3.2.3.2 Requirements Gathering

Basic requirements were gathered through an informal interview process with both THE PROVIDER and THE CLIENT at their respective offices. Despite usability designers often facing a “fight for survival” (Trenner & Bawa, 1998) when gaining access to real users, I had no such issues as THE CLIENT have signed onto the fact that the results of this development work will directly benefit them.

The interviewing method of requirements gathering was chosen since it is relatively quick and easy and allows discussion and clarification of issues at the time of gathering. Interviews were carried out “on location” as “ people tend to be more confident and open when on their home ground” (Sutcliffe, 2002). As interviews can suffer from users not being able to volunteer their tacit knowledge (Sutcliffe, 2002) it was decided to use two other elicitation methods for THE CLIENT—observation and user participation—to gain a richer understanding of the current issues and their derived requirements. Brief observation was carried out for over an hour, with two users, in which time users used the CMS in their normal manner while being observed and voice recorded by myself. Users were asked to verbalise any particularly prevalent issues which may be undetected by myself as a relative newcomer to the system. As the observation was relatively short, users were asked to keep a notepad by their computer and use the system as they normally would over a week, writing down any further issues as and when they occurred. It was hoped that this combination of interviews, observation and user participation would elicit as many traditional and problem-initiated requirements as possible in a relatively short time period.

Further to eliciting system requirements, an issue specific to CMS design is to gain an insight into businesses “corporate processes” (Brown, 2005) to prevent a CMS from “working against” the established functions and workflow of the business. An understanding of how THE CLIENT works and functions is particularly important as user-centred design is “more than simply presenting design solutions for ratification; it requires consultation with users in their own language and terms” (Eason, 1995). An ethnographic study would have been ideal to elicit this kind of qualitative data, but is out of the scope of this thesis. Therefore, THE CLIENT were discussed at length with both the Account Manager at THE PROVIDER, who has dealt with THE CLIENT for over a year, and directly with the Marketing Manager at THE CLIENT prior to the requirements elicitation session.

3.2.3.3 Functional Specification

Appendix B lists the functional requirements gathered and derived from the requirements gathering process.

3.2.4 Summary

This section has clarified the objectives, success metrics, scope and requirements of the new system. With this knowledge, prototypes can now be developed with reference to the structure, skeleton and surface “planes” of user-centred design.

3.3 Prototyping

There are two requirements for successful prototyping: to understand what is wrong and how to improve (achieved through user feedback); and a good starting point (Dix et al., 2004). If you start with a bad design concept and iterate this design through prototyping, the result may just be a neater and more attractive version of the bed design idea. This problem—known as “local maxima”—is overcome by initially developing several design ideas (Dix et al., 2004) and taking proven usability guidelines and advice into consideration when developing design ideas. Creating several design ideas also ensures that the designer doesn’t get emotionally attached to a single idea—which may not be favoured by the user and will have to be “thrown away”—and will be more open to considering the best features offered by each to merge into the next design iteration. It is also better to have frequent small evaluations of prototypes rather than fewer larger evaluations (Nielsen, 2001) to flush out design ideas that don’t work from the outset quickly.

Prototyping for the case study development consisted of three iterations. Each iteration was evaluated by both THE PROVIDER and THE CLIENT and the design decisions for the next iteration were based on this feedback. Appendix C graphically documents the prototyping process carried out.

The prototyping process has raised two important points about employing user-centred design methods. Firstly, the majority of the issues with each iteration were put forward by THE PROVIDER and not THE CLIENT. This corroborates the suggestion by Garrett (2002) and Boiko (2005) that involving all stakeholders is vital for effective user-centred design. It is important to cover as wide a range of stakeholders as possible as users may assume tacit knowledge. Secondly, some of the issues gathered during prototyping were in fact requirements which should have been elicited earlier in the design process. It is possible that THE CLIENT “did not have the current system in mind” (Sutcliffe, 2002) very clearly when requirements gathering was performed. Furthermore, it is thought that the changes introduced with each prototype iteration can prompt trails of thought, and spark further ideas in the user which would not have been apparent during requirements gathering. This proves the point made by (Garrett, 2002) that the “planes” of user experience should not be considered as discrete stages—they should partly overlap and be performed semi-concurrently to prevent decisions made on lower planes (such as strategy) from restricting decisions made on upper planes (Figure 3.8).

---

---

3.4 Implementation

The third iteration of prototypes was developed into a working system as a collaborative effort by myself and the Technical Director at THE PROVIDER. My work focussed primarily on developing the new user interface and the preliminary stages of integrating this with the existing system backend. The Technical Director carried out full integration of the new interface and added functionality where required.

3.4.1 Constraints

There were a few aspects present in the third iteration of prototypes which could not be realised due to technical and budget constraints.

• There were several knock-on effects which affected the stability of loading the content editor in a modal window (Figure C.8). It was decided to keep the content editor “in line” with the rest of the page.
• The image manager (Figure C.9) was n