1. Introduction

Multimedia is a technological achievement that currently lacks a theoretical basis for reasoning about its utility and effects on usability. Using the most advanced technology will not necessarily improve the usability of current designs. Relying upon naive assumptions, beliefs and intuitions alone will not be enough to bring about a widespread improvement in the quality and usability of interactive systems through the use of multimedia. Although multimedia technology can increase the options opened to the user-interface designer (Alty 1997), it has not yet been met by a growth in design criteria and knowledge.

A common view of multimedia is that it is simply the use of more than one medium to present information to users. However, we adopt a wider definition, encompassing both input and output media, and focusing on user-interactions rather than the technological aspects. In this way, we consider interactions with animations, gesture recognition, speech input, speech synthesis, haptic input and output, hypermedia and virtual reality as pertaining to multimedia. As Marmollin (1992) states:

"A user centred definition would characterise multimedia systems as systems enabling the usage of multiple sensory modalities and multiple channels of the same or different modality (for example both ears, both hands etc.), and as systems enabling one user to perform several tasks at the same time ... That is, multimedia is viewed as a multisensory, multichannel, ... approach to system design. In addition multimedia systems put the user in control, i.e. could be described as a user centred approach." (Marmollin 1992).

The problem with multimedia design is that (Bearne et al. 1994):

"Traditional approaches to design for usability from the Human-Computer Interaction discipline do not yet directly deal with the unique characteristics of multimedia systems: while general usability criteria such as learnability, flexibility and robustness apply equally to single media and multimedia systems, they have little to say regarding the specific benefits and drawbacks of concurrent media input and output"

The use of multiple media, when well exploited by designers, potentially makes multimedia interfaces more exciting, more natural, more enjoyable and pleasant to use than traditional mainly text-based interfaces (Petersen 1996). This occurs, because multimedia provides us with richer forms of representing information in human-computer interactions. However, it does not necessarily follow that merely by increasing the richness of the media we will increase the utility and usability of computers and the information (Johnson 1995). While in some cases the addition of more media will allow us to express concepts and information more fully, with greater clarity, and with greater accuracy than before, in other cases it will introduce ambiguity, confusion and contradiction.

Our research aims to define a set of principles to address the complexities of multimedia design and evaluation, in order to make multimedia systems useful and usable, rather than "gimmicky" and ephemeral. The principles that will emerge from this research are expected to support designers in making decisions about the various media so as to maximise the effectiveness and efficiency of the user-computer interactions. This will enable designers to build more usable multimedia systems, moving from a craft style design approach to a more systematic principled-based approach.

In the following section, we present the some of the design issues we are focusing our research upon, and for which we hope to be developing the principles. We include some examples to illustrate them. Finally, we discuss the basic steps involved in the continuation of this research.

2. Towards Multimedia Principles

We consider a principle to be a general truth or belief that is used as a basis for reasoning or action, or for the development of further ideas (Longman 1978). It has a theoretical and empirical basis for its acceptance and can be applied to a prescribed problem area in a well-defined manner and for which there is some indication of what the result of following the principle (or not) will be.

At this stage in our research we are in a position to be able to point to areas where we ought to be developing a more principled understanding of interacting with multimedia and what might be the features that an underlying set of principles for multimedia design have to address. The areas presented here resulted from a literature survey, the main source being Alty (1993). They seem to help to understand and explain the complexities of multimedia systems, and are serving as the basis for us to pursue the search for the aforementioned principles. Some of the features are particular to multimedia, while others are more general to wider areas of HCI.

A related feature to naturalness is realness, or the degree of correspondence to the real thing. Naturalness and realness are similar but not the same. Naturalness here is concerned with the mapping between the stimuli and the senses, taking recognition of the fact that people normally gain information about the world from multiple senses (e.g. hearing an explosion would cause people to look for a cloud of smoke). On the other hand, realness is concerned with how close the representation of the explosion corresponds to the actual explosion.

Two consequences for systems that possess these features appears to be that they show properties of believability (the closer to the reality, the more believable) and fidelity (degree of detail and faithfulness). Hence, in figure 1 the representation of the document has a high degree of realness (i.e. it closely corresponds to the appearance of the actual document) and naturalness (i.e. it is perceived through our visual object recognition system.
 

Figure 1: A smaller version of a document shown in (Corbis 1995)

 

But is it enough to have a knowledge of each medium in order to make an adequate selection? Some argue that it also depends on the user’s knowledge and experience of a domain and task: if the domain and task are new to the user, a concrete representation that allows exploration seems to be best; if the user has a lot of experience in the domain and task, more abstract representation may be adequate (adapted from Marmolin 1992).

Alty (1993) adds that the usefulness of different media in presentation situations is closely related to the complexity of the idea being conveyed. Nevertheless, he also states that the capabilities of the perceiver play an important role on the media allocation problem. Furthermore, according to Johnson (1995), there is an important difference between abstract and concrete concepts. Abstract and complex concepts are more easily and completely represented by words than by pictures. In contrast, more concrete concepts, if represented by pictures and sounds, can improve the speed of understanding and comprehension over that of text representation. Moreover, he adds that the choice of medium also has to consider what information is it intended to convey and what is the intended effect of the information.

As can be seen, it is not easy to define a complete set of criteria to solve the media allocation problem. One aspect that should be investigated in detail is the relation between media and tasks. In other words, the main problem is to establish which media best transmit the information needed by the users to carry out their tasks.

Summarising what was discussed so far, it seems that multiple factors play a role in the media allocation decision (Arens et al 1993):

In a CD-ROM product targeted for orthopaedists – "Semiology of the Knee" (Evolucao 1996), there were several possible ways to "show" the manoeuvres employed to make a diagnosis about a given knee problem. In books, they are usually presented by several abstract drawings, which make it difficult for the novice to correctly interpret the necessary manoeuvres. In the CD-ROM, they are presented as video segments (figure 2), along with text explanations. Choosing video to present this information allowed us to show the manoeuvres in a more detailed and precise way than could be achieved with a sequence of static drawings.

Figure 2: A video frame from the CD-ROM "Semiology of the Knee" (Evolucao 1996)

In multimedia systems, redundancy is achieved through the integration and synchronisation of different media. It can not only produce "real-world" like conditions, but also can reduce the overload on working memory (e.g. video and audio, animated graphics and text overlay (or sound commentary)). Comprehension is directly affected by redundancy, since there is more chance of the information provided being understood. For instance, if there is confusion and misunderstanding as a result of a misperception of information in one medium, then this can be supplemented by providing the same material in another medium, at the same time (or proximally close in time) (Johnson 1995).

Understanding how to effectively use redundancy, is still a challenge for multimedia systems designers. If combined in a congruent (harmonic, synchronised) way, multiple media is far more effective than using a single medium (Hoogeven 1997). However, if combined in a non-congruent way, it is less effective (in this case, disruption, ambiguity and confusion occur). Vetere (1997) states that presently there is insufficient knowledge to help designers manipulate these redundancies to improve interactions. No methodology or criteria on how to apply redundancy in multimedia systems have been developed so far, let alone a theory of redundancy and its effects on usability.

To exemplify the use of redundancy, in figure 3 a document is presented in two ways: as a photographic reproduction and as a textual transcription. Even though the photograph can be zoomed in, the transcription is easier to read and to search for. This is an example of redundancy, by having two representations of the same information presented in different formats.

Figure 3: A document and its transcription (Corbis 1995)

It is important to observe that this kind of problem also occurs in everyday general communications. Our understanding of multimedia can greatly benefit from many communication theories. Grice’s theory of implicature (Levinson 1983), for instance, is concerned with the efficient and effective use of language in conversation. One of its maxims, the maxim of quantity, is related to the fact that, when making a contribution to a conversation, this contribution must carry all and only the necessary information, not more and not less than what is required. Another maxim, the maxim of relevance, states that one should make his/her contributions relevant.

In a hypermedia application - a literature multimedia encyclopaedia (Nemetz et al. 1996) – we can find an example of this feature. Figure 4 shows a passage of a book that illustrates a particular characteristic of an author. This passage is presented in text and, optionally, in audio. The audio is actually composed by two channels: the first contains the reading of the passage by a narrator, and the second contains an audio-effect that resembles the sound of wind. This effect provides an atmosphere to the narration, associating its contents with the name of the book: Time and Wind. In this case, the audio-effect makes a fundamental contribution, because it is actually reinforcing this association, enhancing its semantics and making it more pleasant, but at the same time it is hoped not to hinder comprehension.
 

Figure 4: A passage from a book (Nemetz et al. 1996)

 

V. Exploration

One of the main advantages of multimedia systems seems to be the increased level of interactivity they provide. This happens not only due to the use of our senses in a fuller and more orchestrated manner (Johnson 1995), but also because of a greater flexibility and freedom to explore the information. Ideally, neither the author nor the designer should decide how the information should be processed; the user should be in control, exploring the interface and choosing the best media for the task.

A high level of interactivity improves sensory stimulation, and thus facilitates human information processing (Hoogeveen 1997). Alty (1993) adds that, for ill-defined goals (or goals not well understood), it is better to allow users to exploit the interface and choose the best media for the task. And Bearne et al. (1994) suggest, in their usability guidelines for multimedia systems, that users must be given control over the appearance and the disappearance of each piece of information. The feeling of engagement produced by the property of exploration is an important issue to take into consideration when designing a multimedia system.

One possible explanation for this phenomenon is that we explore our environments in an active way. Quoting Gibson (1966, 1979 cited by Hoogeveen (1997)), "we do not hear, we listen; we do not see, we look around". To provide this feature, a system should explicitly afford exploration, almost inviting the users to explore it, giving them the appropriate feedback of each action that must be reversible, thus providing "safe" exploration.
 

Figure 5: Building a slide show with paintings (Corbis 1995)

 

A good example of the exploration feature is the slide show facility (figure 5) provided in (Corbis 1995). With this tool, the users can prepare customised guided tours of paintings based on their own criteria. Although there are several guided tours available, giving this possibility to users, allows them to explore the system in a more active way according to their goals, rather than being just passive viewers. The only problem is that, in this case, the user is not able to add annotations or audio to the presentations, which would give them a better way to explore the contents of the system.

With today’s technology, current multimedia representations often have poor quality if compared with their analogue counterparts. For instance, a digital video in a small window cannot compete with the quality provided by an ordinary television. Even though for some kinds of applications this quality is enough (e.g. video-conferencing), for others it can be a restrictive factor (e.g. remote diagnosis by a dermatologist). Therefore, we still do not have a full realisation of the potential of multimedia, although the adequate quality depends ultimately on the task the user is performing.

3. Conclusion

The features presented in this paper reflect many of the main aspects of multimedia systems. In the present stage of our research, they can be considered as design problems that would require principles to guide their solution. It is important to note that, although some of the features can be general, i.e. not specifically addressed to multimedia systems (e.g. redundancy, quality of information representation, exploration), they do represent pertinent aspects of multimedia systems design. It should also be noted that the desired principles may not be equally applicable to all classes of systems multimedia and domains.

In order to advance this research, first we need to refine these features into a set of principles, so that they can be expressed in a more complete and systematic manner, including examples, appropriate theoretical and empirical evidence, and to make predictions about their effects on usability.

The next step is to assess and refine the principles on different classes of multimedia systems, domains and tasks. In doing this, we will assess their (i) predictiveness and reliability through experimental testing, and (ii) applicability and usability through use in design context. In this way we will be assessing if they apply to multimedia design problems, if they can predict usability issues and be applied to those issues, and if the principles themselves are usable by designers and evaluators to develop and assess the usability of systems using multimedia.

In the end, we should be able to propose evaluation and design methods or techniques that are principle-based. A method of evaluation would include criteria, data, analysis and interpretation to produce redesign recommendations. In order to support design creation, an environment could be developed, which would include exemplars, guidelines and constraints derived from the principles.

In this paper, we claimed we need a principle-based approach for the design and evaluation of multimedia systems. Then we proposed a tentative set of six features that were elaborated with evidences from the literature. The features are:

i. naturalness and realness
ii. media allocation
iii. redundancy
iv. significant contribution of the media
v. exploration
vi. quality of information representation

This is an on-going research. In order to achieve our goals, these features need to be further refined, tested and used in real-world situations before they can emerge as principles for multimedia design.

Our research aims to develop basic principles for the design and evaluation of multimedia systems. We believe that these principles will provide a consistent basis for user-interface designers to make better decisions and, hence, to build more usable and useful multimedia systems.
 
 
 
 

References