Pivots and Structured Play: Stimulating creative user input in Concept Development

A UNIVERSAL REMOTE CONTROLLER CONCEPT

This section briefly gives some background on the “universal remote controller” project in which we employed the pivot method to generate and refine an interaction concept for a smart home.

Telenor recently built a futuristic home intended as a venue for showcasing tomorrow’s technologies but also as a venue for exploring actual technology usage. The house features a range of smart home technology such as sensor-driven heating, motorized curtains and blinds, window and door sensors, and electronic locks. The house also contains numerous projectors, speakers, and a plasma screen—all connected to sources of entertainment media via a 16- channel media switching matrix for audio and video. Media sources include Satellite TV, Video, DVD, Game

Figure 1: A small selection of dedicated remote controllers from our smart home.

machines, PCs connected to the Internet, surveillance cameras, etc.

Users control smart home and media functionality by interacting with dedicated remote controllers. To start a DVD movie on the wall opposite the living room sofa, say, the user has to interact with both the DVD remote controller and a wall-mounted control panel connected to the media matrix (in addition, the roof-mounted projector would normally be turned on using its remote controller, but in our case a physical link has been established between the media matrix controller and the projector’s power button).  Darkening the room requires interacting with yet more remote controllers. Figure 1 shows some of the remote controllers associated with the living room.

The Unimote concept

We wanted to simplify interaction and control in the smart home by moving away from relying on multiple, dedicated remote controllers. A project group of seven summer students led by researchers at Telenor were given the task of defining and realizing a concept, the Unimote, for interaction and control in the home of the future. The initial, basic Unimote concept was a kind of magic wand that gave users control of the house without needing to always search for the correct dedicated remote controller.

THE PIVOT SESSIONS

This section reports on our experience from employing the pivot method to solicit input from potential users in refining the basic Unimote concept, allowing us to develop a more accurate specification of system features and issues pertaining to interaction design and usability. The next section offers a more in-depth treatment of how the pivot method stimulates creativity.

During the first stages of the design project, the designers had explored several methods for concept development. Their initial work was focused on getting a grip on the basic concept of Unimote, and did not include users other than themselves. At first, they had taken for granted that they were designing a specific device. During this development phase, however, they became increasingly uncertain about that concept. Therefore, we decided to include potential users in the creative process to help iteration in the further development.

Initially, we had spent a day in the future home (which was the intended target environment of the Unimote prototype) in preparation of bringing potential users into that environment. Among other things we discovered that we had difficulties relating to the surroundings in a creative way. When we brought potential users into the house for a pilot session they experienced the same difficulties. Simply walking around in the house and trying to relate to the surroundings proved physically tiresome. The test subjects preferred staying in one place while referring to the surroundings by pointing. They did not interact with the physical surroundings at all. The surroundings seemed to block their imagination instead of inspiring it.

Earlier, the designers had used a miniature mock up of a home to explore their own concepts of home and everyday living. Based on previous work conducted by Iacucci et al. [‎5,‎6] we decided to try using the ‘dollhouse’ concept with test subjects. The idea was that their imagination would be better spurred by a dollhouse, which gives the idea of ‘home’ rather than being an exact model. In this way, we argue, test subjects would easier parallel the test situation with their personal everyday life, combining something well known with an imagined, possible future reality. Further, a miniature mock-up would be easier to manipulate than a live size environment, thus giving a phenomenological dimension to the test situation. The theoretical framework for these assumptions is discussed further on in this paper. 

Given the above, we decided to build a small symbolic representation of a home, using the concept of a children’s dollhouse as a point of departure. We wanted to use as little time and resources as possible building the house, as one of the advantages with this method. The dollhouse was thus built out of shelves and cardboard boxes, with furniture made out of cardboard, paper and bits of plastic. Cut outs from magazines were used as symbols of different home environment articles, i.e., kitchenware, plants, and technical Figure 2: The Golf player, made out of modeling clay and cardboard.

equipment. Small figures made out of cardboard and cut outs from magazines became ‘inhabitants’ of the dollhouse. They represented different types of people, e.g., a middle-aged woman with a vacuum cleaner represented the housewife and a young man swinging a golf club represented the single man (see figure 2).

This was an intended and carefully planned structuration. The test subjects’ choice of role would give us insights which could be useful later on in the analysis.

In accordance with this, we decided to conduct semi-structured interview sessions by the dollhouse, combined with manipulation of the figurines. A semi-structured interview is a qualitative form of research. Given that we wanted creative input from test subjects we wanted to keep the method as qualitative as possible, to ensure the highest possible degree of flexibility. But the main reason we included test subjects was to get an idea of how potential users would relate to the Unimote concept. To ensure that the results would be representative of real user groups, we needed to be able to compare them to a certain degree. Our solution was to construct an interview guide, giving some structure to ensure that the test results would be comparable, and that our conclusions would be reliable.

Based on the themes in the interview guide, the elements of Figure 3: The dog symbolizes all kinds of domestic animals. The large cut outs on the wall represent home entertainment equipment, a central theme of the sessions.

the dollhouse were carefully chosen. The focus of the Unimote was on control and interaction with different kinds of home entertainment equipment, which were represented in the dollhouse as slightly larger than other items. See figure 3 for a picture of one of the dollhouse rooms. We also wanted to draw attention to control and interaction with the home from outside, both from far away (e.g., from the car, from work, or while away on a trip) and from the immediate surroundings (e.g., from the garden). Therefore, we also included a garden and a garage with a car in it. A small plastic dog allowed us to expand the scope of the sessions to include household animals of all kinds, and problems and ideas relating to their interaction with the ‘smart’ environment. Using this method of physical representation ensures that the themes are kept open, so that the creative flow is not hindered by predefined ‘right

answers’ or assumptions. For example, the car could be a symbol not only of a car in the real world, but also of the theme of mobility in a wider sense.

The test team consisted of two designers, one social scientist and two test subjects. The designers decided on the elements in the dollhouse, and on which themes were important to discuss during the sessions. The dialogue was mainly between the social scientist and the test subjects, but the designers were also able to step in and take over the session whenever they felt particularly interested, or when the conversation was coming to a halt. The sessions were videotaped.

The first half of the sessions were structured around the ‘day in the life’ principle. The subjects were told to picture an ordinary day in their lives and use the figurines and the dollhouse to enact typical everyday activities (see figure 4). During this performance they were continuously challenged to see how everyday problems could be solved using imaginary technologies, more or less consistent with technologies deployed in the smart home. This proved to be very useful for two reasons. Firstly, the designers were presented with new views of technology and needs in everyday life. Secondly, the test subjects developed a firm grasp of the imaginary technological environment that they created during this first part of the session. Later sessions focused on control and interaction with the technological environment. The research scientist started asking questions more specifically about how the test subjects wanted to relate to the technology, using scenarios from the ‘day in the life’ stories they had just told.

Figure 4: Test subject enacting a typical morning ritual in his figured futuristic home.

During pilot sessions we became aware of difficulties connected to adults’ abilities to relate to an imaginary setting. We invited two very different subjects to participate in a pilot. One was a young single male, the other a middle-aged mother of six. The subjects were told to imagine themselves living together in the same home and to go through an ordinary day in their life—from getting up in the morning, to going to bed at night. The subjects never related to an imagined coexistence in the dollhouse; they both seemed to strongly prefer to relate to their own lives. Thus, we found that asking adults to play roles other than their own role in the real world proved unsuccessful. That is, adults need to be coached in a more structured way than children to be creative. But adults are also more aware of their own needs and much more able to relate to the coupling of concrete reality and imagination, once they feel that the setting is meaningful.

The two subjects had no problems imagining the use of imaginary technology, and they related differently to technological concepts according to their marital status and their role in the household.

We also found that bringing together very different subjects could be fruitful in yet another way. Subjects with similar backgrounds tended to become creatively impaired because they agreed on most things. Subjects with different backgrounds seemed to relate to each other by juxtaposing their views, thereby extrapolating a wide range of different information. The miniature model helped them to stay in the same context, although it was open enough to support different notions of what such an imaginary home environment might look like. For example, during the day in life routine, a test subject living in a student collective stated that when coming home from work or the university he was often annoyed to find that his roommates were occupying the washing machine. This simple statement spurred the idea of remote-controlling such appliances. However, for the middle-aged mother participating as a test subject this was not a relevant problem as she was the only one using the washing machine in the first place. For her, the idea of reserving the washing machine opened up the idea of coordinating the use of resources in a wider sense, resulting in her stating that she would like to reserve the bathroom from time to time, having a teenage daughter. In other words, the two test subjects’ descriptions of their contrasting life situations and potential technological needs stimulated creative thinking around the potential of the Unimote for both persons. It allowed them to become more reflexive about their own practices, and explore each other’s mode of thought within their own frames of references.

After the pivot sessions, the videotapes were analyzed. We found that test subjects displayed different views on both the concept of home in general, and the concept of bringing a high degree of information technology into the home. They were all concerned with control and had strong opinions about how to ensure that people be in control of the system and not the other way around. Likewise, all subjects agreed that they would like to be able to switch between various interfaces while interacting with the system. Speech recognition would be appropriate for some functions, but not, for instance, when choosing music. When watching TV most of the subjects preferred a traditional remote control which could be manipulated without looking, but they did not want to carry the remote around in the house. Based on session analyses, we decided to refine the initial Unimote concept as follows:

It should be available on a range of devices, including cell phones, PDAs, PCs, etc. As stated, the initial Unimote concept was based on the assumption that we were designing a specific device with one specific interface. During the pivot sessions it became clear that the preconditions for control and interaction with everyday objects vary  greatly with each object, as well as with each situation. As mentioned above, test subjects wanted a tactile interface when watching TV but speech recognition in the kitchen, and they did not want to have to bring the remote with them around the house. This led us to focus on designing a system rather than a device – software rather than hardware, as it were, which could be present in all the rooms, but accessed in different ways according to situation and room functions.

It should employ context-awareness in the form of user profiles. Parents were especially helpful test subjects in this matter. They had their focus mainly on the supervision and control with children. Most parents wanted to prevent their children from performing certain tasks, such as using the stove, or having access to functions that were potentially dangerous for them. They also wanted to be able to monitor their children’s media habits and general behavior, but prevent the children from doing the same. Here, the coupling of parents with singles and younger people was especially fruitful. They often had an alternative view of control and supervision in the home, thereby challenging the views of the parents. The result was a decision by the designers not to give access to all information about the children, as it was argued that taking responsibility for one’s own actions is an important part of growing up, as well as a right to privacy. Nevertheless, different people should have access to different information in different situations. The theme of profiles became important in further work with Unimote.

It should be available everywhere, in the home as well as away. The test subjects were unanimous on this. Being able to start afternoon chores from work, such as cooking dinner or drawing a bath before entering the house were very attractive traits for all of the subjects. Also, monitoring pets, children or house appliances from afar were functions mentioned by some as very useful.

It should give basic support for coordination with respect to accessing limited resources, but using a hands-off approach.

This discussion was brought about by the student and mother example conveyed above. Exactly how the coordination was to be brought about was unclear at that point and needed much refinement. But it was generally agreed that some universal ‘fairness principle’ should govern the coordination, rather than roles or profiles.

Recently, the realization of a prototype implementation of the Unimote concept was completed. The user interface is based on the direct manipulation of graphical icons. The user interface runs on any device with a simple HTML browser, but the user interface is tuned for small, color touch-screens found on most new PDAs. An always-on wireless network connection (e.g., Wireless LAN) is required to communicate with the back-end implementation. Most of the early realization effort focused on implementing functionality and building a flexible and robust back-end software infrastructure. At the time of writing we are realizing an alternative tangible user interface for the Unimote concept [‎1] on top of the said infrastructure.  

THE PIVOT METHOD

The major goal of the pivot method is the refinement of initial product or design concepts towards more concrete design. To achieve this goal, the pivot method focuses on two aspects: the stimulation of creativity and the exchange of contextual information. The creativity aspect seeks to stimulate an alternation and creative exchange of ideas and knowledge between a symbolically represented imaginary world and the real world through the application of a pivot object. The pivot object, or just pivot, which is vital to the method, is a physical miniature model of the design object environment. The context exchange aspect seeks to attain a common appreciation of the design object and the context in which the design object is tested. This context consists of the physical environment and rules of engagement decided on by the designers, i.e. the contextual framework, and the experiential backgrounds of the test subjects and the designers, i.e. their cognitive schemas and figured worlds. This focus is central simply because the different participants experience the context differently, and because different personal experiences and knowledge also enrich this context.

The pivot method, as used in the early design phase of the Unimote, can generally be described as based on structured dialogue and interaction. By structured we mean that the actual testing happens in a (partially) controlled microenvironment where the structural and regulatory framework has already been decided on by the designer. This microenvironment can for example be a miniature model of a house. Further, this structuring also implies that the designer is present during the session and plays a managing part in the interaction. By dialogue and interaction we mean that test subjects perform a pre-defined scenario or situation through structured play. This involves interacting with the other players and the play manager (designer) through dialogue and by using miniatures in the given microenvironment.

The creative aspect

The process of creatively exchanging and alternating between an imagined and real world rests on an individual’s ability to acquire and employ cognitive schemas. Cognitive schemas are conceptual structures and processes, which enable human beings to store perceptual and conceptual information about the world and make interpretations of events through abstraction [‎12]. It might be described as a flexible mental ‘template’ that is used in the interpretation of events and their contexts. Each schema consists of slots of information based on prior experiences. Where such information is missing, culturally specified default values fill the slots. We know for example when we see a dog that it is likely to have a tail, even though we might not be able to actually see the dog’s tail in that specific situation. Because they are based on individual experience, cognitive schemas vary greatly with different cultural backgrounds and social positions. Creative activity can in this sense be understood as ‘opening up’ slots of information in predefined schemas or filling such information slots with new values, thus both reaching a higher level of reflexivity about one’s own mode of thought and allowing oneself to reinterpret information about the world. This allows for new ideas to develop, as seen in the case example with the student and the mother described in the previous section of this paper.

Complexes of cognitive schemas comprise figured worlds and the rules that govern these worlds, be they imagined or real [‎4]. In children’s play imagined worlds serve as arenas for cognitive, emotional, and social development. A child uses a pivot to evoke the imagined world and move back and forth between that world and the real. Basically, a pivot is the object with which the child actually plays. However, the relationship of the pivot to the imagined and real worlds change as the child’s skills develop and the child matures [‎15]. At an early stage, the characteristics of the pivot influence and constrain the imagined world. Later, the child experiences advancement in freedom from the pivot’s form and in the relationship between the imagined world and the pivot. In addition, the formulation of rules that govern the figured world becomes more and more explicit as the proficiency with the play situation increases. Holland et al has showed how this logic of learning and pedagogy apply not only to the cognitive, social, and emotional development of children, but also to the engagement of figured worlds in general [‎4]. Adults can learn in the same way. An individual moves from being a novice to an expert in the figured world through an increase of skill and engagement. Adults, however, are more restrained in their engagement with the figured world than children because the rules connected to that world are already well defined. To be creative and reach a state of play, adults therefore need to be challenged by other individuals’ preconceived notions about their figured worlds.

Vygotsky’s insights into the role of pivots in learning have been applied in the field of computer design before. Kolomyjec et al. argue for example that the way children play should instruct the design of computer toys [‎7,‎8]. In relation to the pivot method, we maintain that the use of a miniature model as a pivot will enable the test subjects to enter imagined worlds connected to the design object and develop the complexity of such worlds through structured play. The test subjects are dependent on the pivot initially, but will be able to free themselves from the pivot form as their expertise with the imagined world increases, allowing for a display of creativity that can be used as a basis for the development or refinement of concepts and designs. The structuring of the play situation, though careful and meticulous, must not impede the creativity of the participants, and must therefore remain as open as possible, relying rather on symbolic representations than exact models. In practice, this means that the microenvironment should be kept rather simple, and that the rules defining the social context or frames for practice should be general. The next question is how the test subjects’ imagined worlds could become shared.

Exchange of contextual information

A common appreciation of the design object and the testing context, that is, a shared understanding of the test subjects’ imagined worlds, can be obtained at a deep level through an exchange of cognitive schemas. As cognitive schemas are essentially based on concrete, often bodily, experience and repetition, they have a phenomenological learn-by-doing character [‎10]. This means that they are continually produced, reproduced, and modified in the course of interaction with other individuals. This interaction is interpreted by means of the learned cognitive schemas, and the interpretation happens in a hermeneutical manner. This means that there is a back and forth movement between the simple part and the whole, between detail and context, between information and schema [‎3]. The movement happens as part of social interaction: through dialogue and concrete actions. Dialogue and observation of actions and events are not, in other words, sufficient by themselves to generate experiences that imbue cognitive schemas and thus thorough contextual understanding. To achieve understanding at this level one has to experience through action, by doing. Furthermore, the pivot plays an important role in reaching a common contextual understanding. Even though the meaning of the pivot, or pivot identity, varies from person to person, the pivots focus and limit the scope of the figured worlds (e.g., Buur and Bødker denote physical props used in a test situation as ‘boundary objects’ [‎1]).

Based on these insights we claim that the pivot method contains certain obligatory features. Firstly, the focus on context sharing necessitates participation from both designers and test subjects in the actual design object testing. With such participation hermeneutical interpretation of actions and dialogue is made possible for both designers and test subjects, and thereby a sharing of meaningful context information. Secondly, a miniature of the physical environment for the design object must be present to constitute the framework for the testing. This miniature facilitates an easier understanding of the contextual framework for the test subject than would an abstract model. Phenomenology has showed us that when people consider socio-cultural aspects of other peoples’ actions, these considerations are not normally based on rational models. They are, however, taken into account through natural enactment because this is both based on and generates experience [‎10]. Thirdly, there must be a high correlation between the (loosely defined) design goals and the structuring of the test environment. This not only includes the physical miniature model, but also the rules of structured play, the situations and events introduced by the play master (designer) and the composition of the group of participants. In the same way as Iacucci et al. [‎5] have pointed out that role-playing games can offer good insights into the dynamics of group interaction in situations of local or remote mobility, the structured play of the pivot sessions can do the same. Iacucci et al. claim, however, that such games offer less information on individual behavior and micro-mobility. This need not necessarily be the case.

As maintained, the design object testing must include a physical miniature model, that is, a microenvironment consisting of symbolic representations of the imagined or real environment for the use of the design object. In this microenvironment the test subjects control human-like figures, which they move around to illustrate their actions. We also propose that the test subjects should wield life-sized dummy devices. Furthermore, it is crucial to the pivot method that the test subjects are instructed to actively alternate between on-board miniature interaction and face-to-face interaction and dialogue. The emphasis on either level of interaction should be determined by the design goals. The reason for this alternation is that we wish to make the pivot method as flexible as possible with regards to capturing micro-, local, or remote mobility [‎9]. In face-to-face interaction, test subjects can capture facial mimics, hand movements, etc, while on-board movement allows for an illustration of movements with wider range. Moreover, through the pivot method the designers influence their informants actively. In the play situation, the designer play-master can use his or her position to actively influence developments. Furthermore, other designer can participate as players with more or less active roles. These players can actively participate if things come to a standstill, and they have the opportunity to connect more directly to the test subjects and thus share important contextual information.

CONCLUSION

Benefits of the pivot method may be summed up as flexibility, creativity, and low cost. We have argued that the pivot method has the potential to capture a wide range of information pertaining to both individual behavior and group dynamics spanning micro-, local-, and remote mobility situations. The main reason for this is the possibility afforded by the pivot method to structure the design object testing in several different ways. We have also argued that the figured microenvironments of the pivot method offers more flexibility in relation to test subject creativity than other methods based on real-world settings. The symbolic representation of the microenvironment allows all participants to creatively interpret the setting and situations. Finally, we have argued that the pivot method is both quick and simple. A session can be conducted in a couple of hours, and a fully developed prototype of the design object is not required. However, it is important to emphasize that the pivot method itself is in an early phase of development. The validity of the method is continually re-examined as we evaluate new tests and test results. Even so, some aspects of the method seem to be relatively well documented. Iacucci et al. [5, 6] have shown how play in a miniature environment can yield valuable information about the context of social interaction. They also propose that this sort of play should be structured, yet highly flexible. In addition, Buur and Bødker [2] have shown the value of involving users at an early stage in the development process, and the value of promoting experiences through enactment.

Consequently, we argue that the pivot method contributes to usability and design methodology in three ways. Firstly, the pivot method suggests that early involvement of potential users may offer valuable usability information even before a working prototype has been made. The pivot method offers guidelines for the solicitation of such information. Secondly, we propose a theoretical framework in which early usability and design methodology can be conceptualized. Finally, we hold that the pivot method has in itself aspects that add value to methodological knowledge. In addition to supporting similar methods in general, the pivot method offers a deeper understanding of the use of pivot objects as a means to induce creative interaction between test subjects.

ACKNOWLEDGMENTS

We would like to thank summer students Morten Berge, Thomas Fossum, Torkjell Hongve, Silja Mustaparta, and Arne Seime for the great work they did, participating in all phases of the development of Unimote. We would also like to thank Kari Hamnes, Silja Nyhus, and Trond Are Øritsland for valuable opinions on the pivot method and the Unimote design process. Finally, we would like to offer our thanks to the anonymous test subjects.

REFERENCES

1. Bjertnes, A., Fysisk brukergrensesnitt for mediekontroll. (In Norwegian), Master’s Thesis, Deparment of Cybernetics, Norwegian University of Technology and Science, May 2002 (expected).
2. Buur, J., Bødker, S., From Usability Lab to “Design Colaboratorium”: Reframing Usability Practice. In Conference Proceedings on Designing Interactive Systems: Process, Practices, Methods, and Techniques (DIS 2000, New York, NY, USA, Aug. 17-19), pp. 292-307, ACM Press, August 2000.
3. Geertz, C., The interpretation of Cultures. Fontana Press, London, 1993.
4. Holland, D., Skinner, D., Lachicotte Jr., W., Cain, C., Identity and Agency in Cultural Wiorlds. Harvard University Press, Cambridge, 1998.
5. Iacucci, G., Kuutti, K., Ranta, M., On the Move with a Magic Thing: Role Playing in Concept Design of Mobile Services and Devices. In Conference Proceedings on Designing Interactive Systems: Process, Practices, Methods, and Techniques (DIS 2000, New York, NY, USA, Aug. 17-19), pp. 193-202, ACM Press, August 2000.
6. Iacucci, G., Mäkelä, A., Ranta, M., Mäntylä, M., Visualizing Context, Mobility and Group Interaction: Role Games to Design Product Concepts for Mobile Communication. In Proceedings of the Fourth International Conference on Designing Cooperative Systems (COOP 2000, Sophia Antipolis, France, May 23-26), pp. 53-65, IOP Press, May 2000.
7. Kolomyjec, W., Williamson, M., Deconstructing Play: Theory and Practice. In Proc. Symposium on Small Computers in the Arts (Philadelphia, PA, November 1996).
8. Kolomyjec, W., Cassell, J., Kafai, Y.B., Williamson, M., The Implications of a Theory of Play for the Design of Computer Toys, In Proceedings of the 24th Annual Conference on Computer Graphich & Interactive Techniques (SIGGRAPH ’97, Los Angeles, CA, USA, Aug. 3-8), ACM Press, August 1997.
9. Luff, P., Heath, C., Mobility in Collaboration. In Proceedings of the 7th ACM Conference on Computer-Supported Cooperative Work (CSCW ’98, Seattle, WA, USA), ACM Press, 1998.
10. Merleau-Ponty, M., Phenomenology of Perception. Routledge, London, 1962.
11. Norman, D., The Invisible Computer: Why Good Products Can Fail, the Personal Computer Is So Complex, and Information Appliances Are the Solution. MIT Press, September 1998.
12. Rosch, E., Cognitive Representations of Semantic Categories. Journal of Experimental Psychology, 104: 192-233.
13. Schmidt-Belz, B., Eisenhauer, M., Evaluation of Adaptive Systems for Nomadic Users. Position Paper for the Workshop on Evaluation Methods for Ubiquitous Computing (Co-located with UbiComp 2001, Atlanta, GA, USA) October 2001.
14. Schrage, M., Serious Play: How the World’s Best companies Simulate to Innovate. Harvard Business School Press, Cambridge, 1999.
15. Vygotsky, L.S., Mind in Society: The Development of Higher Psychological Processes. Harvard University Press, Cambridge, 1978.
16. Weiser, M., The Computer of the Twenty-First Century. Scientific American, 265(3), pp. 94-104, September 1991.

---