Strict Standards: Declaration of Walker_Page::start_lvl() should be compatible with Walker::start_lvl($output) in D:\Groups\HEX-GRP-wi\HEX-PRJ-wi\public_html\journal\wp-includes\classes.php on line 584

Strict Standards: Declaration of Walker_Page::end_lvl() should be compatible with Walker::end_lvl($output) in D:\Groups\HEX-GRP-wi\HEX-PRJ-wi\public_html\journal\wp-includes\classes.php on line 584

Strict Standards: Declaration of Walker_Page::start_el() should be compatible with Walker::start_el($output) in D:\Groups\HEX-GRP-wi\HEX-PRJ-wi\public_html\journal\wp-includes\classes.php on line 584

Strict Standards: Declaration of Walker_Page::end_el() should be compatible with Walker::end_el($output) in D:\Groups\HEX-GRP-wi\HEX-PRJ-wi\public_html\journal\wp-includes\classes.php on line 584

Strict Standards: Declaration of Walker_PageDropdown::start_el() should be compatible with Walker::start_el($output) in D:\Groups\HEX-GRP-wi\HEX-PRJ-wi\public_html\journal\wp-includes\classes.php on line 603

Strict Standards: Declaration of Walker_Category::start_lvl() should be compatible with Walker::start_lvl($output) in D:\Groups\HEX-GRP-wi\HEX-PRJ-wi\public_html\journal\wp-includes\classes.php on line 702

Strict Standards: Declaration of Walker_Category::end_lvl() should be compatible with Walker::end_lvl($output) in D:\Groups\HEX-GRP-wi\HEX-PRJ-wi\public_html\journal\wp-includes\classes.php on line 702

Strict Standards: Declaration of Walker_Category::start_el() should be compatible with Walker::start_el($output) in D:\Groups\HEX-GRP-wi\HEX-PRJ-wi\public_html\journal\wp-includes\classes.php on line 702

Strict Standards: Declaration of Walker_Category::end_el() should be compatible with Walker::end_el($output) in D:\Groups\HEX-GRP-wi\HEX-PRJ-wi\public_html\journal\wp-includes\classes.php on line 702

Strict Standards: Declaration of Walker_CategoryDropdown::start_el() should be compatible with Walker::start_el($output) in D:\Groups\HEX-GRP-wi\HEX-PRJ-wi\public_html\journal\wp-includes\classes.php on line 727

Strict Standards: Redefining already defined constructor for class wpdb in D:\Groups\HEX-GRP-wi\HEX-PRJ-wi\public_html\journal\wp-includes\wp-db.php on line 58

Deprecated: Assigning the return value of new by reference is deprecated in D:\Groups\HEX-GRP-wi\HEX-PRJ-wi\public_html\journal\wp-includes\cache.php on line 99

Strict Standards: Redefining already defined constructor for class WP_Object_Cache in D:\Groups\HEX-GRP-wi\HEX-PRJ-wi\public_html\journal\wp-includes\cache.php on line 404

Deprecated: Assigning the return value of new by reference is deprecated in D:\Groups\HEX-GRP-wi\HEX-PRJ-wi\public_html\journal\wp-includes\query.php on line 21

Deprecated: Assigning the return value of new by reference is deprecated in D:\Groups\HEX-GRP-wi\HEX-PRJ-wi\public_html\journal\wp-includes\theme.php on line 576

Deprecated: Assigning the return value of new by reference is deprecated in D:\Groups\HEX-GRP-wi\HEX-PRJ-wi\public_html\journal\wp-content\plugins\article2pdf\article2pdf.php on line 167

Strict Standards: Declaration of PDF_ImageAlpha::Image() should be compatible with FPDF::Image($file, $x = NULL, $y = NULL, $w = 0, $h = 0, $type = '', $link = '') in D:\Groups\HEX-GRP-wi\HEX-PRJ-wi\public_html\journal\wp-content\plugins\article2pdf\contributed\fpdf\fpdf_alpha.php on line 287

Deprecated: Assigning the return value of new by reference is deprecated in D:\Groups\HEX-GRP-wi\HEX-PRJ-wi\public_html\journal\wp-content\plugins\article2pdf\contributed\fpdi\fpdi.php on line 96

Strict Standards: Declaration of FPDF_TPL::AddPage() should be compatible with FPDF::AddPage($orientation = '', $format = '') in D:\Groups\HEX-GRP-wi\HEX-PRJ-wi\public_html\journal\wp-content\plugins\article2pdf\contributed\fpdi\fpdf_tpl.php on line 383

Deprecated: Assigning the return value of new by reference is deprecated in D:\Groups\HEX-GRP-wi\HEX-PRJ-wi\public_html\journal\wp-content\plugins\article2pdf\contributed\fpdi\fpdi_pdf_parser.php on line 245

Deprecated: Assigning the return value of new by reference is deprecated in D:\Groups\HEX-GRP-wi\HEX-PRJ-wi\public_html\journal\wp-content\plugins\article2pdf\contributed\fpdi\pdf_parser.php on line 94

Deprecated: Assigning the return value of new by reference is deprecated in D:\Groups\HEX-GRP-wi\HEX-PRJ-wi\public_html\journal\wp-content\plugins\article2pdf\contributed\fpdi\pdf_parser.php on line 260

Deprecated: Assigning the return value of new by reference is deprecated in D:\Groups\HEX-GRP-wi\HEX-PRJ-wi\public_html\journal\wp-content\plugins\article2pdf\contributed\fpdi\pdf_parser.php on line 426

Warning: Cannot modify header information - headers already sent by (output started at D:\Groups\HEX-GRP-wi\HEX-PRJ-wi\public_html\journal\wp-content\plugins\article2pdf\contributed\fpdi\fpdi.php:96) in D:\Groups\HEX-GRP-wi\HEX-PRJ-wi\public_html\journal\wp-includes\feed-rss2.php on line 8
Wi: Journal of Mobile Media http://wi.hexagram.ca Wed, 29 Jun 2011 23:36:07 +0000 http://wordpress.org/?v=2.5 en Outside the Laboratory: Mobile Methods and the User Experience — Introduction[1] http://wi.hexagram.ca/?p=113 http://wi.hexagram.ca/?p=113#comments Mon, 20 Jun 2011 20:00:54 +0000 admin http://wi.hexagram.ca/?p=113 In the fall of 2010, the First International Workshop on Observing the Mobile User Experience was held in in Rekyjavik, Iceland. This all day workshop was a part of the annual conference of Nordic Researchers working in HCI (NordiCHI 2010), and brought together researchers primarily from the European Union to share their methodologies, reflections and experiences on engaging with mobile technology users in places outside of the laboratory. As the original call for participation stated:

the usage of mobile devices is rapidly becoming an integrated part of everyday life. This means that in order to understand the user experience and the usability of a product it is in general not enough to perform studies in the laboratory. Instead the mobile context needs to be taken into account explicitly, and one needs to be able to study users and usage “in the wild”.

The articles included in this issue of wi are the outcome of our participation in this stimulating conference.[2]

Included in this volume are essays on eye tracking in the wilderness (Kuparinen and Irvankoski), evaluating mobile location-based applications in realistic settings (Larsen, Peterson, Zandi and Handler; Wang, Young, and Coxon), the use of derive to understand the mobile experience (Tollmar, Harling and Ramberg), and flexible research methods for mobile industry practitioners (Harrison, Medrington and Stransom). The majority of papers are located in the field of Human-Computer-Interaction (HCI) studies, with the exception of our contribution and that of Maria Férnandez-Ardévol. What links all of the work is a focus of users in context. All are engaged with how individuals and groups engage with mobile technologies in specific locations, such as a retail store (Poppinga and Pielot) local urban environments (Crow and Sawchuk) and national and transnational locations (Férnandez-Ardevol).

As we learned, location matters in a myriad of ways. It matters economically and politically. As some of our readers may know, Canada’s entry into the mobile phone market has been quite different from the European Union. The overwhelming majority of Europeans use the cell phone as their primary form of communication. Although most of these papers do not touch directly on politics, the significant reliance and delivery of many public services on mobile devices makes the user an integral subject to researchers interested in citizenship and inclusion.

A unique feature of the workshop was that it brought together people from industry and academia to exchange methods and experience in mobile research methods. And here it is worth noting that in the European context, it is not at all unusual to find PhDs working in industry-lead laboratories (Harrison et al). A rather close relationship has evolved between industry and academic researchers studying mobile, wireless media in Europe. This partnership gives some researchers access to the latest technologies and timely opportunities to explore user experiences in an industry-backed setting. The goal is not merely to make money, but to do research that might lead to more appropriate technologies for consumers (Harrison et al and Wac and Dey). While this may produce timely interventions, these relationships may also bring limitations: the time and cost required for longer-term studies with a wider range of demographic groups are often prohibitive for profit-driven, time-constrained businesses. These partnerships ask us to reflect on the kinds of research that can and will be produced within these settings. Who has access to public/private research? How will this research be used? What are the products being made and who are these devices for?

In this encounter between HCI and media studies scholars studying telecommunications and mobile media technologies and practices disciplinary connection and differences also become evident. For many of the scholars in this collection, the HCI presentation-format has different expectations than social science and humanities peer-reviewed articles. These empirically driven research papers have a mored limited discussion of theory and methodology and concentrate more on data and findings. These difference also come about because of the disciplines that are cognate to most HCI research: psychology, hardware and software engineering, advertising and marketing research  (Harrison et al and Wac and Dey).

Many of the articles raise ethical questions of “surveillance” and the ties of university-based research to data-tracking. In studying users “in the wild”, the attempt is to try and offer scenarios for testing that requires less human input of data, which is filtered, by the relationship of researcher to the researched (Poppinga and Pielot, Tollmar, et al). How is the dream of knowing what users would really do with technology in everyday life mitigated by research methods, processes and protocols? Is automatic data-gathering of user practices and habits possible? What are the challenges in terms of privacy?

The dream of data-gathering “in the world” also raises a number of key epistemological questions. Is it possible to collect “raw data”? Does this create more objective findings? While many of these papers present the usual charts, graphs, tables or photographs one of the challenges is to find the means to visualize and present this data in meaningful and highly creative ways that show different patterns of complex relationships between individual users moving through locations (Larsen et al) or patterns across national borders (Férnandez-Ardevol). In other words, while the world of empirical research may seem like foreign territory to researchers in communication, cultural or media studies we have much to learn by reading research coming from traditions that are far more empirical and statistical than our own.

In closing, these wide-ranging studies bring us out of our labs and away from our desktops into some risky settings, markedly different contexts, and very specific milieus. We hope that you enjoy, and are provoked, by this rich interdisciplinary foray.

Barbara Crow, Benjamin Poppinga and Kim Sawchuk

Notes


[1] We would very much like to thank a number of graduate students who assisted in the copy editing and production of the journal: Ana Rita Morais, Jessica Ranger, Brendan Scott, Sara Swain, and lastly, for web formatting Christina Haralanova.

[2] The abstracts and paper presentations were referred in the Conference proceedings: Observing the Mobile User Experience: Proceedings of the 1st International Workshop Held in Conjunction with NordiHCI, October 17, 2010, Benjamin Poppinga, Charlotte Magnusson, Wilko Heuten, David McGookin, Niels Henze, Ginger B. Claasen, Martin Pielot, Hakan Eftring, and Jorn Peters, http://omue10.offis.de/.

Share and Enjoy Digg TwitThis Facebook del.icio.us Google StumbleUpon E-mail this story to a friend! Print this article!

]]>
http://wi.hexagram.ca/?feed=rss2&p=113
User Centred Design Research Methods for Mobile Industry Practitioners http://wi.hexagram.ca/?p=67 http://wi.hexagram.ca/?p=67#comments Mon, 20 Jun 2011 18:58:52 +0000 admin http://wi.hexagram.ca/?p=67 By Chandra Harrison, Sam Medrington, and Whan Stransom

Introduction

The UCD process should be an iterative cycle of research, design, evaluation and monitoring after release (“International Standards Organisation,” 2009). This process is applied to many different types of products within the mobile space including mobile phones, network providers’ websites, and packaging of mobile devices. Having the right method available to gain the right insights at different phases is vital to successful product development. In practice using the right method can prove problematic due to the complexity of the mobile research space, restrictions imposed by the client, client’s lack of knowledge of methods, and issues around usability and accessibility.

The purpose of this workshop, on mobile user experience, was to bring together people from industry and academia to exchange methods and experiences related to observing mobile device UX. Therefore, in this paper we briefly present specific applied examples of observing the mobile user experience in practice. Over the past two years we have been involved in many activities that provide insight into the relative merits of various methods and highlight the barriers to design. We discuss barriers particular to practitioners in the design of quality mobile experiences. We also present brief examples of applied research during the four phases of UCD (formative research, conceptual design, evaluation and post purchase). Due to non-disclosure agreements we are unable to discuss particulars of the specific clients, but we do discuss the benefits and constraints of our methods. These insights are based on our activities as practitioners in a User Experience agency over the past few years.

Barriers to Mobile Research

Complexity of Mobile Space

One of the key factors to researching and designing quality mobile user experiences is understanding the complexity of the space and the multiple factors that need considering. To design quality products we need to consider:

    · the variety of users (e.g. able bodied and disabled)
    · the hardware (e.g. screen size and type, button placement)
    · the software (e.g. proprietary, open source)
    · the content (e.g. websites, applications)
    · the network provider (e.g. coverage, costs)
    · the network type and speed (e.g. GSM/CDMA, 2G/3G, Wi-Fi)
    · contextual issues (e.g. lighting, glare, noise)
    · functionality (e.g. storage capacity)

As well as the complexity of the space itself, one significant barrier to designing mobile user experiences is that as practitioners we rarely get to explore the entire space for one project. However, the variety of projects we do get involved with does provide some over-arching clarity of the best methods for research, design and evaluation within these conditions.

Client Relationships

A significant barrier to designing quality mobile user experiences is the relationship between clients and practitioners. A client’s location, short time frames, tight budgets, need for secrecy and lack of UCD knowledge can all have negative influences. If clients are based in Asia with a large part of their market in Europe then methods need refining to accommodate their location. The multicultural needs for scaling, and communication within the design team, also need to be considered. Decision-makers are often not the team members that we see, making it difficult to influence design decisions. Bound by non-disclosure agreements practitioners are also severely limited in our use of case studies, which in turn restricts knowledge sharing. Clients also do not always understand or want the most appropriate method. For example, clients may ask for focus groups so they can see sixteen people in one day, when in-situ observation of three people in one day would provide much better data. Improving the client relationship with quality results is often the only way to ensure the best methods are used.

Usability

In our research, one thing is apparent in almost all projects; basic usability is often overlooked in the design of mobile devices, content for these devices and the supporting websites and collateral that accompanies the devices. While products such as the iPhone cash in on intuitive interaction because the actions are familiar, content is often poorly designed, and guidelines are ignored. Design focus is also often on functionality rather than usability. Consumers seem to be willing to overlook usability issues because of the functionality. But basics of noise interference, lighting and glare issues, poor use of screen real estate, the ergonomics of handsets, web content providers using absolute values, connection speed and not designing specific mobile sites all seem to be overlooked.

As well as usability, access for all seems to be almost completely ignored. Mobile devices are difficult to use in a variety of different contexts and these factors are often over-looked. As mobile devices are used in more and more varied locations the manufacturers and content developers need to consider access. For example, glare on screens, operating the systems in noisy environments, and in cold climates, using cocktail sausages for touch interfaces rather than taking hands out of gloves. Many handset manufacturers seem to still be missing the point, designing separate handsets for different demographics. Designing Fisher Price style phones for older adults is not respectful or tasteful. These oversights offer a huge space for improvement and gaining market share if clients are willing to spend the time and money.

Research Methods

User-centered design follows an iterative pattern of research, design, evaluation and release as illustrated in Figure 1. Clients require us to become involved in research at various phases of the design life cycle for different projects. While ideally we would be involved throughout the life cycle, as agency practitioners we are often brought in for one phase or another rather than end to end. Here we present a variety of research methods that we have used and experiences we have had when conducting user research.

Figure 1: The User-centred Design Process

Formative Research
Formative research is necessary to gain insight into the needs and desires of the target market and to gain greater understanding of the context in which products will be used (“International Standards Organisation,” 2009). While formative research is highly valued in UCD for determining user requirements and setting release criteria, in practice it is more rare than we would hope, as clients often mistakenly believe they already have sufficient insight to design their products. However, over the past two years when we have been involved in formative research we have used a variety of methods. These are mainly ‘in the wild’ methods with real consumers to gain insights into their behavior and context of use. For formative research we would definitely encourage research in naturalistic settings or environments. However, time and client needs often mean that this is not possible.

In one study for a mobile phone manufacturer exploring music consumption behavior on the move, observation was a key method used. The practitioner and one of the client design team observed participants in various settings including record stores, commuting on public transport, hanging out at home and university. The observation involved shadowing and a follow-up interview after the session. The observation was augmented with participants completing cultural probe (Gaver, Dunne, and Pacenti, 1999) type activities such as photographing significant moments influenced by music. The primary focus of this research was on contextual and behavioral aspects rather than the fine detail of the interaction with mobile devices making these ethnographic methods ideal.

During another study conducted to better understand blind mobile phone users’ needs, we used several other ethnographic techniques. We used an electronic diary study which blind consumers found difficult to complete, largely due to the time commitment involved which is a common complaint with diary studies. Participants mentioned that they would have preferred to use a Dictaphone to record their thoughts and activities.

Another tool used was a form of experience sampling method (ESM) (Larson, & Csikszentmihalyi, 1983). At various points over a two-week period participants were sent a text message asking them to perform a simple task using the internet on their mobile phone, for example to find a book on Amazon, and then to return a text message with details of how they got on. This technique was very successful with participants finding it much easier to respond immediately via text message rather than having to remember to note activities in a diary later.

Over the years we have also conducted numerous one-to-one interviews about mobile use, either in participants’ homes, neutral locations such as cafés, or in the System Concepts’ labs. One-to-one interviews on location are particularly useful as they illustrate contextual issues around mobile use. For example, we would not see the difficulties experienced by a blind mobile phone user with their Talks software (“Nuance Communications,” 2010) when it is used in a noisy environment, such as a café at a train station, if the interview was in the lab. In this environment Talks users hold the handset up to their ear to listen then move the phone down to press keys. In a quiet lab setting, users would not need to put the phone to their ear, as the phone would be audible.

In addition, the banter and level of connection with someone in their own home is more relaxed than in a lab setting making it easier to discover more personal facts. In Figure 2 we spent several hours learning more about this man, his mobile and the environment he uses it in, but we also learned about his hobbies and life.

Figure 2: One of our participants in his home showing us how he would use his mobile phone using Talks software.

Despite the major benefits of research done on location, interviews in the lab do make it much easier to set up recording equipment, which we will discuss further in Section 3.3. Another problem with ‘in the wild’ research is that travelling to observe people takes longer and involves more transport costs. It is also necessary to have one person facilitate the discussions and another to record the sessions, increasing the resources and possible intrusion. Yet, relying on self-reporting techniques, such as diary studies, can be problematic in general, but even more so for mobile research as the behaviors of interest are often undertaken on the move when pen and paper are not handy. Therefore it is necessary to find ways of making the recording of events simpler for the participants. What is clear from the formative research we have conducted over the years is that the location and method used greatly depends on the objectives of the research. If clients want the data recorded in great detail for later viewing then ‘in the wild’ research is more problematic. However, if the high level qualitative findings of the context and more general behaviors are more important, then ‘in the wild’ observation provides a richer picture.

Design
Once consumer insight and user requirements have been gained from the formative research, conceptual design follows. While designing the product is the focus of this phase, testing conceptual designs with potential consumers and comparing them to design guidelines can help ensure the success of a product. Clients can involve us in this phase as independent researchers to assess other people’s designs or as consultants helping designers using insights gained in previous research, best practice knowledge and applying guidelines.

During one successful design consultancy project, we worked alongside a large online content producer who was adapting their online offering to mobile-specific sites for a variety of handsets. They required consultation regarding usability and accessibility of the sites on various handsets. We conducted expert reviews of preliminary designs using guidelines (Chandler, Dixon, Pereira, Kokkinaki, & Roe, 2005; Rabin, & McCathieNevile, 2008) and heuristics (Nielsen, 1994). Once designs were coded they were evaluated and changes made. The designers were willing to learn as much as possible and we facilitated this through awareness training and allowing the designers to shadow us during the expert reviews. During this research we used a variety of handsets to test the different designs, but it was not possible to consider all variables in Section 2.1.

In many situations it is clear that mobile web content producers and practitioners are unaware of the guidelines that are available and the restrictions of mobile design. During a recent, UK Chapter of the Usability Professionals Association (UK UPA) event relating to mobile design, it was discovered that few practitioners used the W3C guidelines (Rabin, & McCathieNevile, 2008) and none had used the RNIB guidelines (Chandler, Dixon, Pereira, Kokkinaki, Roe, 2005) when consulting. This is in part because guidelines are too specific and do not consider the interaction of the different factors listed in Section 2.1. In addition, few clients encourage use of guidelines preferring to look for innovation rather than solid design patterns. However, if clients can be convinced to involve practitioners who are aware of the guidelines and who can advise about the appropriate methods then better designs can result.

In another recent instance, a mobile manufacturer designing a new mobile phone content browser, wanted to explore how to present photos and video content. Following the technique of Rapid Iterative Testing and Evaluation (RITE) (Medlock, Wixon, McGee, & Welsh, 2005) for early design concept testing, we used low fidelity paper prototypes in the lab. We alternated between a day of testing and a day of workshops with the client to iterate the designs. In the final round of research we used prototypes of the visual design that weren’t interactive to assess the branding and emotion. However, it was clear that participants were happier to criticize roughly sketched designs than to what appeared to be higher fidelity prototypes. They were often distracted by the detail or the specific content.

Encouraging clients to use RITE is a massive victory for practitioners and one that includes consumers early in the design process rather than just for a final evaluation. It is also clear that paper prototypes are much easier to change than higher fidelity prototypes and participants are more willing to criticize them.

Evaluation
Once the conceptual design has been firmly established and higher fidelity prototypes are available evaluation against release criteria is often required by clients. This type of evaluation is usually to confirm that there are no major problems prior to release. Unfortunately clients often only bring practitioners in at this point to say they have considered usability rather than actually considering the user throughout the design life cycle. This often means that poor design decisions cannot be undone.

During a comparative study of a new proposition operating system with the Android and Apple operating systems, we used basic usability metrics to evaluate the products. In this comparative evaluation brand loyalty was a control variable with a focus on the usability of the new proposition operating system. The research was done in the lab, because there were a variety of tasks to cover with multiple handsets and it would not have been feasible to conduct this research ‘in the wild’. Participants did not use their own phones, the tasks were contrived and not all functionality was available due to the prototype. However, client viewing of the evaluation was vital and large numbers of participants were tested and these were better facilitated in the lab.

One evaluation method that we have found extremely useful is the automatic data logging of behavior. During the evaluation of a media player application, an application was installed on participants’ phones to record their activity with the phone and with the media player. In comparison to other diary studies we have conducted it is clear that automated data recording works better as it does not rely on users remembering to provide information.

Figure 3: Recording camera attached to device and the output of the remote high-zoom camera

The recording equipment used for evaluations (and research in general) is another issue to consider. ‘In the wild’ it is important to capture the behavior as naturally as possible. In the lab it is often important for the client to have control to focus on aspects they see as important. We have three different camera set-ups that we use in different situations detailed in Table 1. We are lucky to have a ‘bespoke’ camera that attaches to the phone which is much better for recording the interaction with the device in a natural way rather than some solutions which require the phone to be fixed. However, it does not record the facial reactions, comments and contextual issues. Figure 3 shows the attached camera and the output from a remote zoom camera.

Camera Type Freedom of participant movement Lack of intrusion for participant Client viewing experience
Attached to device

1

3

2

Suspended on Tripod

3

2

1

Remote high-zoom

2

1

3

Table 1: Ranking of different cameras for viewing and recording mobile device interaction.

Recently a group of UCD practitioners gathered for a UK UPA event. They ranged from freelance consultants specializing in mobile through to in-house practitioners at mobile phone manufacturer companies. Many of those present were aware of the W3C guidelines (Rabin, & McCathieNevile, 2008) and had used them for evaluation. Several practitioners stated that the guidelines had been augmented to included alternative wording, additional points to consider etc. None of these amendments seem to be fed back to the W3C or being shared which reduces the usefulness of the guidelines. However, of the approximately 50 people present only a handful were aware of the RNIB guidelines (Chandler, Dixon, Pereira, Kokkinaki, & Roe, 2005) and none of them had had an opportunity to use them.

Release
UCD consideration should not stop once a product has been released. Most research that we are asked to do about mobile devices post-release are about the purchasing process, point of sale research and the out-of-box experience. It is rare that we are asked to do longer term studies into the learning and adaptation that is likely to take place over time.

We usually evaluate the out-of-the-box experience using expert reviews assessing the packing, wires, user-guide and we set up user journeys. We also do expert reviews of handsets and mobile websites. Observing the experience at high street stores is difficult due to recording issues. Assessment might also have an environmental impact focus (reducing packaging and documentation) or a purely usability focus (using heuristics). We have also used focus groups to explore issues that consumers had with phones they had been using for some time.

For blind and visually impaired consumers the purchase and post purchase situation is dire, with little information available about the relative merits of different handsets at high street stores. While we have not done any specific research on accessibility needs post purchase, the RNIB recently organized an event to help members to choose a handset, which we attended in an effort to gain further insight to share with members of the UK UPA. For the participants it was vital to have real hands-on experience with the devices, something that is lacking in high street stores.

Discussion and Conclusion

What is clear is that quality research in each of the phases of UCD helps gain a greater understanding of some aspects of the problem space. In addition, if the research is of a high quality then the practitioner earns the respect of the client and the relationship is improved. Practitioners can then be more assertive about which methods are preferred and can educate the client about becoming involved in earlier research and including more user involvement. Because of the complexity of the mobile device and the contexts in which the devices are used, different research methods are better in certain situations. While ‘in the wild’ research has many benefits, lab-based research can also offer useful insights and improve the overall client relationship by allowing them to participate more actively.

Many practitioners have never used guidelines, which raises the issue of their effectiveness. Content of the guidelines are often too specific making them difficult to use. Many practitioners also do not know about specific guidelines. Many designers also do not know how to apply them and clients often ignore them believing that the functionality will make up for any lack. The complexity of the space makes guidelines too simplistic. More research into how to present the guidelines better may help. In addition, a variety of methods are available to conduct research, but some are better applied at different points in the design process. Practitioners need to help guide clients as to which are the best.

There are a couple of take-home messages from this snapshot of practitioner life. Firstly, there is no single right method for research as each situation is unique and the mobile space is complex. Secondly, the client still needs convincing to do quality research throughout the design life cycle. Long-term relationships between UCD practitioners and business-focused clients will help ensure that the best methods are used every step of the way. Finally, there is still a need to focus on core usability and accessibility when designing products and improve the use of guidelines to ensure quality mobile user experience is designed.

References

International Standards Organisation (2009). ISO 9241.

Gaver, W., Dunne, T., and Pacenti, E. (1999). Design: Cultural Probes, 6 (1), 21 – 29.

Larson, R. and M. Csikszentmihalyi (1983). The experience sampling method. New Directions for Methodology of Social and Behavioral Science, 15, 41-56.

Nuance Communications. (2010). Convenient audio access to mobile phones.

Chandler, E., Dixon, E., Pereira, L., Kokkinaki, A., & Roe, P. (2005). COST219ter: An evaluation for mobile phones. In P. Bust (Ed.) Contemporary Ergonomics (2006) (Taylor & Francis, London).

Rabin, J. & McCathie-Nevile, C. (2008). Mobile Web Best Practices 1.0.

Nielsen, J. (1994). Heuristic evaluation. In JL Nielsen and R.L. Mack (Eds.), Usability Inspection Methods. New York, NY: John Wiley & Sons.

Medlock, M.C., Wixon, D., McGee, M., & Welsh, D. (2005). The Rapid Iterative Test and Evaluation Method: Better Products in Less Time. In Bias, G., & Mayhew, D. (Eds.), Cost Justifying Usability (pp. 489-517). San Francisco: Morgan Kaufman.

Share and Enjoy Digg TwitThis Facebook del.icio.us Google StumbleUpon E-mail this story to a friend! Print this article!

]]>
http://wi.hexagram.ca/?feed=rss2&p=67
On the Dérive for Mobile Experience http://wi.hexagram.ca/?p=72 http://wi.hexagram.ca/?p=72#comments Mon, 20 Jun 2011 16:57:50 +0000 admin http://wi.hexagram.ca/?p=72 By Konrad Tollmar, Linus Harling, and Robert Ramberg

Introduction

This paper describes our deployment of a new method for observing and understanding the mobile user experience. This new method is built upon the concept of the Dérive, a theory and method appropriated from the French Situationists. In our research on mobile user experiences, the Dérive is used to inspire mobile storytelling.

The context for this work is an overarching research program - Contemporaries - where we study how the writing of stories could be integrated into everyday life to support people’s participation in social, economic, political and economic life. The broad aim of Contemporaries is to facilitate a multiplicity of voices in what Marshall McLuhan would call the new global village. One the most important points of our research agenda is the idea that media should be “Accessible to all”. Hence, we specifically focus on multimodal communication using a variety of techniques and tools for the mediation of expression. We ask if the media is an appropriate tool to illustrate and recreate individual or cultural expressions, and examine its costs, reliability, and ease of use. In this paper, we describe a case study investigating how youth, between the age of 14-18, could use mobile phones to express themselves through storytelling, how to make this activity meaningful and valuable, but also how to facilitate ease of use, to make it more efficient and more accessible. Our goal is to motivate and enhance the user’s experience of mobile storytelling.

To facilitate our workshops we used the Dérive method (a walk and an exploration of an environment without preconceptions, described below) as an inspirational tool to initiate this storytelling. Starting with a dérive using a mobile blogging tool, the storytelling workshop continues on a regular computer that is accessible either in the classroom or at home.

Most often storytelling is used to document user experience, however, in this case we have created a double loop where we also try to capture the user’s experience in the doing of the storytelling task.

In the rest of the paper we describe and discuss in more detail the methods we used. First, we elaborate about how to use and motivate school children using the Dérive method. Second, we discuss the outcomes. And- third, we sketch out some ideas on how mobile storytelling could go hand in hand with observing and understanding user experience.

On Dérive

Our research is based on methods that are anchored in both traditional HCI methodologies and action research. As researchers, we prefer to enter into an already established structure to see how actual work is done, rather than creating specific test groups. Our project has been supported by an active school community and municipal development plans in northern suburbs in Stockholm. However, action research methods often needed a catalyst to get started. In their Handbook of Action Research, Peter Reason and Hillary Bradbury (2001, p. 512) refer to the Scottish philosopher John Macmurray who notes that it is critical to emphasize “‘I do’ rather than ‘I think’” to initiate the learning process. Reason and Bradbury argue from this that the doing is the appropriate starting point for action research [ibid]. To emphasize doing, we chose to facilitate our workshops using the Dérive method. A dérive (drift) is an attempt at an analysis of the totality of everyday life through the passive movement through space (Debord, 1981). This method has been used in studies of architecture to explore a built environment without preconceptions. Many situationists have also used dérives for creating “psychogeographical maps”. These maps are built from the small snippets that form an understanding of bigger phenomena. As Simon Sadler (1998, p. 15) describes: “In discovering a small world we discover the whole world”. In a similar way we also read the school children collection of stories as a part of a bigger and shared story that form an identity of their dérive (Sadler, 1998).

Figure 1. Debord’s psychogeographical map: The Naked City (1957).

Situationism has gained some recent popularity in the HCI community (Gaver & Dunne, 1999; Home, 1996), mostly as inspirational tool to engage designers with modernist counter-culture. Our understanding of the dérive most closely resembles the idea of the cultural probe (Gaver, Dunne, & Pacenti, 1999). This uptake from a wide variety of disciplines to understand and design is rather typical for HCI, and its pros and cons have been discussed widely. Some would argue that these methods are too often used without reflection and reference to their intent. Our approach is to focus upon the meaningful and real stories that are generated by using the dérive as an inspirational tool.

Storytelling Workshops

The storytelling workshops conducted within Contemporaries included approximately 80 youths from the ages of 14-18. Our participants had an equal gender balance and came from mixed socio-cultural backgrounds. In these workshops, we used the dérive to initiate a new moment within an ongoing activity. For example, in a class that studied the industrialization period we talked about historical findings from the neighborhood and how these artifacts formed a shared collective memory that capture this place. A representative from Stockholm City Museum helped-out by bringing real objects to the workshop. She animated these objects by telling a small story about how they were initially used and how they were found after hundreds of years. With this in mind, we then demonstrated the method, went through a small training exercise, and then sent the students, equipped with mobile phones, out on a derive on their own.

In our workshops, we used two different set-ups. The first set-up used Android phones loaded with an application for mobile blogging. In the second set-up, we asked participants to use their own phones and standard tools, like SMS and MMS. We gave the participants vouchers to cover their costs, but this worked less well due to the broad range of cell operators that our young group are using. For forthcoming studies we would recommend using some kind of premium SMS/MMS services. We will return to a discussion about the relative trade-offs and advantages of each approach.

One of the difficulties we encountered was to explain what we meant by the dérive without being too unclear or specific. We learned in the pilot phase that some participants get “lost” and needed more specific instructions. However, if the instructions are too specific, then this could compromise the core idea of the dérive and lead to the generation of very few novel stories. We chose to provide some simple examples of how to do a non-planned movement through space, for example, creating non-deterministic rules such as to make turns on certain events (e.g. whenever you see a read car or meet a person with a red jacket). Furthermore, we briefly showed some examples of historical psychogeographical maps without going into the details. As a result, we learned the importance of conducting a small training exercise before the dérive, like sending a text update or taking a picture of themselves. These sessions had a significant positive impact on the users’ experiences of the dérive and influenced how much they used the mobile blogging tool. Once again, doing creates understanding.

We incorporated these lessons from the pilot phase into the first part of our subsequent workshops. In the second part of the workshop, the participants were introduced to the basics of WordPress and given access to personal blogs using the regular computers in the classrooms. An informal discussion with each participants while they was working with WordPress provided us with a rich feedback about the dérive and the meaning of their stories, as well as their experience of the mobile tools and the use of multimodal media and expressions. This discussion, or conversation, consisted both of pre-prepared questions as well as free form, open-ended conversation. The materials gathered during these workshops were of three primary types: voice-recordings from each session, the actual blogs produced, and their answers to the survey questions.

Two different kinds of workshop formats were tested. The first workshop involved a longer dérive, followed by a later session in a computer room, where they edited and added material to their blogs. The other workshop compressed both the dérive and the compilation of their collected material into a three-hour workshop. One substantial difference between the two workshops was the kind of material the participants were allowed to use. In the first workshop, they could use all of the photographs and videos collected, regardless if they had sent them or if they were stored on their mobile phone. In the other workshop, they were limited to working with material sent from their mobiles during the actual dérive. What did we learn from these two different formats, and two different set-ups?

First of all, we learned that introducing a tool, like our mobile blogging tool for Android phones, could fall short of our expectations. We observed in the pilot phase that the mobile blogging tool often hindered the participants in their dérive with technical obstacles. The use of well-known, familiar SMS/MMS services took much less effort and enabled the participants to use their own phones. Most important, this provided better results in terms of their stories; mainly because they where more detailed (e.g. more text) and much more frequent (due to familiarity with SMS). But we also learned that there is a need to provide better feedback through the SMS/MMS services to engage participants into further use. The most common comment was that usually when you send a regular SMS/MMS you will get and then expect a prompt reply. This feedback can also partly mimic the online experience that otherwise is missing when using SMS/MMS services instead of an online mobile app. At one point the use of their regular mobile phones and SMS/MMS was less favorable. It was not possible to follow and comment on other work, their conducting of searches, or for connecting the material that was being generated with other resources. The integration between a mobile blogging tool and other online resources, such as social media and email, becomes more critical to integrate into mobile story-telling projects as the services become more commonly used. One clear observation we had was this: some questions generated in the dérive faded quickly away if they could not be concurrently explored. For example, it was commented that without being able to search for more information, e.g. Wikipedia, while they created the posts they was considered less interesting than others even later on. Nevertheless, the bottom-line is that providing advanced handsets seems to work less well. Most participants had sufficiently advanced phones of their own. Some additional service could be provided, and if needed, they could be hacked together on the server side.

Second, we learned that the longer workshop format resulted in a perceived lack of connection between the gathering of data and the manipulation of it. In the beginning of the project, we thought that the storytelling work would improve if participants were given a chance to let the experience of the dérive sink in, and they were given time to reflect upon it. This worked less well than expected. Many of the ideas gained by the deneed accentrive but not captured by the posts in the blog faded away rather quickly. It seems like there is a natural division between documenting a story and telling a story when using a mobile device. Very few wrote longer pieces of text on their mobile device. Rather, they used other forms of multimodal expressions and tagged their images with a few words that were later elaborated using the WordPress software on a regular computer. Hence, it seems that the storytelling has two natural phases, on mobile part for capture the moments with pictures and small notes, and one using the web-based authoring tool for WordPress on a regular computer.

Figure 2. Storytelling workshop Where are the images?

Figure 3. The Contemporaries web

Figure 4. Examples from the Contemporaries web

The use of WordPress as a blogging platform has worked well. We expected more problems here but almost all participants found it very easy to work with WordPress. This was unexpected. We had, for example, prepared templates that would simplify the WordPress authoring tools, but these where not needed. However, the input we received from the workshops suggested that workshop participants wanted to create and alter posts along a timeline, e.g. being able to control the sequence of the blogs. We also observed that our workshop participants lacked simple tools to help create dynamic groups of users that could follow each other blogs easily. This preliminary result points to the need for the creation of shared experiences in the workshops, as well as the need for tools that can facilitate greater interplay between the developments of the group and individual. In forthcoming studies we are planning to build some new additional WordPress tools along these lines.

Capturing the Mobile Experience

Carrying out research on mobile user experience is a difficult task. In order to observe and understand the mobile user experience, we need to capture multiple aspects of what people do and feel about using mobile phones and services. Most often, within the world of HCI, we develop mobile prototypes, deploy these applications on the personal handsets of real users and then observe what happens. Shifting needs, contexts and the ubiquitous use of mobile phones makes it very difficult to observe naturalistic mobile behaviours and ask intelligent research questions about mobile user experience.

The dérive method overcomes some of these problems. First of all, the method allows a balance between flexible versus closed instructions, and hence constrains some aspects of the context of use, at the same time as it allows for discovery. The bigger question here is whether the method can push for an open use of mobile media in storytelling to facilitate people’s participation in society. Winograd and Flores (1985), among others, argue that language is intrinsically tied to a situation. The context defines what the “words” mean as much as the “true” definition and composition of a, i.e. if you use an image that is easy to relate to that could become the common, and true, representation for a group of people, instead of an established text.

This leads us, secondly, to use the storytelling generated by the dérive as a mean to analyze the user experience. There are a couple of different ways of measure, or test, the quality of the overall user experience. Most common is through various forms of self-reporting methods, such as diary methods (Bolger, Davis, & Rafaeli, 2003) and Experience Sampling Method (ESM) (Csikszentmihalyi & Larson, 1987). In this case we have created a double twist. In the reasoning about the user experience we use a cyclical process, where studying and understanding the user experience becomes an intrinsic part of the actions employed. A dérive is a situation-creating technique aiming at turning the city around. This “turning around” or détournment is a dialectical tool and in this context could be used as a method to debate and discuss the mobile experience.

Notes

Nordic (not incl Sweden):4,6% Europe:14,7% Asia:47,2% Africa;27,5% Others:6,0%

A modified version of Postbot, http://nickthecook.wordpress.com/

References

Bolger, N., Davis, A., and Rafaeli, E. (2003). Diary methods: Capturing life as it is lived. Annual Review of Psychology, 54, 579–616.

Csikszentmihalyi, M., and Larson R. (1987). Validity and reliability of the experience-sampling method. Journal of Nervous and Mental Diseases, 175 (9), 526-536.

Debord, G. (1981). Theory of the dérive. Situationist International Anthology. Berkeley: Bureau of Public Secrets.

Froehlich, J., Chen, M. Y., Consolvo, S., Harrison, B., & Landay, J. A. (2007). MyExperience: A system for in situ tracing and capturing of user feedback on mobile phones. Proceedings of the 5th International Conference on Mobile Systems, Applications and Services (San Juan, Puerto Rico, June 11 - 13, 2007). MobiSys ‘07. ACM, New York, NY, 57-70.

Forlizzi, J. and Battarbee, K. (2004). Understanding experience in interactive systems. Proceedings of the 5th Conference on Designing Interactive Systems: Processes, Practices, Methods, and Techniques (Cambridge, MA, USA, August 01 - 04, 2004). DIS ‘04. ACM, New York, NY, 261-268.

Gaver, W.W., and Dunne, A. (1999). Projected pealities. Proc. CHI, (pp. 600-607). ACM Press.

Gaver, B., Dunne, T., and Pacenti, E. (1999). Cultural probes. Interactions, 6 (1), 21-29.

Home, S. (Ed.) (1996). What is situationism? A reader. AK Press, San Francisco, CA, USA, 1996.

Reason, P. and Bradbury, H. (Eds.) (2001). Handbook of action research: Participative inquiry and practice. Thousand Oaks: Sage.

Sadler, S. (1998). The situationist city. Cambridge: MIT Press.

Winograd, T., and Flores, F. (Eds.) (1985). Understanding computers and cognition. Ablex Publishing Corp.

Biographies:

Konrad Tollmar is an Associate Professor at The Royal Institute of Technology. His main research interest is to better understand how interactive technologies become a part of people’s everyday practice and life. To get there Konrad tries to combine interaction and co-operative design with novel use of technologies, such as computer vision, mobile computing and virtual reality. From 2010 he is leading the Mobile Service Lab at KTH / ICT / CoS with a focus on mobile infrastructure and mobile services. Prior to this, I worked at MIT and The Interactive Institute. Konrad’s most recent research has also expanded into market research and analysis where he is a research director at the Institute for Economic Research.

Robert Ramberg got his PhD in cognitive psychology at the department of psychology, Stockholm University and now holds a position as professor at the department of computer- and systems sciences at Stockholm University (SU). At the department he is the research director of K2-lab (the Knowledge and Communication Laboratory). He has published numerous articles in journals and refereed conferences. He has served as program committee and editorial board member for several international conferences as well as acted as reviewer for several international journals within the field of technology enhanced learning.

In the early 90s his research had a strong focus on trust related issues where aspects such as understanding and learning from information communicated by human and artificial expertise were a part. Over the years his interest in theories of learning (socio-cultural perspectives on learning and cognition), pedagogy and how these theories must be adapted when designing and evaluating technology enhanced learning and training environments has grown. Of particular interest is how artifacts of various kinds (information technology and other tools) mediate human action, collaboration and learning. A current research interest is mobile learning and collaboration

Share and Enjoy Digg TwitThis Facebook del.icio.us Google StumbleUpon E-mail this story to a friend! Print this article!

]]>
http://wi.hexagram.ca/?feed=rss2&p=72
Into the Grey Zone: Seniors, Cell Phones and Milieus That Matter http://wi.hexagram.ca/?p=69 http://wi.hexagram.ca/?p=69#comments Mon, 20 Jun 2011 16:56:20 +0000 admin http://wi.hexagram.ca/?p=69 By Kim Sawchuk and Barbara Crow

Introduction

Within the burgeoning literature on the everyday and innovative uses of cell phones and mobile technologies, there is a concentration of detailed statistical or ethnographic data on those who are young or middle-aged (Ito, 2005; Caronia and Caron, 2004; Thulin & Vilhelmson, 2007). With the exception of a handful of articles (Wong, Thwaites, & Khong, 2008; Lee, 2008), much less attention, scholarly or otherwise, is paid to those who are fifty-five and over: this demographic constitutes a ‘grey zone’ literally and metaphorically (Harris-Decima, 2008). Our research on ‘Seniors and Cells’ rectifies this absence and is intended to contribute, productively, to the discussion of the intertwining dimensions of age, technology, and the everyday practices of citizenship by differentiating between ‘shades of grey’: we highlight what they do, and try to make sense of it in their terms, rather than comparing seniors with more ‘active’ user-groups.

While we cannot claim, at this stage of the work, that we are in a ‘truly mobile setting’, our research has brought us into milieus that matter to our subjects: milieus are spaces of encounter and exchange, and not merely sites of data collecting and gathering. We discuss two sorts of milieu: intimate individual exchanges comprised of one-on-one conversation, and social interactions that break the isolation and loneliness often experienced by seniors. These milieus take shape within broader national contexts of telecommunications infrastructures and policies that influence and structure individual choices. We end with a discussion of some of the practical strategies we have adopted for engaging with these users from a perspective which allows them to transform the research agenda (Walker, 2007). In this paper we describe some of the broader lessons learned from our project to date, and reflect upon our research process and practice.

Canadian Context

Before examining our reasons for engaging in discussion groups in local settings, there are some data to consider when researching age in relation to wireless, mobile media in Canada’s particular national milieu. Here we would like to clarify that we do not see our research in opposition to statistical research, but as complementary to statistical overviews. Large-scale quantitative studies provide a picture of how systemic conditions might impact the everyday practices of cell phone use in elderly populations.

Seniors and Cell Phones in Canada

Figure 1: Bronx Park, Winnipeg

The number of seniors in Canada is predicted to double from 4.2 million at present to 9.8 million by 2038 (Statistics Canada, 2007). This is a dramatic increase in the population, which has led to grave warnings in the Canadian media about the emergence of ‘a grey tsunami’ threatening to bleed dry the resources of the state, with rising costs for medical care, housing or other social services. There is a climate of fear around ageing reinforced by these discourses that our study’s participants challenge overtly in their comments on encounters with ageism, and more subtly, through the liveliness of their engagements with friends and family.

Chart 1: Population 65 years and over, Canada.

Source: HRSDC calculations based on Statistics Canada. Estimates of population, by age group and sex for July 1, Canada, provinces and territories, annual (CANSIM Table 051-0001); and Statistics Canada. Projected population, by projection scenario, sex and age group as of July 1, Canada, provinces and territories, annual (CANSIM table 052-0005). Ottawa: Statistics Canada, 2010

The mobile phone, or cell phone as it is called in Canada, is rapidly displacing the landline telephone for person-to-person communications here, as it is worldwide. Cell phones are also increasing in popularity in our country, which has tended to have lower adoption rates than elsewhere. On average, 72% of Canadians now own a cell phone, a steady increase since 1997. The lowest rates of ownership are in Quebec and amongst those fifty-five and over.

The Wireless Industry in Canada

As a recent report on the cell phone industry indicates (Nowak, 2010), cell phone companies make enormous profits for services that many Canadians feel are overpriced and inadequate. Such media reporting on the industry has been backed by independent inquiries made by digital research institutes that confirm that Canadians are paying extremely high rates for their cell phones. Compared to users in other countries, Canadians are often locked into draconian service contracts, and can experience punitive fees if they break these contracts (Marlow, 2010). The telecom regulation that historically guaranteed reasonable rates for landline phones in Canada has not been applied to cellular services.

Chart 2: Cell Phone Subscriber Growth in Canada

Landlines have been reliable and inexpensive, comparatively speaking. These conditions influence seniors’ responses to our study and to us; they must be accounted for in our analysis of the individual and household choices made on cell phone use. Hence, to quickly summarize the results of our preliminary data analysis, we have found that seniors tend to restrict their practices to a few functions, share the cell phone between spouses, use pay-as-you go cards for monitoring minutes, and consider carefully who is given access to a phone number. These practices do not stem from mere ‘fears’ of entering into the brave new world of mobile technology. Instead, as we have seen, these ‘restrictive’ practices are logical choices given the infrastructural conditions in Canada. Understanding these systemic conditions that influence individual choice also makes us hesitant to use quasi-psychographic terms, based on survey research, to develop user profiles (Lee, 2008). Such profiling does not account for individuals or social groups in the context of their milieus.

Seniors’ Local Milieus

It is within this context that we are conducting our research. To date, we have held formal group discussions with over one hundred and twenty people who are sixty-five and over, accepting invitations into their community centres, legions, church halls, and homes. Over the past three years, we have engaged in a small number of early one-on-one interviews and countless informal conversations with retired individuals in shops, on the street and in cars on the subject of ageing and technological practices. We have received unsolicited emails from retired people who have offered encouragement and their own testimonials upon hearing of our project. While technically, only the interviews have been approved by our University research ethics committee, all of these conversations constitute valuable source materials for understanding the digital desires and frustrations of senior users.

Figure 2: Seniors Centre, Qualicum Beach

Figure 3: Cowichan Seniors’ Centre

Our entry, albeit brief, into these local contexts provides crucial information about our subjects and their lives in relationship to the lifeworld that might not be shared in a survey or interview situation. Entry into these spaces gives us insight into the lives and mobile practices of both users and non-users. This latter group is particularly important to us. Just as we have been concerned with the reasons this population restricts use, our conversations with seniors indicates that the reasons for this ‘non-use’ are extremely complex, and need more attention. In this our project dovetails with the work of researchers such as Sally Wyatt (2003), who see use and non-use as part of a longer continuum of practices.

Iterative Adjustments

In developing a multi-pronged research agenda, we have adopted methods of data collection that draw from our past experience in developing user-tests, guidelines and protocols for artists and engineers that are participatory and iterative in focus and in practice (Crow & Sawchuk, 2008). Participatory research design asks subjects to play a role in setting the terms of the research agenda. Iterative research design suggests a constant re-adjustment of the research strategies over time, as one learns ‘in the field.’

While related to ethical issues in ethnographic research, including feminist empowerment research, these research strategies stress social change, and are based on ongoing dialogue and the researcher’s accountability to participants at all stages and phases of the research plan. Unlike empowerment research, the demands we place on subjects to maintain involvement in our project is minimal. We are not looking to institute change in a community, but to bring attention to those who have become invisible. As word of our study has gone out into the communities we contact, we frequently find that we have more seniors wanting to talk with us than we have time to give.

From an ethical perspective, in a short research note on working in the field of gerontology, Alan Walker (2007) makes the crucial point that the ‘older research subject’ should ideally be an active participant in setting the research agenda for epistemological, ethical and political reasons. This awareness and transformation of the research agenda is imperative when dealing with the elderly because of the rampant existence of age discrimination and social exclusion often experienced by this cohort. Given the lack of satisfactory studies with this cohort of users to act as a comparative benchmark, and our contextual approach, a pilot project with eight elderly intimates was critically important. During the pilot phase, we were able to test interview questions, develop a small survey, and most importantly negotiate our language and central ‘concepts.’ Taking into account the lack of literature on seniors in media studies, we used these intimates to test initial hypothesis and intuitions, and to work out our own biases and presuppositions.

Local Help: from Informants to Mediators

Our local ‘organizers’ are seniors themselves and most often, our initial point of contact has been through family and friends. In ‘Approaching the Elderly,’ John Tulloch (1989) discusses openly the pitfalls and advantages of working with family members, which he sees as valid when working with populations that feel vulnerable. We have also made contact with individuals in existing volunteer and local organizations for seniors: a guild of quilters; a resource centre; a swimming group; a community centre. After retirement, many seniors also get involved in volunteer work for others, and these organizations have been helpful for not only giving access, but as a reminder that active ageing is not only possible, but actual. Local helpers have also provided material comforts for our groups: coffee, juice and snacks. They play the critical role of mediators in the research process (Latour, 2005), and have not acted as informants in the classical anthropological sense.

Figure 4: St. Thomas Church, Qualicum Beach, British Columbia

Figure 5: Yorkgate Mall, Toronto, Ontario

The Old is Always Other Section

What constitutes a senior is a contested category (see Riggs, 2002) and protocols for addressing this cohort are uncertain, given the range in ages from the recently retired ‘young-old’ to the ‘old-old.’ Ageing, we were reminded over and over again, is not only a demographic variable or a biological condition: it is also a question of perception, ‘a state of mind.’ What was interesting to hear was that no one sees themselves as old. The old is always other. Old is associated with a lack of both agency and mobility. We learned that they felt more comfortable with the term ‘senior’ than with other identity categories associated with ageing.

Figure 6: Senior Special

Incorporating Critique

Our seniors were willing to engage in what are arguably insightful critical discussions of our research program, the current literature and cultural presuppositions about age and ageing. In some instances they have acted in a consultative role setting the research agenda, formulating initial questions and helping make contacts with others. In a more recent encounter, we have been told that our ethics forms were too long and complicated and changed them in response. They have made suggestions to our survey, asked us to increase font size to make the text easier to read, and actively worked to set up interviews with their constituents. In discussion, they have corrected us when we have revealed our own ageist presuppositions. One early lesson for us was when we asked if they were ‘still driving’ which lead to a direct confrontation with our use of the word ‘still,’ which implied being incapacitated. Further, what this revealed is that the concept of mobility is about physical mobility, movement through space, as much as it is about a mobile device. The phone is not only a part of media ecology but a whole system for staying mobile and active: driving and public transport; exercise and walking; having and exerting agency. Conversation on other interests also made us initially attentive to emergent patterns, including pre-retirement and post-retirement work and the gravitation to particular digital devices.

Media Ecologies: Phone Alone?

One of the issues in studying any technology is that the focus of the study often isolates the technology from other uses and practices. Many of our early participants did not want to talk only about cell phones, and often diverted from the discussion of this technology to the question of other digital media. This pattern was repeated in all of our discussion groups, where we were reminded continually that technologies do not exist in the lives of individuals or households in isolation; that there was a ‘media ecology’ of multiple technologies for communications.

This led to the important finding that the ‘restriction’ or ‘rejection’ of the cell phone did not constitute a resistance to new forms of communications from these users. The cell phone exists as a choice among several options, including Skype, the landline phone, and email. Choices were made based on expedience and cost, the experience users had from pre-retirement occupations, and the demands of their interlocutors. This type of finding points to the need for long-term contact with participants to track the reasons individuals may adopt new practices, exchange technologies, or exit the cell phone scene altogether. In addition to tracking such changes over time, we have situated cell phone use in terms of income levels: for example, the pressure felt by seniors who balance a home budget and life on a fixed income to keep up with the costs of maintaining services, engaging in upgrades, or using multiple functions.

A Space for Non-Users

Leaving space for the non-users involves accounting for those who would be left out of the conversation if we used more ‘objective’ means of gathering data on our subjects through a technological device (such as a tracking mechanism on their mobile phones). As we have argued elsewhere (Crow and Sawchuk, 2010), one of the other biases our research addresses is the tendency in media studies to focus on the exuberant user of technology. In our study, the perspective of these non-users has become extremely critical, for it challenges the assumption that only active users or owners of a mobile device are affected by the transition to wireless, mobile means of communication. The increasing lack of public telephones is but one example of how non-users are affected by broader cultural shifts. It also means that instead of the research being about us ‘getting information from them,’ the discussion groups have also become spaces where non-users come to find out from users why they should or should not get a cell phone, what plans they might get, what options and features on the phone to look for, and tips about how it might best serve them.

Figure 7:  St. Boniface, Winnipeg

We Are Not Selling Anything

This was particularly important when dealing with our seniors: one of the important points of reassurance we had to offer was that we were not marketing researchers working for cell phone companies. We had to convince them that we were not trying to sell them something. This, we realized, is related to one of the main issues of this group: their distrust of telecommunications companies, and their sense of vulnerability as a population in relation to unscrupulous researchers and scam artists trying to get money from them.

From Focus Group to Discussion Groups

Initially we termed our research as ‘focus group discussions.’ During the course of our research, we have preferred to use the term ‘group interviews’ in order to stimulate a discussion amongst seniors, rather than simply read a list of questions. For this cohort, ‘focus groups’ imply that we are situated within the paradigms and parameters of marketing research, often affiliated with the much despised phone companies. But it also seemed as if we had a definite research agenda. ‘Discussion group’ reframed the terms of engagement as allowing for a much more meandering flow of conversation, guided not only by our questions, but by the participants’ interests and needs, discussed within a much less formal context.

Money Talks

One critical aspect of recruitment, but also part of the politics of our project, that is rarely talked about is the issue of money. We have paid our subjects for their time: CAD$20 per hour, which usually translates into CAD$40 cash in-hand for each participant. We also know from experience that there are class divisions in the doing of research: doctors and lawyers are paid for their time in focus groups commensurate with their income and status. Survey research with ‘ordinary’ people frequently asks of time on a phone for an hour two, but the pay scales are different. Paying our participants for their time was an ethical and political decision that benefited us. Word got around about the compensation, which valorised their time as important and they loved this. Seniors are sometimes seen as people with ‘time on their hands’ and ‘they like to talk’ as if they have nothing better to do. Offering money for their time and talk was an affirmation that their insights were valuable.

But we also had insight into who they were because of what they told us they were going to do with the money. Given the significant socio-economic differences between our subjects, their responses to having their time acknowledged and their participation rewarded in tangible terms was telling. For one group, CAD$40 is a week’s worth of groceries. For another group, this money represented a special lunch with a friend. For another group, our research was used as a fundraiser for their religious organization. For yet another group, this money was coveted as a way to purchase quilting materials for their favoured hobby.

Conclusion

Our foray into the ‘grey zone’ has revealed much to us, from methods of researching seniors to considering mobility and mobile use in more complex and nuanced ways. While we have collected some qualitative data on our participants, our research largely relies on verbal testimony as well as our own observations of friends and relatives who are in this demographic and who have actively assisted us. In our understanding of what we are ‘getting’ in these conversations, we operate from within the perspective of repertoire analysis, defined as ‘recurrently used systems of terms used for characterizing and evaluating actions, events and other phenomenon’ (Potter, & Wetherell, 1987). As Joke Hermes (1996) explains, interpretive repertoires are ‘a storehouse of possible understandings, legitimations, and evaluations that can be brought to bear on any number of subjects.’ We are in the midst of this analysis of our data, using the TAMs analyzer open source software program to systematically document emergent terms and themes from the volumes of data we have collected: each group discussion is comprised of text ranging from forty to seventy pages in length.

Statistical portraits draw attention to larger discursive and social patterns, but this type of ‘survey’ can operate effectively in collaboration with qualitative data to offer insights and construct categories that are meaningful to the population studied. The technical collection of data from the devices themselves may also not be appropriate for this cohort. There is much to be said for studies that track mobile users and do not require them to fill in details, instead relying on software programs (such as Mobitrak) and the phone itself to gather data. However, we are not sure if such studies would be either possible or desirable given the specificities of our group of participants. Considering that this cohort does not use or want many of the functions of the cell phone, imposing such a device will only give us access to some respondents. There are important cultural differences between this generation and younger users in their feelings about the need for privacy, as our discussions on these matters have indicated.

Our research suggests that the methodologies for mobile users and usage in the ‘grey zone’ are enriched when we engage with them in their milieus on their terms. As we enter into the next phase of the research, it is the insight gleaned through contact, conversation and entry into these milieus that matters. This insight will guide the analysis, interpretations, and positions we inevitably must put forth as the authors of this study as we seek to make seniors matter within the ever-shifting terrain of mobility studies.

Acknowledgments
Our thanks to the Social Sciences and Humanities Research Council of Canada for their support of this project (#410-20091553).

Notes

Population projections use a medium-growth scenario (M1) based on interprovincial migration trends from 1981 to 2008. For further information see: Statistics Canada. Population Projections for Canada, Provinces and Territories (2009-2036). (Cat. No. 91-520 XIE).

References

Caronia, L., and Caron, A. (2004). Constructing a specific culture: Young people’s use of the mobile phone as a social performance. Convergence, 10 (2), 28-61.
Crow, B., and Sawchuk, K. (2008). Shaking hands with the user: principles, protocols, and practices for user-integrated testing in mobile design. In M. Ladly & P. Beesley (Eds.), Mobile Nation (pp. 37-42). Toronto: Riverside Architectural Press.
Crow, B., and Sawchuk, K. (2010, May 8-10). Embracing the restrictive user. Paper presented at Cultures of Movement Conference at Royal Roads University, Victoria, BC, Canada.
Harris-Decima. (2008). 2008 Wireless Attitudes Study. Ottawa: Canadian Wireless Telecommunications Association.
Hermes, J. (1996). Reading women’s magazines: An Analysis of everyday media use. London: Polity.
Ito, M. (2005). Mobile phones, Japanese youth and the replacement of social contact. In R. Ling, & P. Pedersen (Eds.), Mobile communications: Renegotiation of the social sphere`(pp. 131-148). London: Springer.
Latour, B. (2005). Reassembling the social: An Introduction to actor-network theory. Oxford: Oxford University.
Lee, Y.S. (2008, March). Older adults’ user experiences with mobile phones: user cluster identification. In C.W. Khong, C.Y. Wong and B. von Niman (Eds.), Proceedings of the 21st International Symposium on Human Factors in Telecommunication: User Experience of ICTs (pp. 39-47). Englewood Cliffs, NJ: Prentice Hall.
Marlow, I. (2010, August 10). Cell phone contracts slammed. Globe and Mail.
Nowak, P. (2010, July 19). Canadian wireless firms still tops in profit: report. CBC News.
Potter, J., and Wetherell, M. (1987). Discourse and social psychology: Beyond attitudes and behaviour. London: Sage.
Riggs, K.E. (2002, June). The new, new deal. Global Perspectives and Partnership on the Information and Communication Technology Divide. Panel Presented at Informing Science: InSITE – Where Parallels Intersect, Cork, Ireland.
Statistics Canada. (2007). A Portrait of seniors in Canada. Ottawa: Government of Canada.
Thulin, E., and Vilhelmson, B. (2007). Mobiles everywhere: Youth, the mobile phone, and changes in everyday practice. Young, 15 (3), 235-253.
Tulloch, J. (1989). Approaching the audience: the elderly. In E. Seiter, H. Borchers, G. Kreutzner, and E.-M. Warth (Eds.), Remote control: Television, audiences and cultural Power (pp. 180-203). New York: Routledge.
Walker, A. (2007). Why involve older people in research? Age and Ageing, 36 (5), 481-483.
Wong, C.Y., Thwaites, H., and Khong, C.W. (2008, March). ‘Oh! My battery was drained because I forgot to press the end call button.’ In C.W. Khong, C.Y. Wong, & B. von Niman (Eds.), Proceedings of the 21st International Symposium on Human Factors in Telecommunication: User Experience of ICTs (pp. 31-38). Englewood Cliffs, NJ: Prentice Hall.
Wyatt, S. (2003). Non-users also matter: the construction of users and non-users of the internet. In N. Oudshoorn & T. Pinch (Eds.), How users matter: The Co-construction of users and technology (pp. 67-79). Cambridge: MIT.

Biographies

Kim Sawchuk is a Professor in the Department of Communication Studies at Concordia University. A feminist media studies scholar, Sawchuk is interested in new media art, wireless and mobile media technologies and the politics and culture of health and biomedicine. She is a co-founder of Studio XX (1996), the Mobile Media Lab (2006), wi: journal of mobile media (2008) as well as the former Editor of the Canadian Journal of Communication (2005-2011).

Barbara Crow is Associate Dean, Research in the Faculty of Liberal Arts & Professional Studies at York University, Toronto. Her research interests are in the areas of gender, digital and mobile technologies. She is currently the co-director of the Mobile Media Lab and one of the co-founding editors of wi.

Share and Enjoy Digg TwitThis Facebook del.icio.us Google StumbleUpon E-mail this story to a friend! Print this article!

]]>
http://wi.hexagram.ca/?feed=rss2&p=69
Interactions With and Through Mobile Phones: What About the Elderly Population? http://wi.hexagram.ca/?p=66 http://wi.hexagram.ca/?p=66#comments Mon, 20 Jun 2011 16:55:48 +0000 admin http://wi.hexagram.ca/?p=66 By Dr. Mireia Fernández-Ardèvol

Introduction

Age plays a role in the adoption and uses of mobile telephony. This evidence has been discussed since the first stages of popularization of this technology (see for instance, Ling, 2002; Castells, Fernández-Ardèvol, Qiu, & Sey, 2006). Furthermore, as we have argued elsewhere, there is a general trend “toward the general diffusion of mobile communication within the whole population, with age continuing to specify the type of use rather than the use itself” (Castells et al., 2006, p. 41).

It seems that elderly persons are less inclined to use mobile communication; however, they are “catching up to the levels of mainstream innovation, but largely lag behind in the use of new services integrated into the technology” (Karnowski, von Pape, & Wirth, 2008, p.191). Recent statistics on the use of mobile phones and the use of advanced mobile services from Eurostat confirm this trend (Eurostat, 2010).

Despite lower acceptance rates of mobile telephony among the elderly population, which indeed are quite high compared to other ICTs, seniors must be carefully studied when it comes to understanding the use and appropriation of mobile communication. Ageing is a key characteristic of European societies (Giannakouris, 2008). In 2008, 17% of the total population in EU27 was aged 65 years old or over; while this proportion will increase to 20% in 2020, and up to 30% by 2060. Therefore, it is of great interest to study the current situation and the future evolution of adoption and use in this age cohort. Future studies, as well, should take into account the evolution of mobile use as those that became mobile users during their youth get older. At present, however, most of the research focus is on individuals who have been introduced to mobile communication later in their lives.

This short paper is organized as follows; in section two I present the conceptual framework based on the most relevant current knowledge on the field. In section three I discuss the most recent quantitative data from Eurostat on how different generations use mobile telephony. Finally, in section four I conclude with an analytical explanation that gathers the available evidence regarding the elderly population in a developed context like Europe. I will argue that among senior populations, mobile telephony is an extra layer in the whole set of communication tools but it is not perceived as being central to their everyday life.

Mobile Phones and The Elderly: What Do We Know?

Personal communication is affected by age, and so are the information and communication technologies (ICT) mediating these communications (Charness, Parks, & Sabel, 2001). Ageing is related to socio-cultural aspects; thus personal values and interests change over one’s lifetime. Moreover, ageing shapes physical characteristics as well: from cognition or reading capacity, to more basic abilities, like handling small featured devices. As argued by Charness and Boot (2009, p. 255), “in a very literal sense, older adults may perceive technology differently than younger adults do.”

Hence, the main concerns regarding elderly users are ergonomics and usability. Since the early launch of the Japanese raku-raku (or “easy-easy”) in 2001 by Docomo, the market has experienced an increased uptake of mobile handsets specifically designed for the elderly. A number of different operators and handset producers have introduced “non-frill” mobile phones in the market. For instance, the “Vodafone Simply” handset appeared in 2005; while in 2006 Jitterbug Wireless was created, a USA company focused on “easy-to-use” services and mobile phones. An increasing number of studies are devoted to the identification of features and characteristics a mobile device should have to properly fit elderly attitudes and aptitudes (for instance, Duh, Do, Billinghurst, Quek, & Husueh-Hua, 2010; Kurniawan, 2008; Kurniawan, Mahmud, Nugroho, 2006; Mohd, Hazrina, & Nazean, 2008). While these studies discuss ergonomic issues and propose solutions, an interesting result also comes up: few elderly people buy “non-frill” handsets (Karnowki et al., 2008) because they are not interested in mobile phones targeted to aged people (Oskman, 2006).

The effective use of mobile devices is not only related to ergonomic and usability issues, but also to communicative habits, which among the elderly, are mainly centered on the maintenance of family relationships (Oskman, 2006; Kurniawan, 2008; Kurniawan et al., 2006). In terms of use, older people are more likely to use mobile phones in emergencies or unexpected situations (Hashizume, Kurosu, & Kaneko, 2008; Kurniawan, 2008; Kurniawan et al., 2006; Mohd et al., 2008) when they consider it to be the most efficient tool to communicate with. They usually do not use their mobile phones for casual conversations, except when they need to call another mobile phone and the cheapest way to establish contact is by using a mobile (Kurniawan et al., 2006). Thus, even though other means of communication seem to be preferred among this age cohort, older people will tend to use a mobile phone largely when it is perceived as necessary. For elderly people, as well as for teenagers, mobile phones need to be useful, social and enjoyable in order to be adopted (Conci, Pianese, & Zancarano, 2009). However, despite the common supporting logic explaining the adoption process of mobile phones, the final result is not the same in each age cohort because intensity and patterns of use differ. As Karnowski et. al. remark, “it seems that the elderly are always behind [regarding innovative services] while the younger are always ahead, already using the latest technologies when the elderly are still trying to catch up on yesterday’s innovation” (Karnowski et al., 2008, p.189). Hence, the significance of the mobile phone is different for an adolescent than for a senior citizen (Oskman, 2006).

The most important mobile phone service for elderly people is voice calls, whereas they are less likely to text (Ling 2002, 2004, 2008; Lenhart, 2010; Kurniawan et al., 2006). Oskman observes that, “initial use is characterized by caution” (Oskman, 2006, p.14). Once the elderly person is accustomed to it, the device is incorporated into everyday use. However, very often it is the members of the elderly person’s personal network who are the proactive part of the communication (Ling, 2008; Mohd et al., 2008). This is the case at least in the first stages of adoption while some differences in the pattern of use have been described for different countries. For example, in northern Italy (Conci et al., 2009) or in England (Kurniawan, 2008), reported uses by the elderly are more basic than those reported in Finland (Oksman, 2006). From the elderly perspective, use depends on personal willingness as well as on the expectations that others put on them to use mobile features. However, such reluctance could turn into acceptance if the service meets the needs of the person (Ling, 2008).

To be incorporated into everyday life, mobile phones must demonstrate an acceptable level of usability, compared to other means of communication that would satisfy similar communicative necessities of the individuals. Indeed, the use of mobile phones should be considered as one element of the personal system of communication channels. We define this as the set of communication channels that are used on a regular basis: fixed phone, mobile phone, Internet, face-to-face communication and even letters or telegrams. Each person will identify a different set of channels in their everyday life. The set of channels might be framed by individual attitudes and aptitudes, as well as by personal interests and socially imposed interests or pressures. Accessibility and availability of communication tools become critical aspects, as it is use and not ownership which is the key element that defines the personal system of communication channels.

The personal system of communication channels is framed by individual attitudes and aptitudes, as well as by personal interests and socially imposed interests and pressures. While among teenagers the mobile telephone plays a central role (see, for instance, Ling 2004; Castells et al., 2006), for the elderly population this does not seem to be the case, as can be deduced from available evidence discussed above. The senior population is quite likely to dispose of a fixed phone line at home, which could occupy a central position in the system. Adults having a fixed phone line at home would prefer it to a mobile phone as the cost of communication is lower, both in monetary terms and usability terms. Regarding monetary cost, elderly persons might only call with a mobile phone when it is necessary (emergencies or last minute coordination) or when calling another mobile phone. Regarding cost of use, ergonomics is important. Fixed phones might be more user-friendly for some elderly people partly because fixed telephony is a well known technology for them.

Mobile Adoption in Europe: What Does the Data Say About Elderly Users?

In 2008, there were 121.7 active mobile subscriptions per 100 inhabitants in EU27 while in 2009 the penetration rate increased to 124.6 (ITU, 2010). Does that mean that every European citizen is a mobile phone user? Statistics from the industry fail to give details on the socio-demographic distribution of mobile telephony. However, household statistics enable us to answer this question and to better understand the distribution of this general purpose technology (Jovanovic & Rousseau, 2005) among different segments of the population. As well, not all the mobile subscriptions in a country correspond to those living there, as any non-resident, for instance a tourist, can easily buy a prepaid mobile. But none of these surveys address whether mobile users have more than one mobile subscription.

European Union (EU27): An Aggregate Description

In 2008, 87% of the EU27 population between 16 and 74 years old claimed to be mobile users (Table 1). Indeed, almost every young person (16-24 years old) and adult (25-54 years old) use mobile telephony (97% and 93%, respectively). In this landscape of very high diffusion, the senior population shows a sizeable difference as only 79% of those between 55 and 64 years old declare being mobile users, while the figure falls to 62% for elderly seniors (65-74 years old).

Table 1. Use of Mobile Phone by Age Cohort in European Union (EU27), Year 2008.

 

Age cohort

All

Youth

Adults

Seniors

16-74

16-24

25-54

55-74

55-64

65-74

 

 

 

 

 

 

 

Mobile user (% individuals)

87

97

93

72

79

62

Male

88

96

93

75

-

-

Female

86

97

92

69

-

-

Low education

79

94

87

63

-

-

Medium formal education

91

98

93

77

-

-

High formal education

95

100

97

87

-

-

-: not available.

Source: Eurostat (2010).

On the other hand, the allocation of users between genders is quite balanced (88% of men and 86% of women) while the most uneven situation is observed among senior citizens. In the 55-74 age cohort there are 6 percent points of difference between men (75%) and women (69%). In addition, differences regarding education level are most pronounced among seniors (55-74), as in this age cohort mobile users are 87% among those with high formal education and 63% among those with low education (odds ratio equals to 1.4). Finally, it should be noted that socio-demographic differences among youth are markedly low because of mobile telephony saturation with practically every person from 16 to 24 years old using them.

Table 2. Type of Subscription to Mobile Telephony, by Age Cohort in European Union (EU27), Year 2008

 

Age cohort

All

Youth

Adults

Seniors

16-74

16-24

25-54

55-64

65-74

 

 

 

 

 

 

Subscription (% of users)

 

 

 

 

 

Pre-payment

39

42

35

45

55

Post-payment

47

47

52

41

29

Post-payment with flat rate for Internet access via mobile phone

4

5

5

2

-

Note: in the original, figures do not add 100.

-: not available.

Source: Eurostat (2010).

 

In terms of billing plans (Table 2), it can be seen that more than one third of the users have prepaid subscriptions (39%). Youth (42%) are slightly over the average, as when first introduced, the mobile phone tends to be a prepaid one. The young population turns to postpaid plans as they get older which are associated with higher consumption capacity levels (Castells et al., 2006). The senior population ranks even higher than youth, with 45% of subscriptions being prepaid among the 55-64 cohort and 55% in the 65-74 cohort. Elderly people are later adopters of mobile telephony, which may be partially explained by their use and ownership of fixed telephones. Therefore, as in the case of children and teenagers, it is more likely that the first mobile phone will be a postpaid one. In the future, it would be interesting to study the evolution of billing plans, when those adults that used to have prepaid plans become seniors. Finally, a recent billing package of postpaid bills with a flat rate for Internet access are just beginning to take off (4% on average).

From ITU (2010) data, we also know that 56% of all active mobile subscriptions were prepaid in the European Union in 2008.1 This figure would suggest that more than one half of the users have prepaid plans. However, figures from Eurostat do not seem to support this hypothesis. These two indicators seem to be reporting contradictory results even though they measure different aspects of the same phenomenon. One possible interpretation is that second or third phone lines that some individuals may use could be prepaid, although not reported in the household survey (which only gathers the use of at least one mobile telephone). Moreover, the survey is addressed to the population living in private households between 16 and 74 years of age. This excludes children up to 15 years who are heavy users of prepaid mobile phones;2 as well as those persons trapped in the economic margins, such as the homeless, who may only access prepaid mobiles.

Table 3. Use of Mobile Phone Advanced Services in the Previous 3 months, Percentage of Individuals in Each Age Cohort in European Union (EU27), Year 2008

 

Age cohort

All

Youth

Adults

Seniors

16-74

16-24

25-54

55-64

65-74

 

 

 

 

 

 

Sending (photos, video…)

20

41

20

7

3

Internet browsing

7

13

7

1

0

e-mail

5

8

6

3

1

Uploading (photos, video…)

4

11

3

1

1

Receiving subscription-paid information (news, weather forecast, sports results…)

4

6

4

2

1

Personal navigation, use of location-aware services (receiving nearby travel, shopping and event information)

3

4

3

1

1

Watching/downloading TV or video

1

3

1

0

0

Payments (instead of credit card or cash)

1

2

2

1

0

Source: Eurostat (2010).

 

Basic uses, voice calls and SMS are widespread across the whole population, with the elderly preferring voice communication. However, the use of advanced services in the European Union seems to be more restricted. Table 3 shows that sending pictures or videos is the most popular advanced service, as some 20% of mobile users have shared these kinds of files in the previous three months. This activity is followed by Internet browsing (7% of mobile users) and e-mail (5%), while the rest are practiced by less than 5% of users. Now, we begin to see more pronounced age differences as youth double the average regarding some services (for instance, 41% photo or video sending, or 13% Internet browsing) while seniors hardly show users in the 55-64 cohort, with a notable exception being the 7% of mobile users that send photos or videos, and almost no users in the 65-74 cohort.

Scandinavian Countries Lead in the Adoption of Mobile Phones by the Elderly

While this data provides an overview of the European Union, can we identify any country specifically regarding the level of mobile telephony use among the elderly population? To answer this question, a cluster analysis was conducted including all the country-level data available from the household survey in 30 countries (Eurostat, 2010; see Table A.1 in the Annex). In this instance, two variables were considered: share of mobile users in the 55-64 and 65-74 age cohorts. Both are quantitative variables with similar range width, so there was no need to standardize the data. The selected method included average linkage between groups with squared Euclidean distance as distance measure. The resulting dendrogram helped to identify four clusters.

Table 4. Mobile Subscriptions and Percentage of Mobile Users in the Indicated Age Cohort in 30 European countries, Year 2008

 

Age cohort

Subscriptions

All

Seniors

per 100 hab.^

16-74*

55-64*

65-74*

 

 

 

 

 

Cluster 1: markedly over the average

N= 5 countries

122.2

96.6

94.6

86.6

Cluster 2: over the average

N= 11 countries

131.3

91.5

486.3

71.3

Cluster 3: below the average

N= 8 countries

113.7

87.5

78.1

56.3

Cluster 4: markedly below the average

N = 6 countries

124.6

78.7

62.5

35.0

Total sample

N = 30 countries

123.7

88.7

80.7

62.5

Reported figures are simple averages in each group.

Statistical significant differences between groups at 1% level: *; at 5% level: **; at 10% level: ***.

^: Differences are not statistical significant at usual levels.

See Table A.1 for country-level detailed data and sources.

 

The first cluster brings together the European societies in which senior citizens show higher use of mobile telephony and is mainly comprised of Scandinavian countries: Finland, Iceland, Luxembourg, Norway and Sweden. They have an average of 94.6% mobile users among younger seniors (55-64) and 86.6% among older seniors (65-74). Table 4 shows that the cluster also stays markedly over the average in terms of the whole population, which reaches 96.6% in cluster 1, above the 88.7% of the total sample. Cluster 2 shows values above the total sample average, with 86.3% of users in the 55-64 cohort and 71.3% in the 65-74 cohort. This cluster includes 11 countries (Austria, Belgium, Czech Republic, Denmark, Estonia, Germany, Hungary, Italy, The Netherlands, Slovakia and United Kingdom). In terms of the whole population, cluster 2 is the second one in terms of diffusion (91.5% users).

Cluster 3, on the other hand, is below the sample average but close to it, with 78.1% users among young seniors and 56.3% among older seniors. The eight countries in the cluster are Cyprus, France, Latvia, Lithuania, Malta, Portugal, Slovenia and Spain, and the average of mobile users in the whole population is 87.5%. Finally, cluster 4 is markedly below the average with 62.5% users in the 55-64 cohort and just one third (35.0%) in the 65-75 cohort. Its six countries (Bulgaria, Croatia, Former Yugoslav Republic of Macedonia, Greece, Poland and Romania) have an average share of 78.7% users in the whole population.

Figure 1. Users of mobile phone (%), by age cohort. 30 European countries, year 2008.

In parenthesis: assigned cluster.

See Table A.1 for data and sources.

These results reveal that the higher the average diffusion of mobile telephony at an individual level, the higher it is among the elderly population. Cluster 1 has almost reached a situation of saturation, and this is the path followed by seniors despite the fact that the most aged in the sample (65-74 years old) still show notable differences compared to the young senior cohort (55-64). These differences increase when diffusion drops, as can be seen in cluster 4. Moreover, Figure 1 shows that youth (16-24) and adults (25-54) are always above the total population average, while seniors (55-64 and 65-74) are always below it.

However, penetration rates do not shape the same general trend described for mobile users. On the contrary, Cluster 1 ranks below the total sample average (122.2 vis-à-vis 123.7 mobile subscriptions per 100 inhabitants) and has lower penetration than cluster 4 (124.6, see Table 4). Therefore, we can state that penetration rates are not good predictors of effective individual access despite the fact that in a couple of years, it seems that throughout Europe every person between 16 and 75 years old will use a mobile phone.

Table 5. Use of Mobile Phone Advanced Services in the Previous 3 Months, Percentage of Individuals in European Countries by Cluster, Year 2008

 

 

Cluster 1

Cluster 2

Cluster 3

Cluster 4

All

 

 

 

 

 

 

Sending (photos, video…)**

28.6

18.7

18.6

16.3

19.9

55-64 years old*

15.4

6.6

4.9

3.0

7.0

65-74 years old*

8.3

2.9

1.1

0.3

2.8

Internet browsing*

14.8

5.9

7.5

3.7

7.4

55-64 years old*

6.0

1.9

0.7

0.3

2.1

65-74 years old***

1.0

0.4

0.0

0.0

0.3

e-mail*

10.2

5.8

5.6

3.5

6.0

55-64 years old*

6.8

3.3

2.0

1.0

3.1

65-74 years old**

2.8

1.1

0.5

0.0

1.1

Uploading (photos, video…)^

4.0

5.3

5.6

4.0

4.9

55-64 years old**

2.5

1.7

0.7

1.0

1.4

65-74 years old**

1.0

0.6

0.0

0.0

0.4

Receiving subscription-paid information (news, weather forecast, sports results…)^

5.4

2.9

4.1

2.5

3.6

55-64 years old^

3.0

1.4

1.0

1.0

1.5

65-74 years old^

0.8

0.4

0.6

0.3

0.5

Personal navigation, use of location-aware services (receiving nearby travel, shopping and event information)*

9.2

2.4

1.9

1.4

3.3

55-64 years old*

4.0

1.1

0.3

0.3

1.5

65-74 years old**

1.0

0.1

0.2

0.0

0.3

Watching/downloading TV or video^

1.8

1.0

1.4

1.0

1.3

55-64 years old*

1.0

0.1

0.2

0.0

0.3

65-74 years old^

0.3

0.2

0.0

0.0

0.1

Payments (instead of credit card or cash)*

5.4

2.1

1.4

2.3

2.6

55-64 years old*

2.8

1.0

0.0

0.5

0.9

65-74 years old^

0.5

0.3

0.0

0.0

0.2

Source: Eurostat (2010).

Reported figures are simple averages in each group.

Statistical significant differences between groups at 1% level: *; at 5% level: **; at 10% level: ***.
^: Differences are not statistical significant at usual levels.

The analysis of the use of advanced mobile services reveals these differences as well (see Table 5). Cluster 1 ranks first among the two cohorts of senior population, always clearly above the sample average. However, for the whole population (16-74 years), uploading photos or videos to Internet is the only service that is not over the average in cluster 1. On the opposite extreme, cluster 4 lies below the average with lower levels of users in all service categories, especially among the senior population. Cluster 2 usually ranks higher than cluster 3 with values closer to the sample average. Finally, in each of the four clusters the use of advanced services markedly decreases among the senior population.

These four clusters, exclusively built on data of elderly users, are associated with the diffusion in different age cohorts and in the whole population under study. Hence, the identified clusters are homogeneous and provide information on the situation at a country level.

The reach of the present analysis, however, is conditioned by two limitations. First, Eurostat individual statistics on advanced mobile uses are based on research that covers only one year, therefore only cross-sectional analysis is possible. Furthermore, 74 years old is set as the standard upper limit, so there is a lack of information regarding an older European population and making it difficult to obtain more in-depth knowledge of the issue under study.

Conclusion

Three main issues arise from the analysis of Eurostat data on mobile adoption and use by individuals aged between 16 and 74. First, elderly people can be considered to be the last adopters of mobile telephony in aggregate terms, while they are likely to become users. Young seniors (55-64 years old) show a faster path adoption than older seniors (65-74 years old). They always constitute the age group in which penetration is lower but seem to reach saturation (that is, almost all of the individuals are mobile users) once the younger cohorts do. Therefore, in countries where diffusion is comparatively lower on average (below 80% of users), senior mobile users have a smaller presence (around 30% in the 65-74 cohort, and around 65% in the 55-64 age group). In those countries where an average of 95% of the population are mobile users, diffusion in the 65-74 cohort is above 80%, and above 90% in the case of the 55-64 group. This is confirmed as well by the cluster analysis.

Second, the three most popular advanced mobile services in Europe are sending pictures or videos, Internet browsing and e-mail. However, their use remains low and shows a high correlation with mobile use rate. In such context, senior mobile users show distinctly lower percentages of advanced services use. And lastly, penetration of mobile telephony, the indicator that accounts for the active mobile subscriptions per 100 inhabitants, is not associated with the percentage of users among individuals and, therefore, is not a valid predictor of the effective importance of mobile telephony among different age segments.

In analytical terms, taking into account the obtained results together with the qualitative evidence discussed above, we can state the following hypothesis: Within the personal system of communication channels of the European elderly population, the mobile telephone occupies a peripheral position. The peripheral situation of mobile telephony would justify the higher rates of prepaid billing among the elderly population. Mobiles tend to be introduced for safety and security reasons; they are not always likely to be used extensively but only on limited occasions as an extra layer in the communication system. The mobile phones, indeed, might be perceived as a distant device. Thus, it seems that there is not an economic reason for changing to postpaid billing, as younger users do once the budget devoted to mobile telephony increases.

All in all, secondary data show that interactions through and with mobile phones among the elderly population in Europe follow a distinctive pattern than younger age groups. This different appropriation process might be due to the peripheral position that the mobile telephone has in the personal system of communication channels.

Notes

1 EU27 data, without Latvia.

2 For instance, in Catalonia 66% of children between 10 and 15 years old had a mobile phone in 2008. Source: INE (2008).

This is an improved version of the paper presented at ECREA Conference 2010. 3rd European Communication Conference Hamburg, 12-15 October, 2010.

References
Castells, M., Fernández-Ardèvol, M., Qiu, J. L., & Sey, A. (2006). Mobile communication and society: A Global perspective. Cambridge, MA: MIT Press.
Charness, N., & Boot, W. R. (2009). Aging and information technology use: potential and barriers. Current Directions in Psychological Science, 18 (5), 253-258.
Charness, N., Parks , D.C., & Sabel , B. A. (Eds.). (2001). Communication, technology and aging: Opportunities and challenges for the future. New York: Springer Publishing Company.
Conci, M., Pianese, F., & Zancarano, M. (2009). Useful, social and enjoyable: Mobile phone adoption by older people (INTERACT, Part I, LNCS 5726, 2009), 63-76.
Duh, H.B.-L., Do, E.Y.-L., Billinghurst, M., Quek, F., & Husueh-Hua, V. C. (2010). Senior-friendly technologies: Interaction design for senior users. CHI, April 10-15, 2010, Atlanta, USA.
Eurostat (2010). Statistics on the use of mobile phone. Special Module 2008: Individuals - Use of Advanced Services, (last updated 09-08-2010).
Giannakouris, K. (2008). Ageing characterises the demographic perspectives of the European societies. Eurostat Statistics in Focus.
Hashizume, A., Kurosu, M., & Kaneko, T. (2008). The choice of communication media and the use of mobile phone among senior users and young users. In S. Lee et al. (Eds.), APCHI (2008, LNCS 5068), 427-436.
INE. (2008). Encuesta sobre equipamiento y uso de tecnologías de la información y comunicación en los hogares. Instituto Nacional de Estadística.
ITU. (2010). World Telecommunication/ICT Indicators Database 2010. Geneva, Switzerland: International Telecommunication Union.
Jovanovic, B., & Rousseau, P. L. (2005). General purpose technologies. In P. Aghion & S.N. Durlauf (Eds.), Handbook of Economic Growth Vol. 1B, (pp. 1181-1224).
Karnowski, V., von Pape, T., & Wirth, W. (2008). After the digital divide? An appropriation-perspective on the generational mobile phone divide. In M. Hartmann, P. Rössler, & J. Höflich (Eds.), After the mobile phone? social changes and the development of mobile communication, (pp.185-202). Berlin: Frank & Timme.
Kurniawan, S. (2008). Older people and mobile phones: A multi-method investigation. International Journal of Human-Computer Studies 66, 889-901.
Kurniawan, S., Mahmud, M., Nugroho, Y. (2006, April 22-26). A study of the use of mobile phones by older persons. CHI 2006, Montréal, Canada.
Lenhart, A. (2010). Cell phones and American adults: They make just as many calls, but text less often than teens. Full Report: Pew Internet & American Life Project.
Ling, R. (2002). Adolescent girls and young adult men: two sub-cultures of the mobile telephone. Revista de Estudios de Juventud 52, 33-46.
Ling, R. (2004). The Mobile connection: The cell phone’s impact on society. San Francisco, CA: Morgan Kaufmann.
Ling, R. (2008). Should we be concerned that the elderly don’t text? The Information Society 24, 334-341.
Mohd, N., Hazrina, H., & Nazean, J. (2008). The use of mobile phones by elderly: A study in Malaysia perspectives. Journal of Social Sciences 4 (2), 123-127.
Oskman, V. (2006). Young people and seniors in Finnish ‘mobile information society’. Journal of Interactive Media in Education 2, 1-21.
Pew Research Center (2009). Pew internet & American life project methodology. Interviewing conducted by Princeton Survey Research Associates International (November 20 - December 4, 2008).

Biography

Dr. Mireia Fernández-Ardèvol holds a PhD in Economics (University of Barcelona). She is a full-time researcher at the Internet Interdisciplinary Institute (UOC, Open University of Catalonia) where she is the co-director, together with Manuel Castells, of the Research Program “Mobile Communication, Economy & Society.” Besides, she is part time lecturer at the Department of Econometrics, Statistics and Spanish Economy (University of Barcelona). Mobile communication has been one of her main areas of study since 2003. With a combined sociological and economic focus, she approaches to this issue both with qualitative and quantitative methodologies. Her interests are set both in developed and in developing countries. Mobile communications among the elderly constitute her main research focus in developed areas, while the contribution of mobile communication to social and economic development in Latin America is her focus of interest regarding less developed contexts.

Highlighted books:
Mobile communication and society: A global perspective. Cambridge, MA: MIT Press. Castells, M, Fernandez-Ardevol, M., Linchuan Qiu, J. and Sey, A. (2006).
Comunicación móvil y desarrollo social y económico en América Latina, Ed. Ariel Fernández-Ardèvol, M.; Galperin, H.; and Castells, M. (dirs.) (2011, in press)

Share and Enjoy Digg TwitThis Facebook del.icio.us Google StumbleUpon E-mail this story to a friend! Print this article!

]]>
http://wi.hexagram.ca/?feed=rss2&p=66
Understanding Touch Screen Mobile Phone Users by Taxonomy of Experience (ToE) http://wi.hexagram.ca/?p=71 http://wi.hexagram.ca/?p=71#comments Mon, 20 Jun 2011 16:54:17 +0000 admin http://wi.hexagram.ca/?p=71 By Wen-Chia Wang, Mark Young, Steve Love, and Ian Coxon

Introduction

User experience is one of the most important elements of mobile phone design and in recent decades has received increased attention in the HCI community. The user experience should include considerations of the usefulness and usability of a product (Alben, 1996; Shedroff, n.d.), the ‘user’s internal state, the context, and the user’s perceptions of the product’ (Väänänen-Vainio-Mattila, Roto, & Hassenzahl, 2008, p.1). It is generally the case that the user’s experience is highly subjective, situated and dynamic in nature (Väänänen-Vainio-Mattila, Roto, & Hassenzahl, 2008). Therefore, efficient methodologies to obtain and to evaluate user experience accurately are essential for improving product design. Norman indicates that an understanding of user experience should be able to evaluate the user’s experience in a circumstance that is similar to the actual using situation to avoid the user imagining the experience (Anderson, 2000). The data collection process should record the user’s experience as it happens rather than relying on the user’s memory of the experience. Moreover, the user’s experience should be understood through his/her subjective information about the experience (Isomursu, 2008). Methodologies for evaluating experience have been established based on the user’s attitudes and expectations (Isomursu, 2008), emotion (Hole & Williams, 2008), concept of the object (Al-Azzawi, Frohlich & Wilson, 2008), judgment of the product (Karapanos & Martens, 2008.), and through comparing the user’s reference to different interfaces (Aula, Granka, Heimonen & Hutchinson, 2008). These studies capture and analyse user experience by experimental pilots (Isomursu, 2008), emotion sampling (Hole & Williams, 2008), multiple card sorting (Al-Azzawi, Frohlich & Wilson, 2008), and repertory grids (Karapanos & Martens, 2008.). In addition, the approaches of diary (Bolger, Davis, & Rafaeli, 2003), focus groups, surveys and competitive analysis are also tools that are commonly used (Kuniavsky, 2003). Whilst many researchers attempt to transform user experience into quantitative data, Coxon’s (2007) Taxonomy of Experience (ToE) and its analytic approach of SEEing, uncover an understanding of the user experience through qualitative analysis. The term ‘SEEing’ attempts to analyse users’ experience by an alternative angle with its unique themes (Coxon, 2007). The nine steps of the SEEing process aim to clarify the user’s experience. It begins by transforming the user’s verbal commentary and ends by synthesising them into super-ordinary metaphors. This study applied the ToE to capture the user experience of trialing an unfamiliar touch screen mobile phone, and provides an alternative consideration for the interface design of touch screen mobile phones.

Taxonomy of Experience (ToE)
The purpose of ToE is to understand users’ experience with a product via analysis of their verbal commentary to explore and consider meanings hidden from verbal commentaries. The structure of the ToE is based on philosophy, methodology and design theory to provide a multi-layered method to understand user experience. It was first developed in the study of the user experience of transportation vehicles (Coxon 2007). In Coxon’s (2007) review, the video recording of his own trial experience of an electric vehicle helped him to recall the deep aspects of the experience that he was not particularly conscious of while personally trialing the vehicle. The process of transcribing the sequence of the dialogue and other activities brought the experience more vividly into consciousness. Figure 1 depicts the framework for understanding an experience with considerations to temporality, spatiality, relationality and corporeality (Coxon, 2007).

Figure 1. Framework of an experience

These ‘lived experience descriptions’, should be able to a) describe the experience as it is lived without asking why; b) describe the experience from the inside: the feelings, mood, and emotions; c) focus on a particular example of the experience and describe it; d) focus on an example that stands out, as it was the new experience; e) recognise how the body senses: smells, sounds and so forth; and f) avoid complicating the illustration with flowery language or jargon. Overall, the ToE-SEEing process provides a way to make abstract concepts comprehensible and visible. It comprises nine steps (Coxon, 2007):

Step 1: Gathering data and establishing structures
It is important that the researcher ‘gets to know’ the experience, and becomes familiar with the experience by understanding its ‘language.’ This stage emphasises that the researcher must be immersed in the experience completely (Csikszentmihalyi, 1991; Hanington, 2000). The data of people’s experience can be collected from observations, interviews, and contextual studies. Information that might be useful to the researcher, including images, sounds, and samples, is collected for the researcher to recall the user’s experience.

Step 2: Descriptive narratives
This is the process of transforming the data collected in Step1 into a textual format and breaks the experience into fragments as small as a single word, or a phrase for referring the experience to the themes of the SEEing process in step 3.

Step 3: Sorting fragments into themes
This step includes cataloging data into meta-themes and sub-themes. Meta-themes in the SEEing process include somatic, affective, cognitive and contextual factors; the sub-themes include senses, positive-negative affect, internal-externalised cognition, and many contextual categories. Each theme has a collection of fragments, and provides the researcher with the feeling of the story that they are telling.

Step 4: Developing meaning(s)
This step requires the researcher to look at each fragment of the information carefully and to find deeper ‘meanings’ behind the fragment. This process helps to ‘tease out’ the text’s different meanings. At this point the researcher is not concerned with establishing what the meanings of the fragment ‘should be’. Instead, the aim of this process is to accept all ‘possible’ meanings that are contained within the fragment.

Step 5: Essential elements
This step helps to filter out the less important meanings. The researcher has to determine if the meanings in Step 4 are incidental or vital to the essential nature of the experience. It is necessary to know the importance of the experience, whether the element is essential to the experience or the experience might be different without the element.

Step 6: Super-ordinary elements
This step distills the super-ordinary essence of the experience: the unexpected, novel and hidden aspects of the experience. This stage focuses on the everyday experience, and isolates those elements of the experience that might not have been seen as an important part of the original design. However, those elements are still an important part of the experience. This process searches for the surprising elements, the unintended impacts of the experience.

Step 7: Weighting of super-ordinary elements
This is a weighting process to consider which super-ordinary elements are the most ‘powerful.’ The researcher evaluates the super-ordinary elements by his or her understanding of the language of the experience, to give a subjective numerical scale using a Likert rating (1-7, 1 is low) to determine a relative level of intensity.

Step 8: Super-ordinary summary words
The super-ordinary elements, sorted into descending order, provide a ranking of the essential super-ordinary elements of the experience by intensity. This stage uses word metaphors to synthesise ‘what is the collective meaning behind these elements?’ For example, the super-ordinary element of ‘no risk means no fun’, could essentially be a statement about ‘freedom to enjoy danger.’

Step 9: Summary word descriptions
The previous step summarised the super-ordinary elements; this stage focuses on ‘explaining’ the summary. It concludes the work of Step 6-8. One or two narrative paragraphs help to represent the understanding of the experience to someone who does not understand the meaning of the super-ordinary words.

Design guidelines for the interface design of mobile phones have been well established by mobile phone manufacturers and include design principles for elements such as content, layout, colour, font size, text and terminology. It is also necessary to concern the user’s requirements of an object from their experience. As the market for touch screen mobile phones continues to grow, understanding experienced user’s thoughts and novices’ expectations of the touch screen mobile device is essential to providing a better design. This study applies the ToE and its process of SEEing to generate deeper understandings of users’ experience in order to provide extra design principles for mobile phone interfaces.

Experiment Design
This study attempts to develop extra criteria for designing mobile phone interfaces based on user experience. This is the first trial of applying ToE-SEEing to mobile phone user experience. Participants were required to trial the touch screen mobile phone in the laboratory. Twelve participants were recruited from a British University. Half of them currently use a touch screen mobile phone, whereas the other half currently use a 12 keypad mobile phone. A Vodafone 541 mobile phone was used in this study (Figure 2).

Figure 2. Vodafone 541

This model is the previous generation of touch screen mobile phone. The hardware and software are not advanced enough to compete with new generation phones such as the iPhone. The aim of choosing this model was to push the participants to talk more about the using experience. Before starting the data collection, the observer demonstrated to the participant how to apply the approach of ‘think aloud’ by trialing a touch screen camera. Participants then were required to practice a ‘think aloud’ protocol by trialing the camera. The practice was intended to help the participants to get used to expressing their experience while trialing the Vodafone 541 mobile phone. Participants had five minutes to free trial the phone as they wished. Their interaction behaviour with the mobile phone was filmed for the ToE-SEEing analysis. The camera only focused on their hands and the mobile phone, and recorded their verbal commentary without showing their faces.

Results

The data was transferred into Step 3 of ToE-SEEing which includes two layers of themes (meta-themes and sub-themes). Firstly, each participant’s verbal description of the experience was coded into different themes. The meta-themes include the body-somatic experience (sound, touch, feel, sight, smell, taste, comfort-ergonomics, and appearance-aesthetics); the heart-affective experience (positive-negative emotions); the head-cognitive experience (conation, reflective-thought-external-doing, conscious cognition-reflective thought-internal-thinking) as well as a range of contextual factors (environmental, regulatory, social), and existential factors (time, space, corporeality, and the body’s relationship to others). Most of the participants’ experiences with the touch screen Vodafone 541 mobile phone strongly relate to the sub-themes of sight and cognitive experiences. The following section presents the super-ordinary elements and the summary of participants’ user experience with trialing the Vodafone 541.

Understanding-from the head
It is important to see that the ‘graphic icon and its title are consistent, and represent the function clearly.’ Clear feedback is given confirming whether or not the operation was successful. It is essential to show instructions for unique features of the phone, maybe to demonstrate how to operate the feature, or to make it easy to get ‘help’ information.

Sensitivity of the touch screen is crucial, and should fit the user’s pace when operating the phone. The user would like to dominate, to trust the phone, and to fully understand the operation process before using the phone.

Experienced and familiar-from daily life and history
The way to operate the scroll bar on Vodafone 541 should be the same as using the scroll bar on a computer.
Based on previous experience using mobile phones with 12 keypads, it would be good to see that the icon becomes highlighted when browsing the icon on the menu. It would help to reduce mistakes if the phone can highlight what the mistake was, to detect the failed task automatically, and then provide help and instructions to complete the task correctly before the user has to ask for help.

Freedom-from the operation
The phone should provide links between different functions, rather than having to go to the menu to execute another function. The size of the phone provides the freedom for the user to carry it all the time, and allows users to hold the phone in their hand without worrying that the phone might slip from their grasp.

The three super-ordinary elements above had the highest score from participants. The other super-ordinary elements were ‘specific’, ‘share’, ‘intimacy’, ‘comfortable’, ‘enjoyment’, ‘flexible’, and ‘logic.’

The ToE-SEEing process helps to transform and categorise the raw meaning of an experience: to find the meanings behind the user’s commentary; to sort the importance of those elements; and to summarise super-ordinary elements of the experience. It provides an overview of the user’s experience and describes whether it is the user’s previous experience or the experience that was produced when trialing the object. The categories in Step 3 help to clarify the key themes of users’ experience, and to establish a good foundation for further analysis. In this case, the summarised super-ordinary elements not only reflect the user’s expectation of Vodafone 541, but also highlight the components that the user cares about most. This study indicates that it is not only necessary to follow design guidelines to design interfaces for mobile phones, but also to concern user experience as part of the design.

Conclusion

This paper presents the process of executing the methodology of ToE-SEEing to understand user experience with a touch screen mobile phone. The validity of ToE has been examined with extensive observation data from video clips and interviews during the development process (Coxon, 2007). This method might be questioned due to its explicit subjectively; nevertheless, as mentioned earlier, the nature of an experiential encounter is subjective, situated, complex and dynamic. Therefore, the ToE-SEEing process is a useful tool for distilling the true meaning that lies behind the verbal description of such a complex event. This short paper emphasises the importance of understanding user experience before design begins. The result provides alternative considerations to achieve the goal of making things ‘easy to use.’

References

Alben, L. (1996). Quality of experience: Defining the criteria for effective interaction design. Interactions, 3 (3), 11-15.                                                                                                                                           Anderson, R. (2000). Organizational limits to HCI: Conversations with Don Norman and Janice Rohn. Interactions, 7 (3), 36-60.                                                                                                                     Bolger, N., Davis, A. & Rafaeli, E. (2003). Diary Methods: Capturing life as it is lived. Annual Review of Psychology, 54, 579–616.                                                                                                                      Brezet, H.P.D., Vergraght, P.P.D., & Van der Horst, T. (eds). (2001). Kathalys: Vision on sustainable product innovation. Amsterdam: BIS Publishers.                                                                                                    Aula, A., Granka, L., Heimonen, T. & Hutchinson, H. (2008, April 5-10).Comparing the user experience of search user interface designs. CHI 2008, Florence, IT.                                                                            Coxon, I. (2007). Designing (researching) lived experience. Ph.D. Thesis. University of Western Sydney. Csikszentmihalyi, M. (1991). Flow: The Psychology of optimal experience. New York: Harper Collins.        Hole, L. & Williams, O. (2008, April 5-10).Emotion Sampling and the Product Development Life Cycle. CHI 2008, Florence, IT.                                                                                                                             Feenberg, A. (2000). From essentialism to constructivism: Philosophy of technology at the crossroads. In E. Higgs, A. Light, & D. Strong (Eds.), Technology and the Good Life (pp. 294-315). Chicago: University of Chicago Press.                                                                                                                                     Glanville, R. (1999). Re-Searching design and designing research. Hong Kong: School of Design, Hong Kong Politechnic University.                                                                                                                     Hanington, B.M. (2000, May). Innovation and method in design research. Design (plus) Research, Politecnico di Milano, Milan, Italy. Higgs, J. (1997). The Context of qualitative research. In J. Higgs (Ed.), Qualitative research: Discourse on methodologies. Five Dock, NSW, AUS: Hampden Press Five Dock.                                                                                                                                                Kuniavsky, M. (2003). Observing the user experience: A Practitioner’s guide to user research. San Francisco, CA: Morgan Kaufmann.                                                                                                                        Schmitt, B. (1999). Experiential marketing: How to get customers to sense, feel, think, act, and relate to your company and brands. New York: The Free Press.                                                                                Shedroff, N. An Evolving Glossary of Experience Design.                                                                             Karapanos, E. & J. B. Martens. (2008, April 5-10). The Quantitative Side of the Repertory Grid Technique: Some Concerns. CHI 2008, Florence, IT.                                                                                            Al-Azzawi, A., Frohlich D. & Wilson, M. (2008, April 5-10). User Experience: A Multiple Sorting Method Based on Personal Construct Theory. CHI 2008, Florence, IT.                                                                          Isomursu, M. (2008, April 5-10). User Experience Evaluation with Experimental Pilots. CHI 2008, Florence, IT.
Väänänen-Vainio-Mattila, K., Roto, V. & Hassenzahl M. (2008). Towards practical user experience evaluation methods. 5th COST294-MAUSE Open Workshop on Valid Useful User Experience Measurement (VUUM). Reykjavik, IS.
Van Manen, M. (1997). Researching lived experience: Human science for an action sensitive pedagogy (2nd ed.). London, ON: Althouse Press.

Biography:

Wen-Chia Wang completed her BA in Commercial Design from Ming Chuan University in Taiwan in 1999. She then worked as a graphic designer and an event organizer across the design and entertainment industries.  In 2004 she started an MA in Design Management at Ming Chuan University, and upon completion, she worked as a part time lecturer in the the University’s School of Product Design from 2006 to 2008. Her passion for understanding human interaction with machines led her to start her PhD at Brunel University in 2008. During her PhD, she has worked on several projects with the telecommunication company 3 in the UK to understand user experience with mobile phones.

Her PhD research addresses the importance of understanding users before conducting design; especially how individual differences of cognitive styles (holistic - serialistic) affect human operational behaviour with mobile phone interfaces. More importantly, her research seeks to establish the linkage between psychology and design, and to provide design guidelines that are based on individual differences. She is expected to complete her PhD in 2011. Meanwhile, Wen-Chia continues to work as a lecturer in Graphic Communications, which she’s been doing since 2009.

Share and Enjoy Digg TwitThis Facebook del.icio.us Google StumbleUpon E-mail this story to a friend! Print this article!

]]>
http://wi.hexagram.ca/?feed=rss2&p=71
Mobile User Experience Beyond the Laboratory: Towards a Methodology for QoE-QoS Evaluation in Natural User Environments (A Position Paper) http://wi.hexagram.ca/?p=70 http://wi.hexagram.ca/?p=70#comments Mon, 20 Jun 2011 16:53:02 +0000 admin http://wi.hexagram.ca/?p=70 By Katarzyna Wac and Anind K. Dey

Introduction

The growing availability of diverse interactive mobile applications, envisaged to assist us in different domains of our daily life, make their perceived quality of experience (QoE) increasingly critical to their acceptance. Comments such as, “If it’s slow, I won’t give my credit card number” indicates the QoE expectations of a typical mobile commerce application user [1]. These expectations can be different given the user’s previous experiences with an application or an application’s criticality to the user’s task at hand. To date, the evaluation of QoE has been mainly conducted with qualitative methods that focus on an applications’ usability [2]. These studies are typically conducted within a limited time span in controlled laboratory environments, conditions that do not resemble users’ natural daily environments. The results of such evaluations may help to discover the mobile application’s serious and immediate usability issues in a design, but they may not help to uncover issues that are relevant to real life situations in the world outside the lab.

Issues relevant to the world outside the lab involve, amongst other things, a non-deterministic quality of service (QoS), and, in particular, the performance of the underlying network infrastructures that support the execution of an application and mobile service delivery (Figure 1). This quality of service can be quantified by measuring delay, jitter and network capacity. This quality of service usually is provided at a ‘best-effort’ level; that is without any guarantees by a company or service provider that it will work at an optimum level. Yet, as we would argue, this quality of service is critical to the mobile user’s quality of experience, especially for highly interactive mobile applications, delivery of which depends on frequent data transfers over the underlying network infrastructures.

Fig. 1: A concept of QoE and QoS in a mobile service delivery

A common practice for quality of experience provision is that mobile application designers use their own judgment and perception of an application’s ease of use as a bellwether to gauge an application’s perceived quality of experience by an imagined mobile user [2]. But can the designer, who may have created the system and thus have an intimate knowledge of its capabilities and embedded logic, really stand in for the imagined user? The overall effect of this situation is that users whose QoE expectations are not satisfied, may simply stop using the applications or switch to another provider. For example, it is estimated that there are, on average, 200 new applications available daily in the online store for Apple’s iPhone platform. However, due to the low quality of experience, more than half of these applications do not achieve a critical mass of user acceptance and are withdrawn from the store’s list of offerings within some months from the launch.

The challenge for designers and researchers studying these technologies and new applications is that no rigorous and robust scientific methods, tools, and systems exist for evaluating an application’s perceived QoE in the user’s natural environment(s) [3]. Rather, there are separate methods for usability evaluation in the HCI community [2, 4] and separate methods for the evaluation of the quality of service and performance of an application’s underlying network infrastructures in the data networking community [5-7]. The former methods are largely qualitative, while the latter are largely quantitative. Both methods can acquire quality results in their dedicated areas of applicability. However, due to the dichotomy between these two scientific communities, there are no scientifically proven methodologies that combine both of these approaches.

Objective and Approach

Given this chasm between methodologies, approaches and the study objects, the focus of our research into these methods is to find a viable way to bridge this gap. To this end, we are in the process of developing a set of methodological procedures for the reliable real-time evaluation of an interactive mobile applications’ perceived quality of experience in user’s natural environments. This investigation of the quality of experience will be conducted in conjunction with an assessment of the variable quality of service provisions. In our approach we will focus on already implemented and operational interactive mobile applications, which are now available to a typical mobile user. We assume that these applications have undergone a cycle(s) of (re)design and usability tests in a laboratory environment, although we do not necessarily have access to the results of these.

Our approach in this paper is as follows. We identify and analyze existing and emerging qualitative methods for the evaluation of usability, as well as quantitative methods for the evaluation of QoS and performance of mobile computing applications. Based on these methods, we propose a novel set of procedures for the real-time quantitative evaluation of users’ perceived QoE of mobile applications in natural settings. This methodology is based on our long-standing successful history of research on measurements-based quality of service and performance evaluation methods for interactive mobile applications [8, 9]. We have successfully used this methodology in a healthcare domain, specifically in the creation of interactive applications for health telemonitoring and teletreatment depending on delay and capacity (i.e., quality of service metrics) of the underlying network infrastructures [5, 10, 11]. What is new is our desire to transfer, expand towards QoE and test this methodology in new areas and domains of mobile experience.

To quantify the mobile user’s quality of experience, the methodology first requires defining a set of dependent (i.e., target) variables. We then define a set of mutually exclusive and collectively exhaustive variables influencing this quality of experience. These are the independent variables that can include, for example, user context like location, time, social settings, etc. Both sets of variables should be based on the existing scientific literature and documented expert knowledge.

Furthermore, for a given interactive mobile application, the methodology requires a set of qualitative methods to derive new independent variables that are not indicated in the HCI or the networking communities so far, but are important to the experience a mobile user has in her natural daily environments. One qualitative method that can be used for this purpose is the Experience Sampling Method (ESM) [12]. The ESM is based on occasional user surveys, which can be administered over specific time intervals, after particular events, or at random. Since we aim to evaluate a user’s perceived quality of experience while interacting with a mobile application, the ESM could be implemented in the form of a short, mobile-device based survey given to the users after each use of this application. The survey will pose some open-ended questions to get the user’s in-situ real-time, spontaneous opinions on their mobile experience. These new independent variables will be ‘grounded’, as they are derived from the answers acquired from this user [13, 14]. The ESM method must be designed and deployed such that it does not influence the experience and behaviour of a mobile application user, but enables us to gather information that is relevant and predictive for this user’s quality of experience evaluation.

As the evaluation will be conducted in the user’s everyday environments, the methodology must provide requirements and guidelines for the effective and efficient implementation and application of (software) modules necessary for measurements of the mobile application usage, the QoS and performance of its underlying service and network infrastructures. In this way the state of these variables (including those generated by the ESM), is continuously and accurately logged (i.e., measured) in real-time in an automatic manner; that is, in a way that is non-intrusive to the mobile user.

Moreover, having defined sets of dependent and independent variables, and having the modules implemented and operational for their measurement, the methodology would require reusing the existing analytical methods to discern relationships between variables and possibly establish causality. An example from our healthcare domain includes delineating boundaries for network delay and its jitter (i.e., independent variable), for which application data may have a clinical value for real-time diagnosis (i.e., as a part of dependent variable) provided to patients [5].

To analyze possible relations and causality between variables, the methodology requires the occasional involvement of a mobile user in the data analysis process. Namely, a mobile user needs to be interviewed about his application’s usage patterns and experience. This data must then be matched to the data automatically logged in the application and service infrastructure. The interviews we propose to conduct will be based on the completion of a detailed diary of the previous 24-hour period, as suggested by the Day Reconstruction Method [15]. This breaks the day into episodes described by activities, locations and times, and the mobile application usage and experiences during these times. During the interview, users can explain, in more detail, their responses in the ESM, and these results will be compared to the state of other independent variables logged in the system. This way causalities and relations specific to this particular user can be identified, while any inconsistencies can be clarified.

The combination of these methodologies, which are both qualitative and quantitative, will then provide adequate guidelines on how to statistically analyze and interpret the acquired (qualitative) survey data and (quantitative) measurement data for analysis by a user for one or multiple interactive mobile applications. Focusing the analysis on a single user implies the idiographic approach putting the effort to understand the meaning of contingent, unique, and subjective phenomena of quality of experience state for this particular user. Given a population of users, an analysis could then be conducted within this for one mobile application, or between populations of users of different mobile applications. The subsequent data analysis might then be implemented using advanced statistical methods, such as a multivariate analysis, or machine learning techniques for pattern recognition in data. Example machine learning techniques include logic-based techniques (trees, rules), density-based techniques (Bayesian networks) techniques based on non-linear functions (neural networks and support vector machines), as well as so-called ‘lazy’ techniques (‘k’-nearest neighbours). These techniques automatically learn to recognize and model complex patterns in the collected data, based on which they can predict the target variable, i.e., quality of experience state for a given application user in a given context.

Contributions and Evaluation

Our preliminary research on this topic, and approach, brings together and expands upon recent advances and methodologies in key scientific and technical areas, including the evaluation methods for human computer interaction, quality of service and performance evaluation methods and tools for mobile computing, real-time machine learning and prediction. By laying out this set of principles and guidelines, in the future we will conduct research on critical issues such as:

  1. The definition of quality of experience that is expected and required for interactive mobile applications. This definition must integrate multiple views including an examination of the application and its underlying infrastructure views, such as the interactions and provided QoS and performance, and the user view, such as past experiences and expectations, current application’s perception and its criticality to the task at hand, as well as the user’s context. The definition must also delineate a role of the user’s affective response for an interactive mobile application use.
  2. Implementing this method in future research involves a second challenge: that of gathering reliable real-time capturing of a user’s perceived quality of experience and the parameters and conditions that influence this quality of experience as it is lived in their natural daily settings. This includes identifying the variable ‘best-effort’ state of quality of service for underlying service and network infrastructure.
  3. The third aspect of our research in these areas will be to identify methods for documenting an automated and accurate inference of the user’s quality of experience state based on the gathered data. An accurate inference of this state is challenged by the fact that a) quality of experience state may be temporally indirect, i.e., there may be a lag between a cause (e.g., user context change) and the change of the user’s quality of experience state; b) sensors and modules used for real-time capturing of a user’s quality of experience may be unreliable: their reading may be influenced by, e.g., a user’s bodily position and movement artifacts; and, c) there are individual differences in quality of experience perception and evaluation of state for each human.
  4. The fourth challenge focuses on the accurate and real-time recognition of QoE patterns based on data mining and machine learning techniques. These challenges become even more complex if the system is required to be accurate and operational in real-time and to generalize to novel situations, e.g., in the case of novel mobile applications, or novel user’s interaction patterns.
  5. The final critical issue includes the ethical aspect of continuous real-time logging of possibly private information, implying that adequate security and privacy mechanisms must be in place for deployment of our methodology for a range of mobile application users. The users themselves must be informed about how and by whom the data is being collected, handled, stored, analyzed and used.

With use of the proposed complex multi-methodological approach, we hope to gain a deeper understanding of the use of interactive mobile applications, and to be able to quantify a user’s quality of experience and her relationship with the underlying quality of service, and point out ways to improve the usability of these applications and generate higher user acceptance. On one hand, application providers could use such methods to improve the provided applications, and, on the other hand, consumer advocacy groups could use the methods to monitor the quality of provided applications.
It is our intention to test our methodology on a set of widely available mobile applications for leisure, entertainment, communication or information, whose users expect to be able to easily stream multimedia content, such as YouTube, or use Internet-based radio; those engaging in highly interactive instant messaging or web browsing, such as e-banking or e-commerce; those playing multiplayer online gaming. Finally, another area where quality of service should match expectations is in VoIP video-conferencing, such as Skype or Google Talk.

Concluding Remarks

In this paper we have presented our proposed research approach towards defining a methodology for quantifying a mobile user’s experience (QoE) in their natural daily environments and relating this experience to the performance (QoS) of the underlying service and network infrastructures. This methodology is a blend of both quantitative and qualitative procedures. We propose a twofold methodology for evaluating user experience, where the user becomes an active participant in the research. First, it requires gathering in situ spontaneous information about the user’s mobile experience by employing the Experience Sampling Method for interaction with the user directly after each mobile application usage. Second, it requires a retrospective analysis of the user’s experience and of the state of factors influencing it, by employing the Day Reconstruction Method to assist with the recollection of the past 24-hours. While our current work focuses on defining the methodological steps, our future research includes an evaluation in a large-scale mobile user study for a set of widely used mobile applications.

Acknowledgments
Research conducted by K. Wac was sponsored by Swiss SSER (C08.0025) and Swiss NSF (PBGEP2-125917). This work is also partially supported by and the US NSF FieldStream project (0910754).

References

Bouch, A. et al. (2000). Quality is in the eye of the beholder: meeting users’ requirements for Internet quality of service. in the SIGCHI conference on Human factors in computing systems. The Hague, The Netherlands.

Bults, R. et al. (2005). Goodput Analysis of 3G wireless networks supporting m-health services. 18th IEEE International Conference on Telecommunications (ConTEL05). Zagreb, Croatia.

Dix, A. (2003). Human Computer InteractionPrentice Hall.

Hornbæk, K. (2006). Current practice in measuring usability: Challenges to usability studies and research. International Journal of Human-Computer Studies.

Hektner, J. M., et al. (2006). Experience sampling method: Measuring the quality of everyday life.Sage Publications, Inc.

ITU-T (2008). Definitions of terms related to quality of service, Recommendation E.800.

ITU-T (2001). Communications Quality of Service: A framework and definitions, Recommendation G.1000.

Kahneman, D., Krueger, A., Schkade, D., Schwarz, N. and Stone, A. (2004). A Survey Method for Characterizing Daily Life Experience: The Day Reconstruction Method. Science306 (5702), 1776-1780.

Kjeldskov, J. & Graham, C. (2003). A review of mobile HCI research methods. Lecture Notes in Computer Science(pp. 317–335).

Martin, P. Y. & Turner, B. A. (1986). Grounded theory and organizational research. The Journal of Applied Behavioral Science. 22, 141.

Michaut, F. and Lepage, F. (2005). Application-oriented network metrology: Metrics and active measurement tools. IEEE Communications Surveys & Tutorials,, 2-24.

Salamatian, K. and Fdida, S. (2001). Measurement Based Modelling of Quality of Service in the Internet: A Methodological Approach. International Workshop on Digital Communications: Evolutionary Trends of the Internet. (pp. 158-174).

Wac, K. et al. (2005). Measurements-based performance evaluation of 3G wireless networks supporting m-health services.12th ACM Multimedia Computing and Networking Conference. San Jose, CA, USA.

Wac, K. and Bults R. (2004). Performance evaluation of a Transport System supporting the MobiHealth BANip: Methodology and Assessment. MSc Telematics. University of Twente, Enschede, the Netherlands.

Biography

Anind K. Dey is an Associate Professor in the Human-Computer Interaction Institute at Carnegie Mellon University. His interests lie at the intersection of human-computer interaction, machine learning and ubiquitous computing. He has spent the last decade developing techniques for building context-aware applications, and for improving the usability of such applications. He is very interested in the development of truly smart systems, systems like sensor-enabled phones that can autonomously collect a vast amount of information about users and use that information to improve user experiences. Anind is the author of over 100 articles in the area of ubiquitous computing, has served as the Program Chair for several conferences on ubiquitous computing and serves on the editorial board for IEEE Pervasive Computing, Personal and Ubiquitous Computing Journal and the Journal of Ambient Intelligence and Smart Environments. Before joining Carnegie Mellon University, Anind was a Senior Researcher at Intel Research Berkeley and an Adjunct Assistant Professor at the University of California-Berkeley. He holds a PhD and a Masters degree in Computer Science, as well as a Masters degree in Aerospace Engineering, all from Georgia Tech, and a Bachelors of Computer Engineering from Simon Fraser University

Katarzyna Wac is a senior computer scientist currently associated with Institute of Services Science at University of Geneva (Switzerland). In 2009-2010 she has visited the Human-Computer Interaction Institute at Carnegie Mellon University. In 2003 she has received a BSc and MSc degree in Computer Science from Wroclaw University of Technology (Poland), and in 2004 MSc in Telematics from University of Twente (the Netherlands). In 2009 she has defended her PhD thesis in Information Systems at University of Geneva. Her research focuses on measurements-based methodologies for an evaluation of performance of interactive mobile applications and its relation with the end-user perceived experience. She builds tools that predict application’s performance and hence facilitate development of mobile computing applications that improve end-user perceived experience.

Share and Enjoy Digg TwitThis Facebook del.icio.us Google StumbleUpon E-mail this story to a friend! Print this article!

]]>
http://wi.hexagram.ca/?feed=rss2&p=70
Observing the Context of Use of a Media Player on Mobile Phones using Embedded and Virtual Sensors http://wi.hexagram.ca/?p=68 http://wi.hexagram.ca/?p=68#comments Mon, 20 Jun 2011 16:52:11 +0000 admin http://wi.hexagram.ca/?p=68 By Jakob Eg Larsen, Michael Kai Petersen, Nima Zandi, and Rasmus Handler

Introduction

Mobile phones have become ubiquitous and an integrated part of our everyday life. In the last couple of years smart phones have received increased attention as app­li­cation sto­res (e.g. Apple App Store, Android Market, and Nokia Ovi Store) have enabled easy distribution of mobile ap­pli­cations. New smart phone features and sensors have enabled a wide range of novel mobile applications, especially within games and media consumption domains. In the area of music appli­ca­tions a number of different mobile applications have been created. One example is mobile applications for the Internet radio Last.fm, which enable recommendation of music si­mi­­lar to the user’s favorites based on social net­work­ing features. MoodAgent is an example of an app­li­ca­ti­on that enables the user to navigate a music collection in terms of mood, rhythm, and style of the music.

However, getting an understanding of actual behavior and use of such mobile applications is difficult as the situations of use are inherently mobile. This makes it relevant to observe music listening habits “in- the-wild”, as laboratory studies may prove insufficient for reproducing realistic sce­narios to capture and getting insights into actual con­tex­tual user experience. Thus methods and techniques to ac­qui­re in­formation about actual mobile use in context are needed to obtain a better un­der­standing of the mobile user ex­pe­ri­ence.

In the study described in this paper we use a software framework we have developed, which is capable of acquiring contextual data from embedded mobile phone sensors during daily life use by a mobile phone user. Our software runs transparently in the background on the mobile phone and is log­ging data from multiple em­bedded sensors. This allows us to describe and understand information about surrounding people (devices) and places (locations), as well as application and media usage. In the present study our focus is on studying the use of the media player application on mobile phones, and in particular to under­stan­d the context in which it is being used. We hypothesize that contextual information obtained from embedded mo­bi­le phone sensors can offer useful information in terms of understanding the situations of mobile use involving the media player app­li­ca­ti­on. Fur­ther­more, we discuss how such contextual information can pro­vide inspiration for new ty­pes of context-aware mobile user interfaces and in­ter­ac­ti­on, such as in the present case user interfaces for music recommendation applications.

Related Work

Field versus laboratory evaluation of mobile applications has been debated in the HCI literature. Kjeldskov et al. [7] has shown that few additional usability problems were found in field ex­pe­­ri­ments compared to laboratory ex­pe­ri­ments and suggested that establishing the right laboratory settings can provoke similar findings [6]. On con­trary Duh et al. [2] found that significantly more (and more severe) usa­bility problems were iden­tified in field ex­pe­riments. This is supported by Bernard et al. [1] that found that contextual changes had a strong impact on behavior and performance. Roto and Olasvirta [12] studied mo­bile users on the move using web-browsers on mobile pho­nes by employing mul­tiple cameras worn by the test par­ti­ci­pants and a moderator stay­ing in proximity of the test par­ti­ci­pant to monitor the ex­pe­riment.

Shorter attention span was observed when using applications on the move com­pa­red to use in la­bo­ra­to­ry settings [11]. Froehlich et al. [3] took an approach re­sem­­bling ours in combining quan­ti­ta­tive and qualitative me­­thods for “in-the-wild” col­lec­tion of data about usage, in­­clu­ding device logging of app­lication use and context using embedded sensors. In [], [] and [] this was con­si­dered in music scenarios. Such studies under­line the im­por­tance of undertaking studies of observed “ac­tu­al use” in con­text, instead of “learning to use” an un­fa­mi­li­ar app­li­ca­tion in a controlled environment as described above. Such stu­dies ty­­pi­cal­ly only cap­ture use over a short period of time and re­veal little about actual use or use pat­terns over an ex­­ten­ded time pe­riod where learning and ha­bi­tu­a­tion has ta­ken place.

Mobile Context Logging

To observe test participants while using mobile phones and app­lications in real-world settings, we have created a mobile con­text logging software framework for mobile phones. The Mobile Context framework for mobile phones [8] allow us to carry out “in-the-wild” experiments where acquiring data from mul­ti­ple em­bed­ded mobile phone sensors is required in order to esta­blish information about the context of use. The software fra­­me­­work installed on a mobile phone can obtain in­for­ma­ti­on from embedded sen­sors including accele­ro­meter, GPS, Blue­­tooth, WLAN, micro­phone, call logs, calendar, and ad­di­­tional sensors can be added for specific experiments. An overview of the software frame­work as used in the present study is shown in Fig. 1, and illustrates the em­pha­­sis on aggregating multiple sensor data into higher-level con­­text descriptions.

Fig. 1. Mobile Context Framework taking raw sensor data, extracting features, and translating into contextual labels describing people, places, and music

Method and Experiment

In our experiment 7 participants (4 male and 3 female, ages 18-29) were provided with a Nokia N95 mobile phone with our software framework installed. They were instructed to carry and use the mobile phone as they would normally use their own mobile phone for a two-week duration. In addition they were told to use the mobile phone as their MP3 player device and en­cou­ra­ged to upload their own music collection to the mobile phone. Thereby encouraging them to listen to the music that they liked and they would typically listen to on a daily basis. The software was run­ning silently and con­ti­nuously acquiring data from the em­bed­ded mobile phone sensors for the duration of the expe­riment. The framework included a virtual sensor component capable of ac­qui­ring data from the embedded media player application on the mobile device. The component obtained data about music tracks being played, the duration of the song, and current playback po­si­ti­on. In addition ID3 meta­data from each song including artist and title information was retrieved. As shown in Fig. 1, in­for­ma­tion was ob­tai­ned from the Blue­tooth sensor and phone logs in order to extract features describing the user’s so­cial re­lations. Features describing places were extracted from in­formation acqui­red from Wi-Fi and GSM cel­lu­lar net­work in­formation. Each of these features was trans­lated in­to la­bels describing people and places.

Results
The data acquired during the two-week duration for the 7 par­ticipants in the experiment is shown in Table 1. The par­ti­cipants were fairly active using the media player listening to 94-292 songs in total, meaning 7-21 songs on a daily basis on average. The played tracks indicate the number of tracks that were started on the media player, but we only denote a track as listened to when more than 50% of the track has been played. The number of unique music tracks in­di­ca­tes that some participants listened to a smaller set of tracks repeatedly. Each track played was logged with a time-stamp meaning that the time of day where the media player was being used could be analyzed.

Partici-

pant

Gender

Age

Tracks

played

Listened

tracks

Unique

tracks

Known

places

1

M

25

337

160

85

7

2

M

26

474

153

100

11

3

M

26

375

190

48

5

4

F

18

524

292

68

6

5

M

29

173

110

58

7

6

F

23

742

167

124

6

7

F

24

198

94

65

7

Table 1. Overview of music listening for the 7 test participants

The information on music listening was coupled with the analysis of the contextual labels acquired from the logs of em­bedded sensor data. The GSM cellular information and Wi-Fi access points were analyzed in order to determine lo­ca­­­tions. Based on the analysis it was possible to determine the places in which the participants spend the most time. Thus it was possible to determine if a participant was at “ho­me”, in a “known place” (a place where time was spent re­peatedly), an “unknown place”, or “in a transition” bet­ween places (continuous changes in GSM and Wi-Fi data in mi­nu­te size time windows). As an example Fig. 2 illustrates 7 known pla­ces discovered for a participant by analyzing the co-occur­rences of Wi-Fi access points. Each node re­pre­sents a Wi-Fi access point and edges denote access points dis­covered concurrently. In this example access points seen below a threshold has been excluded for clarity. Using this method an average of 7 known places were identified for the participants, as shown in Table 1. The time where most time was spent was assumed to be the “home” place.

Fig. 2. Seven known places P1-P7 discovered for a participant by analyzing the co-occurrences of Wi-Fi access points over two weeks. Each node represents a unique Wi-Fi access point.

The social relations were mapped based on the data ac­qui­red from correspondence logs (phone calls and SMS mes­sa­ges) in terms of who was calling and sending mes­sages to whom. Based on Bluetooth device discoveries (of par­ti­ci­pant mo­bile phones) it was possible to map out when the par­ti­ci­pants were in physical proximity of each other or in proximity of other people. The social relations of the par­ti­ci­pants based on the mapping of the Bluetooth data are shown in Fig. 3. The numbers on the edges denote the num­ber of Bluetooth dis­co­ve­ries indicating the time spent in proximity and the number of correspondences (calls and SMS messages).

Fig. 3. Social relations of the 7 participants mapped based on Bluetooth-based proximity (left) and call/SMS logs (right)

Based on this data it was possible to establish the context of use of the mobile media player on the mobile phone de­scri­bed as time, places, and people. It was possible to de­ter­mi­ne the time and places in which the music was being played and it was possible to determine the people pre­sent when the media player was being used for music play­back.

The analysis of the music used the track metadata that was ac­qui­red from the media player during playback. This was enhanced by social tags, which offer a large-scale source of descriptive labels about music exceeding the information available in typical track meta­data. Based on the artist and song title (ID3 tag) metadata the col­la­bo­ra­tive tagging of mu­sic tracks available from Last.fm was used to enhance and capture the underlying semantic aspects of the music. Based on a latent semantic analysis of co-occurring tags Levy and Sandler [] have described a method to enrich the original metadata with ad­di­tio­nal descriptions of related genres that facilitate a com­pa­ri­son of the songs. The model involves 90 specific semantic aspects describing the music based on social tags. For each aspect the ratio of tags present was calculated and com­bi­ned to obtain a track signature. Subsequently we collected tracks into sessions if the participant has been listening to at least 3 tracks in a row. Thereby combining the above track signature into a session signature constituting an average of the ratios of tag co-occurrences defined in the track sig­na­tu­re. For clarity we generalized the tag co-occurrences into 8 broa­der categories of musical style, which allows us to ob­ser­ve the chan­ging genre characteristics over time [], as il­lustrated in Fig. 4.

Fig. 4. Example five track sequence profile derived by enhancing the track metadata with Last.fm tags. Each color represents a music genre, as annotated at the top.

This approach introduces a way to describe the music being listened to in terms of a few generalized genres. This allows us to consider and compare which genres of music are being played over time by the participants and the particular contextual settings in which it is being played.

Our analysis of the data on music listening behavior indicated inte­res­ting patterns. First of all, the genres of mu­sic that were lis­te­ned to over time highly depended on the context of the user. In some places one set of genres were typically played, whereas in transition bet­ween places a dif­ferent set of genres were being played. The level of inter­ac­tion (e.g. skipping songs) also depended on the context of use. Transitions between places were cha­racterized by fre­qu­ent interaction (skipping and choosing tracks), whereas in known places the interaction was less frequent. The time of day also had an influence on the music lis­te­ning behavior.

Discussion

Our initial findings have indicated an inter­play of time, con­text (places), so­ci­al relations, and the media (music) being played. Based on these initial findings we propose that me­dia play­er user interfaces could benefit from involving para­me­ters including time, places, people, music genre as mul­tiple dimensions along which the user could navigate the avail­able music in a music collection. This means that the mobile context could potentially play a much more pro­mi­nent role in mobile applications, such as the media player. In this case, an obvious example is re­com­men­dation sys­tems that not only recommend music based on music si­mi­la­ri­ty, but also contextual similarity. Current music recom­­men­­dation interfaces typically rely mostly on music si­mi­la­ri­ty, but for instance Last.fm partly involve the social di­men­sion enabling the user to discover music that similar users have listened to. We find the po­ten­tial of involving mo­­re contextual parameters discussed in this study in­tri­gu­ing and suggest that these findings could inform designers and inspire new kinds of context-aware music navigation in­ter­faces for navigating and discovering content in the media player.

Fig. 5. Illustration of the 4-dimensional feature space that could shape novel context-aware music navigation interfaces

Fig. 5 outlines a feature space that could be considered to capture the four contextual parameters identified as shaping the music listening behavior in the study. Next generation interfaces designed along these lines, could allow users to browse playlists purely based on the similarity of tracks, incorporate listening habits found within similar contexts or alternatively take into account what other relevant users are listening to based on contextually derived social graphs. Thereby allowing the user to discover music based on si­mi­la­rity along multiple dimensions.

The initial findings in this study indicated the importance of studying the “actual use” of mobile app­li­cations “in-the-wild” over labo­ratory studies should be em­pha­sized. Studying the mobile user experience in con­text could potentially be a valuable source of inspiration for de­signers. This study has provided valuable insights on how the context of use had im­pli­ca­tions for the in­ter­action with and the content chosen in the media player application. Since our study is based on two weeks of data acquired from 7 participants, it has pro­vi­ded some initial indications on the potential for alternative con­text-aware music inter­fa­ces. Future work could involve lar­ger-scale studies with mo­re par­ti­ci­pants, as well as ex­plore novel context-aware mu­sic navi­ga­tion interfaces. It would also be relevant to ap­ply these methods in the study of other application domains.

Conclusions

We have ob­ser­ved 7 mobile phone users using a mobile pho­ne application in real-world set­tings. Based on logged information from multiple embedded mobile phone sensors we have been able to establish information about the time and context of use of a media player application on mobile phones and we have been able to identify how the context has implications for music listening behavior and the use of the application. We conclude that contextual information can offer valuable insights to the where and when of mobile use and provide valuable insights forming input for new approaches to mobile context-aware applications and user interfaces.

Acknowledgments

We would like to thank the participants that took part in our experiments. Also thanks to Nokia Denmark and to Forum Nokia for the equipment used in the experiments.

References

Barnard, L., Yi, J.S., Jacko, J.A. and Sears, A. Capturing the effects of context on human performance in mobile computing systems. Pers. & Ubiq. Comp. 11(2) (2007).

Baumann, S., Jung, B., Bassoli, A. and Wisniowski, M. Bluetuna: let your neighbour know what music you like, in CHI’07 extended abstracts on Human factors in computing systems (2007) 1941–1946.

Duh, H.B.L., Tan, G.C.B. and Chen, V.H. Usability evaluation for mobile device: a comparison of laboratory and field tests. Proc. of the 8th conf. on HCI with mobile devices and services (2006) 186.

Froehlich, J., Chen, M. Y., Consolvo, S., Harrison, B. and Landay, J. A. MyExperience: a system for in situ tracing and capturing of user feedback on mobile phones. Proc. 5th Int. conf. on Mobile systems, applications and services (2007) 57-70.

Kjeldskov, J. and Skov, M. B. Studying usability in sitro: Simulating real world phenomena in controlled environments. Int. Journal of HCI 22(1) (2007) 7-36.

Kjeldskov, J., Skov, M. B., Als, B. S. and Høegh, R. T. Is it worth the hassle? Exploring the added value of eva­luating the usability of context-aware mobile systems in the field. In Proc. MobileHCI (2004) 529-535.

Larsen, J. E. and Jensen, K. Mobile Context Tool­box: an extensible context framework for S60 mo­bi­le phones. In Proc. EuroSSC, Springer LNCS 5741 (2009) 193­206.

Levy, M. and Sandler, M. Learning latent semantic models for music from social tags. Journal of New Music Research 37(2) (2008) 137–150.

Oulasvirta, A., Tamminen, S. and Roto, V. and Kuorelahti, J. Interaction in 4-second bursts: the fragmented nature of attentional resources in mobile HCI. Proc. of the SIGCHI conf. on Human factors in computing systems (2005) 928.

Reddy, S. and Mascia, J. Lifetrak: Music in tune with your life, in Proc. of the 1st ACM Int. workshop on Human-centered multimedia (2006) 25–34.

Roto, V. and Oulasvirta, A. Need for non-visual feed­back with long response times in mobile HCI. Special interest tracks and posters of the 14th Int. conf. on World Wide Web (2005) 775-781.

Seppänen, J. and Huopaniemi, J. Interactive and context-aware mobile music experiences. Proc. of the 11th Int. Conf. on Digital Audio Effects (2008).

Zandi, N., Handler, R., Larsen, J. E. and Petersen, M. K. People, Places and Playlists: modeling soundscapes in a mobile context. 2nd Int. Workshop on Mobile Multi­media Processing (2010).

Biographies

Michael Kai Petersen’s research focus is cognitive modeling of digital media, combining “bottom-up” machine learning elements of latent semantics with “top-down” behavioral aspects of emotions, in order to emulate how we perceive content. An approach that might potentially be applied in applications ranging from web 2.0 social network interaction and sentiment analysis to cognitive neuroscience.

Petersen has 30 years of experience within digital media and has since 2004 been associated with DTU where he received his PhD doctorate in 2010 and was appointed Assistant Professor in the Cognitive Systems Section at DTU Informatics. He has a combined technical and creative background, holding M.Mic master degree in mobile internet communication from DTU 2004, while previously being trained in digital sound engineering as a producer in DR Danish Broadcasting Corporation, after completing his studies in music graduating from the soloist class of the Royal Danish Academy of Music in 1982. As an entrepreneur he has founded three start-up companies over the period 1993-2003 and has furthermore produced 100 plus CD albums. At DTU he is working with aspects of personalized context awareness at the MILAB mobile informatics lab, as well as teaching courses within the Digital Media Engineering master program related to building collective intelligence based on metadata, sensors and web 2.0 mobile interaction.

Share and Enjoy Digg TwitThis Facebook del.icio.us Google StumbleUpon E-mail this story to a friend! Print this article!

]]>
http://wi.hexagram.ca/?feed=rss2&p=68
Unsupervised User Observation in the App Store: Experiences with the Sensor-based Evaluation of a Mobile Pedestrian Navigation Application http://wi.hexagram.ca/?p=73 http://wi.hexagram.ca/?p=73#comments Mon, 20 Jun 2011 16:50:52 +0000 admin http://wi.hexagram.ca/?p=73 By Benjamin Poppinga, Martin Pielot, Niels Henze, and Susanne Boll

Introduction

To observe the mobile user experience various observation techniques exist. For field studies ethnographic observation techniques, like shadowing, are often used. In shadowing, an experimenter follows a participant and takes notes on the observed behaviour. Shadowing is known to be highly situated [3, 5]. However, this technique does not scale very well. Additionally, because of its obtrusiveness, it could change the observed participant’s behaviour.

To overcome the disadvantages of low scalability and high obtrusiveness, new observation methods are being developed. In theory, passive automated logging through sensors seems to reach the same “situatedness”, while being scalable and unobtrusive [3, 5]. In practice logging has rarely been applied to mobile observation during the last years. One reason for this might be that suitable data sources, e.g. sensors, were not available on a common mobile device. However, the extension of smart phones through external sensors showed that sensors are able to infer users’ everyday situations [2].

Nowadays a commercial mobile smart phone, like the iPhone, has a variety of sensors integrated. Thus, principles were earlier specialized hardware was required, can now be ported to the phone (e.g. a pedometer). McMillan et. al. [4] successfully applied logging in the large scale in a mobile game, which they submitted into the App Store. Hence, these sensors make logging more and more interesting as scalable, unobtrusive, and situated observation technique.
However, while there are some well-known concepts, like a pedometer algorithm available and ready for instant application, a holistic view on how to use, combine, and apply sensors to log a specific user action is missing. In this paper we present our approach towards unsupervised in-market studies and identify three major challenges based on our preliminary findings.

Figure 1: The PocketNavigator is a mobile pedestrian navigation application. Our integrated sensor-based observation technique is invisible to the user. However, the participation within the user study is defined as opt-in to maintain ethical correctness.

Experiment Design

Originating from the interest in providing tactile feedback as an additional navigation aid, we developed the PocketNavigator. The PocketNavigator is a personal navigation application, available for free in the Android market (see Figure 1). It is designed as a traditional map-based application, providing a map surface, the user’s location, and a waypoint-based route towards an arbitrary destination [6].

In addition, the application is complemented by a concept, which encodes the direction towards the next waypoint in vibration patterns. If the waypoint is straight ahead of the user, two vibration pulses of equal length are shown. If the next waypoint is on the right, the duration of the second pulse increases. The same happens to the first pulse, if the waypoint is on the left. If the waypoint is behind a user, three pulses are shown.

The additional values we assumed for the tactile feedback are that a user will need to watch the display less often, will commit fewer navigation errors, and will be less disoriented. These three assumptions serve as hypotheses for an experiment we decided to conduct remotely and unsupervised in the Android Market. If a concrete research question should be answered, it is recommended to define the hypothesis right before any sensor data is gathered.

Then, for each hypothesis the observable values need to be identified. Therefore one should think about what are observable events, supporting or not supporting the hypothesis. Comparable studies in literature already propose a definition for how a specific parameter can be observed. In case of the PocketNavigator, we decided to measure e.g. if the user looks at the display be using the roll and pitch angle, determined through the accelerometer.

In the last step the to-be-measured values will be assigned and represented through available sensors. In the exemplary case, if the user is watching the display we decided to use the accelerometer, which is able to provide the required values roll and pitch. As one can imagine, every matching of a hypothesis to an observable behaviour and then to a set of sensors induces some noise and inaccuracy. Thus, it is necessary to design and validate the sufficient representation of a to-be-observed behaviour iteratively. At some time if the selected representations are reasonably accurate, the experiment can be released to the market.

Identified Challenges

The PocketNavigator is still available and the study (i.e., the logging) is still ongoing. At this point, more than 500 people have participated in the study. In this section we transfer our experiences into general challenges, which need to be approached to further establish sensor-based observation in mobile applications. We identified three challenges: recruiting, analysis, and the question of internal validity.

Recruiting

In the participant recruitment process, a good application title and description needs to be provided to attract participants. Further, a well-designed application icon and some screenshots could also attract users. Without question, the application should provide the advertised functionality, be robust and reliable.

To fulfill ethical requirements, the study must be announced to the user in a clear and transparent way. Thus, the mentioning of the study in the application’s general terms and conditions will not suffice. A separate menu entry should be made available clarifying the purpose and frame of the study. The participation in the user study should be opt-in instead of opt-out. A user should be able to withdraw at any time, at least by uninstalling the application.

Early releases of the PocketNavigator presented the study in a separate info view, selectable through the application’s menu. If interested in participation, the user must explicitly check a checkbox. However, under this condition the acquisition of participants proceeded quite slowly. In an updated version, we proactively announced the study through a simple and short pop up dialog. If the user declines to participate in the study, a more detailed info screen on the study is shown, trying to convince the user. This approach has lead to an increased participation rate of about 5 to 10%.

Data Analysis

The recording of sensor values within the application is one thing. However, the gathered data of each client must be available to do analysis. Therefore we used a custom made server where each client connects via sockets and transmits the gathered data in chunks. Alternatively a script, running on an existing server can be used, such as PHP. This can also be easily combined with encryption algorithms, like SSL. To avoid loss of any data, a backup and watchdog is recommended.

Once the application is in the market and the participants are sending their data, one can begin the analysis. From our personal experience, we recommend doing the analysis on a regular basis, to identify overlooked aspects or strange application behaviours, which can be solved by adapting the logging algorithms. With every adoption it is important to monitor the version a participant is using and to not confuse different types of data during analysis.

The actual analysis is done by custom made tools, as universal analysis tools probably do not exist for a specific use case. In case of the PocketNavigator we build one application, which does a summary over the data of all participants and prepares an output file, which is readable by e.g. Microsoft Excel. Second, we have built an application, which is able to replay the behaviour of an individual user by displaying the values of the sensors in real time. The first tool is more suited for quantitative analysis, while the second tool can give insights in individual’s situations, which can be treated as qualitative data.

Internal Validity

In controlled experiments, internal and external validity are two contrasting aims. Internal validity is the validity of the inference of causal relationships, or how confident the observed effects can be attributed to the experimental manipulation. External validity is the validity of the generalization of experimental findings, or how confident the observed findings can be generalized beyond the experiments setting.

Typically, experiments (especially those conducted in the lab) focus on internal validity. The disadvantage of this approach is that the experimenters often generalize their findings to actual usage scenarios. Studying applications in ”real” use by making them available to a wide range of users -as we did with the PocketNavigator -stresses external validity at the expense of the internal validity.

In the case of the PocketNavigator we identified two factors threatening the internal validity: the design as quasi-experiment and the unpredictable usage.

Experiment vs. Quasi-Experiment

In a true experiment, conditions get allocated randomly. As we are studying the effect of the vibro-tactile feedback technique, in a true experiment, half of the participants would be chosen to use the tactile feedback and the other half not. However, in our actual study design, we allowed the participants to choose for themselves if the tactile feedback should be turned on or off. We were afraid that people would get annoyed by the tactile feedback, giving the application bad ratings in the Android Market, and in consequence deterring potential future users.

Thus, the experiment is not a true, but a quasi experiment. Due to the lack of randomization it is harder to rule out confounding variables and unsystematic variance. In our study, people that decide to use the tactile feedback could have certain traits or be in certain situations which favour or disfavour the usage. For example, if only people with lots of experience use the tactile feedback, because they are more open to new innovations, their navigation performance could be disproportionally better than average because of either their experience or the tactile feedback.

Unpredictable Usage

Another problem that turned up is the unpredictable usage of the application. In a typical experiment the task is well defined and well known to the person analyzing the data. In the case of the PocketNavigator, we neither have a way to dictate a certain usage pattern to the users nor can we completely understand the usage at a certain time. In what follows, we will provide a few examples of unpredicted usage patterns that could have threatened the internal validity if we had not identified them:

Example 1: Lying on table. In the first stream of data we received from our participants, we had many situations where no navigation took place. Having a close look at the data, the accelerometer indicated that the device was oriented parallel to the surface and the GPS signal showed no walking speed. From this data, we inferred that many users might be testing the application indoors, possibly leaving the device on the table and probably keep running the application in the background.

Example 2: Car Driving. At a later stage we were investigating the effects of the tactile feedback on the average walking speed. However, we were surprised by the huge variance in the walking speed averages. Taking a closer look at the individual data we found that some walking speeds were unnaturally high (e.g. > 70km/h in average) for pedestrians, hence, we inferred that people had used it in their cars or another moving vehicle.

Example 3: Background idling. Android offers parallel and background executing. As the PocketNavigator is expected to run in the pocket, we designed it to continue running when the screen saver is activated or another application is pushed to the front. The problem is that the Android OS does not really terminate applications but only pushes them into the background until the resources are needed otherwise. Thus, in a few cases the application kept running in the background producing nonsense data.

Conclusion

In this paper we report on our experiences, applying a sensor-based virtual observer to the Android Market. We identify three major issues that should be considered in future developments: recruitment, data analysis, and internal validity.
In our future work, we want to extend and apply the in-market observation methodology for true experiments, as well as for more open research questions, which cannot be answered within an experiment. Additionally, we want to apply logging as an observation method in a traditional field study to prove the validity of the method. Finally we are interested in the advantages, disadvantages, and limitations of the virtual observer in different settings.

Acknowledgments

The authors are grateful to the European Commission which co-funds the IP HaptiMap (FP7-ICT-224675). We like to thank our colleagues for sharing their ideas with us.

References

Choudhury, T., et al. (2008). The mobile sensing platform: An embedded activity recognition system. IEEE Pervasive Computing, 7(2), 32–41.

Consolvo, S., et al. (2008). Activity sensing in the wild: a field trial of ubifit garden. In Proc. of CHI.

Froehlich, J. et al. (2007). My experience: a system for in situ tracing and capturing of user feedback on mobile phones. In Proc. of MobiSys. 57–70.

McMillan, D. (2010). Further into the wild: Running worldwide trials of mobile systems. In Proc. of Pervasive, 6030, 210–227.

Mulder, I. et al. (2005). Socioxensor: Measuring user behaviour and user experience in context with mobile devices. In International Conference on Methods and Techniques in Behavioural Research.

Pielot, M. et al. (2010). Pocketnavigator: Vibro-tactile waypoint navigation for everyday mobile devices. In Proc. of MobileHCI.

Biography

Benjamin Poppinga is a scientific researcher in the Intelligent User Interfaces Group at the OFFIS – Institute for Information Technology, Germany. He is also pursuing his PhD studies in the Media Informatics and Multimedia Systems Group at the University of Oldenburg, Germany. He graduated in 2008 in Computer Science from the University of Oldenburg. Benjamin is working in the European research project HaptiMap which aims at making geospatial data and location-based services more accessible via non-visual interfaces. Benjamin is one of the lead developers of the PocketNavigator, one of the selected HaptiMap demonstrators. This Android-based application is able to guide a user to an arbitrary destination by providing tactile feedback.

The evaluation of these mobile navigation aids is a challenge. Thus, Benjamin’s research focuses on mobile ‘in the wild’ context sensing and evaluation techniques, especially in the domain of mobile navigation, exploration and location-based applications. Until now he focused on the quantitative recognition of certain navigation behaviours. His future research plans focus on the contextual enrichment of qualitative user observation techniques. Benjamin has more than 10 peer-reviewed workshop and conference publications. Further, he organised the international OMUE (Observing the Mobile User Experience) workshop and did reviews for several HCI conferences, like CHI, MobileHCI, UIST, INTERACT, and Ubicomp. He was selected for a Microsoft travel grant to participate and present his work at the SenseCam 2010 symposium.

Share and Enjoy Digg TwitThis Facebook del.icio.us Google StumbleUpon E-mail this story to a friend! Print this article!

]]>
http://wi.hexagram.ca/?feed=rss2&p=73
Experiences from the Use of an Eye-Tracking System in the Wild http://wi.hexagram.ca/?p=90 http://wi.hexagram.ca/?p=90#comments Mon, 20 Jun 2011 16:50:19 +0000 admin http://wi.hexagram.ca/?p=90 By Liisa Kuparinen and Katja Irvankoski

Introduction

According to Renshaw and Webb [10], the benefits of eye-tracking include the independence of data from user memory, the indication of problem solving strategies and a large amount of quantitative data. Examples of situations where the use of an eye-tracking system would be useful are when there is a need to get information about the most important objects used in navigation or to identify which objects in traffic a driver of a car notices or misses. In addition to eye-tracking, other methods such as interviews, observation and performance accuracy are applied to validate or complete the findings from the eye-tracking data.

Another issue is the need to research mobile user experience in the field instead of the laboratory. For example, Nielsen et al. [8] state that the field setting elicits a significantly increased amount of usability problems, as well as problems with interaction style and cognitive load that are not identified in the laboratory setting. If the research target is to investigate wider user experience in a natural context as well as to identify usability problems, the importance of a field study is even more evident.

The use of eye-tracking systems has been very sparse in the research of mobile user experience. Along with stationary environments, they have been used for example in the research of shopping behaviour, infants’ natural interactions, and various everyday tasks [2][4][5]. To our knowledge, the research of mobile user experience in a forest environment is virtually non-existent.

In this paper, we focus on using an eye-tracking camera in a typical Finnish rural environment – a forest. Our focus is more in the validity testing of the eye-tracking method than on the use of mobile devices in order to discover the issues that must be considered when planning eye-tracking tests in the wild.

Tests in the Wild

We executed multiple pilot eye-tracking tests in a forest environment with different tasks in different conditions. The eye-tracking system we used was iView X™ HED from SensoMotoric Instruments. This monocular system consists of an eye camera and a scene video camera which are attached to a bicycle helmet. The first tests were executed without a mobile phone. In that phase, the goal was to assess the feasibility of using an eye-tracking system in a forest environment and to pilot test task settings for future studies. During the tests, we took the users to the forest area to do simple navigation tasks. The tasks included, for example, walking through a certain route with a little guidance (no maps, paper or mobile applications were used), describing what he or she saw, describing how he or she located him/herself and describing the route in such a way that another person could follow it.

After completing the first experiments, a test with a mobile map service was executed. In this single experiment, the user walked a route according to given instructions and located herself on the map. The user was also asked to navigate on foot to a certain position pointed on the map. The composition of the test is presented in Figure 1.
In addition to recording eye-tracking data and interviewing the user during the test situation, the users were interviewed after the tests as well. These post-experiment interviews were conducted to validate and complete the eye-tracking data and observations made in both of the field test cases.

Figure 1. The goals of the test tasks were to resolve the current location on the mobile map and to navigate to a predefined position. The eye-tracking camera was attached on the bicycle helmet and the laptop used for data recording was carried in the backpack.

Challenges

In this section, we present the main findings of using an eye-tracking system in a mobile context.

Some problems concerning the use of eye-tracking systems are commonly recognised in stationery environments. Those issues include, for example, the difficulties of tracking a person’s eye movements if he or she wears glasses, if his or her pupil size is very small (e.g. when tired), the colour of the iris is light or if the person has very long, downward or made-up eyelashes [3].

Along with these problems, we also discovered some special issues that should be considered when conducting eye-tracking research in a mobile context.

Data Quality

There are some issues in using an eye-tracking system in the wild that may risk the quality of data. Perhaps the most challenging issue in executing an eye-tracking test in a field setting is that the off-the-shelf eye-tracking systems are unable to provide definite information about distance of focused gaze in a three-dimensional setting [9]. The monocular system we used provides data consisting only of a gaze cursor on the recorded scene video, that is, a gaze position relative to the head (and video frame) [7]. Therefore, we faced situations where we could not be sure whether the user focused his or her gaze on a tree three meters ahead or to the lake that could be seen between the branches of the tree.

There are only a few commercial binocular eye-tracking systems available, such as NAC Image Technology’s EMR-9, which has some parallax error compensation. In addition to these, different labs using eye-tracking methodology have been developing eye-tracking systems that resolve the parallax problem and head movement both in natural environment and virtual reality [9][11]. One solution to this problem is the use of the thinking-aloud protocol. In addition to the lack of head tracking and depth information, the features of a forest environment make it difficult to define explicit areas-of-interests on recorded scene video data.

Calibration of an eye-tracking camera is much more difficult in the mobile context than in stationary conditions. In a mobile context, especially when investigating mobile device use, the gaze distance varies from a few dozen centimetres to hundreds of metres. However, the gaze data is the most accurate at the calibration distance due to parallax errors [7]. We handled the calibration by using a large rectangular area, a wall or a large paperboard several metres away from the user in the same environment where the test was going to take place. The calibration was then tested by comparing the equivalence of what the video showed and what the user said he or she was looking at. Generally, the calibration needed to be corrected several times. We discovered that calibration should be repeated during the test because it quite easily weakened in motion even though the helmet with the eye-tracking camera was strapped very tight.

Due to the unreliability of the calibration and parallax errors the eye-tracking system may not be trustworthy enough to examine eye movements in  mobile device’s small screen. However, the eye-tracking system is very suitable for tracking situations in which a user takes a mobile device in hand and checks it for location or direction.

Experimental Conditions

Regarding the experimental conditions, the most obvious ones concern weather conditions, which differ from the stable environment of a research laboratory. It is important to take into account that, for example, rain may prevent the execution of tests at the planned time. The use of eye-tracking cameras also requires adequate light, thus, it is typically impossible to execute tests early in the morning or late in the night – at least during the winter. Moreover, the lighting conditions may vary during one single experiment session.

Wearing a helmet or other attachable objects with an eye-tracking camera, which has multiple hanging wires, and carrying a laptop in a backpack or a shoulder bag, handicaps the movements of the user and influences his or her behaviour, at least until he or she gets used to the equipment. For this reason, it is recommended that the actual test is not performed until the user has had some time to become familiar with the equipment. Improvements to the mobility of eye-tracking systems are being made, but to the best of our knowledge, the current solutions remain obtrusive to the user. For example, in 2008, a research project executed with a new kind of eye-tracking solution, lightweight EOG goggles, was reported by Bulling et al. [1], where the user also has to carry a laptop with him or her. On the other hand, Tobii Technology has recently introduced Glasses Eye Tracker, which uses a smaller recording unit instead of a laptop.

One limiting factor in eye-tracking tests in the mobile context is the low battery capacity that applies to many eye-tracking systems. Keeping that in mind, it is impossible to plan a user test that would last for hours. With our test equipment, the maximum duration for test recordings was about a half of an hour. The weather conditions (e.g. cold or hot) as well as the bag for the recording laptop also influence the duration.

Finally, it is essential to pay attention to the careful design and definition of test tasks in order to be aware of the user’s goals and to interpret the gaze data [5].

Underlying Cognitive Processes

One should be aware that eye-tracking data does not give all-encompassing data about the allocation of the user’s attention. Eye movements can be an indication of a shift in attention (overt attention); on the other hand, a user may shift his or her attention to another target without moving his or her eyes (covert attention) [6]. In our study, the dissociation between where the user looked and what she paid attention to was evident in the picture recognition test as well. After the user had walked the route in the forest, she was asked about what she saw. She was then shown pictures and asked whether they were taken of the route. The user was shown 16 pictures, five of which were from the route (see example in the Figure 2) while nine were from other forest scenes. The recognition rate was very low; only a couple of the pictures were recognized properly. The results of our recognition test cannot be completely trusted though because they are based on a very small amount of data.

Figure 2. One of the pictures used in the recognition test. The task given to the user after walking a certain route in the forest was to identify whether the pictures shown were taken on that particular route.

Conclusions

Despite the many challenges of using eye-tracking systems in a mobile context, they provide a valuable method for gathering data that could not be reached by any other method; for example, behavioural methods such as think-aloud verbal reports and reaction-time based methods lack the kind of data that can be gathered by eye-tracking solutions. The problematic issues presented should be considered when preparing a test with an eye-tracking system in the wild. Issues such as the weather and light conditions are easy to take into account. Some of the problems identified in this study, such as the difficulties of defining area of interests in three-dimensional data, should be remedied by the eye-tracking systems’ manufacturers.

Please note that this is a position paper. Many of the findings presented still require validation.

Acknowledgments

This work was supported by the Graduate School in User-Centered Information Technology (UCIT), the Nokia Foundation and Academy of Finland (project 1129346). Great thanks also go to Antti Nurminen, Mikko Berg, Ville Lehtinen and Tuomo Nyyssönen for helping with the research.

References

  1. Bulling, A., Roggen, D., and Tröster, G. (2008). It’s in your eyes: towards context-awareness and mobile HCI using wearable EOG goggles. In Proceedings of the 10th International Conference on Ubiquitous Computing (Seoul, Korea, September 21-24, 2008).
  2. Castagnos, S., Jones, N., and Pu, P. (2009). Recommenders’ influence on buyers’ decision process. In Proceedings of the Third ACM Conference on Recommender Systems (New York, USA, October 23-25, 2009).
  3. Duchowski, A. T. (2007). Eye tracking methodology: Theory and practice. London: Springer-Verlag.
  4. Franchak, J. M., Kretch, K. S., Soska, K. C., Babcock, J. S., and Adolph, K. E. (2010). Head-Mounted eye-tracking of infants’ natural interactions: A New method. In ETRA ‘10: Proceedings of the 2010 symposium on eye-tracking research & applications. ACM.
  5. Hayhoe, M. and Ballard, D. (2005). Eye movements in natural behavior. Trends in Cognitive Sciences. 9, 4, 188-94.
  6. Henderson, J. M. (2003). Human gaze control during real-world scene perception. Trends in Cognitive Sciences. 7, 11, 498-504.
  7. iView X System Manual. Version 2.2. SensoMotoric Instruments.
  8. Nielsen, C. M., Overgaard, M., Pedersen, M. B., Stage, J., and Stenild, S. (2006). It’s worth the hassle!: the added value of evaluating the usability of mobile systems in the field. In NordiCHI‘06: Proceedings of the 4th Nordic Conference on Human-Computer Interaction (Oslo, Norway, October 14-18, 2006).
  9. Pfeiffer, T., Latoschik, M. E., Wachsmuth, I., and Herder, J. (2008). Evaluation of binocular eye trackers and algorithms for 3d gaze interaction in virtual reality environments. Journal of Virtual Reality and Broadcasting, 5, 16.
  10. Renshaw, J. A. and Webb, N. (2007). Eye tracking in practice. In Proceedings of the 21st BCS HCI Group Conference HCI 2007 (Lancaster University, UK, September 03-07, 2007).
  11. Wagner, P., Bartl, K., Günthner, W., Schneider, E., Brandt, T., and Ulbrich, H. (2006). A pivotable head mounted camera system that is aligned by three-dimensional eye movements. In Proceedings of the 2006 symposium on eye tracking research & applications.

Biographies

Liisa Kuparinen is a doctoral student of information systems science in the University of Jyväskylä, Finland. She graduated as a master of economic sciences in 2008 from the University of Jyväskylä’s Department of Computer Science and Information systems. The working title of Kuparinen’s doctoral thesis is “Designing Mobile Map Services: the Viewpoint of Spatial Cognition in Navigation”. In her thesis she focuses on the problem of perceiving the virtual view in contrast to the physical environment. She has found out that the current mobile map services do not always support the user’s location-awareness very well. There is also a risk of user ignoring the real world while trusting only the guidance of the mobile map service.

Previously Kuparinen has worked in the research projects concerning user psychology and user-centered design and she has been coordinating a network of cognitive science and cognitive technology. Kuparinen was awarded a Nokia Foundation scholarship in the year 2009 and got a four-year funding from the Graduate School in User-Centered Information Technology UCIT starting from the year 2010. She has also had her own company since year 2005 concentrating on producing web solutions and IT support. In 2010 Kuparinen accomplished a series of eye-tracking pilot studies in the forests with a group of researchers from the University of Helsinki and Aalto University, both from Finland. The results of the experiences are reported in this paper and should be used when planning a research with an eye-tracking system and when refining the systems.

Share and Enjoy Digg TwitThis Facebook del.icio.us Google StumbleUpon E-mail this story to a friend! Print this article!

]]>
http://wi.hexagram.ca/?feed=rss2&p=90