Overview

My research is concerned with the human aspects of ubiquitous, pervasive and tangible computing. I am interested in both theory and practice: understanding how everyday, learning and work activities can be augmented and the intellect extended through interactive technologies. A focus is on how to design, build and evaluate interactive representations that support external and distributed cognition. Many of my ideas, research projects and theoretical developments have come about as a result of the close synergy and long collaboration I had with the late Mike Scaife.

Research themes:

External cognition

Technology-enhanced learning

External representations and collaboration

Shareable interfaces: mobile, tangible and ubiquitous computing

Theory and reflections

External cognition

The continuing thread running through my research is to consider how we interact with external representations - be they diagrams, sketches, animations, multimedia, virtual environments or other. In particular, my research is concerned with developing a theoretical account of the 'external cognition' that occurs when we create, interact with and use different and multiple representations for various kinds of activities (e.g. learning, problem-solving). I am also interested in how new media and novel technologies can be designed and appropriated to represent information in novel ways - that cannot be achieved using 'traditional' media and technologies (e.g. books, film, TV). A recent interest has been to look at the value of interacting with physical artefacts (i.e. tangibles) that are augmented with computation and digital representations in interesting ways.

An example of a novel interactive technique we developed is dynalinking, where multiple representations are dynamically linked to each other. This computational mechanism has potentially a number of cognitive benefits. For example, in one of our earlier research projects, called ECO-i, we developed multimedia software to teach complex concepts to children by dynamically linking difficult abstract representations with familiar concrete ones. The software (called PondWorld) was designed to enable children to construct and change aspects of one representation, which resulted in corresponding changes being made to an interlinked representation.

 

 

Pondworld also enables children to create their own abstractions (food webs) of a pond ecosystem and to visualise the outcomes of making both incorrect and correct decisions. To the left is a frame from the software showing what happens when a child incorrectly places the weed above the fish in the food web formalism - an animation is played with sound effects depicting a 'bloodbath' whereby the weed is seen eating the fish.

Technology-enhanced learning

Our more recent research has explored how different forms of external representations that are presented in different modalities (including sound and touch) can be designed to support playful and reflective learning. The Equator project was a six-year interdisciplinary research collaboration between eight British universities, exploring the relationship between the physical and the digital. A number of projects have been conducted that explore how novel learning and playing experiences can be designed that move beyond the desktop to promote reflection, collaboration and a sense of wonderment among children. An example is the Ambient Wood project, an outdoors digitally augmented learning experience. A variety of handmade and off-the-shelf devices (e.g., PDAs) were designed, built and deployed using a wireless network in a real woodland. A main goal was to enable children to reflect upon various invisible biological processes (e.g., photosynthesis) through discovering various physical aspects of the environment (e.g., light, plants, insects). The medley of learning tools included probing devices, an ambient horn and a periscope device (see belowthat when interacted with provided a range of contextually relevant information. Our studies showed children exploring the Ambient Wood in highly collaborative, imaginative and reflective ways.

For more information see Equator project.

 

External representations and collaboration

I am interested in the functional and changing role different kinds of external representations play in complex activities distributed over time, space and people. I have been studying how the combination of physical and electronic representations are used and created in a number of workplace settings, including the creation and editing of content in the news and media industry, the construction of building plans in engineering companies and the construction of booking forms and ticketing in travel centres.

An initial project that investigated how best to integrate multiple representations was Espace. This project was concerned with how to present information via multiple interlinked displays that can be interacted with in different ways by different social groupings. A focus was on the spatio-temporal aspects of how best to display co-linked representations on multiple displays such that the individuals can follow the flow of information that is being presented and also know intuitively how to interact with the information presented on different displays.

A particular research issue was how individuals in different groupings know which display (or part of) to look at and find and interact with the information they need (or which someone else is referring to) for a given task or stage of an activity. We have carried out a number of experimental and ethnographic studies (Rodden et al, 2002; Rogers et al, 2002) that have shown how orientation and positioning of different kinds of interactive displays can have profound effects on the nature of the coordination and collaboration that unfolds.

For more information, see Espace project.

Espace Project

Shareable interfaces: Mobile, tangible and ubiquitous computing

This line of research is concerned with developing new frameworks, principles and concepts for mobile, tangible and ubiquitous computing. We are also exploring how these technologies might be used in the real world. We investigate the security and privacy aspects in ubiquitous computing. Another strand of our research focuses on augmenting and extending everyday learning and work activities with interactive technologies that move "beyond the desktop". We have designed a number of enhanced user experiences through appropriating and assembling a diversity of technologies including mobile, wireless, handheld and pervasive computing. Domains we are looking at are collaborative learning, creative play, working, and decision-making. We are also investigating how a variety of surfaces can be integrated, including large mixed paper-electronic wall displays, tangibles and ambient displays.

For more information see ShareIT Project.

 

Theory and reflections

A central part of our research involves adopting a critical stance towards theory and conceptual frameworks. For example, we have critiqued the role of theory in HCI and its application; explored the role of philosophical argument in experimental design; and argued for the need for new challenges and agendas for UbiComp. We also adopt theoretical frameworks from other disciplines for informing our research analysis and design.