Yesterday, UTS Integrated Product Design student Keisha Jayaratne took second place in the CHI 2016 conference’s Student Research Competition. After being shortlisted based her Extended Abstract paper and poster at the conference, she presented her work on Memory Tree, a design that supports reminiscing using sound recordings. It was developed, prototyped, and tested with participants last semester as part of Keisha’s Honours programme, during which she was supervised by Elise van den Hoven. At CHI, she took second place among the undergraduate research submissions.
We’re happy to see Keisha’s work on supporting remembering was well received and allowed her to present in front of quite a crowd at CHI. For those of you not at CHI, her paper is already available for download from the ACM Library.
Credit for the photo up top goes to Berry Eggen, who was in the audience. The other image was a slide by Keisha and grabbed from the CHI session webcast.
Recently, the new home for our Sydney members, the Faculty of Engineering and IT, sat down for an interview with the newly appointed Professor Elise van den Hoven. The following interview was done by Jen Waters of Origami Communications, with photographs by Images for Business. We received permission to publish a copy of the interview, first published internally within FEIT in late March.
Elise van den Hoven draws on a range of disciplines when working to improve people’s everyday lives through enhanced recollection of memories.
“My research sits between three fields: design, psychology and computer science,” she explains. “It is about people-centred design and human- computer interaction (HCI), and how we can use or appropriate technology to meet people’s needs, and make their lives better and easier.”
Elise leads the international research program Materialising Memories, which takes a design-led research approach to studying people’s everyday remembering activities and experiences. A key focus of the project is around voluntary and involuntary memory cues, how people are affected by them, and whether they can be purposely created; for example, looking at how people use media, such as photographs, in everyday remembering situations. The aim is to inform and create innovative, interactive media products that help people with both remembering and forgetting.
The impacts of her work are broad, from everyday remembering activities to helping people with conditions such as dementia. “Remembering has different types of functions; for instance, people use sharing memories to build amazing relationships, so there’s a social function. You can also use it to solve problems through recalling how you solved similar issues in the past, or to form opinions about things. People use it all day, every day – but we’re usually not aware of it.”
The project is partially funded by a five-year personal fellowship, awarded by the Netherlands Organisation for Scientific Research (NWO) to give talented researchers the opportunity to develop their own line of research and build a research group. Comparable to an ARC Fellowship, her grant is valued at around AU$1 million over five years, and receiving it was quite a coup.
“For researchers to get funding from a design background is rare, to say the least,” she explains. “I was the second person from a design faculty to get such a grant in the history of The Netherlands. As a personal fellowship, it gave me the confidence that I was doing something right.”
Elise’s initial tertiary studies were in biology, specialising in neuroethology – the science and neurology of behaviour. A designer at heart, she found her place at the nexus of science and creativity, undertaking a post-master in technological design, and then a doctorate, at the Eindhoven University of Technology. It was here that Materialising Memories was born.
She spent ten years as an Assistant Professor in Eindhoven’s Department of Industrial Design, during which time she was a visiting scientist in Atlanta’s GeorgiaTech, the University of Sheffield in England, and the Faculty of Design, Architecture and Building (DAB) at UTS. When the opportunity arose for both Elise and her partner to relocate abroad, they moved to Sydney, and she took up a role as Associate Professor in UTS:DAB.
With human-computer interaction as the foundation of her work, FEIT seemed a natural fit for further developing her research and teaching in the field, and she commenced as a full professor in the School of Software on 1 January this year. Her work is interdisciplinary and collaborative, and Elise hopes to build partnerships with the many FEIT researchers working across HCI and interaction design. She is also keen to explore the potential of the UTS Data Arena in facilitating tangible interaction with her research.
“I think there are a lot of opportunities here. I’m always open to collaboration, and would welcome people to contact me if they’re interested in exploring collaborative opportunities, in particular in relation to the Materialising Memories research program.”
A few fast facts about Elise
Elise grew up in The Netherlands in a village called Best, about 10 kilometres from Eindhoven. With Eindhoven as her base, she has been lucky enough to live in a few different places through her research: the Caribbean, while researching her graduation project for her undergraduate biology degree, and sabbatical periods in the United States, England and Australia. She moved to Australia in 2012, settling in the Sydney suburb of Pyrmont.
Reading is a favourite pastime, along with meeting up with friends and getting out in nature. She loves hiking – long-distance walking in particular – and enjoys playing games, both board and computer. Elise has a fondness for travel, especially to islands; she has just returned from Sumatra.
Elise had a range of unusual jobs during high school. She fondly recalls her after-school job delivering mail for the local notary, and the challenge of racing on her bicycle and planning the best routes to deliver all the mail personally in the half-hour allotment. She also spent two weeks working in a meat factory in her village and says she saw some things she would prefer never to share. Consequently, she no longer eats meat.
Over the past week, most of us moved over to our new offices in the ‘cheese grater’ building of the Faculty of Engineering and IT. While staff and visiting fellows found a home among other staff offices on level 7, the postgraduate students moved to desks in the Interaction Design and Human Practice Lab on level 6 (room 406). There, we are obviously close to the IDHuP group whose lab we co-occupy now, as well as the Creativity and Cognition Studios, our nearest neighbours.
Of course, the contact information on this website is already up-to-date in case you’d like to call or drop in.
Materialising Memories group Faculty of Engineering and IT University of Technology Sydney
Building 11, Level 7, Room 217 81-115 Broadway (postal address: PO Box 123) Broadway NSW 2007 Australia
After a breakup, what happens to shared possessions is often a prickly issue. Even if one person clearly owns something or has access to it, such as copies of digital photos, those possessions still carry the legacy of the other now gone. What happens to those traces of a digital life spent together after people break up? Could the technology that supports the generation and collection of those photos, messages, and other media also support the process of two people going their separate ways? Those are in essence the questions in the thesis of Daniel Herron, who was recently interviewed by UTS Newsroom. The article, available online, focused on his position as a joint degree student supervised by Wendy Moncur in Dundee and Elise van den Hoven at UTS in Sydney.
Our project leader, Elise van den Hoven, started as a full Professor in the Faculty of Engineering and IT in January 2016. With her promotion comes a move to the School of Software within FEIT, where the work on Materialising Memories will continue. The new faculty formally announced the move last month via their newsletter in a bid to strengthen the growing number of people working on Human-Computer Interaction and interaction design. There are also plans to deepen the collaboration between FEIT and our current Faculty of Design, Architecture and Building (DAB).
The Visiting Fellows and postgraduate students will also make the move from DAB to FEIT early this year. Our current projects will remain as they are after the trek across campus. Once we have settled in we’ll give another update!
Last week, we had our third annual Making Memories day. The one day getaway from research led us to dive into the history of Cockatoo Island and explore Sydney’s Museum of Contemporary Art. As the accompanying image shows, it was also a good opportunity to get a picture of (a large part of) our research group as we currently host several overseas visitors in our space in Sydney. Quite a few of us will also be making the trek to OzCHI in Melbourne next week, which is sure to pad our collection of group photos.
Most of us who have followed a story felt attachment to the main characters. Maybe you yelled at the puppet show as a kid to warn them of the character in the dark who was about to knock, or you revelled in the danger and you felt a little empty after the story was over. Such feelings of attachment can be even stronger when you take up the role of the character, that is, to become the one who knocks, or jumps, or solves puzzles, and saves the dragon. Yes, kids, save the dragon. Rescuing the damsel in distress is so yesteryear.
The avatars we play may not be us. But in some games we get a lot of options to tune the avatar to our liking, and perhaps play a bit with our own understanding of our identity. They may look better, behave far better, or act as indiscriminate rogues, and you may think of them as more dateable than you are. Despite differences between the virtual world and real life (here on earth dragons are merely some sunbathing lizards), in-game experiences can transfer beyond the fantasy realm into everyday life and contribute to our understanding of ourselves. This touches upon our project interests, because we do have memories of those experiences.
In fact, those memories pop up in everyday life. A recent study by Poels, IJsselsteijn, and de Kort (2014) surveyed players of the online role-playing game World of Warcraft to see in what way the game influenced the thinking and dreaming of frequent players. They indeed found evidence for elements of the game to play a role in daydreaming and mundane thoughts. Additionally, their results indicated that objects in normal life may remind people of their virtual exploits. Unfortunately no examples of such objects and thoughts were included. It would have been interesting to see how real things relate to experiences in another realm. We do not need to wear our ‘Epic Hat of Uncertain Principles’ to reminisce about experiences we had while wearing that hat.
Three years ago I started to take screenshots of a game I was playing at the time, the role-playing fantasy game Skyrim. I thought it strange that we do photograph a day trip to a theme park, but not keep evidence of the countless hours we spend in an elaborate fantasy world. Perhaps we are too busy just being entertained by the game, or perhaps it is a social stigma to not show those unfettered glimpses into what we really prefer for enjoyment. After all, there is something of us in the choices we made for an avatar, even if it’s just to explore an alter ego. For example, a fairly large proportion of men play as female characters and certainly not always in skimpy, revealing outfits as you may expect. Similar experimenting happens the other way around as well. Maybe it is exactly this playing with identity that keeps it separate from our idea of who we are, and therefore any screenshots do not end up in a visual narrative of our life.
Yet, here I am writing this because I did take some screenshots that ended up alongside my other, personal pictures. Looking over my photos, there was no distinction between my life and that of my game characters.
One of the screenshots I took more than a year ago. Yes, that is my beloved character named Cruela with whom I spent somewhere close to 270 hours in-game. Make of that what you will.
This leaves me with a few questions. Do people capture any visual material from in-game experiences? Do they construct a narrative to go along with it? To what extent does that story relate to their real lives? Do they ever look back? If they do look back, is it ever shared with others? I wonder. For a games industry that builds on giving us enjoyable experiences, obtainable achievements, and in MMO’s also some amount of social status, seemingly little flows over into our everyday lives when it comes to remembering all those things.
While the actual gaming experience and avatars used to attain those are perhaps under-represented in the publicly observable realm of memory-supporting media, cosplay has seen quite a bit of popularity in the past decades. Cosplay, or dressing up as your favourite game, comic, movie, or series hero, is a way of expressing fandom through often elaborate self-made costumes. Even though doing so does not express the personal experiences of playing a game or watching fiction, it does indicate the cosplayer was in some way infatuated with that piece of popular culture and not afraid to show it.
Looking into personal memories of gaming and other forms of fandom provides a nice bridge between media studies and the study of everyday remembering. Would you be willing to share your stories? And do you consider those part of your personal past?
How time flies! I have been in Sydney for a month and everything here is fresh for me. Sydney is really a vibrant, beautiful and pleasant city. I am very lucky to know so many nice people in the Materialising Memories Lab. What’s more, they are so hard working people which really made me shocked. All these give me a good impression about life and experience here. I would like to share these good memories with my family and friends. How should I do? It seems that there are so many ways I could do. I could call them, send messages and pictures, or make a video call or send a postcard. I am thinking whether we could use a special media to communicate with our family and friends, sharing these good memories with them and giving them a comprehensive sensory experience.
In my previous research and design, I didn’t pay much attention to the emotion. Now I find that the real design could make people moved which means they can transmit emotion, bring back memories and give people surprises. Good design is a poem about life and it could bring people into deep thinking. The emotional memory is people’s memory of emotional experience in the daily life and it has the feature of vividness and impressiveness, and it is one of the most effective way to help people remember. So I would like to do some research on emotional memory and the design of media products.
Items are able to stimulate vivid experience of the past. Interactive devices can stimulate similar functions through cues, for example, photos, audio recording, the objects he used before. I will explore which objects/use behavior would give people positive emotion memory (e.g., wonderful childhood memory) and how do these cues impact on people’s positive emotion memory in the next step. Finally, I hope these study could be used in the design of media products for the sharing of good memories in our daily life.
My PhD research is focussed on Digital Separations – looking at what happens when a couple come to the end of a relationship with regards to their digital memory cues, and how technology can be used to support the individuals.
Although I am primarily based at the University of Dundee in (chilly) Scotland, I have had the good fortune to also enrol at UTS and become part of the Materialising Memories programme, spending time in (the much sunnier) Sydney. While preparing to spend time at UTS earlier this year I stumbled across an ironic problem – though I was investigating how technology can support people ending relationships, I needed to find technology to help me maintain my relationship with my partner while we were halfway around the world from one another.
Suddenly I had a vested interest in how technology could support people staying together.
The more I thought about it, the more I came to realise that the ways in which Shauna and I stay in contact during a typical day might fall short across nine and a half thousand miles, an 11 hour time difference and a reversal of seasons. Writing an email about what I’m up to, or receiving texts from Shauna about the project she’s working on would keep us up to date on each other’s lives, but we would want more than just various updates scattered across apps.
We decided that we wanted to communicate with one another using a variety of media within one channel to give each other a fuller picture of our time apart; this is what led us to Couple.
Couple is an app that’s billed as “An Intimate Place For Two”; upon creating an account you choose a partner to communicate with through the app. You can only have one partner linked to your account, creating a private space on your device that is separate from all other communications.
While in Sydney, everything Shauna and I sent to one another through Couple was stored in a timeline – text, images and video were all displayed in chronological order in one place. We shared voice recordings, location updates, and sketches with one another, as well as a handy pre-set “thinking of you” illustrated message.
One of my favourite features of the app was the Thumb Kiss. Both users press their thumbs to the same point on their respective screens and once the thumbs have been held in place for a few seconds, the screens glow red and the devices vibrate, mimicking physical contact. It’s beyond cheesy, but also kinda cute, and a strangely tangible way of connecting across a digital medium. It’s one of the few features that required both me and Shauna to be actively using the app at the same time, creating an intimate little interaction.
When Shauna joined me in Sydney for the last two weeks of my trip we continued using Couple to document our time together in Australia. Once we returned to Scotland, we came to realise that our Couple Timeline, which once acted as a very successful communication medium, had taken on a different role. It now acts as a diary of our adventures in Australia together, my first experiences of being at UTS and Shauna’s time as a single parent to our dog. The timeline contains cue after cue for memories of our time apart and together in Oz, and has ended up becoming a powerful tool for reminiscing.
In the end, creating digital memory cues and sharing them with Shauna helped us maintain our relationship during my time in Sydney. I gained valuable first hand experience in just how important digital memory cues can be to a relationship, and I can now more realistically imagine how difficult it must be having to confront them when a relationship comes to an end.
Ahead of the 2015 CHI conference in Seoul, Korea (coming up later this month), I made a short video preview to go along with a 10 page paper titled ‘Things That Make Us Reminisce: Everyday Memory Cues as Opportunities for Interaction Design.’ It’s only 30 seconds and can be seen below.
Behind the scenes
It takes a surprisingly long time to make a short video like this. It took me about a full working day, including the editing, adding overlays, and exporting the final result. So here’s a bit on the making of, including tens of cast members, hundreds of extras, and a couple of undisciplined dragons.
First up, the setting. I wanted a home-like environment for the video, with enough light to get a good image, plus an environment that would be quiet enough to get a decent audio recording. Eventually I settled on using my own studio apartment as I would have everything on hand there. The clear downside is of course having to use my bed and empty wall as the enigmatic backdrop for my narration. Rather than doing just a voice over I decided to show myself, tell why the paper is relevant, and show a bit of our method. As such, the video is more of a teaser from an information point of view.
I faced a few challenges in getting my video recorded. With only myself on deck (all the others ended up hunting loose dragons), how to hold my camera phone steady? I have another still camera that mounts to my tripod, so I opted to tie my phone to the bigger camera with elastic cord. A voice recorder was placed on my office chair just outside camera view, with a notepad acting as my cue sheet. I couldn’t actually read my script this way, so it took quite a few takes to get it right (I would make a bad actor). The desire to wear decent clothes while the room temperature reflected the heat and humidity of an Australian Summer didn’t help things either. I had to take a few breaks to cool down, and yes, it does explain my expression during the first second or two.
The method section was filmed in our MM lab, with the diaries and other snippets and pieces from the analysis spread out along a table. I stood behind the camera, did the diary browsing, and then panned the camera to get the other items recorded. Later, in editing the footage was sped up. The rest of the footage was cut to fit only the most important bits within the limit of thirty seconds. Finally, I added the text overlays and a blur and vignetting effect to move the focus away from the somewhat lacklustre setting.
Looking back, there are a few things I’d like to improve about my little video. The location isn’t great, and I feel I could get more information into the thirty seconds. Perhaps I could have shot a couple of things that made people reminisce for an introduction, and only briefly show how we got there with our diaries. It seems I need to get another paper accepted to put these ideas to the test!
The actual paper presentation will be during a session titled ‘Digital Collections, Practice & Legacy’ on Thursday, 23 April, starting at 9:30 in Room E1/E2. If you happen to be at CHI 2015, come and have a look.