Week 12 log

Week 12

Beginning 26/12/2011

The week the general relationship between memory and HCI was explored. Our interaction with digital artefacts is governed by our memories, perception and learning abilities. Cognition i.e. reasoning, understanding and awareness etc. plays a part in all interactions. I explored different ways of interacting with digital artefacts; tangible and tactile interfaces i.e. touch field interfaces, chromarium cubes (tangible interfaces) and SAMSUNG i900 motion sense interface.  I also explored my work through prototype concepts; using music interpretation for social interaction.

Different types of data collection were explored and a few quick concepts developed. A concept explored was ‘emotional dating’-using heart rate monitors to give wearers true feelings away. Problems were identified though; that people may dislike such honesty. Another concept was titled ‘I’m sorry I missed you’ which identifies where users geographic overlap occurs to plan meetings. Another concept was ‘I’m missing you’ where two users get a tangible touch-activated interface. User 1 touches the interface when they’re missing the other person and person 2 is notified of it. This is an example of objects storing, sharing and reinterpreting memories.

Environmental attributes that could be captured by objects were explored and compared them with attributes that we usually capture for memory i.e. location, light and the idea of ‘souvenirs’. Memory doesn’t have to be static. Drawing and traditional means capture multiple states in time whereas photography only captures a single moment. Memory streams can be captured using photography.

The next week will involve examining the four areas of interest in more depth, doing more research and developing some working prototypes and mock-ups.

Exploring Early Concepts

1. Exploring tangible interfaces lead to the development of the 'touch cube' concept. The idea is that a user can literally 'touch' a memory, and that you can 'spark' its existence by interacting with a box. It is a subversion of the concept that memories are 'sparked' by objects. The object itself stores various memories. Below is a youtube visualisation of the concept. It was made using stock images from the internet.



2. The concept of being able to create a physical representation of a tangible memory as well as the use of objects to create and reinterpret memories (type 2 and type 4). It is titled 'I'm missing you' and it connects two people together. Person 1 misses person 2, so touches their device and it sends a notication to person 2's device which lights up.

It could also be implemented as a wearable interface.

Below is a storyboarded scenario for the concept.

3. 'I'm sorry i missed you' - identifying where users journeys meet up and allowing them to use this information to plan meetings.

4. 'Music Dating'

a. musically talented users create their own pieces of music to go alongside their dating profiles OR users have their own music created based on physiological input i.e. heartbeat. These are broadcast among other users and it interacts with other users personal music. The idea is that if peoples music sounds good then they will be good together. A downside is that it could be gimmicky and irritating.

b. Emotional dating. Heart rate monitors give users true feelings away. Problems: users may dislike such honesty, and heart rate does not necessarily indicate attraction.

Memory and HCI

Memory forms a critical part of human computer interaction (HCI).

Our interaction with digital artefacts is governed by our memory, perception and learning abilities. Our acceptance of new interfaces and their effectiveness is closely linked to memory. Creating engaging experiences is an important aim of HCI.

A lot of research has gone into HCI and understanding how people work to improve how computers work. My User Interface Design lectures with Kate Devlin have been invaluable in improving my understanding of how memory influences design and how designers have worked with memory to create engaging user experiences. Designers use memory as a resource for their designs as well as use their understanding of human memory and cognitive processes to inform the way that their designs come to fluition.

Cognitive Process in HCI

• Cognition plays an important role in Human Computer Interaction.

• The term cognition includes understanding, remembering, reasoning, attending, being aware, acquiring skills and creating ideas.

INPUT SENSES: CURRENT Sight, Touch, Sound. FUTURE: Taste & Smell

There are lots of new ways emerging to engage with computers, and increasingly more tactile ways methods of interaction for example the gesture based input of the paper phone.

Tangible Interfaces: UID THEORY

Physical object memory & its relation to user interfaced design. Points and images taken from Kate Devlin's UID Lecture slides. Related slides found here http://www.doc.ic.ac.uk/~frk/frank/da/hci/Interaction%20Types%20and%20Paradigms.pdf

Sensor-based interaction where physical ojects i.e. bricks are coupled with digital representations

When a person manipulates the physical object/objects it causes a digital effect to occur i.e. an animation. Digital effects can take place in a number of media and  places or embedded in objects.

A physical exploratory system embedded in objects.

Examples

Chromarium cubes

– when turned over digital animations of color are mixed on an adjacent wall

– faciliates creativity and collaborative exploration

Flow Blocks

– depict changing numbers and lights embedded in the blocks

– vary depending on how they are connected together

Urp

– physical models of buildings moved around on tabletop

– used in combination with tokens for wind and shadows -> digital shadows surrounding them to change over time

Benefits-can be held in both hands and combined and manipulated. Multi person exploration possible & encourages alternative ways of exploring and representing a problem space. People see and understand situations differently-can lead to greater insight, learning and problem-solving than with different kinds of interfaces. Creativity and reflection.

Design issues-how to map between action and effect. What physical aspect to use.

HCI Resource, Gesture Interfaces & More

http://www.sigchi.org/

A fantastic resource for novel HCI projects and research. Taken from Kate Devlins UID lectures.

Shows different ways of interacting with objects and creating memories.

http://www.huffingtonpost.com/2011/05/05/paperphone-smartphone-e-ink-display_n_858114.html

Gesture interfaces.



Touch field interfaces

http://www.pcworld.com/article/227509/invisible_touch_interface_creates_multitouch_force_field.html

Touch screen in a 'frame'. Can be mounted on a surface or hung mid-air. Extremely futuristic and multi-touch. A new way of interacting with objects-also senses the size of the interaction i.e. width of the finger and reacts accordingly.

Garret Quote: Objects as experience creators

“every product that is used by someone has a user experience: newspapers, ketchup bottles, reclining armchairs, cardigan sweaters.” (Garrett, 2003)

http://www.jjg.net/about/

Author of The Elements of User Experience: User-Centered Design for the Web

Source: Kate Devlins UID Lectures

Objects are considered of sources of experience and therefore memories. Designing objects to create a specific user experience is a common practise and forms part of every design process.

WEEK 11 LOG

Following a tutorial last week, four main areas of interest have been identified that need to be narrowed down;

TYPE 1: Memories stored in objects -> Intentional/Non-Intentional, Digital/Non-Digital
TYPE 2: Memories created by objects -> how humans and other objects interact and create memory
TYPE 3: Objects subverting memories -> Physical form & Informational
TYPE 4: Objects reinterpreting memories -> Purpose?? Methods?? Subject Matter??

Related designs and the importance of Memory within the development on design will be investigates and communities will eventually be identified. Mock-ups need to be created as part of the design process and continued investigation into technology is vital i.e. Pachube and Arduino (Pachuino). It was deemed beneficial for a Supervisor to be attained for assistance on the technical side of things although it is not necessary.

Achievements this week: Investigation into current works & research.

Goals for next week: Working mock-ups using Pachuino, Testing Wrangler, Models of concepts and ideas.

Goals for January; Narrow area of focus; define project proposal.

Data Manipulation Tools: Wrangler

As i will be probably collecting large amounts of data for my memory dissertation project, i will need tools to manipulate this data.

Wrangler is a tool which allows greater flexibility than Excel. Data can be imported, manipulated, than exported. This would come in very handy, and i will try it out.

http://vis.stanford.edu/wrangler/

Wrangler Demo Video from Stanford Visualization Group on Vimeo.

The Big Dripper: Arduino Project

http://vimeo.com/20636301



source code https://github.com/vishnubob/bigdripper/tree/master/bigdripper

'Big Dripper is a robotic sculpture created by Giles Hall in 2011. It is a concept based on Harold Edgerton's Piddler. Edgerton's Piddler, also known as a "Time Fountain", uses a stroboscopic light source to highlight individual drops of water in a constant stream of liquid. With the strobe off, the stream looks like a solid cylinder of falling water. With the strobe on, and correctly synchronized with the actuation of the pump, the individual drips of water that compose the stream are exposed. The drips appear to hang in space as if frozen in time. By modulating the phase relationship between the frequency of the pump and the frequency of the strobe, the device can generate optical illusions of motion. For example, the individual drops can appear to fall slowly, or even crawl upwards.

Big Dripper expands on this idea by adding eight individually controllable pumps. By adjusting the phase relationships between the pumps and LEDs, a variety of visual effects can be achieved. '

Technology used:

Arduino; Lighting 

Memory theory:

Gives the effect of being able to isolate specific droplets of water and change their motion. Makes it look like time is being slowed, reversed and stopped. 


It is a different way of intepreting memory. Memory is not stored, but a concept is drawn attention to. Perhaps an object can represent the concept rather than being an example of it.

Credit: 

Big Dripper from Giles Hall on Vimeo.

Be your own souvenir; Rep Rap & Predicting the future

http://blog.reprap.org/2011/04/be-your-own-souvenir.html


Be Your Own Souvenir! from blablabLAB on Vimeo.


A 'snapshot' memory of your physical state at a specific point in time. 3D scanning equipment connected to a 3D printer prints out a tiny model of a person which can then be taken home! It's an incredible idea, and shows how technology can be used to create and subvert memory or interpret it in different ways. The creation of a mini sculpture as a fast service is only really possible using technology. This physical 'memory' is different to how humans would perceive it. We could even create a multi-layered memory where the printers are continuously receiving input and that the individual is moving around.

Technology used: It is a hack; no purpose built technology is used.

• Three 'Kinects'
• RepRap (a plastic printer). 

http://reprap.org/wiki/RepRap
RepRap is an open source plastic printer, and because many parts are plastic it can actually self-replicate.





Memory theory:

This is the use of human physical form as a vessel for memory; and transferring it into an object. The model is an encapsulation of various memories; your pose, your clothing, the object you're holding. What if more memories were stored like this, creating a mini world version of the world, and printing updates to the world to replicate real world events. If this much data could be collected, some people say you could predict the future.

Is predicting the future anything more than establishing patterns? For example using the theories of the quantified self i could analyse my habits i.e. web usage and from that predict what i was going to do tomorrow. Various factors affect my web usage i.e. which course i'm revising for, and therefore i might search for different things. Memory is a large part of this; memory being nothing but data interpreted in various ways.

An interesting video: Fate vs. Free Will. The beginning of the video is most interesting to me.

Tutorial/Presentation

I met with Kate Devlin and Edd Bagenal last week to discuss my progress and research.

Feedback: Areas of interest currently in a number of different areas, narrow these down. Start making things and investigating relevant work.

Plans


Narrow down my area of interest, and start developing more specific concepts.
Look at existing relevant work, analyse and change it
Start making mockups-Kate Devlin stresses the importance of prototyping in her lectures; even making things that dont really work and videoing them.
More tactile research-video, meeting people etc

Current Action

I have contacted my Physical Computing lecturer Brock Craft and requested his supervision on my project as i am interested in the physical world and ways that memories can be reinterpreted/subverted. Arduino is a pwerful tool.

Overview of my current work and future plans

2011-2012 Dissertation project Siobhan McKenzie

My current brief: ‘Memory; Hiding in plain sight’

What it’s all about:

Investigating unnoticed memories, and looking at ways that these can be noticed. Finding novel ways to reinterpret this information and draw attention to it. My personal manifesto stated that design must have a use, so what I design must always be beneficial in some way. My focus is objects and form; so I am investigating the ways that physical objects can store memories that aren’t noticed and what these memories are. Both interactive and non-interactive objects have been investigated.

My process:

My Plan for developing a complete proposal by January

1.      Examine ways that objects are currently collecting memories

2.      Examine the various ways that memories are processed, stored, reproduced/presented and shared

3.      Continue looking at ways that memories are lost and what these memories are-try  to profile the individuals to help categorise them

4.      Look at new ways to represent these: we have five senses so five different ways to receive stimulus

Methodology

Main note: Separate ‘objects’ into digital and non-digital

1.      Observation: Participant and non-participant. Observing people and understanding their habits; what they notice or don’t notice

2.      Trial current data collection software/hardware on test subjects to see how they work and how data is currently interpreted

3.      Build my own software/hardware and test them on users

a.      Use arduino

b.      Use Pachube to tap into current data streams, subvert them. See what data is currently being collected and why.

c.       ideas can be trialled or simulated in real like and scenarios created

d.      scenarios and trials on non working prototypes can be really useful-used this methodology in Kate Devlin’s UID lectures and they are successful in identifying problems before a real prototype is build. Test usability of any products I design

4.      Look at the ways that non digital artefacts could be ‘enriched’ i.e. the camera oyster card

5.      Investigate object personification and the creation of smarter objects: objects that can create their own memories via interaction with other objects/the world that are separate from users

a.      Objects having an agenda

b.      Objects being able to help by contacting other users

6.      Investigate form further

a.      Objects storing memory in their physical form

                                                              i.      Ageing

                                                            ii.      Scarring

                                                          iii.      Healing

b.      Archaeology

c.       Healing/Growth

d.      Cold reading

e.      Perception

Technologies that I am going to use

1.      Arduino; smart objects

2.      Smartphones

3.      Social network API

Theory:

1.      The quantified self - theory

2.      Internet of things; pachube

Music visualisation techniques

Continuing on with my research into ways to reinterpret memory data, visualisation is an option.

Below is a link to a general description of the steps it takes to visualise music. A possibility is using a data stream to create a 'visualisation' of some-ones memories. These could be broadcast and combined with other peoples. Algorithms could interact with eachother to provide an approximation of the music.

http://knol.google.com/k/music-visualization-techniques#

The ability of objects to interpret unusual data: AUDIOVISUAL INTERPRETATION

There is a huge range of information that could be collected from normal human interaction that isn't easily formatted by computers i.e. movement, speech. 

HCI has taken some massive steps forward, with the incorporation of new ways on communication i.e. movement and 3D scanning and projection. This data could be collected and interpreted.

http://www.cs.rutgers.edu/~vladimir/pub/pavlovic97pami.pdf

Gesture recognition: how it works.

a gesture recognition system can be made of different components

•  Gesture Modeling

•  Gesture Analysis 

•  Gesture Recognition

•  Gesture-Based Systems and Applications 

These can be modelled in 3D space, and reinterpreted.

Speech is also something that computers have been used to analyse. Audio recorders and interpreters were utilised in the following study to detect how many words men and woman said respectively on average per day.

http://www.sciencemag.org/content/317/5834/82.full

Women are generally assumed to be more talkative than men. Data were analyzed from 396 participants who wore a voice recorder that sampled ambient sounds for several days. Participants' daily word use was extrapolated from the number of recorded words. Women and men both spoke about 16,000 words per day.

Findings

Sex differences in conversational behavior have long been a topic of public and scientific interest (1, 2). The stereotype of female talkativeness is deeply engrained in Western folklore and often considered a scientific fact. In the first printing of her book, neuropsychiatrist Brizendine reported, “A woman uses about 20,000 words per day while a man uses about 7,000” (3). These numbers have since circulated throughout television, radio, and print media (e.g., CBS, CNN, National Public Radio, Newsweek, the New York Times, and the Washington Post). Indeed, the 20,000-versus-7000 word estimates appear to have achieved the status of a cultural myth in that comparable differences have been cited in the media for the past 15 years (4).

AUDIOVISUAL INTERPRETATION

Sound and visuals can be generated from  data using various algorithms. What has not been done yet is having different objects respond to eachother and change the data interpretations.

http://www.australianmusiccentre.com.au/article/living-songs-music-law-and-culture-in-aboriginal-australia

Aborigines interpret the information about their environment and share this information through song: how can the collection of data be transferred into sound and shared? 

What data would be used? How would it be interpreted? A new way of making music. Can it be made into visual data?

Object personification & agendas

I am very interested in how objects can tap into memory streams, and interpret them in different ways. Objects are an extension of the owner, and can create novel ways of interpreting experiences to benefit the user, and possibly to also project the identity of the user. I am interested in how these objects can then interact with other objects and create memory streams of their own. I am interested in what information can be collected, innovative ways to interpret or subvert it in and then to 'tap in' to other users objects and affect them. Then to be able to view what is made. Agendas in objects also interest me; how their form affects us.

Objects create memories constantly, and we have associations which spark memories. Objects have 'agendas' whether this is overt ie. a functional use, or covert i.e. a function which is designed to secretly help us, or hinder us. I am interested in personifying objects and treating them as more than inanimate objects; how they can interact with people in new ways. Both non interactive and interactive objects create memories. I am most interested in items which people are often in contact with or have emotional connections with.

Perception

http://joeltalks.com/index.php?p=1_22

Top down processing i.e. the way that individuals project themselves onto their experiences and stimuli to make sense of it. Explains how false memories can be created as a side effect of normal mental processes.

Top-down processing

Top-down processing is a core part of how the brain builds meaning for you to understand. In top-down processing, your mind actively imposes meaning on your perceptions, memories, and understanding of the world. The brain uses top-down processing to create meaning by applying what you already know or expect or believe. This process is what allows you to make sense of the world even if the information you get from the world may be a bit misleading or may be miss some details. 

I talk about top-down processing a lot in my presentations.

Here are a few illustrative examples.  

READING

The literal letter-by-letter reading of the following words would be nonsense. But, you can probably read these words and follow the meaning pretty well. The reason you can do so is that you apply your knowledge of words to impose an understanding that the letters themselves do not otherwise give you.

Arocdnicg to rsceearch at Cmabrigde Uinervtisy, it deosn’t mttaer in waht oredr the ltteers in a wrod are, the olny iprmoatnt tihng is taht the frist and lsat ltteer are in the rghit pcale. The rset can be a toatl mses and you can sitll raed it wouthit pobelrm. Tihs is buseace the huamn mnid deos not raed ervey lteter by istlef, but the wrod as a wlohe.

PERCEPTION

You probably think you see a white triangle below. There is no white triangle in the actual light that your eyes sense. There are only black shapes that have parts missing. You perceive a triangle by imposing that perception on the sensory data that your eyes are giving you.

You probably perceive the middle stimulus below (blue) differently on the first line than the second line, but it is exactly the same stimulus both times. Your eyes see the exact same data each time. So, the different perceptions on the two lines cannot be due to the stimulus itself. It has to be that the different perceptions are because you use the context to impose a different meaning on the two otherwise identical stimuli.

MEMORY

Since your initial perceptions of your world are already themselves in part due to top-down processing, then it follows that your memory of those perceptions may too be influenced by top down processing. But, it may also be that the act of recalling a memory may itself also involve more top-down processing. That is, when you recall an earlier experience, at the moment when you do the remembering you again impose meaning on your understanding. This type of process explains the fun of the party game where one person whispers a story into the ear of another person, and then that person whispers the story to another, and so on. At the end, the last person "remembers" a story that has been very much changed. I talk about several examples of top-down processing in memory in many of my presentations.

Intelligent Objects & Memory: Arduino Project

I am in a group of three working on a Physical Computing Project. Ive been in charge of doing all of the coding and component building of the project, and the other two are in charge of the exterior design. I've really enjoyed making this smart object, and it is a good example of object memory: storing information, processing it, and outputting it accordingly.

There is an aspect of physical computing that is called ;making things talk'-there is a book on this. I am very interested in the idea of objects having their own identities sometimes operating outside of human interaction for periods of times. This will be further explored very soon.

I was able to program the arduino to 'remember' which side had been most recently interacted with, and to 'forget' these things when the arduino was tilted onto its side.

Pachube was one of the technologies that i was originally interested and there are lots of ways to integrate this with arduino. http://community.pachube.com/arduino/usb/pachuino You can also integrate processing for visualisation.

The process of the project was in four stages

1. Concept: What we wanted it to do (as a group)
2. Giving it these functionalities (programming done by me)
3. Building plans for the housing, and putting it together (Other group member)
4. Putting the arduino into the housing and soldering it together (All group members)

I really enjoyed this project as it developed me technical skills and it also gave me a taste of interactive object memory. I am very interested in object memory and experience, and the identity of objects.

The project is in its 3rd week, and we are on step 4.

The project is a box with three light sensors on the outside. When the user hover over each side, the lights change colour. There is a tilt switch so when the box is tipped over it turn off. There is a push button with two music programs: The Simpson's and Eastender's theme tune. When the button is pressed it cycles through these two tunes.

We are currently at the stage where the internal parts and external parts are fully functioning, but still need to be put together.

Internal:




I also drew a schematic diagram using Fritzing. I taught myself to use this program so that i could record and share my Arduino designs and replicate them.

Finished housing:

We used the Mitre Cutter and a Laser Cutter to cut six sides out and cut the text out. We had an induction on it courtesy of the Design Department staff. Now able to use this machinery we were able to produce our housing to a very high standard.

Noticing the unnoticed: Objects Capturing Memories and Creating Them

Accessing my google web history was incredibly surprising. I am very active, even up to 2am at night and 6am in the morning. Trend statistics allowed me to access this. This is quite similar to the quantified self. Now that i know how much I'm on my phone, and at the most antisocial hours, i would like to cut down. Before this i wouldn't have said if asked that i use my phone and use google that much.

Some statistics:

Total Google Searches: 25308

I find it interesting the way in which technology and certain objects interact with us; creating data-streams that we could tap into and use. Sometimes this data is not captured or thee are no systems in place to do so. My iPhone is a multifunctional object that has a multiple 'agendas'. It is made to be obsolete: the iPhone operating system degrades quickly and it is not damage-resistant (easily crackable screen etc), it wants to be able to send me things (push notifications) and more. It creates false experiences to draw me in (virtual reality, augmented reality).

Are digital experiences and augmented reality false memory? Yes, they are. We are often so immersed in digital experiences and they have so many aspects of 'real' experiences that we can be deeply affected by them. Take film for example: these experiences are not real; we know we are not in danger, but horror films can have a lasting psychological impact even on adults. The simplest example of this is that some people don't want to sleep alone after watching horror films. 

 Do objects create memories, interact with each-other and have 'agendas'? Yes, they do. 

A branded pen wants to make people buy other branded pens... And it has a variety of ways to do this... By being a better quality than non branded pens or by looking better...

I am also interested in the way that objects could interact without people: creating their own memories separate from us. My phone could tap into other phones, creating a digital footprint of its own.

I am very interested in the memories that objects create. 

By digging into my google history I rediscovered a design book that i wanted to read:

The Design of Things to Come: How Ordinary People Create Extraordinary Products[Hardcover]