USER TESTING

User testing proposal

 

Guest Testing A large part of designing any interactive experience for children is putting said experiences through the rigors of repeated playtesting “in the field.” This is especially important for the Virpets Theater team, since we have very little in common with our intended guests. We must examine everything from the input devices to the backdrops we create from a child’s point of view. We have developed a preliminary plan to address this need that includes a minimum of 4 live tests at the Museum. What follows is an enumeration of said plans along with an examination of our target audience, as well as our findings so far.

Target Audience & Inherent Challenges

Ours is an exciting challenge – design a technology-heavy interactive museum exhibit for children from 4-7 years of age. A few of the unique challenges that our young user base presents:

- Reading & Writing Ability
? Most children at this age will not be able to read
? Those that are literate will not be sophisticated readers.
- Attention Span
? Children will probably not want to sit through multiple tests.
? Even if they do, they cannot quantitatively express their preferences.
- Gathering Data
? It is unlikely that written surveys would be effective in this situation.
? The best way for us to obtain information is observation and video interview.
- User Consent
? Parents must give permission for children to appear on video.
? Our users are not legally capable of providing permission themselves.

As a result, our methodologies will differ somewhat from standard user testing methods. We will not be attempting to gain quantitative knowledge from these tests – rather, we will concentrate on gathering as much from observation as we can. When appropriate (and permissible), we will also tape interviews with the children to get their reactions to the system.

 

Overall Research Questions:

General
- Are some puppets chosen more than other puppets?
- What do children take away from the exhibit about puppetry?
- Which methods are most effective for conveying information?
- Would a “mascot” be a valuable tutorial tool?

Creative
- How does a backdrop influence a child’s storytelling capabilities?
- Does a themed set limit their stories to a particular genre?
- What effect does removal of backgrounds have on storytelling?
- How much time do children spend telling stories?
- Are children willing to perform with an unfamiliar child?

Interface
- Is the MS Sidewinder Pro too intimidating as an input device (joystick)?
- Could simple aesthetic changes make the Sidewinder more appealing?
- Is one joystick sufficient?
- How much functionality do kids need to feel in control?
- How much time do kids spend choosing a character?

Technology
- Can we project puppets and scenes from separate computers?
- How durable are the components? (joystick, screen, mouse, touchscreen)
- What abnormal demands will children place on the system?
- How much stress can the system handle?

 

Proposed Playtesting Schedule

This schedule is subject to change, especially in regards to the subject matter concentrations for each test. This is intended as a rough guide to content creation and materials procurement so that our team will be fully prepared for each of the tests.

Playtest 1: Week of February 20th

High-Level Goals:
- Observe effect of backgrounds on children’s play
- Observe 2-player dynamic
- Test intuitiveness of joystick interaction
- Basic observations about reactions to interaction

Conditions:
- Minimum of 2 puppets in system
- Minimum of 3 worlds (blank, generic, themed)
- Soliciting kids to play in the world (not walk-up and play)

Materials:
- Minimum 2 laptops or desktops/monitors
- Networking equipment (router, cables, etc.)
- 2 joysticks (one per station)

Playtest 2: Mid-March

High-Level Goals:
- Test potential input schemas for puppets
- Test child-facing UI
- Test revisions from Playtests 1&2

Conditions:
- 1-player or 2-player setup
- Solicited play (not walk-up-and play)
- Child-facing UI in place (puppet / world selection as applicable)

Materials
- Minimum of 4 puppets in system
- 2 joysticks per system
- Minimum 1 control computer, 1 server

Playtest 3: Early April

High-Level Goals:
- Test educational value of system
- Observe unfettered play on the system
- Preliminary tests of final system

Conditions:
- 2-player setup
- Full system loop (idle, selection, play)
- Tutorial elements in place
- All completed models to date

Materials:
- Recorder for theStage (if possible)
- 3 computers (Server, 2 control)
- Networking equipment
- All communications software

Playtest 4: Late April/Early May

High-Level Goals:
- Stress-testing final setup
- Enjoyability of final setup

Conditions:
- Setup display area
- Walk-up-and-play (little soliciting)
- All software should be in place
- Most models in place

Materials
- Minimum 3 computers (Server, 2 play stations)
- Large projection screen
- Digital projector
- 2-4 joysticks or other input devices
- Touch Screen displays

Observations from the First Guest Test
February 22, 2003 – Basement of the Children’s Museum

Conditions:
Technology:
2 Networked systems – 1 client, 1 client/server
1 small monitor for display of world
1 joystick per character
Content:
Same puppet for both players
One world – the old Garvey family “house”
Real “creepy smile” puppet was on display near our station
Methodology:
Videotaped interactions
Minimal interaction with the children
Answered any questions kids had

Observations:
1. On the joysticks:
a. Kids are not scared of the Sidewinder joysticks.
b. They are quite willing to press all the buttons
i. They watch intently to see what each buttonpress does.
c. They are also willing to abuse the heck out of the joysticks.
d. To some kids, the joysticks seem to suggest a military or flight-based experience.

2. On multi-player behavior:
a. Kids were willing to play with strangers as well as their own parents.
b. Children also shared their knowledge about the controls, teaching one another.
c. Behavior was often imitative – kids were more willing to come up and play if someone else was first, and if one child was rowdy, their compatriot would most likely follow.
d. Some of the best interactions were a parent and child learning together.
i. Moms were just as willing to play as dads!

3. On the presentation:
a. Kids didn’t seem to notice that collision detection wasn’t implemented.
b. The size of the monitor may have been a factor – it was so small that the joystick almost dominated the experience.
c. Kids wanted the puppets to cross into each other’s physical spaces.
d. System does not attract people when idle. Need idle animation, etc.

4. On storytelling:
a. Children didn’t know that we wanted them to tell stories
i. They thought this was a video game
b. Some kids seemed to be telling one-player stories
c. Having 2 different puppets might help
d. Adults were must more emphatic about the storytelling/roleplaying

5. On context:
a. The kids that did draw a parallel between the real puppet and the virtual one were much more interested in the experience.
b. “I want to play with a real puppet now!”
i. Real puppets should be in VERY close proximity in final exhibit

6. Quirky behavior:
a. Kids universally loved the slider that made the puppets “fly”
i. Closest many got to telling stories