uontimemockup.png

UOnTime

Improving the experience of student commuters in Sydney

Project Type: Mobile application design and experience

Time: August 2019 - November 2019

Role: User Researcher

Skills: User Research, User Testing, Wireframing, Prototyping, Iterative Development

Tools: Balsamiq

Note: This research was completed as a group project in the course Human-Computer Interaction at The University of Sydney

Overview

During my semester at The University of Sydney in Australia, I took the course Human-Computer Interaction. A major component of this course was using the methods we learned about on our own project. Our group focused on a mobile app to help student commuters. I found this topic really interesting because there aren't many student commuters at my home university. I hadn't realized it was such a problem! As it turns out, most Australian students don't move away to university, but instead, commute from home.  I ended up learning a lot about both HCI practices and Aussie student culture. 

The Challenge

Goal: 

To design a smartphone app helping university students to plan their daily commute to the university from home and vice-versa.

Key Features: 

  • Allowing students to plan their commute according to their calendars and timetables, taking into account transport mode (bus, train, ferry) and personal events

  • Helping students to find the ideal route, transport mode and stops depending on the time of day, start and end locations

  • Allowing students to provide real-time feedback and comments on buses, transport modes, routes, and stops, so that other students can benefit from the information

Phase 1 - Establishing Requirements

1 - The Art of Observation

In order to further understand the problem space, we were tasked with observing people that take public transport. To get a holistic representation of commuting habits and different types of users, we decided to observe students in different contexts (singles, couples, groups, time of day, location, method of transport). We recorded our observations and took notes about the characteristics and qualities of the users, their goals, how they currently reach them, and what problems they encounter. At the conclusion of our observations, we came back together as a group to collect our thoughts and to summarize our findings.

Screen Shot 2020-03-03 at 2.15.40 AM.png
Screen Shot 2020-03-03 at 2.15.46 AM.png
Screen Shot 2020-03-03 at 2.16.06 AM.png
Screen Shot 2020-03-03 at 2.16.18 AM.png
Screen Shot 2020-03-03 at 2.16.26 AM.png

Our group notes. Some of us preferred to take notes digitally on our phones, while others preferred pen and paper.

A Summary of Our Findings

  • The majority of the students we observed paid for their journey using a variety of Opal cards.

  • Many students use a variety of applications to support their public transport journey (e.g.

    Google Maps, Sydney Transport, Tripview, Messenger).

  • Students traveled alone more frequently than together.

  • Users of public transport get frustrated at unexpected delays.

  • The students listening to music seemed the most comfortable and relaxed about their journey,

  • While worried students and students running late did not listen to music.

  • Students preferred when transport was less busy so they could sit (together if in a group).

  • Students typically took only one train or bus line and didn’t have to transfer.

  • Train commuters going to University got off at Redfern the most.

  • Students used maps apps both before boarding and during the journey.

  • Many students seemed concerned with getting off at the right stop during their journey.

  • Both individuals that took the train stayed on the Darlington side of campus.

  • Students taking the bus tended to have shorter trips (<20min).

  • Students that took the train usually had longer trips (>20min) and were more likely to try and

    fill their time during the trip (ex. Have a conversation, take a nap)

  • Students want to know what to expect on their journey (ex. Upcoming stops, delays, how busy/full a train or bus is)

  • Students want the most time-efficient route possible.

2 - Interviews

After our observations, we had a lot of new information to process.  Accordingly, we also had some questions. Our next task was to conduct user interviews. As a group, we came up with relevant questions based on our observations. We then ordered them to create a script for a semi-structured interview. To test it out and to identify any pain points, we ran a pilot interview.

Pilot Interview Questions  

  • What modes of public transport do you take to University?

  • Do you have alternative ways to get to University?

  • Do you use any tools to assist in your journey?

    • If yes:

      • Which app(s) do you use?

      • Why do you find this app useful?

      • Are there any additional features you wish this app had?

      • Do you have any grievances about this/these tools?

    • If no:

      • How do you find information regarding stops, routes, and timetables?

      • Is there anything you find frustrating when planning your journey this way?

  • Do you communicate with friends to try and catch the same train/bus/ferry?

  • Do you have any grievances about public transportation you have to take to Uni?

  • Do you take the bus frequently?

    • If yes:

      • Why do you choose to take the bus over other transport options?

      • Please describe a typical bus journey for you (location, time, distance, etc)

    • If no:

      • Why do you tend not to take the bus?

  • Do you ever feel frustrated while taking the bus? Explain why

  • How well do you feel the bus system caters to students?  Explain why

Our pilot interview helped us figure out points for improvement. The main issue we ran into was that our script was too long. While we wanted to make the questions open-ended so the subject ended up repeating themself a lot of the time, and it wasn't an efficient use of time. Together, we revised our questions to be more concise and came to this final script.

Final Interview Questions  

  • Do you normally take public transportation to University?

  • What modes of public transport do you take to University?

  • Do you use route planning apps during your commute?

  • Is there any way these could be improved?

  • What problems do you encounter when taking public transport to uni?

  • What is a typical commute like for you?

Conducting the Interviews

Based on the different contexts (single, couple, group) that we had observed, we interviewed 5 different people about their experiences. We then used our raw interview notes to summarize key findings.

Screen Shot 2020-03-03 at 2.53.43 AM.png
Screen Shot 2020-03-03 at 2.53.57 AM.png
Screen Shot 2020-03-03 at 2.53.20 AM.png
Screen Shot 2020-03-03 at 2.53.32 AM.png

Some examples of our typed up interview transcriptions

A Summary of Our Findings

  • Coordinating journeys with friends can be difficult (interview 1, interview 2)

  • User get frustrated by overcrowded public transport (interview 1, interview 4)

  • Students that live closer to campus don’t tend to take public transport (interview 2, interview 5)

  • Buses are the public transport option most often used by students (interview 1, interview 2, interview 5)

  • Updated, real-time travel information is important to users (interview 2)

  • Integrating more features into one app offers convenience (interview 2, interview 5)

  • Users want an app that adjusts to unforeseen circumstances like human error, timetable

  • delays, etc (interview 4 and 5)

  • Travelers like to be notified when approaching their destination (interview 3)

  • Users don’t want to repeatedly enter the same details into an app (interview 5)

3 - Researching Existing Approaches

Based on our interviews, we learned about the apps that students are currently using to plan their commute. We decided to look into these apps more to figure out what problems they were solving, and how it related to the behavior we had observed.

App #1 - Tripview

  • Tripview tries to resolve the need to re-enter details every time for regular/recurring trips, by allowing users to save trips.

Screen Shot 2020-03-03 at 3.08.48 AM.png

App #2 - Opal Travel

  • Opal Travel solves the problem of not knowing exactly when a bus will arrive at your stop, by providing live location and countdown boards.

Screen Shot 2020-03-03 at 3.10.45 AM.png

A Summary of Our Findings

  • Interviewee 5 takes the same route repeatedly and mentioned getting frustrated at having to re-enter trip details into google maps every time.

  • Interviewee 4 uses bus timetables on stop to find departure times and encounters problems when the bus falls behind schedule, and there is no way to know how long until it arrives.

4 - Exploring The User Journey

With all of our initial research, we had a generally good idea of what our typical user would look like. It was now time to create personas, scenarios, and use cases to explore what features our app might need. 

Identifying Stakeholders

We identified our app's main stakeholders so that we could base our personas on them.

  • User / Passenger

  • Transport Sydney Bus employees

  • Transport Sydney Trains employees

  • Opal business managers

  • Local Council(s)

  • University staff

  • Students

  • App developers

  • App Investors

Screen Shot 2020-03-03 at 3.19.47 AM.png
Screen Shot 2020-03-03 at 3.22.19 AM.png
Screen Shot 2020-03-03 at 3.20.03 AM.png
Screen Shot 2020-03-03 at 3.22.29 AM.png

Some examples of our personas

Scenarios and Use Cases

Using our personas, we created short scenarios of using the app. We then generated an essential use case of a student leaving lecture and getting home from university. We also created an alternative use case for if the student chose to get home from a different route or method of transport.

Screen Shot 2020-03-03 at 3.29.22 AM.png
Screen Shot 2020-03-03 at 3.29.30 AM.png

Some examples of our scenarios

Essential Use Case - Student Traveling Home from University

  1. User leaves lecture

  2. User opens app

  3. User presses “take me home” button

  4. App uses GPS location and home address to find best routes home

  5. The app displays options for routes home

  6. The user selects their desired route home

  7. App displays route overlayed on a map

  8. The user follows the map to walk to the departure bus stop

  9. User arrives at the bus stop

  10. The app displays departure board for the bus stop, highlighting the bus the student should take

  11. User boards bus

  12. The app displays how many stops until the student should disembark

  13. The phone vibrates when the student is approaching the stop, and should press the stop button

  14. User presses stop button

  15. User disembarks bus

  16. App provides walking directions to home address

  17. The user follows walking directions

  18. User closes app

Possible Extensions

3. If no home address stored:

  1. the app prompts the user to provide a home address

  2. the user provides the home address

  3. return to step 4

 

4. If no GPS location available:

  1. app displays error message

  2. the app prompts the user to enter the address of the current location

  3. user provides address

  4. return to step 5

15. If the user misses the stop:

  1. the app notifies the user that they have missed their stop

  2. the app finds a new route

  3. return to 12

Alternative Use Case - Request Journey Sharing

  1. User 1 wishes to locate user 2 on public transport

  2. User 1 opens the app

  3. User 1 selects trip sharing option

  4. App provides options to share, or request another user to share the journey

  5. User 1 selects “request journey sharing”

  6. The app displays a list of friends to request journey sharing from

  7. User 1 selects user 2

  8. App sends journey sharing request notification to user 2

  9. User 2 accepts or declines

  10. The app notifies user 1 of response

Possible Extensions

6. If user 2 is not displayed in list :

  1. user 1 selects “add friend”

  2. the app prompts user 2 to enter the name and mobile number of user 1

  3. user 1 enters details

  4. user clicks “send request”

  5. return to 9

10. If the trip share request is successful:

  1. The app displays user 2’s route and location to user 1

A Summary of Our Findings

  • Our research, scenarios, and use cases helped us come up with requirements for our app.

5 - The Requirements

Functional Requirements

  • Provide public transport departure/arrival times MH

  • Provide station locations MH

  • Provide passenger capacity information for specific busses/trains/ferries NTH

  • Notify user of any delay/disruption of services SH

  • Allow NFC scanning of user’s Opal Card to notify users of their Opal Balance and other Opal benefits e.g. How far they are from reaching Daily/Weekly travel cap. NTH

Data Requirements

  • Details of users recent journeys should be stored SH

  • User’s ‘Home’ and ‘Work’ addresses should be stored SH

  • ‘Frequent trips’ routes should be stored SH

  • User’s Opal Card information (Balance, Auto top-up status, etc) NTH

Environmental Requirements

  • Be able to share location/journey/trip with other users NTH

  • Able to run on popular phone operating systems (Android, iOS) MH

User Characteristics

  • Cater to the travel habits of students SH

  • Trip planning shortcuts for frequent users SH

* Priorities:

   MH - must have

   SH - should have

   NTH - nice to have

Phase 2 - Design & Low-Fidelity Prototype

1 - Object/Operation Analysis

In order to better understand what visual icons our users would find most beneficial, we practiced analyzing different objects and creating a sketching vocabulary of common icons.

Screen Shot 2020-03-03 at 4.11.08 AM.png
Screen Shot 2020-03-03 at 4.11.26 AM.png

Our object analysis and final sketching vocabulary

Screen Shot 2020-03-03 at 4.11.34 AM.png
Screen Shot 2020-03-03 at 4.11.39 AM.png
Screen Shot 2020-03-03 at 4.11.56 AM.png
Screen Shot 2020-03-03 at 4.11.47 AM.png
Screen Shot 2020-03-03 at 4.12.06 AM.png

Practices for our sketching vocabulary

2 - Initial Conceptual Model

Our next task was to create an initial conceptual model.  First, we identified possible interface metaphors for each task and sketched a visualisation of that metaphor.

Main Task 1 : Plan Route

Use 'Add Friend' metaphor because it is familiar to users from social media apps. Used to plan/share a journey with another app user. 

Main Task 2 : Provide Feedback

User feedback is displayed using the metaphors of red and green. If user feedback says a carriage is full, it is colored red. Otherwise it is green.

Main Task 3 : Manage Habits

The user's frequented locations (commuting habits) are stored in an 'address book'.

Screen Shot 2020-03-23 at 10.15.50 PM.pn
Screen Shot 2020-03-23 at 10.15.56 PM.pn
Screen Shot 2020-03-23 at 10.16.01 PM.pn

Next, we wanted to identify possible interaction modes to use. We came up with:

Instructing​ - It can be used in the app to quickly plan journeys by instructing the app to 'take me home' or 'take me to class'. This is likely to be the most efficient way to interact.

Manipulation- It can be used to edit journeys by dragging and dropping points along the route. This provides an intuitive and easy way to reroute.

 

Feedback- When providing feedback, users can respond to pop-ups asking how their journey was. Asking a user to respond to a pop-up could help the system gather more feedback on routes

We also wanted to determine the interface type that would fit our identified interaction modes.

Speech Interface​ - A speech interface could be used when instructing. This would be especially useful when students are in a hurry to get to class and can use a voice command instead of stopping to look down at their phones.

Touch Screen Interface - A touch screen of a mobile interface would be most suitable for manipulation, allowing users to directly manipulate their route on the go. However, users could experience the fat finger problem.

 

Users could have the option to use a speech of touch screen interface when responding to feedback, depending on their preference.

After discussing our options as a group, we ended up deciding that the red/green metaphor was the most relevant to our app, as it can have multiple applications in showing carriage space, highlighting transport delays, indicating low Opal balance, the flow of traffic, etc.

We also decided that instructing was the most relevant interaction style, as users would most likely open the app wanting to find specific information (for example, directions home), and can efficiently instruct the app to provide them with that information.

3 - Expanding the Conceptual Model

With an initial conceptual model in place, it then became time to expand upon it. Based on the use-case and requirements we found in ​Phase 1, we had to figure out what functions our product would perform. Specifically, we had to allocate the tasks to the product or the human, and determine which options are under the control of the user.

Functions the System Should Perform

  1. Plan routes based on what users value most (fastest route, least changes, restrictions on the mode of transport)

    1. ​Users must enter ‘from’ and ‘to’ locations first.

    2. The system will provide multiple routes based on user’s preferences

  2. Plan routes based on user’s university timetable

    1. ​The system should do this automatically.

    2. Users must provide university timetable login information.

  3. Allow users to plan routes with friends.

    1. ​User must allow system access to user’s contacts

  4. Allow users to edit the planned routes

  5. Replan route if a user misses a stop or bus/train/ferry

  6. Use the GPS signal to help users locate themselves on the transport network

  7. Help users locate their friends on the transport network

  8. Provide directions to users on how to follow a route

  9. Provide shortcuts to frequented journeys and locations

  10. Collect feedback from users on their journey

  11. Advise users on where most space can be found on public transport

  12. Notify users of delays

  13. Provide live departure boards for buses/trains/ferries

  14. Track the location of individual busses/trains/ferries

  15. Provide transport timetables

  16. Allow users to add opal cards to their profile

    1. ​User must enter card details (via NFC or card no. & security code)

  17. ​Allow users to manage their opal balance in-app

    1. ​Top-up requires the user to enter payment details

  18. ​Provide estimate times for each part of the journey

  19. Provide street-by-street directions for walking journeys

  20. Provide station-to-station directions for train journeys

  21. Provide wharf-by-wharf directions for ferry journeys

  22. Provide stop-by-stop directions for bus journeys

Next, we started to ask ourselves how the functions are related to each other.​ We ended up deciding that:

  • Function 11 (find space) relies on user feedback from function 10 (collect

    feedback)

  • Function 13 (track bus/train/ferry) allows functions 12 (live departures) and 11

    (delay notification) to be performed

  • unction 6 (track GPS) is needed for function 8 (provide directions)

  • Functions 3 (plan with friends) and 7 (locate friends) work together to make

    commuting with others easier

  • Functions 19-22 should be able to work synergistically for journeys that span

    multiple transportation modes.

  • Function 12 (Notify of delays) information should reflect in Function 18

    (provide estimate times)

Finally, we wanted to determine what information would need to be available.

  • Access to TransportNSW’s routes and timetable information.

  • Access to TransportNSW’s Travel alerts and trackwork information

  • Permissions to access user’s GPS

  • Permissions to access user’s contacts

  • Permissions to sync calendar

  • For the opal functionality of our app, must have access to the user’s opal card information (via NFC or manually entering card details)

4 - Validating the Conceptual Model

Finally, with all of our information and data, it was time to start storyboarding! As a team, we created storyboards for each main task and then showed them to potential users to gain some feedback. From there, we refined them based on the feedback.

Main Task 1 : Plan Route

Original Storyboards

Screen Shot 2020-03-27 at 8.18.35 PM.png
Screen Shot 2020-03-27 at 8.18.42 PM.png
Screen Shot 2020-03-27 at 8.18.49 PM.png

Storyboards with Feedback

Screen Shot 2020-03-27 at 8.18.58 PM.png
Screen Shot 2020-03-27 at 8.19.07 PM.png
Screen Shot 2020-03-27 at 8.19.14 PM.png

Main Task 2 : Plan Commute According to Timetable and Calendar

Storyboards with Feedback

Screen Shot 2020-03-27 at 8.25.58 PM.png
Screen Shot 2020-03-27 at 8.26.29 PM.png
Screen Shot 2020-03-27 at 8.26.07 PM.png

Final Storyboards

Screen Shot 2020-03-27 at 8.30.32 PM.png
Screen Shot 2020-03-27 at 8.30.16 PM.png
Screen Shot 2020-03-27 at 8.30.39 PM.png

5 - Paper Prototypes

With storyboards completed, it was finally time to create the paper prototypes! We kept them pretty low-fidelity at first, showing just the core UI elements, so it would be easier to discuss and revise. 

Prototype 1

Feedback

  • Simple design, easy to find the route

  • 3 options for routes displayed clearly

  • Feels pretty basic

  • Self-explanatory icons

  • Some tasks might take a lot of pages to complete - since it’s so simplistic things might be ‘buried’

Screen Shot 2020-03-27 at 8.38.05 PM.png
Screen Shot 2020-03-27 at 8.38.14 PM.png

Prototype 2

Feedback

  • Home Screen cluttered

  • Icons are intuitive

  • Seeing routes overlaid on a real map would be better than seeing the transport network map

  • Not much information given on routes and journey options

  • How to navigate back a page?

Prototype 3

Feedback

  • Cluttered

  • Search bar and bottom menu bar are consistent

  • Intuitive interactions (button, slider, drop-down)

  • Choose route screen has a lot of information at once - it’s cramped

  • Choose route people icons aren’t clear - meant to show how crowded a route is - a better way to show this info?

  • Missing Opal functionality

  • Doesn’t let you pick ‘From’ location

Screen Shot 2020-03-27 at 8.38.23 PM.png
Screen Shot 2020-03-27 at 8.38.29 PM.png

Prototype 4

Feedback

  • Friends function

  • Too much stuff on the main page

  • Friend and profile page icon ambiguous

Prototype 5

Feedback

  • Search functionality for location is restricting. It can only give directions from the current location.

  • User might want directions from a different location than the one they are currently at

  • Too simplistic design, too sparse of information.

Screen Shot 2020-03-27 at 8.38.40 PM.png

Time to start refining! We showed our initial designs to some users, and from our discussions, we came up with the feedback above. Then, based on the feedback, we refined 2 prototypes even further

Refined Prototype 1

Changes Made

  • Based on prototype 5

  • Might be better options for persistent buttons at the bottom, we have a different symbol for the share ride detail functionality.

  • Information too cramped when displaying the multiple routes

  • Card option buttons might be too small to press easily

  • No homepage is nice. Gets straight to the most important features straight away.

  • The location selection feature is well placed and allows for directions from a location different from yours.

Screen Shot 2020-03-27 at 8.50.38 PM.png

Refined Prototype 2

Changes Made

  • Based on prototype 3

  • Added a From location option

  • Added Opal Card

  • Not all buttons are clear

  • Changed map icon to compass- don’t like it

  • The arrow on login and Timetable isn’t clear

  • Words are too bulky on routes (ex. Train and bus)

  • Feedback - missing bottom menu, disorienting

  • A to B on to and from could use an icon instead, make it more clear/intuitive

Screen Shot 2020-03-27 at 8.51.01 PM.png
Screen Shot 2020-03-27 at 8.51.07 PM.png

With 2 improved prototypes, we brought then back to some users for additional feedback.  Using all of this information, we went in and made 2 sets of more detailed prototypes, that would show the user all the screens needed to complete the tasks sketched in the storyboards.

Further Refined Prototype 1

Changes Made

  • Based on Refined Prototype 2

  • Changed map icon

  • Buttons and ‘x’ buttons are consistent

  • Typing input - should standardize text box with rounded or squared edges

  • Routes - cleaner, but the user has to ‘dig’ for details now

  • Added a menu below feedback form

Screen Shot 2020-03-27 at 9.02.06 PM.png
Screen Shot 2020-03-27 at 9.02.13 PM.png

Further Refined Prototype 2

Feedback

  • User’s liked the slide-out (accordion) menu.

  • Found that the less cluttered routes page now does not provide enough information (how long each journey takes)

  • Interaction on the feedback page is unclear - the user didn’t know whether they were meant to click or slide on the scale

  • The information on the ‘cards’ page was too sparse - couldn’t easily see the card balance)

  • The ‘route directions’ page, users were unsure what some of the symbols meant (original thought was that the three people silhouettes were meant to represent how busy the vehicle is)

  • User was unsure how to access the feedback page - was meant to pop up as a response to user boarding a vehicle.

Screen Shot 2020-03-27 at 9.02.25 PM.png
Screen Shot 2020-03-27 at 9.02.33 PM.png

Almost there! At this point, we once again brought our prototypes to potential users to get their feedback. Based on a culmination of all our work and feedback, we decided on which design was strongest and we made some final refinements.

For our final prototype, we decided on the former of the two further refined prototypes. We implemented the changes outlined in the previous users feedback once again:

  • We clearly place the opal card balance underneath the respective card

  • Feedback is now also accessible through the slide-out menu (in addition to the feedback popping up when the app detects that the user is aboard the transport via the phones GPS)

  • The feedback busyness scale now has a slider that clarifies to the users how it is used.

  • Symbols that represent how busy the vehicle is now corresponding more to our color interface metaphor i.e. red = full, green = empty, as opposed to the more confusing previous symbol of the people silhouettes.

Final Paper Prototype

Screen Shot 2020-03-27 at 9.13.41 PM.png
Screen Shot 2020-03-27 at 9.13.51 PM.png

Phase 3 - High-Fidelity Prototyping

1 - Horizontal Prototype

With all of our initial work done, it is finally time to create a high-fidelity prototype (!!!). ​ For these prototypes, we used the program Balsamiq so that we could design the interface and also make it clickable. To start out, we created a horizontal prototype to get a better idea and sense of our product. 

horizontal2.png
horizontal1.png
horizontal3.png
horizontal4.png
horizontal6.png
horizontal5.png

Beyond the design decisions we had made for the paper prototype, we also made some decisions based on the specific device, as well as our use of colors. For example, a device-specific decision we made was to include an Android nav bar on the bottom of the screen to allow user functionality to go back. In terms of color, the screen with the different route options shows the routes as color-coded based on the line of train or bus used. In addition, the modes of transport are differentiated by color as well, making it easier for the user to see their options and make a decision more effectively. To differentiate the top menu bar from the rest of the page, we consistently kept it yellow.

2 - Vertical Prototype

Our next task was to extend our horizontal prototype into a vertical prototype covering the functions of commute planning, visualizing routes, and giving real-time feedback/sharing information.

We determined that the 8 most important commute planning functions were configuring beginning and end destinations, configuring timing information, managing frequent routes, entering and visualizing of shared information, utilizing calendar information and personal preference, visualizing real-time information, managing Opal card balance, and providing feedback about the route. Based on our concept from Phase 1 and sketches from Phase 2, the following are the screens that we needed to implement in order for users to click through our scenarios.

Screen Shot 2020-03-28 at 12.49.09 AM.pn
Screen Shot 2020-03-28 at 12.49.17 AM.pn
Screen Shot 2020-03-28 at 12.49.26 AM.pn
Screen Shot 2020-03-28 at 12.49.36 AM.pn
Screen Shot 2020-03-28 at 12.49.47 AM.pn
Screen Shot 2020-03-28 at 12.49.55 AM.pn
Screen Shot 2020-03-28 at 12.50.03 AM.pn
Screen Shot 2020-03-28 at 12.50.12 AM.pn
Screen Shot 2020-03-28 at 12.50.22 AM.pn
Screen Shot 2020-03-28 at 12.50.32 AM.pn
Screen Shot 2020-03-28 at 12.50.41 AM.pn
Screen Shot 2020-03-28 at 12.51.01 AM.pn
Screen Shot 2020-03-28 at 12.51.09 AM.pn
Screen Shot 2020-03-28 at 12.51.18 AM.pn
Screen Shot 2020-03-28 at 12.50.51 AM.pn
Screen Shot 2020-03-28 at 12.51.28 AM.pn

While designing this vertical prototype, the most important designs decisions were:

  • Getting rid of the bottom menu bar - we decided it was redundant due to the pull out menu, and it was taking up unnecessary space. This was a big design decision from our paper prototypes, but we chose to prioritize keeping the interface simple since there is already a lot of information being shown to the user.

  • On the route options page, we moved the visual details of the route to the right of it instead of squeezing it in underneath. This gave the text there some breathing room and still left sufficient whitespace. Additionally, the images are now more visible and recognizable to the viewer due to their different colors.

  • In comparison to our paper prototype, we’ve simplified our Share Journey page significantly. Before, there were a lot of details that the user had to fill out, making the task tedious and unnecessary. We’ve simplified it to be a simple button, and we also added a search feature in order to find more people than the eight initially displayed.

Phase 4 - Evaluation

For the final phase of our project, it was time to evaluate the work we had done by preparing an experiment​ and running trials to ensure our app was as intuitive as possible. As a group, we came up with the plan to instruct participants through a typical workflow using our app, and then follow the same instructions in a different app. By doing this, we wanted to see if our app was intuitive enough for users to complete tasks easily without help or information. By contrasting it with another app, participants could tell us what they liked and disliked in comparison to one another.

1 - Procedure of the Experiment

This is the sequence of actions we decided on for participants to complete.

  1. Open the application

  2. Open trip planner

  3. Add the starting location as Circular Quay Station

  4. Add the destination as Redfern Station

  5. Search for the route options

  6. Swap the starting point and destination

  7. Change the start time to 5:35 PM

  8. Change the destination to home

  9. Change the starting location to Circular Quay Station

  10. Add first route option to starred routes/save a route

  11. See the route information for the first metro/train/bus on the plan

  12. Go back to the route search results

  13. Select the second route option

Screen Shot 2020-03-28 at 3.08.18 AM.png

2 - Participant Consent Forms

Before running the actual experiment, it was important for participants to consent to the experiment, and to inform them on what it involves for them. We drafted up a Participant Consent Form (PCF) and Participant Information Statement (PIS) to provide more clarity and information.  As part of an excerpt, here is how we described the experiment to participants:

"You will be asked to complete a series of tasks in order to plan a route on an app. The study will take place on campus at Fisher Library, and you will meet with one or more members of the student team. You will be provided with a smartphone and/or laptop to use for the duration of the study. The researcher will briefly interview you about your basic information, and habits commuting to and from the university. You will then get one minute to explore and practice using the first application. The researcher will then read you a list of tasks one by one and will time how long it takes for you to complete each task. They will also take note of any difficulties you may encounter while completing a task. This procedure will then repeat with the second application."

3 - Questionnaires

Once we found suitable participants, we wanted to collect some basic information about them and their habits. To start out, we had participants fill out a simple demographic questionnaire that would remain anonymous and only refer to the participant as their assigned number.

Screen Shot 2020-03-28 at 2.32.35 AM.png

After completing the experiment, in order to gain data, we also had participants fill out a questionnaire regarding their experience with the app. 

Screen Shot 2020-03-28 at 2.32.42 AM.png

4 - Prepare the Trials

With all of our materials set, it was time to have a trial run ​to work out any problems with the experiment before testing our participants.  We created an entire procedure description, including the actions and cues for the interviewer, so that each trial would be the same.

Screen Shot 2020-03-28 at 2.40.56 AM.png
Screen Shot 2020-03-28 at 2.41.04 AM.png

Next, to thoroughly test our procedure, we ran a pilot experiment with one of our group members. Based on the results of that, we found some difficulties with our initial plan. We had realized that we were missing some steps like screening the participant, explaining the PIS, and giving contact information. Additionally, the app we were testing against allowed functionality of auto-generating addresses, while our prototype did not.

To fix this, we added steps in to cover all screening criteria. We also slightly changed the procedure so future participants would choose from preset locations, to avoid reliance on the address auto-generation.

5 - Conduct the Experiment

Finally, it was time to conduct the experiment! We selected 4 typical users and conducted the experiment according to our procedure description. Here are some examples of the questionnaires we received, regarding experiences with the apps.

Screen Shot 2020-03-28 at 2.49.43 AM.png
Screen Shot 2020-03-28 at 2.49.53 AM.png

Our next step was to analyze the data we had collected, and summarize our results and findings. As we saw, through field study, we could have a closer look on how users actually use and interact with our app, and retrieve feedback from users about what and how they feel our app can be improved.

Screen Shot 2020-03-28 at 2.47.25 AM.png

A Summary of Our Findings

  • Participants had an average age of 21

  • Opal Travel had an average longer time to complete the task compared to Opal Travel than (approx. 4 seconds)

    • This could be a result of Opal Travel having many levels to the app, which users had to go through to complete a task. As well as not storing recent or frequent searches in memory, and having user’s type out addresses until the address auto-generation could predict the location.

  • UonTime had on average more errors than Opal Travel (approx. 0.6 more errors)

    • Common errors were user’s missing the save journey feature for UonTime as it is hidden within the drop-down options button. Another common error was user’s misentering their specified times, this could be attributed to having to use Android’s TimePicker Dialog on a computer interface instead of a mobile interface.

  • Trialist found Balsamics cursor very cumbersome, attributing to a higher level of frustration in the NASA TLX mark.

  • Opal Travel’s inability to store recent search locations meant that users had to manually type out addresses, attributing to a higher level of physical demand in the NASA TLX mark.

6 - Design Decisions

After running our experiment, we made many changes to our prototype.

Change: Replaced 'go home' with 'home' and 'go to class' with 'class'

Reason: When selecting a start location, seeing “go home” confused users. They believed they were picking a destination, not start location. Home and Class both work for start location and destination

Change: Added “frequent locations” when searching locations

Reason: Users liked the recent searches feature, as a way to save search time. We believed adding frequent locations would add similar convenience. 

Change: Separated detailed route view, and starting a journey 

Reason: Users were confused regarding whether they were viewing route details, or commencing a journey. We separated the states into 2 screens, with a clear “Go” button visible when viewing route details 

Change: Remove share journey button from the map 

Reason: One user made an error by accidentally clicking to share journey. We made the share journey feature available when viewing detailed route instead of overlaying on the map when on a journey, to prevent similar errors in the future 

Change: Remove arrows between transport icons in recent/saved journeys

Reason: Removed for consistency, as we did not use similar arrows when searching routes

Change: Added back button to the sidebar

Reason: One user spent time trying to recover from the error of accidentally opening the sidebar. Added back button to make error recovery easier 

Reflection

During this semester-long process, I learned a lot about the process of collecting research and then making informed decisions based on those results. I think that getting to experience this all firsthand was a very informative and rich experience, and I'm glad that I had the chance to do it, especially in another country. I learned a lot about what is possible through User Research, and I definitely have grown an interest in it, as I didn't know much about it before. It was gratifying to see the lifecycle of a product from beginning to end, and it definitely excites me about future projects!

Screen Shot 2020-03-28 at 3.07.52 AM.png
Screen Shot 2020-03-28 at 3.08.07 AM.png
Screen Shot 2020-03-28 at 3.08.00 AM.png

Thanks for reading!