Skip to main content

Usability Study

The Usability Study is an opportunity to collect in-depth quantitative data about the workplace and its use today. DORIS looks for overarching patterns about when, where, and how people use their current workplace. DORIS collects data on the general activities within each space, the number of people, and the items people interact with. There are nine rounds per day, starting at 8:30, and the last tour begins at 4:30. The touring of spaces is done hourly, and that data enables us to show when/how much areas are used in general and during given dayparts. 

DORIS uses Vantage Space to collect the data. The program allows us to indicate which spaces will be studied and assign RAs to workspaces, which they can access on the Vantage Space app. 

  • A typical usability study occurs over 10 business days or two weeks.

Not every research project includes a usability study; assessments do not include a usability study.

What to Expect in Usability Study
  • DORIS researchers walk a set path through the workplace once every hour
  • The usability study takes place during the workplace’s standard business hours over 2 weeks unless special days or time requirements have already been established with the leadership team
  • Data is recorded at spaces including workstations, offices, meeting rooms, and common areas
  • Between rounds of walking the path, researchers sit at a home base location

Start

Review client-provided floor plans

Client floor plans are located in PDF format in Box under the project's folder > 0 - Project Info > Client Floor Plans. 

Accessing Client Floor Plans in Box

Most floor plans can be used as-is, however, very complicated floor plans need to be traced and simplified using InDesign. 

Plan

Prep Vantage Space

We use Vantage Space to collect usability study data. By uploading floor plans, we are able to assign departments, space types, and capacities to each studied workspace. RAs are assigned spaces so they know which spaces they are studying and simply have to fill in the info fields on each tour.  

How to Use Vantage Space

Sign into doris@dorisresearch.com account (login info is in LastPass)
*Note LastPass automatically populates the account info for the ra2 account, be sure to click the drop down and choose the doris@ account

Vantage Space login page

  1. Create Project
    a. Due to Vantage Space's setup, you will need to create a new Organization for each project. 
    b. Name it the project's organization and enter DORIS's information into the address fields. 
    c. Click on Study Options on the left bar menu and edit the default Activities, Technologies, and Workspace Types +      save.
    d. Click Create New Study. In the Study Title Field, enter the Organization's name. Enter a number for the Study Reference, set the Time Zone, Study Dates, Meghan Tooman as the Study Manager and click Save. 

  2. Upload Floor plans
    a. In the Draft Studies page, select Buildings and Floors. 
    b. Click Add Building to input building name + address. Then click the map icon to upload floor plans.

How to Create Organizations, edit Study Options, and upload floor plans

  3. Creating + editing workspace points 
      a. Click on the space in the floor plan you want to make an area of study (e.g., an office)
      b. Click add new workspace and enter Workspace Type, Capacity, and Department. Click Save. 
      c. To edit a singular workspace, click on it, select "Edit workspace" and make necessary changes.

After the walkthrough has been completed, add/edit workspaces to reflect any changes. Then assign RAs to workspaces. The Operations Manager will provide you the number of RAs the project is using (generally between one and seven).   

   4. Assigning spaces to Research Assistants
       a. Go to the Manage Team menu on the left side menu. Click Invite New Team Member(s) on the top right of the page.
       b. enter in the RA emails separated by commas (ra1@dorisresearch.com, ra2@dorisresearch.com, etc). Select the                           External - Data Collection & Study Specific Access.
       c. Click Invite New Team Member(s) and inform the Operations Manager they need to accept the email invites for each                 RA. 
       d. Click the "Select Multiple" button and click/drag over the desired points. You can also directly click on each individual                 point. Click "Reassign Spaces" repeat as necessary. Tell the Operations manager when completed. 

Creating, Editing, and Assigning Workspaces in Vantage Space

Adding Research Associates to Studies

Once the study and dates are fully confirmed, purchase the Vantage Space study by selecting Payment, the study size (based on number of workspaces. Select pay by card and ensure the receipt is sent to Pat (pat@dorisresearch.com). Once purchased, post it in the project's Slack channel and tag the Operations Manager. 

Conduct usability walkthrough on-site 

The walkthrough is conducted to ensure the existing space matches the floor plans we received. Download the floor plans from Vantage Space with the IDs labeled and print them so the person conducting the walkthrough can note and report back any necessary edits.  

  • Update Vantage Space to reflect changes.
  • Assign points to RAs.
    • Before the points can be assigned to RAs in Vantage Space, the Operations Manager must first determine the required number of RAs based on the total data points, avoiding assigning over 200 points to a single RA.
      • Once the necessary number of RAs is established, the Operations Manager will proceed to hire and allocate RAs to data points using Google Sheets.
      • Maintain a comprehensive file containing each RA's contact details, assigned data sites, addresses, and any pertinent information.
      • When assigning RAs, consider the number of data points, the presence of multiple sites, and whether these sites are walkable. Prioritize assigning RAs to sites that don't require driving between locations for efficiency.

Gate: Review and finalize Vantage Space setup

Prep People

Hire contract RAs as needed

At DORIS, our standard practice is to recruit contract research assistants (RAs) to conduct usability studies. Ideally, we prioritize sourcing RAs from our contractor pool. However, in cases where this isn't feasible, we engage local temp agencies.

When hiring temporary staff, it's paramount to select individuals committed to daily attendance, physically capable of the job, and capable of upholding DORIS's professional standards.

Here are some best practices:

  • Ensure a surplus of 2-3 additional temps during the training phase to account for potential dropouts.
  • Conduct interviews with each temp candidate for suitability. We need to have people that can be at every scheduled tour, will be unobtrusive but able to answer any questions asked on site, and will not be overwhelmed by using mobile apps.
  • Clearly communicate to the agency our need for high-quality candidates. Our studies have strict time limitations, and any absence impacts our data collection significantly. The agency must understand that missed tours due to absences cannot be made up the following day.
  • In case of data collection issues at a location leading to a study extension, consider utilizing staff from HQ to maintain reliable data collection.
Create materials for RA orientation 

For each project, a usability orientation presentation and packet are created. These materials give RAs an overview of who DORIS is, the organization being studied, what is expected of them onsite, what a usability study is, and how to use Vantage Space.

Creating Training Materials for RAs

The templates for both the Presentation and Packet are found in Box under the Data Collection > Usability subfolder 

Opening the usability training files 

Usability Orientation Presentation

  1. Open the Orientation Presentation on InDesign and input the organization name on the first slide and the name and logo on slide 7
  2. On Slide 30, insert correct start date for the study  
  3. On Slide 31, put contractor's names, their assigned RA number, and the location they are studying.
  4. Save changes back to Box.

RA Orientation Packet 

  1. Open the Orientation Packet on Acrobat and update the Introduction section (on Page 3) with a couple of sentences describing the organization / goal of the project.  
  2. Save Changes back to Box. 

 Gate: Get content approved from the Project Lead and Operations Manager

Conduct RA orientation

Orientation consists of two parts:

  1. A virtual Zoom meeting covering DORIS history, job expectations, and guidance on using the Vantage Space app.
  2. On-site training involving familiarization with the space and data collection techniques.

Process:

  1. The Operations Manager schedules and invites RAs to orientation meetings.
  2. Research Associate prepares training materials. 
  3. Either the Operations Manager or Research Associate conducts the virtual meeting, following the presentation outline and allowing introductions.
  4. A follow-up email with the electronic Training Manual and Vantage Space login information is sent.
  5. The Operations Manager will collect the physical training manual and DORIS name tags, and print the floor plans for the onsite training.
  6. During the walk-through, distribute the printed Training Manual, ensure Vantage Space access, conduct a trial study, and clarify roles for all RAs.

Execute + Monitor

Conduct Usability Study

Start of Day Checklist:

  • Do you have your badge, name tag, DORIS shirt, etc?
  • Is Vantage Space working and loading properly?
  • You know who to contact in case of any issues...
  • Arrive to the location 15 minutes early

Walking rounds Checklist:

End of Day Checklist:

  • Save all responses in Vantage Space
  • Backup all data
  • Communicate any weirdness to DORIS lead

Check Vantage Space + RAs daily during the study

  • The operations manager will go to Vantage Space to check the previous day's tours to ensure they are complete.
  • Daily check-ins: The Operations Manager will check in twice daily via text at 8:30 AM and 5:00 PM to make sure that the RAs are present and if they have any issues with the Vantage Space app.
    • It's common to get a text from an RA saying the app is not saving their data. The internet is likely slow, or their phones don't have enough storage for the Vantage Space mobile app. It usually helps if they shut off the phone and restart it to clear the phone cache to create space.
  • The operations manager will need to keep track of any missed tours in case we need to schedule additional tours at the end of the study.

Finish

Thank RAs and document any information collected outside of Vantage Space

No comments to display

Back to top