Usability Study
The Usability Study is an opportunity to collect in-depth quantitative data about the workplace and its use today. DORIS looks for overarching patterns about when, where, and how people use their current workplace. DORIS collects data on the general activities within each space, the number of people, and the items people interact with. There are nine rounds per day, starting at 8:30, and the last tour begins at 4:30. The touring of spaces is done hourly, and that data enables us to show when/how much areas are used in general and during given dayparts.
DORIS uses Vantage Space to collect the data. The program allows us to indicate which spaces will be studied and assign RAs to workspaces, which they can access on the Vantage Space app.
- DORIS generally hires contract research assistants to conduct usability studies.
- Ideally, we would use RAs from our contractor pool, but we pull people from local temp agencies when that is impossible.
- It's crucial to hire temps who are committed to coming in every day, can physically manage the job, and will represent DORIS appropriately.
- Best Practices:
- Hire 2-3 more temps than you need for the training process so that if someone drops out, you will have a backup.
- Interview each temp candidate.
- Make it clear to the agency that we need high-quality candidates as our studies are time-limited and every hour counts. The agency must understand that when people don't show up, we must extend the study. We are not able to make it up the next day.
- A typical usability study occurs over 10 business days or two weeks.
Not every research project includes a usability study; assessments do not include a usability study.
What to Expect in Usability Study
- DORIS researchers walk a set path through the workplace once every hour
- The usability study takes place during the workplace’s standard business hours over 2 weeks unless special days or time requirements have already been established with the leadership team
- Data is recorded at spaces including workstations, offices, meeting rooms, and common areas
- Between rounds of walking the path, researchers sit at a home base location
Start
Review client-provided floor plans
Client floor plans are located in PDF format in Box under the project's folder > 0 - Project Info > Client Floor Plans.
Accessing Client Floor Plans in Box
Most floor plans can be used as-is, however, very complicated floor plans need to be traced and simplified using InDesign.
Plan
Prep Vantage Space
We use Vantage Space to collect usability study data. By uploading floor plans, we are able to assign departments, space types, and capacities to each studied workspace. RAs are assigned spaces so they know which spaces they are studying and simply have to fill in the info fields on each tour.
How to Use Vantage Space
Sign into doris@dorisresearch.com account (login info is in LastPass)
*Note LastPass automatically populates the account info for the ra2 account, be sure to click the drop down and choose the doris@ account
- Create Project
a. Due to Vantage Space's setup, you will need to create a new Organization for each project.
b. Name it the project's organization and enter DORIS's information into the address fields.
c. Click on Study Options on the left bar menu and edit the default Activities, Technologies, and Workspace Types + save.
d. Click Create New Study. In the Study Title Field, enter the Organization's name. Enter a number for the Study Reference, set the Time Zone, Study Dates, Meghan Tooman as the Study Manager and click Save. - Upload Floor plans
a. In the Draft Studies page, select Buildings and Floors.
b. Click Add Building to input building name + address. Then click the map icon to upload floor plans.
How to Create Organizations, edit Study Options, and upload floor plans
3. Creating + editing workspace points
a. Click on the space in the floor plan you want to make an area of study (e.g., an office)
b. Click add new workspace and enter Workspace Type, Capacity, and Department. Click Save.
c. To edit a singular workspace, click on it, select "Edit workspace" and make necessary changes.
After the walkthrough has been completed, add/edit workspaces to reflect any changes. Then assign RAs to workspaces. The Operations Manager will provide you the number of RAs the project is using (generally between one and seven).
4. Assigning spaces to Research Assistants
a. Go to the Manage Team menu on the left side menu. Click Invite New Team Member(s) on the top right of the page.
b. enter in the RA emails separated by commas (ra1@dorisresearch.com, ra2@dorisresearch.com, etc). Select the External - Data Collection & Study Specific Access.
c. Click Invite New Team Member(s) and inform the Operations Manager they need to accept the email invites for each RA.
d. Click the "Select Multiple" button and click/drag over the desired points. You can also directly click on each individual point. Click "Reassign Spaces" repeat as necessary. Tell the Operations manager when completed.
Creating, Editing, and Assigning Workspaces in Vantage Space
Adding Research Associates to Studies
Once the study and dates are fully confirmed, purchase the Vantage Space study by selecting Payment, the study size (based on number of workspaces. Select pay by card and ensure the receipt is sent to Pat (pat@dorisresearch.com). Once purchased, post it in the project's Slack channel and tag the Operations Manager.
Conduct usability walkthrough on-site
The walkthrough is conducted to ensure the existing space matches the floor plans we received. Download the floor plans from Vantage Space with the IDs labeled and print them so the person conducting the walkthrough can note and report back any necessary edits.
- Update Vantage Space to reflect changes
- Assign points to RAs
Gate: Review and finalize Vantage Space setup
Prep People
Hire contract RAs as needed
Using either our existing contractor pool or through a Staffing Agency.
Create materials for RA orientation
For each project, a usability orientation presentation and packet are created. These materials give RAs an overview of who DORIS is, the organization being studied, what is expected of them onsite, what a usability study is, and how to use Vantage Space.
Creating Training Materials for RAs
The templates for both the Presentation and Packet are found in Box under the Data Collection > Usability subfolder
Opening the usability training files
Usability Orientation Presentation
- Open the Orientation Presentation on InDesign and input the organization name on the first slide and the name and logo on slide 7
- On Slide 30, insert correct start date for the study
- On Slide 31, put contractor's names, their assigned RA number, and the location they are studying.
- Save changes back to Box.
RA Orientation Packet
- Open the Orientation Packet on Acrobat and update the Introduction section (on Page 3) with a couple of sentences describing the organization / goal of the project.
- Save Changes back to Box.
Gate: Get content approved from the Project Lead and Operations Manager
Conduct RA orientation
Process:
- The Operations Manager creates orientation meetings and invites the RAs
- Research Associate prepares training materials.
- Either the Operations Manager or the Research Associate conducts the meeting by following the presentation outline and give people a chance to introduce themselves.
- Send a follow-up email containing the Training Manual and the necessary Vantage Space login information.
- During the walk-through, hand out the printed Training Manual and ensure everyone can access their Vantage Space account and can go through the trial study and understand their role.
Execute + Monitor
Conduct Usability Study
Start of Day Checklist:
- Do you have your badge, name tag, DORIS shirt, etc?
- Is Vantage Space working and loading properly?
- You know who to contact in case of any issues...
- Arrive to the location 15 minutes early
Walking rounds Checklist:
End of Day Checklist:
- Save all responses in Vantage Space
- Backup all data
- Communicate any weirdness to DORIS lead
Check Vantage Space + RAs daily during the study
- The operations manager will go to Vantage Space to check the previous day's tours to ensure they are complete.
- Daily check-ins: The Operations Manager will check in twice daily via text at 8:30 AM and 5:00 PM to make sure that the RAs are present and if they have any issues with the Vantage Space app.
- It's common to get a text from an RA saying the app is not saving their data. The internet is likely slow, or their phones don't have enough storage for the Vantage Space mobile app. It usually helps if they shut off the phone and restart it to clear the phone cache to create space.
- The operations manager will need to keep track of any missed tours in case we need to schedule additional tours at the end of the study.