Problem
Research
Wireframing
Testing
Solution
Results
Takeaways

Medical Question Bank

Lead Product Designer - Jun 2018 - Aug 2018

Role

Sole designer leading end-to-end process

Team

1 PM, 3 Engineers, 2-4 Doctors

Results

120% increase in avg session duration

Impact

More incentive for users to engage with platform

I led the design of SketchyMedical's question bank to enable students to test their recall speed and knowledge after watching videos.

  • Collaborated with Co-Founders, Lead Engineer to outline feature requirements
  • Participated in QA to review new feature, creating test cases and reporting bugs
  • Facilitated user testing, coordinated with Marketing to recruit participants

Problem

Medical students are averaging 8-10 minutes on our platform

Although medical students are opting out of in-person lecture for online learning, our average session duration for the previous quarters was less than 10 minutes. As a subscription service, SketchyMedical needed to increase user engagement as a subset of user retention.

Hypothesis

Account personalization encourages individual membership

Creating a tool that 1) enables students to test their information recall and 2) is unique to SketchyMedical as a platform will drive user engagement and encourage membership

It is possible that we can get users to use the platform more if we provide account personalization and more content for testing knowledge. So far, SketchyMedical has proven to be a great resource for learning, but we wanted to take it to the next step, where students start to encode and recall stored information over time.

Research

1. User behavior and personas

I interviewed 5 medical students in their first two years of school for 30 minutes each. Most of them studied independently, even if they were with a group of friends. Usually after studying material, they will take practice questions to see if they've mastered concepts.

2. Competitive audit

Faculty usually suggest or require medical students to subscribe to an online question bank to begin preparing for the USMLE Step 1. I audited the most recommended question banks to understand what features were common in preparing a student for the board exam, including UWorld, Amboss, and Kaplan. Commonalities included:

  • Option to time their test to get in the habit of answering quickly
  • Option for tutor mode, which will show answers after answering a question
  • Select topics and view available number of questions per topic
  • Testing experience similar to that of the USMLE Step 1 (which is online)
  • Record of all tests ever made, with scores and details for each entry

3. Creating a roadmap for versions

Based on the audit, I created 5 major sections that represented the most important tools for students: Create, Take, Review, Performance, and Notes. Create, Take, and Review are highlighted because they represent the core functionalities of a question bank.

Wireframing

1. Low fidelity screens

Create test

Initial wireframes were first designed as "Create Test" before the doctors decided to switch to "Quiz". Customization options included selection of specific units and sections, or videos. A user could also filter the type of questions he or she wanted the quiz to have.

Take test

A user can view details about their test while answering questions, including customizations they chose and the ID of the quiz. They're able to take notes, use a calculator, and navigate to any other question in their test.

Review test

When a test is completed and or submitted for grading, a user receives an overview of how they performed with individual breakdowns of each question. This wireframe shows question origin, status, performance compared to average population, and time spent to answer each question.

2. High fidelity screens

I designed high fidelity screens in Sketch, using the dark interface theme of the redesigned website. Some of the components followed styling from UIKit.

Testing

Identifying UI changes to improve user experience

Through marketing, I recruited participants for in-house user testing. We recorded task completion and user interactions for 5 participants, 45 minutes to an hour each. Participants already relied on our videos for their study routine (regardless of how they attained access) and said that they'd be more likely to use the platform because of the question bank.

"I think it's a great way to learn the sketches better. Because currently what I do is watch the Sketchy, then do the Anki [flashcards] for it to make sure that I know everything in the sketch. Now I have another way to review material."

— Lucas F. (Stritch School of Medicine, Class of 2022)

After reviewing transcripts and screen recordings from user testing, I identified UI changes that we could address with the next iteration. These included updates to how a user could navigate through creating a quiz and taking a quiz.

Solution

Our solution gave students the opportunity to test themselves through another effective method of recall. The final solution of the question bank allowed users to:

1. Customize quiz content

Create a quiz

A user can toggle Tutor, which contains SketchyMode, courses, and question types in the filter bar. Once the user has selected a course (micro, pharm, or path), the Units content will update with the appropriate videos and available questions.

View an updated list of topics in "My Quiz"

As a user selects topics that they'd like to include in his or her custom quiz, the "My Quiz" module will update immediately. Videos are categorized by the course type and unit within that course. At the bottom, a user can view the total number of questions available and customizations.

2. Eliminate options and view rationale

Taking a quiz in "Tutor Mode"

Tutor mode allows users to submit a selected option for grading or omit the question. Users are also able to view SketchyMode on the right hand side, which is a panel that shows related artwork in the SketchyMedical universe.

Connecting concepts in SketchyMedical

Once a user submits an answer for grading, rationales for each option are shown below, including reasons why choices are correct or incorrect. Underlined terms in the rationale are connected to symbols in artwork across available courses in SketchyMedical.

3. View history and results

History of all quizzes

"Quiz History" records all quizzes that have ever been created. A user can filter quizzes by status (incomplete or complete) or look up a specific Quiz ID. Completed quizzes have scores, with the option to review questions, view results, or retake the same quiz.

Review quiz results

If a quiz is completed, a user has the option of viewing its results. This record shows a detailed breakdown of each quiz question, including grade, ID, origin, time spent on the question, and whether it was marked.

Results

Increased average session duration to 10-22 minutes

A few months after the release of SketchyMedical's question bank, we saw an increase in average session duration from 10-22 minutes. We also received and addressed feedback concerning latency and interaction with subsequent updates to the platform, contributing to an increase in the number of quizzes generated through the question bank.

120%

Increase in average session duration

125%

Increase in quizzes created after optimization

Takeaways

Ask questions first

Ambiguity can be navigated by asking meaningful questions that introduce different angles. This showed my team how a design perspective could help us move in a more clear direction.

Bring everyone into the same room

I encouraged teams to step out of their silos so that all stakeholders could actively participate in the conversation. They found that this improved our communication and understanding.

Get feedback, no matter what

Our company had little to no experience in collecting and analyzing user feedback. I voiced my concerns as a designer, convincing senior leadership that we needed to keep our ears open.