JOURNAL TRANSCRIPT
Comparing Cohorts of Event Sequences A VISUAL ANALYTICS APPROACH presented by Sana Malik with Fan Du, Catherine Plaisant, and Ben Shneiderman May 26, 2016 — HCIL 33rd Annual Symposium, College Park
often, analysts compare cohorts within datasets
s p u o r g any , s r e s u of , s t n e i t s pa d r o c e or r
often, analysts compare cohorts within datasets
?
FREQUENT PAT TERNS
ABSENCE OF EVENTS
DURATION
Data Collection
Cohort Selection
Statistics
Data Collection
Cohort Selection
Visual Analytics
Statistics
Data Collection
Cohort Selection
Visual Analytics
Statistics
EVENTFLOW
Monroe et al. Temporal event sequence simplification. IEEE Transactions on Visualization and Computer Graphics (TVCG 2013).
EVENTFLOW
Monroe et al. Temporal event sequence simplification. IEEE Transactions on Visualization and Computer Graphics (TVCG 2013).
EVENTFLOW
? Monroe et al. Temporal event sequence simplification. IEEE Transactions on Visualization and Computer Graphics (TVCG 2013).
Data Collection
Cohort Selection
Visual Analytics
Statistics
Aspirin -> Emergency -> Normal Floor Bed -> ICU -> Exit 38.37, 0.0, 4.11e-123 Emergency -> ICU -> Exit 24.61, 0.0, 2.11e-73 Emergency -> Normal Floor Bed -> Exit -> ICU 5.26, 0.0, 4.12e-15 Emergency -> Normal Floor Bed -> ICU -> Exit 5.26, 0.0, 4.12e-15 Aspirin -> Emergency -> ICU -> Normal Floor Bed -> Exit 7.89, 3.22, 1.20e-06 Aspirin -> Emergency -> ICU -> Intermediate Care -> Exit 4.15, 0.0, 4.22e-12 Emergency -> Normal Floor Bed -> ICU -> Intermediate Care -> Exit 2.97, 0.0, 7.02e-09 Aspirin -> Emergency -> ICU -> Exit 2.80, 0.0, 2.02e-08 Emergency -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 1.44, 0.0, 9.83e-05 Emergency -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 1.01, 0.0, 0.00 Aspirin -> Emergency -> Normal Floor Bed -> ICU -> Intermediate Care -> Exit 0.76, 0.0, 0.00 Aspirin -> Emergency -> Normal Floor Bed -> Exit 0.0, 1.61, 3.37e-05 Aspirin -> Emergency -> ICU -> Normal Floor Bed -> ICU -> Exit 0.0, 2.37, 2.84e-07 Aspirin -> Emergency -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 3.82, 6.45, 0.00 Emergency -> ICU -> Normal Floor Bed -> ICU -> Exit 0.0, 4.07, 7.17e-12 Emergency -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 0.0, 4.24, 2.48e-12 Emergency -> Exit 0.0, 11.88, 9.00e-34 Emergency -> ICU -> Normal Floor Bed -> Exit 0.0, 16.12, 2.18e-46 Aspirin -> Emergency -> Exit 0.0, 47.79, 2.49e-162 Aspirin -> Emergency -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 0.0, 0.59, 0.02 Aspirin -> Emergency -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit Aspirin -> Emergency -> ICU -> Exit 2.80, 0.0, 2.02e-08 Emergency -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 1.44, 0.0, 9.83e-05
5.26, 0.0, 4.12e-15 Emergency -> Normal Floor Bed -> ICU -> Exit 5.26, 0.0, 4.12e-15 Aspirin -> Emergency -> ICU -> Normal Floor Bed -> Exit 7.89, 3.22, 1.20e-06 Aspirin -> Emergency -> ICU -> Intermediate Care -> Exit 4.15, 0.0, 4.22e-12 Emergency -> Normal Floor Bed -> ICU -> Intermediate Care -> Exit 2.97, 0.0, 7.02e-09 Aspirin -> Emergency -> ICU -> Exit 2.80, 0.0, 2.02e-08 Emergency -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 1.44, 0.0, 9.83e-05 Emergency -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 1.01, 0.0, 0.00 Aspirin -> Emergency -> Normal Floor Bed -> ICU -> Intermediate Care -> Exit 0.76, 0.0, 0.00 Aspirin -> Emergency -> Normal Floor Bed -> Exit 0.0, 1.61, 3.37e-05 Aspirin -> Emergency -> ICU -> Normal Floor Bed -> ICU -> Exit 0.0, 2.37, 2.84e-07 Aspirin -> Emergency -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 3.82, 6.45, 0.00 Emergency -> ICU -> Normal Floor Bed -> ICU -> Exit 0.0, 4.07, 7.17e-12 Emergency -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 0.0, 4.24, 2.48e-12 Emergency -> Exit 0.0, 11.88, 9.00e-34 Emergency -> ICU -> Normal Floor Bed -> Exit 0.0, 16.12, 2.18e-46 Aspirin -> Emergency -> Exit 0.0, 47.79, 2.49e-162 Aspirin -> Emergency -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 0.0, 0.59, 0.02 Aspirin -> Emergency -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit Aspirin -> Emergency -> ICU -> Exit 2.80, 0.0, 2.02e-08 Emergency -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 1.44, 0.0, 9.83e-05 Emergency -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> ICU -> Normal Floor Bed -> Exit 1.01, 0.0, 0.00 Aspirin -> Emergency -> Normal Floor Bed -> ICU -> Intermediate Care -> Exit 0.76, 0.0, 0.00 Aspirin -> Emergency -> Normal Floor Bed -> Exit
SAS
SAS Business Analytics Software. Vers. 9.4. SAS Institute, 2014. Computer software.
STATA
StataCorp. 2015. Stata Statistical Software: Release 14. College Station, TX: StataCorp LP.
Data Collection
Cohort Selection
Visual Analytics
Statistics
Visual Analytics
Data Collection
Cohort Selection
Statistics
HIGH-VOLUME
Hypothesis Testing
HIGH-VOLUME
Hypothesis Testing
Systematic Exploration OF RESULTS
HIGH-VOLUME
Hypothesis Testing
Systematic Exploration OF RESULTS REAL-WORLD
Case Study
HIGH-VOLUME
Hypothesis Testing
Emergency Room
Normal Floor Bed
ICU
Discharged
start and end of record
non-consecutive (contains other events between)
1 SHORT SEQUENCE 14 UNIQUE PATTERNS non-consecutive (contains other events between)
RECORD COVERAGE
Does this sequence occur in more records in one cohort than the other?
DURATION
On average, does this sequence take longer in one cohort than the other?
FREQUENCY
On average, does this sequence occur more frequently per record in one cohort than the other?
RECORD COVERAGE Does this sequence occur in more records
14 DURATION On average, this sequence take X 3does METRICS longer in one cohort than the other? 42does HYPOTHESES On average, this sequence occur FREQUENCY in one cohort than the other? UNIQUE PATTERNS
more frequently per user in one cohort than the other?
HIGH-VOLUME
Hypothesis Testing
Systematic Exploration OF RESULTS REAL-WORLD
Case Study
Systematic Exploration OF RESULTS
Demo
HIGH-VOLUME
Hypothesis Testing
Systematic Exploration OF RESULTS REAL-WORLD
Case Study
REAL-WORLD
Case Study
MULTI-DIMENSIONAL IN-DEPTH LONG-TERM CASE STUDIES (MILCS) Entry Interview & Training (1 session) Partners Use Tool Partners Provide Feedback
(3 months)
Researchers Refine Tool Exit Interview (1 session) For Researchers
Demonstrate utility, refine tool
For Partners
Papers, insights, discoveries
B. Shneiderman and C. Plaisant. Strategies for evaluating information visualization tools: Multidimensional in-depth long-term case studies. In BELIV ’06: Proceedings of the 2006 AVI workshop on BEyond time and errors, pages 1–7. ACM, 2006.
CASE STUDY PARTNERS
CASE STUDY PARTNERS
PARTICIPANTS & DATASET
Three analysts at Adobe • •
One experienced user Two novice users
Users’ events on a product website • • •
viewing the display ads signing up for promotions or free trials purchasing products
Dataset Size • •
6,999 users 124 events types / 81,563 events
GOAL
Compare users who purchased a product with using trials versus without using trials to understand ad-related behaviors
SYSTEM USE
SYSTEM USE
SYSTEM USE
“Event filtering was the most helpful to focus the analysis”
SYSTEM USE
“Reduced metric calculation time provided a much better user experience for data analysis”
RESULTS: FOR PARTNERS
Users who had a trial viewed display ads more than the other group & contained more retargeting events.
Analysts hypothesized trial users were “explorers” and non-trial users were “experienced users” based on event pattern differences
Future Work • Extensions to other data types (e.g., networks)
• Interval events • Cohort selection
HIGH-VOLUME
Hypothesis Testing Systematic Exploration OF RESULTS REAL-WORLD
Case Study
Comparing Cohorts of Event Sequences
A VISUAL ANALYTICS APPROACH
Y R T
C
C O
! O
presented by Sana Malik
email
[email protected] www http://hcil.umd.edu/coco Support from Adobe, Oracle, and the University of Maryland’s Center for Health-related Informatics & Bioimaging (CHIB).