Portfolio: $100k

Improve product perception while cutting costs by...

$5,000

$20,000

$50,000

$100,000 👈

client

Indiana University

role

Project lead

Let’s get in touch!

Summary

Overview

Indiana University implemented a vended, multi-factor authentication system that demanded a significant change in user habits. On top of causing many people daily frustration and earning a bad reputation, it also accumulated a quarter of a million dollars in annual telephony fees.

$250,000

/year

in fees

At the request of the request of the manager of the identity team and the security committee, I was given two broad goals:

Improve product perception
Reduce annual telephony costs by $100,000 before 2021

In the summer of 2016, the Indiana University community came under the attack of a well-crafted phishing campaign. Over a thousand people were seriously affected, some of whom had their paychecks rerouted. Fast-forward nine months, and all students, faculty, and staff were required to use multi-factor authentication to login to any IU service. Two-Step Login, the user-friendly name given to the vended product by Duo Security, had stepped in to save the day, but at a cost. While most login methods were free, including a mobile app, every use of a call or text (SMS) to Two-Step had an associated fee. Within a year and half, the annual cost to support these telephony methods was up to a quarter of a million dollars.

But the financial burden wasn't the only cost. Two-Step was rolled out quickly and required a significant change in user habits. While the Two-Step requirement was applied initially to personal and financial systems as a soft launch to acclimate users, it was a struggle for many across the university when the time came to needing a device for every single login. When forgetting your Two-Step device meant a student couldn't get to their in-class quiz or faculty weren't able to pull up their slides to teach in the first place, it isn't an exaggeration to say that there were instances of outrage. It wasn't long before the security savior had a PR problem.

Discover

Uncover user habits and attitudes through guerrilla interviews, data analysis, a user survey, and peer review.

Interviews

Data analysis

user Survey

Peer review

Ideate

Propose solutions to address user concerns and misconceptions while increasing user convenience.

Prototyping

testing

Refine

Receive client feedback on proposed solutions, iterate on approved strategies, and validate with testing.

client feedback

Iteration

validation

Implement

Redesign the onboarding experience, educate incoming students, send targeted communications, and assess results.

html/css

copy writing

collaborations

assess outcomes

Step 1

Discover

Guerrilla interviews

Purpose

Uncover the challenges and pain points felt by the users while gaining a better understanding of their attitudes toward this service as well as account security.

Methods

What do you think of Two-Step Login?

I posed this question to well over 50 friends, family, coworkers, classmates, and even new acquaintances. Living and working in a small midwestern town with a large university community meant opportunities were everywhere to hear from users from all backgrounds.

Findings

These informal interviews helped me understand that users:

Had concerns the mobile app would track/monitor their device
Assumed their phones didn't have enough storage for the app
Were unaware the mobile app could be used without Wifi/cellular service
Thought the Remember Me feature posed security threats

And, most importantly, it helped me understand the frustrations of users when Two-Step really got in the way of accomplishing their work.

Data analysis

Purpose

After listening to individual users' experiences and opinions, I wanted to capture a high-level view of user behaviors through the millions of monthly service logs.

Methods

Initially, this meant exploring device enrollments, types, and usage as well as login frequency. My analysis quickly expanded into device replacement patterns and feature adoption.

Findings

Telephony methods (calls or texts) were utilized equally by the entire IU community (faculty, staff, affiliates, and students). Despite offering a mobile app,

1 of 5 authentications was completed with a call or text.

One notable feature allowed users to skip Two-Stepping by selecting a "Remember me" checkbox. The time period for this feature had expanded from 12 to 24 hours, and then to 7 days. The switch to 7 days had taken recently, providing an opportunity to review the Remember Me usage rates. I wanted to know if extending the time period from 24 hours to 7 days increased adoption.

130%

increased adoption

when changed from 24 hours to 7 days

User survey

Purpose

Combining feedback from individual users with the broad analysis of general user habits, I created a survey to:

Understand the device ownership and usage of call and text users
Find out if those who replaced a device switched to telephony methods
Uncover awareness of and attitudes toward the Remember Me feature

This method was chosen as it could quickly collect information around device practices and measure concerns around features with a broad set of users. Additionally, it provided an opportunity to get feedback from users about specific features through open-ended questions.

Methods

The online survey was sent to 100 students, staff, faculty, and affiliates in various departments throughout the university and had a 75% completion rate.

Preview survey

Findings

Over 20% used calls or texts as their primary Duo login method; 80% of them owned a smartphone
17% of those surveyed had never used Remember Me; all of them indicated they would use it in the future
45% of those who were aware of Remember Me said they would use it going forward
60% of primary call/text users always use Remember Me, which was above the average (42%)

Peer review

Purpose

While IU was one of the first large universities to significantly increase account security with multi-factor authentication for all users on all logins, other institutions had also adopted the Duo product. To better understand their user habits and security tactics, a peer review of was conducted around three main areas:

Ease of use
Telephony approaches
Remember Me settings

Methods

Collect and compile messaging, settings, and practices at other universities through website research and peer correspondence.

Findings

The review revealed that many institutions were guiding users away from telephony methods but, more importantly, offering longer Remember Me timespans:

days

at Arizona, Carnegie Mellon, and MIT

days

at Stanford

Step 2

Ideate

A. Educate throughout product lifecycle

The Discovery phase revealed that many users:

Were uncomfortable or concerned with the Duo product
Did not want to engage with the Duo as often as they were
Did not understand how the features functioned, or were not aware of them altogther.

Two-factor authentication had become more common place as financial institutions broadened adoption but standard methods and practices had not coalesced. The Discovery phase revealed that many users:

Onboarding

Nudge users to set up the mobile app; begin collecting device registration metrics

UI Design

Data analysis

Pre-engagement

Alert users of Remember Me feature at time of login; deploy A/B testing on messaging

UI design

testing

copy writing

Post-engagement

Target telephony users with information about mobile app benefits and offer a contest to win one of five iPads; track device conversion and telephony usage

copy writing

Data analysis

Address common misconceptions through various social and direct channels

copy writing

B. Extend Remember Me to 30 days

Improve ease of use & convenience
With such a positive response to the first major extension of Remember Me from 24 hours to 7 days, this would be a major win for users. This would relieve much of the pressure to always have a device handy while boosting the product's likability through ease of use and convenience.

Drastically reduce Two-Step costs
Further, it was also clear that allowing users to skip authentications would also reduce the number of calls and texts. After assessing the daily cost differences between the 24-hour Remember setting and the 7-day setting, and then forecasting that for a 30-day setting, the results were astounding.

$101,500

/year

estimated savings

Step 3

Refine

Client feedback

The SafeIT Committee was pleased to move forward with email communications and use of social media channels to educate the community around common misconceptions and valuable features. However, it did not agree to the iPad giveaway. Both the identity team and SafeIT readily accepted the proposed onboarding changes to increase mobile app adoption. Further, IMS was willing to put logging in place to better track which methods users intended to set up based on their selections.

Educate throughout product lifecycle
Approved
- Onboarding
- Pre-engagement
- Post-engagement
Extend Remember Me to 30 days
Not approved
- While filled with intrigue at the estimated cost savings, SafeIT concluded that it was uncomfortable extending Remember Me beyond the current 7-day timespan.

Expand communication outlets

Through my regular collaborations with the Support Center, I was invited to participate in planning for New Student Orientation. This partnership led to Two-Step Login being prominently displayed in a variety of marketing materials.

The key to getting a seat at the planning table was using data to demonstrate that Two-Step was the most challenging support issue at IU.

Graph showing Duo with most support tickets

Educate pre-onboarding audiences
Approved
- Feature in orientation videos for incoming students
- Full-page explainer in the redesigned IT brochure
- Royal card in the playing deck given in every swag bag

Test Remember Me alert

Based on the updated login concept that integrated IU's new design system, over two dozen messages were created to put before users for testing. While personal favorites like "Coach Miller does this one trick" and seasonal quips like "It's like Spring Break for Duo" got laughs, the most effective message was simple and straightforward.

Message selected based on user testing

Tracking Two-Step setup devices

In order to gain insight into which devices users would like to use and then measure how these devices were actually used, I implemented logging on the account creation tool. This would reveal whether smartphones and tablets that were enrolled with the mobile app were later being used with calls or texts. The initial results were surprising as a strong majority of users were enrolling a device that supported Duo Mobile.

of users enrolled app

This led to the a few conclusions, both of which could be true:

Users were very capable of utilizing the app but chose telephony methods
Telephony usage was significantly higher among those who had not enrolled their device during account creation

Combined with the write-in comments from the user survey expressing confusion around the app, it was clear users were struggling with how or why to use the app. Educating users at the time of setup could increase adoption.

Step 4

Implement

Targeted telephony user campaign

Method

By providing data demonstrating telephony usage, the SafeIT Committee grew more comfortable targeting telephony users with email communications. I drafted a couple of different emails for various audiences to be released around the New Year. Each population had 5 or more calls and/or texts in the last 30 days, and the total audience was over 60K people.

New Year's campaign with 45% open rates

Results

The open rates were the highest IT Communications had ever seen—between 42-48%—and click rates of at least 5%. This meant that tens of thousands of people were getting the message. But more importantly, anywhere from 3% to 11% actually stopped using calls and texts.

+25,000

users reached

with direct emails

Orientation collaboration

Method

For the past three years, I've had the privilege of working closely with the IT arm of IU's New Student Orientation. Providing feedback on storyboards for Stranger Tech videos and reviewing Two-Step swag has been a welcome change of pace.

The most rewarding contribution I made was re-configuring the entire onboarding experience to get new students up and running with Two-Step as 2020's orientation went completely virtual due to COVID-19.

Results

Two-Step Login has been at the forefront of all IT-related communications to new students, educating them on the benefits of the mobile app and Remember Me.

Incoming students were coached with best practices or even guided through the process in-person
Expectations were set to prepare users for login practices

9,000

incoming students

reached through partnership

Redesigned Two-Step setup

Method

Before the end of 2018, a resigned Two-Step page was rolled out to better encourage use of the devices that could be paired with Duo Mobile.

Additionally, to help users better understand the mobile app, a post-setup workflow was proposed to improve adoption. While the original prototype could not be found, the goals were threefold:

Prompt smartphone/tablet users to complete a Two-Step push notification immediately after completing the account creation process
Direct them to select Remember me for 7 days
Prompt the user to change their settings to enable auto-push, so this is set as the default option.

Conclusion

Outcomes

Increased satisfaction

Between the start of these initiatives and their completion, customer satisfaction rose at least 8% among the 20,000 users surveyed across IU's two largest campuses. It was clear that these efforts had improved experiences across the the board.

1-star rating decline

Over fiscal year 2020, the number of 1-star ratings for Two-Step declined 50% from over the same period the previous year. This also supports the survey findings that customer perception has improved.

Fees down $100K

Before the end of 2020, telephony fees had been reduced to an annual cost below $150K, achieving our goal. All of the efforts worked together to drive down telephony use. This is not only a huge win for IU's bottom line but the security and ease of all users.

Go to Portfolio: Guest

client

role

Overview

Discover

Ideate

Refine

Implement

Discover

Guerrilla interviews

Purpose

Methods

Findings

Data analysis

Purpose

Methods

Findings

User survey

Purpose

Methods

Findings

Peer review

Purpose

Methods

Findings

Ideate

A. Educate throughout product lifecycle

Onboarding

Pre-engagement

Post-engagement

B. Extend Remember Me to 30 days

Improve ease of use & convenience

Drastically reduce Two-Step costs

Refine

Client feedback

Expand communication outlets

Test Remember Me alert

Tracking Two-Step setup devices

Implement

Targeted telephony user campaign

Method

Results

Orientation collaboration

Method

Results

Redesigned Two-Step setup

Method

Outcomes

Increased satisfaction

1-star rating decline

Fees down $100K

My Digital Store

Creado UI Kit

Floria Agency Template

Persona Template

Looking for a custom website? Let’s build something great together

Single-Page Website

Multi-Page Website

Ecommerce Website

Clients usually ask me

Where do I look for “Frequently Asked” Questions?

Can a FAQ page help with SEO?

Where should I put my FAQ section?

Do you have any templates?

Which template is the best for business?

I work with clients globally

Let's connect