user research + a/b testing + strategy
Director of Information Architecture – Proposed, led, and completed all activities.
The University announced the CMS would be replaced due to usability and performance issues. I requested to be involved and evaluate the usability of the candidates since I was the only Usability Expert on staff. Eventually, this led to a UX Strategy to create a unified vision of what needed to be built, and how it would be used to guide not only the selection of a CMS but its implementation.
Design Thinking was used to evaluate the CMS Candidates. A Discovery phase included: identifying detailed, realistic needs for both stakeholders and users to inform business decisions, finding and designing solution, gathering quantifiable and qualitative feedback, and vetting and projecting success.
I utilized the UX Methodology to inform a definition of what is a useful and usable CMS for GW from both a user and stakeholder perspective. My plan was to identify user groups, their abilities, and tasks, conduct multi-variant testing on the replacement candidates, and define user requirements. I also expected to identify latent business requirements as UX does that very often. I was granted 6 weeks to do it. In that time I:
Conducted User Profile Survey to identify and draft user groups.
Conducted contextual inquiries and stakeholder interviews to identify latent user groups, flesh out user groups, and identify primary tasks finalize user and business requirements.
Created Personas and defined user requirements to communicate end-user needs.
Designed and conducted multivariate, scenario-based Usability Testing of potential replacement products to evaluate their ability to support user and business needs. Testing metrics were defined and agreed upon by my team. The standard was set high. Efficiency, Effectiveness, Ease of Learning, Error Tolerance, and User Satisfaction measurements were gathered from both my testing and user satisfaction surveys (Weighted Likert Scale).
Designed satisfaction surveys for users, stakeholder, technical, product teams (Weighted Likert Scales - Quantitative and Qualitative) to focus each group on their relationship and use of the CMS rather than everyone weighing in on how usable and useful it would be for users and the organization overall.
Regular updates were made to keep the University teams updated to progress, insights, and opportunities for input to foster ongoing support and gather needed feedback.
Conducted Participatory Design Sessions for critical CMS screens – Dashboard, content creation/edit to gather latent requirements and ideas, while fostering buy-in and consensus.
The A/B Usability Testing revealed that the final 3 CMS candidates would not meet user needs in terms of common tasks, and the usability of each was low for the CMS users. Additional candidates were sought with a greater understanding of the organization's needs, and Drupal was selected for its flexibility and the ability to customize and evolve the functionality. I was also asked what CMS I would recommend, and I concluded that Drupal would offer the effective and efficient solution immediately with the ability to evolve over time to effectively support all the organizational and user needs to be revealed during this effort.
Each organization had specific desires, and wanted individual workflows. I identified patterns from the interviews, and created a large workflow chart (5' x 3') and conducted walk-throughs with stakeholders who I thought had similar workflows. The session flowed smoothly with groups exchanging ideas on how to work more efficiently and agreed upon this workflow. This also depicted what the CMS needed to support.
One of the latent requirements this effort uncovered was the need for content sharing. Several university level organizations had trouble managing content that was used by different groups in the organization such as Policy. The ramifications of the inconsistent and/or outdated information had legal and financial implications.
During this effort I introduced GW to Design Thinking - Making business decisions with a realistic understanding of the organizations' and users' needs, working collaboratively and iteratively. The established methodologies were used to identify detailed, realistic needs for both stakeholders and users to inform both business decisions, finding and designing solution, gathering quantifiable and qualitative feedback, and vetting and projecting success. I established User Experience as a trusted methodology and established my team as a collaborative partner in managing GW's digital properties.
Report of Findings
Work flows & Interaction Models
CMS interface direction and functional requirements