Role and Responsibilities
All research, analysis, and testing; information architecture creation; findings presentation; led persona building workshop
Scopes & Constraints
No budget, COVID shutdown occured mid-testing
Timeframe
January-March 2020
Product
Certified Analytics Professional, or CAP, is a certification attainable by analytics professionals to distinguish themselves as highly knowledgeable and ethically minded within their field.
Problem
This project began with my employer’s Professional Development team wanting to make significant changes to the certification website due to low pageviews, a low number of certifications being achieved, and a frustrating amount of customer service calls and emails.
What Happened
Audience Creation
The previous CAP website was operating with no real audience in mind beyond “analytics professionals.” This made marketing efforts difficult to target and increasing pageviews and conversions would be hard to complete without more focused marketing efforts, more targeted language on site, and implementing an SEO strategy.
To start building an audience profile, I requested the data gathered from submitted applications, some 3,500 records. I used this to come up with a good set of demographic data on which to base the audience profile.
I also pulled a number of reports from Google Analytics to compare to what I had found in the records. I also used Google Analytics to begin forming the psychographic portion of the audience profile.
Once compiled, I presented my findings to the Professional Development team and two Marketing team members. I then lead a workshop to create two personas for the CAP brand going forward, one focusing on the primary audience and a second focusing on the second decision-maker audience. The second persona was also important for driving industry acceptance of the CAP certification as a standard.
Content Audit
Audience and personas completed, I started an audit of the website’s content and functionality. I quickly discovered that most of the customer service calls were likely because important information such as price and the requirements were buried under extra clicks or in the FAQs page, not on the main landing pages where users were likely looking for them.
From there, I questioned the Professional Development team on what their wish list items were. Outside of the basic information (price, requirements, etc.), what questions did they receive the most or what information was frequently requested?
I also began talking with IT about what they would need for functionality requirements and whether the website redesign would live on the same framework the first had been constructed on. The website had been built on pure PHP and was difficult to update outside of two people in the company, neither of which were in the Professional Development team. I wanted to ensure that whatever came next would be able to have its content easily updated by the people in charge of it. This, however, led to the discovery of old user flow information and business rules that had not been updated since the website was initially created.
To update these and ensure the process still worked, I sat down with the Professional Development team to begin revising the business rules and updating the user flows to certification and maintaining certification. While they took over re-writing the business rules separately, this was incredibly useful in really understanding how they wanted the process to work to be able to communicate it later in the design and with IT for the backend.
Testing
Those conversations complete and the Professional Development team busy updating the business rules, I began setting up a card sorting exercise to determine how the CAP website content should be organized. I drafted a set of thirty cards from the available content on the website and some of the wish list items within OptimalSort, an online testing platform. From there, I sent the test to three groups: the certification board and Professional Development team, CAP and aCAP certification holders, and a group without prior knowledge of the certifications. Using the results of the card sorting activity, I drafted a new sitemap for the website.
Next, I created a treejack activity, again within OptimalSort, to test the new information structure for all the non-logged in content with user expectations. Because I was using the free version of the software, I had to set up five individual tests with three tasks each, with a total pool of eight tasks to test. The task list was created from what questions Professional Development and Customer Service were most frequently asked in relation to CAP and aCAP as well as wanting to make sure respondents explored as many of the navigation avenues as possible. I sent the link to perform the activity to the same groups again, receiving different respondents this time. After running the test for a week and receiving thirty-one respondents, it looked safe to conclude that the navigation was good and usable for this content.
Overall, respondents reached a task’s identified correct response on an average of 67% between all eight tasks. However, I also created an adjusted average including an Acceptable category of answers for respondents who landed on the correct page but selected another block of text as where they would look or for respondents who navigated to another page that would likely have a link to or summary of correct information such as a FAQ page. This adjusted average raised the success rate to 93%.
One question drastically brought down the average by having only a 27% correct answer rate, however the respondents were consistently looking at one particular page for that answer. I decided to address this by adding a link to the actual correct answer onto the focused page so that going forward, users would be able to locate that information.
Outcomes & Lessons Learned
Overall, the client came away from each presentation happy and excited to move forward. They found the discovered information valuable and highly praised their inclusion in the persona building workshop.
With the new sitemap, customer service calls and emails drastically lowered after the website redesign. Having information unhidden and easily locatable was a big step forward for this product. Additionally, I worked with the rest of the Marketing team to implement marketing workflows into the website when I developed the design and website wireframes to build funnels to raise conversion rates.
Many steps in this project helped drive home the importance of bringing people along with me in the process by tailoring my delivery to their style and needs. The Professional Development Director was a numbers person and enjoyed the attention to detail and effort I put into the Audience data presentation, including visualizied breakdowns of the data I had analyzed to reach my conclusions. The Marketing Creative Director, more concerned with stories and the buyer’s journey, appreciated how I kept bringing the focus of the project back to discovering what was most useful to the customer. Bringing both of them together for the persona building workshop was a huge hit.