Before implementing our new user testing initiative, we first needed to address several issues that were limiting the effectiveness of our previous testing methods:
1. Most testing was conducted with the product support (PS) team rather than actual users.
2. Testing sessions were often treated as user interviews rather than usability tests.
3. Many sessions focused on demonstrating or selling a feature rather than evaluating its usability.
4. There needed to be clear metrics or scoring systems to evaluate the success of testing sessions. Testers were selected based on availability rather than specific user criteria or personas.

To address these challenges and improve the accuracy of our user testing, we introduced quarterly user testing days and invited our users to our offices. Leading up to each testing day, our design team carefully selected the features to be tested and prepared unbiased scenarios, questions, and prototypes. We also established UX metrics we wanted to measure during testing. In addition, to ensure the success of our user testing days, we invited specific users based on our user personas. This approach helped us gather more accurate and relevant feedback from individuals who closely resembled our target audience.

Before our first user testing day, I led a workshop for our design team on conducting effective user testing. This workshop gave our team members the necessary knowledge and skills to facilitate successful testing sessions. We also conducted internal pilot testing to further prepare for the actual user testing day. This allowed us to refine our questions and ensure that we were asking them in the most effective manner possible.