Vipnet usability testing, education and consulting

  1. Internal education and consulting about usability
  2. Usability studies and research of cloud-based, IPTV and IVR systems
  3. Corporate website user experience audit and research
  • Client: Vipnet is the first private mobile phone operator in Croatia known for its innovative business solutions.
  • Type: Usability studies and research; Consulting; Education; UX design; Software development
  • Industry: IT and telecom; Management consulting

We’ve been working with UX Passion on several projects related to usability of telecom and IT products and services. And I have to say that the name of the company suits them perfectly – people at UX Passion are really passionate about usability and overall UX. To sum it up, it is always a pleasure to work with UX Passion because you know that they will do the best possible job.

Author imageMarko Popovic
Market Strategy and Development Expert, Vipnet

Problem

Being the first private mobile phone operator in Croatia, VIPnet is known for its innovative business solutions that often change the game of the entire telecommunication industry in Croatia.

Owned by Telekom Austria Group, VIPnet became a Vodafone partner in 2003, spreading its services to 30 countries across Europe.

As our cooperation with VIPnet has continued to grow over the course of several years, we’ve been involved in a number of different projects covering the areas of usability testing and expert reviews, management consulting and education.

1) VIPnet offers a number of cloud-based services for SMEs and individual customers, ranging from car and ship fleet management systems to CRM/ERP based systems, among many others. Several external vendors have developed those solutions, and even though they’ve been following a common visual framework, there have been numerous discrepancies.

The user flow has been inconsistent and the overall user experience has been sub-optimal with serious usability issues.

Also, VIPnet offers it’s own IPTV service. We were asked to perform a wide scale usability research and expert analysis of its newly built IPTV software, identifying possible obstacles for users satisfaction, as well as pain points in its current IPTV experience.

Being in a telecom business, VIPnet’s users are regularly interacting with their IVR (Interactive Voice Response) system. It’s one of those things you hear when calling the helpdesk (“For subscribers press 1, to talk with operator press 0 or hold”). VIPnet noticed a number of areas where users were struggling and we were asked to asses the situation and help.

VIPnet asked us to help with the usability of their website.

We were asked to conduct a series of user testing sessions in pre-paid, post-paid, online shopping and across a range of various smartphones, in order to identify pain points for their website and then figure out a way to improve user experience satisfaction and increase conversion rates as well as the profitability of the company.

2) Lacking the practical knowledge of working UX principles, VIPnet design and development teams have been producing sub par products and services, as well as wasting a lot of time and resources in correcting those errors.

We were asked to educate, create and support an internal UX task force that would take care of usability principles, customer experiences and decrease the amount of time needed for development of new products and services.

Solution

The scope of the projects and challenges we accepted and described in the previous section was quite wide and the applied solutions varied greatly. Here’s the summary of key solutions and approach we undertook to deliver the best possible service both for client and their users.

1) We prepared and executed a large-scale usability operation. The first phase was an expert heuristics-based usability review and presentation of findings. After that we recruited test participants and performed user testing based on tests and walk-throughs.

We used advanced screening and recording techniques and captured all mouse movements, verbal communications, facial expressions, eye focus and other forms of non-verbal communication. The users were located in our testing lab with a testing moderator, and in a separate room connected via video link were client representatives and UX Passion’s usability consultants, who followed the tests in real time and discreetly sent their suggestions to the moderator where there was a need for further clarification.

After the tests and their analysis we prepared a report with suggestions for how to improve the systems. We prepared visual style guides and a “visual playbook” to help developers and engineers implement our suggestions.

A similar approach was used for IPTV system evaluation as well: we conducted expert usability review and analysis in two tiers. The first tier was focused on information architecture, interaction design and the overall functionality of the system for the end users. The second tier examined visual style, language and metaphors, identifying design flaws and inconsistencies within the user experience.

Almost the same approach was used for the client’s website, with a total of 26 users testing their website through several different scenarios over the course of six days using our proprietary UX Passion usability methodology.

Again, building on previous experiences we used our knowledge and experience for the IVR system usability testing project, but on somewhat smaller scale. We mapped user’s cognitive walk-throughs and prepared tests as well as a prototype of a new IVR system and recorded users’ behaviours and responses, as well as error rates and number of other, crucial usability metrics.

2) It was important to make sure that our solutions were not left in a vacuum after the series of tests and that VIPnet’s team had enough knowledge to at least assess basic UX needs.

In order to accommodate those needs we ran a number of tailored training courses for their key staff members. The focus of these workshops was on future efforts to improve the usability of the company’s website and platforms, covering all major aspects of usability from key principles, research and monitoring methods and approaches, SUS scores, heuristics, user testing and so on.

After the training days, every group had a half-day workshop where we compared internal usability findings with the results from our previous usability study.

Results

VIPnet is very receptive to our suggestions regarding usability of their key products and services. Where technology and time allowed, almost 100% of our suggestions were implemented. The SUS (System Usability Score) increased by 60% (measured internally) after implementing the proposed changes. The visual style guides have been rated as excellent and will further help the planned consolidation of the brand as well as the overall user experience.

IPTV project results also reflect significant improvements. During the research phase and data analysis it became evident that many of the issues were correlated, so we presented our findings in a side-by-side layout allowing a 360° view of all of the identified issues within their broader correlations, thus facilitating the solution of specific problems. On top of that, all of the issues were graded by their relative importance, with clearly-outlined risks to help the decision makers in prioritisation of future upgrades. The same approach was followed for IVR system. Given our long-term cooperation with VIPnet we are following their progress. Changes are still being implemented and rolled out in phases. We are actively tracking progress and providing consulting on an as-needed basis.

By creating a task-force within VIPnet, UX Passion trained staff are now working with VIPnet development teams from the concept phase through all of the projects lifecycles, aiming to recognise and test functionality and usability early in the process, thus saving time and other resources and facilitating production of better services and products.

Due to the non-disclosure agreements in place, we are not able to disclose more about future or full-scale plans inspired by our collaboration and proof-of-concepts.

Results

Related work