We frequently hear words like ‘usability’ or ’user experience”, but still very few developers have conducted user tests for their apps.
So says Daisuke Hiraishi, the CEO of InnoBeta Inc., the up-and-coming startup behind the crowdsourced app testing service UIScope. Innnobeta announced today that it would be partnering with DeployGate, a smartphone app test platform run by Japanese social network operator Mixi. As we mentioned in our previous feature interview, DeployGate has acquired users from over 93 countries and been used for testing more than 4,000 apps worldwide.
With this integration with DeployGate, Innobeta provides a one-stop service to be known by the name of ‘DeployGate Scope.’ It allows developers to provide crowdsourced usability testing not only for live apps but also for prototypes still under development.
UI Scope was launched last October, and its team consists of three full-time workers, a freelance designer, and an engineer (who is attending grad school). The CEO Hiraishi used the bulletin board service WishScope to invite potential co-workers, including his CTO Takakiyo Aritaka. They’ve had been running a design-oriented app service, but faced many issues in the process of user testing. That experience made them realize that many app developers were likely facing the same problem. And this led to them to eventually launch the crowdsourced usability testing service, making use of a pool of testers to provide feedback to developers and designers.
Similar services exist, but there’s no testing service specifically focused on testing for smartphone apps. The majority of the startup’s clients are big Japanese companies such as Recruit or NEC Biglobe, but users from startups also on the rise.
Since launch last October, the service was used for 200 projects by 40 companies. 70% of developers have the service to get feedback on upgrades for their apps, and the rest are for testing initial releases.
In terms of specifying the layer of testers you want to ask, you may choose them according to age, gender. You can also select testers by other criteria, such as people who have tried out a specific service, or even someone who has recently traveled to a certain place. On average, every case takes about one week and with 10 testers. According to
global measurement company Nielsen Jakob Nielsen, a test conducted with 10 five people is enough to find and identify 80% of all problems .
The testing process is an interesting one. Participating testers will receive a webcam for free from UI Scope. Testers are requested to record their testing task, speaking their impressions as they proceed step-by-step. By collecting those recorded videos, you can understand how testers thought and felt, why they pressed a certain button, or why they abandoned a screen on the app. These videos will then be passed on to clients. Pricing is 3,000 yen for testing, and clients will pay 15,000 yen on average, typically asking for around 5 testers. Analysis reports are also available as an additional option.
Strict requirements for testers
More than 4,000 people have signed up for the service to be testers. The startup is devising ways to get more applicants, although they have a very strict filtering process for those who want to be qualified testers. Potential testers receive two to three dummy tests a week, and only are few of them who yield good results and display a certain literacy level will finally qualify. After that they can can receive 500 yen compensation every time they finish a testing task. When compensation reaches 2,000 yen, it will be transferred to your bank account.
Housewives were invited to be testers through the startup’s ‘invite your friend’ campaign, and many students have signed up through an introduction from the startup’s partner companies. UI Scope has also partnered with Mamion, a chain operating PC training courses for the elderly, thus succeeding in acquiring some older testers as well. This broad variety of testers helps the startup to receive many testing orders from a wide range of companies.
For future problems they want to solve, Mr. Hiraishi further explains:
We need to some time to educate developers on the necessity of the usability testing. That’s why we’re organizing events like the “Smartphone Design Conference”. For usability testing, it usually requires 10 to 15 minutes for a testing case. Conducting the case with 10 testers, that means 100 minutes at least. We want to improve the service so that it allows users to see each [step] of the testing task. We look into providing more specifications about our testers, so that our clients will be able to conduct more targeted tests.
For now, most testings are conducted at home, but the startup is planning to develop an environment where people can do tests outside their home (useful for testing GPS-enabled apps, for example) and real-time testing (for testing social network apps). For developers who would like to try out the service, UI Scope is giving a 50% discount until the end of this month.
The company previously raised 5 million yen (about $53,800) from Movida Japan, and has acquired 120 developers and 2500 testers during the last six months. The video below will give you a better idea of how the service works.