TestArmy ebook QA trends 2019

Be sure to download our new report and follow the latest trends in QA, Cybersec, IoT, DevOps and more!


The Proliferation of Mobile Devices Pt. II: Finding the Best Solution for You


If you read our previous blog about the proliferation of mobile devices, you may have been left wondering what would be the best solution to this problem. If you haven’t read our last blog, you probably should to get a better idea of the problem we’re discussing!

Considering the 24,000+ unique Android devices and the 30+ unique mobile Apple devices on the market, testing can be quite a daunting task. That doesn’t even bring into consideration the various versions of those operating systems, network operators and memory of each device.

We’re here to tell you that it’s not all gloom and doom, and below we’ll take a look at your three major options.

Replicate your Target Audience by Buying their Devices

Let’s start with the most obvious answer, replicate your target audience. If you have an extremely specific target audience, then this solution may be perfect for you.

While it’s usually extremely costly; buying multiple devices and testing your project on those devices can give you or your testers a wide-open window into what your users might experience.

One problem lies in the fact that if you test on devices that aren’t yours, or that you haven’t used before, your proficiency with those devices would practically be 0. This can effect the testing process and affect the number of bugs that are found.

Another problem is that buying various devices and testing your projects on different operating system versions and network providers can be financially and physically impractical, especially if you plan for your product to be used in different markets.

While great for small, isolated products, buying devices and testing on them will likely result in you throwing your money into the pit of doom; especially if the proliferation of devices continues.

Emulators: Cheaper, faster and smarter?

Naturally, the next thing that comes to mind would be to use some type of emulator to test your project across a multitude of devices.

Using an emulator allows you to test your product on many different devices in a simulated environment. Emulators eliminate the headache of having to buy 50+ devices and physically testing your product on each of those devices.

However, emulators don’t always catch everything or account for each unique setting that one can discover on physical devices.

In our experiences, we have noticed that emulators usually lack the savvy to properly test navigation/GPS technologies and often cannot properly test product-related sounds, which can lead to an incredibly frustrating experience for a client.

We have also noticed that while emulators are great at testing the technicalities of the product itself, some things can be overlooked.

For instance, we once tested a mobile application where the keyboard was horribly inconvenient and difficult for the customers to use. While it looked ok to the emulator, in reality it was very problematic for the users.

Emulators are great for doing tests across multiple operating systems and various versions of operating systems. Although, it should be noted that while this ‘blanket’ approach is used by lots of companies, there are always some holes in the blanket and it never can be used as a substitute for a quality test on a real device.

Outsourcing to a Testing Company

Finding a company to outsource testing to can be a scary, time consuming and expensive process. One bad partnership can leave a rancid taste in your mouth and wallet for a long time.

However, there are some upsides to outsourcing that may be overlooked, especially if you’ve never tried it before. In general, outsourcing should be a cheaper, quicker and better way to get an unbiased test of your project.

Testers are cheaper than developers and specialize in finding bugs, which should cut your testing time down to almost nothing. Plus, if you hire a company to test remotely, they won’t have any loyalty to your product, which should result in a “cleaner” test since there are no biases involved.

In addition, a third party testing company will work remotely and does not require any extra investment in training or orientation. Plus, you will not have to provide them with any devices, thereby saving more money.

Just like with any other industry, there are good companies and bad companies. Usually if it’s too good to be true, it probably is, and you should look elsewhere. But for every bad guy out there, there’s always a company who stands by their work and will prioritize your needs to give you the most bang for your buck.

So What’s the Answer?

As you can see above, there’s no ‘one size fits all’ solution for your new product.

Each potential solution has its own upsides and downsides. And the goal for everyone in this situation should be to stay financially smart and think about the final product.

While outsourcing would probably be the most comprehensive way to get the best of both worlds, it will usually cost a couple pennies more than an emulator.

The nice thing about outsourcing is that you can dictate exactly how you would like your product tested on different devices/platforms. For instance, if the majority of your target audience uses a couple of devices, you can request extensive testing on those devices and automated testing on all secondary devices.

This gives you the pros from both ways of testing without requiring a small fortune to begin with. Plus, if you have a quality partner, they will also make suggestions that would be better for your final product or for your budget.

Having a partner that you can trust during the testing process is the key to success in this situation. If there seem to be some red flags with potential partner, keep moving.

Hopefully, once you identify a quality partner, they will understand your target market and can help you choose the best options for your product. With over 24,000 unique devices on the market, it helps to have someone guide you through the testing process.

Rate the article:

Redakcja TestArmy

Redakcja TestArmy 15.01.2017

Redakcja testarmy.com

Leave a comment

Your email address will not be published.