I do most of my testing on platforms like BrowserStack. It works super well—having access to a ton of devices and browsers have been useful in finding bugs and ironing out some of the kinks that inevitably come up. Unfortunately, I’ve found it to be a bit lacking when I’m trying to test the “feel” of a website: if the animations and interactions are smooth, if interacting with the app feels natural, and if the buttons are too small for my stubby fingers.
While I obviously have my own personal devices to test on, they’re not enough. Not everyone has the latest and fastest phone, and not everyone uses the same browser. Ethan Marcotte explains this well:
Newer devices are often the best-represented among the teams I work with, which means they usually have, well, the best representation in the design process. But it’s the older ones that need our time and attention: because even though these devices are of a vanishing breed, if not an outright obsolete one, they can help us understand how our design decisions will impact devices that don’t look like ours. How they’ll impact users who aren’t quite like us.
Mozilla goes into the details:
Many developers believe the browser they use is the only browser that anyone really uses, therefore they should just develop for it. By some measures, 70% of web developers use Chrome on the desktop. But only about 50% of web traffic across all device types is on Chrome, and only about 62% of web traffic on the desktop is on Chrome. Building and testing only on Chrome alone ignores almost half of global users. (It’s worth pointing out here that different browser share trackers use different methodologies and produce different numbers, and the numbers change quickly and often.)
And browser use varies by geography. Chrome, Firefox and IE/Edge are the top browsers in many locales, but the proportion of users on each varies. German users favor Firefox over Chrome. IE is big in Japan. Quite a few Australians choose Safari. More than 1 in 5 Vietnamese users run a fork of Chromium called Cốc Cốc. Building and testing on just one browser ignores these market differences.
I looked into device labs online and saw how intimidating Perth Device Labs and Helsinki Device Lab were. I thought about not going through with it at first because I definitely didn’t have the money to buy that many devices. But then I thought that I could start small and cheap by asking for device donations from people that I knew.
I started off with my girlfriend’s old LG G5. It had some charging problems and a roughed up screen, but I thought it was good enough! It was fortunately fairly straightforward to repair it. (It was straightforward, but it wasn’t easy! Prying it open was a constant reminder of how easy it is to break these things.)
Then I got my mom’s old iPhone 6S. And then finally an iPhone SE for $80 from a second-hand electronics website called Swappa. Definitely a steal considering Apple is currently selling refurbished iPhone SEs for $249.
I installed a ton of web browsers on each of them, and also registered a separate Gmail account in the App and Play stores just in case someone wanted to borrow it. So far it’s been useful. I’ve not only tested websites, but also apps.
I’m not close to being in the lab stage yet—and I’m far from running into problems that Etsy’s device lab has—but I think it’s a start. Another plus for buying second-hand, fixing, and asking for device donations is that we’re also puts less strain on the environment:
The little computer you carry with you requires a lot of energy to assemble. The production of an iPhone 6, for example, released the equivalent of 178 pounds of carbon dioxide, or about as much as burning nine gallons of gas, according to a 2015 study. Instead of buying a new phone, try to keep yours in working condition for as long as possible (here’s some advice on how to extend its life). But if you must get rid of yours, recycle it or consider buying a used one.