Application
development for mobile devices is a fast growing phenomenon.
Users
want mobile applications to be simple and fast. Just one nagging bug
or usability issue can spoil the entire experience. And with so much
competition in this new space, if users don’t have an excellent
experience with your application, they will switch to a rival product
faster than you can say “There’s an app for that.”
When
planning your testing effort for a mobile device application, in
addition to the usual functional testing, it is also important to
consider the following areas and how they differ from desktop or
regular web applications:
User Interface Testing
Mobile
devices have unique user interfaces like smaller screens that can be
re-oriented, touch screens and soft keyboards, and navigation methods
like hard keys and trackballs.
The first area to explore in your
test plan is the user interface. While Smartphone applications are
relatively new, there are already accepted guidelines for look and
feel and overall behavior. It is your job as the tester to confirm
that your application.
Overall
color scheme/theme of the device
Style
and color of icons
Progress
indicators when pages are
loading
Menus.
How they are invoked and
typical items they contain
Overall
responsiveness of
applications on this device
Touch
screens
Multi-touch
vs. single touch screens:
If
your device and application support multi-touch features, like the
pinch-to-zoom effect on iPhone, be sure to include lots of test cases
involving touching the screen in more than one place simultaneously,
especially while typing on the soft keyboard.
Long
touch vs. short touch
While
there is usually no concept of a double-click on touch screen devices
(although there could be if specifically implemented in your
application), some, like the Android smart phones, distinguish
between long touches and short touches: pressing and holding an item
will bring up a context menu in the middle of the screen, while
short-clicking the same item will automatically perform the first
action in that context menu.
Button
size and position
Ensure
that buttons and icons are large enough and far enough from the edges
of the screen to be easily clicked by a large fingertip.
Trackballs,
track wheels, and touchpad’s
If
your device doesn’t have a touch screen, it is even more important
to verify that screen navigation is as painless as possible for the
user. In these cases, the user may rely on a trackball, track wheel,
or touchpad to move from object to object
Soft
keyboards
There
are often various special cases and corner cases that are important
to end users.
1.
Does the soft
keyboard automatically appear if the user’s main action is to enter
some text?
2.
Does the first layer of the soft keyboard include shortcut “@”
and “.com” keys if the highlighted field is for entering email
addresses?
3.
Does a long touch on a soft character key bring up several different
character choices for that input?
4.
Can the soft keyboard be dismissed and re-displayed easily?
5.
Can the soft and hard keyboards be used interchangeably (if the
device has both)?
6.
Do soft keyboard characters entered in password fields only show up
as ****?
Hard
keys
Be
sure to include a lot of testing around the use of the device’s
available hard keys such as Start, Home, Menu, and Back. These should
all interact with your application similarly to how they interact
with the device’s native applications.
External Factors Testing
Mobile
device applications must also contend with interactions and
interruptions from other device features like various network
connection types, SD cards, phone calls, and assorted device
settings.
Network
connection Types
App
going to be used on devices in various locations with various network
connection speeds, it is important to plan testing coverage for the
following scenarios:
1.
Only Wi-Fi connection
2.
Only 3G connection
3.
Only 2G connection
4.
with no SIM card in the device
5.
In Airplane mode (or all connections disabled)
6.
Using the network through a USB connection to a PC
7. And most interestingly, test
intermittent network scenarios that a user might encounter in the
real world:
a.
Walk out of Wi-Fi range so the connection automatically switches to
3G/2G (for example, in a Large building like a hospital or airport,
or outdoors).
b.
Ride in an elevator or on a train where the network connection may go
up and down
c.
No network connection available at all
SD
card interactions
If
your application can potentially store or retrieve items on the
device’s SD card, then it is important to test the application’s
behavior when there is an SD card present or not. At a minimum, the
application should provide user-friendly error messages when a
function cannot be performed due to a missing SD card.
Phone
calls and other interruptions
If
device you're testing with is also capable of making and receiving
phone calls, be sure to test
The
following scenarios,
1.
Your application is interrupted by an incoming call, originator hangs
up the call
2.
Your application is interrupted by an incoming call, terminator hangs
up the call
3.
Your application is interrupted by placing an outgoing call,
originator hangs up the call
4.
Your application is interrupted by placing an outgoing call,
terminator hangs up the call
Also
consideration such interruptions as,
1.
Text messages
2.
Voicemail notifications
3.
Calendar events
4.
Social media notifications (Face book, Twitter, etc)
5.
Alarm clocks
6.
Low battery notifications
Device
options
1.
Sound profiles
2.
Device password/unlock
pattern
3.
Font
4.
Screen timeout/Auto on, off
5.
Screen orientation
6.
Connections
Stress Testing
Mobile device
applications have much less overall device memory and power available
so must handle them very efficiently.
Stress
testing is a must to find exceptions, hangs, and deadlocks that may
go unnoticed during functional and user interface testing.
Load your application with as much
data as possible to try to reach its breaking point
Perform the same operations over
and over again
Perform the repeated operations at
varying speeds – very quickly or very slowly
4.Leave your application running for
a long period of time, both interacting with the device and just
letting it sit idle, or performing some automatic task that takes a
long time.
(E.g. – Slideshow)
5. Randomly send screen taps and
keystrokes to your application.
6. Have multiple applications
running on your device so you can switch between your application and
other device applications often.
Security Testing
Mobile device
security will become more and more important as the user base grows,
so it is essential to test the security of your mobile web
applications, sensitive data storage, and how your application
behaves under various device permission schemes.
Most mobile devices assume one
user; they do not have multiple accounts. (Except in the case of
multiple email addresses configured per device, which should be
tested especially if your application deals with the device’s
Contacts or Email functions). But in general there is no concept of
user switching, different profiles, or permissions based on user
level.
It is up to the user whether or
not they configure a password/unlock pattern for their device at
all.
Outside communication of any
sensitive data should be done over SSL/TLS because it can’t be
assumed that the user will configure their Wi-Fi to be secure.
Emulator Use
Emulators
can be a great asset when it comes to achieving testing coverage on
multiple devices, but your test plan must also respect the fact that
sometimes there is just no substitute for the real thing.
Not all activities can be
realistically emulated, like switching network connections, or
taking a picture or video.
Some activities just don’t work
at all on emulators, like streaming video on a Blackberry emulator
Due to lower device power and
memory, your application could exhibit slower performance overall
when run on an actual device.
If
the emulator and actual device have different dpi resolutions, your
screens may not display as you expect.
In general, it is a good idea to use
some combination of real device and emulator testing,
