Choose your language:
A leading worldwide provider of communications services needed to transition their existing address book application over to a new third-party platform and selected TEKsystems to provide quality assurance of the address book functions across all legacy devices, new devices, Web and server.
With more than 100 million mobile subscriptions in the U.S., the client is one of America’s leading carriers of wireless voice, messaging and data services, and through carrier partnerships, it supports over 200 countries worldwide. The client’s legacy is within the telecommunications industry, as it is a subsidiary of a multibillion dollar telecommunications corporation and can be credited with nearly half of the business of its mogul parent organization. The client offers both B2B and B2C wireless services, providing solutions to customers such as enterprises, government agencies and academic institutions. TEKsystems has had a partnership with the client’s corporate entity for over 20 years.
Mobile technology has developed into a critical asset for businesses to leverage as a means of maintaining relationships with customers. In fact, mobility was ranked as one of the top three technology priorities across most industries in 2011, according to Gartner. Though it is difficult for businesses to remain relevant while ignoring the mobile trend, it is also challenging to keep pace with the ever-changing technological landscape. Organizations must first establish themselves on mobile platforms and then fine-tune their presence over time in alignment with the evolution and advancement of mobile technology
Mobile applications are the vehicle through which the power of mobility can be maximized. To get the most out of an application, a business must develop and test properly to ensure its functionality and usability meet business objectives. An effective quality assurance (QA) program must be put into place to test the application across a complex set of variables, ensuring all technical requirements and specifications are integrated and functioning appropriately prior to introducing the application to market.
Businesses can take advantage of the dynamic technology environment by implementing a QA program that runs parallel with their mobility strategy. QA allows for the identification of defects and their resolution to enhance the application and further strengthen the organization’s presence in the mobile space.
As a provider of mobile devices, landline voice devices and landline data devices, the client has countless applications that are compatible with the different models of each type of these platforms. With their address book offering—a standard application found on most personal devices—the client presented multiple books for each platform the application could run on. In turn, the application was inconvenient for customers using more than one platform (e.g., mobile, landline voice or landline data) and lacked stability, posing potential aggravation from the client’s customer base.
In an effort to improve the functionality, automation and value of its address book offering, the client sought to streamline their address books, converging them into one product that can be used across all of its devices. A unified address book application, compatible on all client platforms/devices, would yield better functionality and subsequently, happier end customers.
During the development of the single address book, it was critical to ensure that quality was not compromised and all defects were identified and resolved prior to formal roll-out. Securing a QA strategy would help in the delivery of an enhanced product for customers and in safeguarding the client’s bottom line. Before moving toward this end goal, there were several roadblocks for consideration:
In-house Expertise: The client was utilizing our subject matter expertise in a staffing solution and did not have the space, bandwidth or desire to retain the QA team for this project.
Location: The client also did not have additional space to support the growth of the QA team needed to deliver on the requirements. Paired with this was the fact that numerous devices were securely housed at the client’s testing lab in the Seattle metropolitan area and could not be removed from there.
Scope Control: The scope of conducting quality assurance was reliant on the client as well as its third-party vendors. Scheduled release dates were subject to change, as were the specifications of what was being tested. The selected QA partner was working in more of a reactive role, operating within unpredictable timeframes.
Scalability: A project of this size would require a variable capacity model utilizing a proven, knowledgeable, flexible QA partner. A successful QA strategy would need to be handled by a partner that could ramp its resources up and down to align with the client’s expectations and accommodate the variability of release dates and test cycles.
The quality assurance strategy TEKsystems recommended involved increasing the team to include a delivery manager, program manager, analysts and manual testers, all while providing an off-site facility within close proximity to the client’s testing lab. TEKsystems would be responsible for the decision-making in terms of overall hiring of QA team; this would include creating job descriptions, recruiting, ensuring skill sets were appropriate to meet the demand and determine whether candidates were a strong technical and cultural fit with the organization.
Additionally, we would leverage one of our specialty consultants as a subject matter expert to provide advisory counsel to the client throughout the engagement. This TEKsystems consultant had eight years of experience working with the client and 25 years of QA experience, with 15 years of expertise in mobile applications—and SyncML service in particular—which would prove useful in overseeing the address book QA program.
The delivery manager would lead all financial tasks associated with the program and coordinate project change requests, while the program manager would handle the day-to-day oversight of streamlining the application. The team of leads, analysts and testers would handle a range of responsibilities, including:
Creating new test cases and test plans
Reviewing requirements and wireframes
Updating existing test cases due to change in requirements and to keep the cases current
Executing tests on all devices; server and Web testing
Documenting all results including defect reporting and tracking
Providing status reporting and maintaining weekly status calls
SIM set up for lab
Manage overall testing schedule, resource management and device management
The client selected TEKsystems as the sole solution provider for this quality assurance program for three key reasons:
The scalability and agility of TEKsystems resources
Our ability to conduct the testing off site but within close proximity to the client’s testing lab in the Seattle area
Our subject matter expert held a trusting relationship with the client for nearly 10 years, providing additional confidence that TEKsystems was the right selection as a strategic partner
Our QA program of device(s), server and Web platforms was successfully completed within the client’s desired timeframe. We supported the testing to identify defects that were documented prior to the production release, providing a cost savings for the client.
The QA team grew from just six people to more than 20 over the course of the engagement; this included the overall program lead (our consultant with subject matter expertise), test leads, delivery manager, QA service manager, analysts, and testers. Because the client did not have the space to conduct the testing in-house, the team worked at an off-site facility we provided.
The off-site facility, outside of Seattle, allowed analysts and testers to test devices in the secured TEKsystems Global Services facility.
In the first 18 months alone, the QA team wrote more than 2,500 unique test cases for over 150 mobile devices. Also during this time, the team ran approximately 74,000 test cases—including over 4,000 updates; through this testing, nearly 4,900 new defects were identified from across the old address book applications as well as the replacement contact application.
We also successfully streamlined and enhanced the intake request process. The previous intake request process practiced by the client’s 20+ program managers was counterproductive; requests were not standardized, there was no formal prioritization process and as a result, our team was faced with competing priorities. Taking an upstream approach to this process, we guided the client to clearly define requests and establish a prioritized request structure. By improving the former intake process, the client realized increased productivity, and as the upstream model eliminated potential duplication of requests and subsequently freed up testers to work on new requests.
Additionally, the team dedicated itself to completing a variety of tasks outside the scope of work defined at the onset of this program. This included:
Reproducing issues for various development groups and third-party partners
Testing issues against the application’s former platform
Preparing and revising device worksheets
Retesting and logging bugs for the former platform and testing as requested by third-party vendor
Handling ad hoc requests including additional scope, devices and changes to priorities
Dropping off and picking up devices from the client’s testing lab to our off-site facility
Troubleshooting device connectivity issues
Managing device inventory
Sequentially through the duration of this program, we worked with the client’s third-party vendor to test devices on different releases within the environments. We provided an off-site outsourced solution and was accountable for producing an impact assessment identifying gaps and a way to track performance against deadlines established. We provided additional value to the client by helping refine the work intake process, status reports, schedule impacts, stakeholder meetings and a better overall QA governance process.
We continue to support QA initiatives for applications outside the scope of this program, further demonstrating the success of this effort.
Key Success Factors
Three critical factors laid the groundwork for the successful implementation of this QA program:
QA Expertise. The client lacked the expertise and bandwidth to take this project on internally. With a dedicated Quality Assurance practice and experience in mobile device testing, TEKsystems understood the client’s challenges and leveraged its talent pool to source, screen and place resources with the unique skill sets required by this program. Our expertise enabled us to identify individuals who understood two-way sync, possessed working knowledge of SyncML specifically and could write test plans and test cases.
Flexibility. Since a vendor partner worked with the client in the development of the code for the address book application, quality assurance had to fit within the time constraints established by the middle partner. Due to the fluctuating workflow, unanticipated tasks surfaced for TEKsystems; devices often required testing and re-testing multiple times, time crunches were frequent and working overtime was necessary to get the job done. TEKsystems maintained flexibility and resilience in terms of scope by relying on an open communication model and holding respective parties accountable for meeting the client’s expectations.
Impact Assessment. Collaboration between the developer and quality partners was required in order to ensure the application was effectively tested and all bugs were fixed, and TEKsystems had little control over release dates and timing for testing of each address book iteration. To address this, we created an impact assessment monitoring tool to provide visibility into the tester’s progress over time and to map dates back to intended schedules.