Automated Testing Versus Manual Testing

Brand/Sector:

Leapfrog – Educational toys, books and electronic games.

Project:

• LeapFrog Enterprises produces award-winning educational toys, interactive books, and hand-held electronic games. For nearly a decade LeapFrog employed a team of approximately 35 quality assurance (QA) engineers and 75 seasonal contractors to test their devices and software. Most testing was performed onshore and proved to be a costly operational challenge. It quickly became apparent to LeapFrog that in-house software testing had become an inefficient practice so the company began outsourcing QA to multiple vendors from around the world.
• After several years with mixed results, LeapFrog hired DWS parent company LogiGear for software testing services in 2006.
• LeapFrog embarked upon an ambitious new market leadership strategy. Key components of the overall corporate strategy called for aggressive new product introductions and the web enablement of all existing products. Critical to the success of this strategic initiative was the ability to improve operational efficiency. The new strategy required increased research and development productivity and throughput while preserving the existing QA budget. LogiGear immediately began working together with LeapFrog to develop the fundamental changes necessary to transform their approach to software testing.

Solution:

• LeapFrog’s pilot project with LogiGear proved to be a success, raising the effectiveness of LeapFrog’s overall QA efforts by a significant margin. Due to the successful pilot project, LeapFrog launched an ambitious initiative to outsource 80% of all software testing. To ensure the success of their strategy, LogiGear developed a custom growth, training, and execution plan that enabled the company to outsource 100% of their functional software testing. Upon review of LogiGear’s plan, LeapFrog’s Vice President of QA made the decision to partner solely with LogiGear for software testing citing the quality of service, open communication, low cost, and staffing scalability. Software testing efforts included manual and automated tests as well as team and process improvement. A wide variety of software was tested including PC applications, web integrations, embedded devices, downloadable and cartridge-based games, and the databases used to run LeapFrog’s international sites
  • Test Design & Execution:
    • The LogiGear team formulated test plans, wrote test cases and executed those test cases. Test case execution was both manual and automated. In addition, a fully accessible defect database was continually maintained.
  • Action Based Testing:
    • In addition to software testing products and services, LogiGear provided QA training on testing methods such as Action Based Testing. Action Based Testing™ (ABT) is a methodology for device testing as well as database testing. ABT is a keyword driven methodology pioneered by LogiGear that facilitates the creation and maintenance of automated test cases by reducing the effort needed to automate and maintain test cases. TestArchitect was the specific tool used during the testing process.

Outcome:

• Reduced hourly costs by 30%
• Eliminated seasonal hiring.
• Record product releases.

Technology:

• TestArchitect
• Software testing services QA, consulting and training.

Brand/Sector:

Hewlett Packard, Technology Solutions

Project:

Hewlett-Packard, the world’s leading provider of technology solutions to meet business and consumer needs, was updating a worldwide portal site deemed ‘mission critical’ by senior management. Time-to-market was essential, and testing was identified as the critical path item preventing an earlier release. Previous efforts at test automation had not been successful, with only a small percentage of tests being automated at a very high expense. HP sought a test automation solution that would:

• Automate the critical user acceptance tests designed by key business stakeholders
• Provide a stable, reusable long-term test automation framework that could be expanded to support other teams and projects.
• Reduce the overall testing time to improve time-to-market.

Solution:

• DWS parent company LogiGear designed a comprehensive global testing solution, including the TestArchitect™ toolset for integrated test design, automation and management, and testing teams based in the United States, Vietnam, and HP’s India facility. HP’s critical business requirements were translated into test requirements and test cases designed using the Action Based Testing™ method.
• A library of reusable action keywords was built, providing complete automation coverage of the application, and allowing 100% automation of the application, as well as future expansion with little or no maintenance.

Outcome:

• LogiGear delivered a suite of fully-automated test cases allowing the critical user acceptance tests to be run in a matter of hours, rather than days. The testing cycle was drastically reduced, removing QA from the critical path.
• The automation framework developed by LogiGear was so powerful and easy to use that it was adopted by HP’s internal test team midproject, leading to full automation of all systems combined with, functional and integration testing for a turnkey test solution that far exceeded expectations.

Brand/Sector:

Halliburton / Landmark Graphics. Oil & Gas Exploration

Project:

• Landmark Graphics is a wholly owned subsidiary of Halliburton and is the premier provider of software and technology services for the upstream oil and gas industry. Their software solutions help geoscientists and engineers make highly complex technical and business decisions.
• The specific product line involved in this case study is Decision Space Nexus – a next generation reservoir simulation software suite.
• In a move to an Agile environment, the development team were able to identify a number of key issues, one of these being the importance of having an automated regression test suite to exercise functionality under the graphical user interface.
• While the team did not have extensive unit tests, they did have a good set of functional tests that provided overall feature coverage.
• Prior to any commitment being finalized, the development regression suite was run. In addition to the development regression suite, the team relied on a customer regression suite, a manual smoke test, and additional exploratory testing. The developer regression suite safety net paid off well for the team and they had general confidence in their check-ins.
• Despite the successful check-ins, the developer tests were not finding issues that were showing up in the more complicated customer models. Additionally the big challenge with the customer regression suite was that it took almost a week of computation time on a high-end cluster to determine if the tests passed or failed.
• In addition to the challenges of long computation times, reservoir simulation presented an additional challenge – the problems were being solved by approximation techniques. In other words, small perturbations in either data or algorithmic code, or even running on a slightly different processing environment, could create different output results. The development team built an intelligent differencing tool to help understand whether generated results were within engineering accuracy of the baselined results but the team still struggled with quality issues.
• Developer tests were catching many regression issues, but the more complex customer regressions were still encountering many problems. Besides taking a long time to run, the problems discovered with the more complex customer data were also difficult to debug. As things were, both the developers and the testers on the Nexus team were barely keeping up with the defect backlog. And, when they did think they had things under control, customers would invariably find issues in beta testing or once deployed.

Solution:

• Landmark selected DWS parent company LogiGear due to the availability of cost effective engineering talent and test automation expertise which included TestArchitect; LogiGear’s keyword driven test automation platform designed for large and complex software.
• While Landmark had been interested in performing more GUI test automation internally, their team simply did not have the bandwidth or expertise to do it well. This situation presented an opportunity to augment Halliburton’s domain talent with LogiGear’s proven test automation services and Agile project experience.
• The first project kicked off with a small team of three testers in Vietnam and a part-time project manager based in California. The objective was to increase coverage through automation while, at the same time, freeing up Landmark’s reservoir engineers so that they could perform more exploratory testing. The initial smoke test automation pilot project targeted the top six integrated workflows used for Nexus software training. Test leads for the pilot were established with workflows prioritized and accountability for initial automation split across assigned testing resources. . In addition to providing experienced test automation engineers, LogiGear also provided a maintainable test automation platform in TestArchitect. LogiGear worked with Landmark to customize the tool to fit their very specific needs. This customization was critical so that all of the test cases could be automated.
• Using the Action Based Testing approach, LogiGear created test cases as a series of keywords (actions) with arguments. The automation focused not on automating test cases, but automating the actions. Since there are many fewer actions than test cases, and action implementations tend to be shorter than test case implementations, the automation effort is much more manageable. This manageability is especially evident when the application under test changes. Using TestArchitect, only a few of the actions had to be modified when the application changed. To ensure productivity through reusability and maintainability of tests, the test engineers were well trained in test case design by LogiGear.

Outcome:

• 84% decrease in defects found in beta and a 97% reduction in the known issues found at product shipment.
• LogiGear’s products and services enable Halliburton’s domain experts to dedicate more time to exploratory testing which is critical for complex software. With positive results through the combination of in-house management, outsourced testing, and TestArchitect test automation, Halliburton has increasing confidence in its ability to continue to deliver a high quality product to their clients.

Technology:

• TestArchitect
• TestArchitect for Visual Studio
• QA training and consultancy.
• Software testing services QA.

Brand/Sector:

Commscope, telecoms.

Project:

Commscope were manually testing core, back-end functionality of their networking software via a high-cost on-shore manual testing partner. Commscope were also not testing the GUI component of their solution with the result that users were finding GUI bugs with every release.

Solution:

Commscope implemented an automated testing program that made use of:

• Our off-shore testing resources in Vietnam and
• TestArchitect – a toolset that integrates the latest methodologies and technologies in one easy-to-use package

Outcome:

We created an automated software testing program that:

• Implemented 4,000 automated software test cases for the GUI software.
• Expanded testing coverage from 0% to over 90%
• Runs the entire automated test suite in only 20 machine hours.
• Identifies bugs that were previously uncovered by the company’s customers, thus helping to improve customer satisfaction.

Technology:

• TestArchitect

Brand/Sector:

Data management & Storage

Project:

The client is a data services provider who is able to ingest, clean, transform and aggregate individuals and organisations data for business purposes. A new data platform/warehouse was required to replace an existing legacy solution. We completed an initial discovery and definition project which then led to the architecture specification for the project.

Solution:

• Build out the base architecture in Azure Cloud and automate the deployment with pipelines.
• Build automations for import, transformation, output and other integrations.
• Test environment and automations.
• Consultancy covering how to support and maintain the platform.
• Write procedures for client’s team to re-do maintenance stored procedures and assisting the client with source control for SQL server stored procedures.

Outcome:

• Project successfully delivered.

Technology:

• Terraform, Scrapy, Asure Logic Apps
• Azure DevOps, Azure Data Factory
• SQL server, T-SQL
• Storage Blob

Brand/Sector:

Fintech, Crypto Trading Exchange Platform

Project:

• To assist the client in the creation of a crypto-currency platform (website & mobile App) from scratch.
• To enable users to trade, store assets in the wallets and withdraw/deposit funds – securely.
• To integrate with the TradingView platform to enable real-time updates.

Solution:

• Responsible for platform architecture and design and the development of the crypto currency exchange web and forthcoming mobile Apps in line with business requirements
• QA/software testing of various functions, real-time data chart, custom UI interface and cases of internet connection and disconnection.

Outcome:

Successful delivery of the project featuring:

• User authorisation (Google Authenticator)
• Real-time data chart updates.
• User interface customisation.
• Crypto wallet integration and security.
• Deposit and withdrawal via bank transfer and other digital methods.
• Trading and analytic tools for users.
• Matching engine.

Backstage management system:

• Admin management.
• User management – asset management, abnormal behaviours alert, VIP user set-up.
• Initial Exchange Offering listing and management. Blockchain currency (Blockchain currency management, transaction records, reports and dashboards.
• General system management including third-party platform set-up, handling fees set-up, log information.

Technology:

• Back-end – Java (Spring). Front-end – Angular
• Mobile – Flutter
• Integration with Fiat Payment Systems and live chart data (TradingView).
• Database – MariaDB/MySQL
• Blockchain – BTC, ETH, TRX, BCH, LTC, USDT, XUSD

Brand/Sector:

Fintech, Credify.

Project:

• Southeast Asia fraud rates in finance and e-commerce are amongst the highest globally.
• Credify provides secure and cost-effective blockchain-driven Universal Identity and Trust System solutions to companies active in the e-commerce and digital finance sectors.

Solution:

Engaged as an official technology partner for Credify to develop areas including:

• Privacy preservation – user controlled unique and universal identity solution where third parties can securely attach rich sources of information including KYC results and proprietary credit scores.
• Tokenised asset wrapper and issuance platform.
• Patent Pending vouching system, easily integrated into various online ecosystems and digital finance.

Outcome:

Worked in collaboration with the in-house Credify team, delivering a product with the following features:

• Secured by Blockchain – personal data can only be managed by the user themselves. Secured by Blockchain the data cannot be leaked or breached.
• Universal – users can use the same ID across multiple services.
• Unique – one user, one ID.
• Cost reduction – As personal information is not centrally managed, account management costs are reduced.

Technology:

• EOS (Blockchain Software Archtecture)
• U-Prove (Microsoft)
• OpenID
• Kubernetes

Brand/Sector:

ALUMNI – Professional Private Network

Project:

To provide consultancy, design and development services to build the ALUMNI mobile App from scratch.

Solution:

• Provision of a full service team – project management, front and back end development, testing and implementation.
• Development of various features including messaging, message board and investment to enable connections between educational institutes, business partners, their alumni and employees in order to build networks through events and business opportunities.

Outcome:

Delivery of the Alumni Mobile App featuring:

• Real-time integrated geolocation for a personalised user experience
• Full user security and privacy.
• Easy to use and intuitive UX

Technology:

• Back end:
  • NET, Core 2.1 & Web API Core 2.1, SignalR, Entity framework Core 2.1
• Front end:
  • Angular 5, HTML5/CSS3, Bootstrap 4, Google Map API
• Mobile:
  • React Native 0.54.2, Redux & Redux-Saga, Google Map API

Brand/Sector:

Fintech – one of the largest commercial banks in Vietnam.

Project:

Part of the client’s overall digital transformation – enabling digital banking services to be offered to their customers on both Web and Mobile platforms.

Solution:

Working on-site with the client’s existing team:

• Design and code new features, integration with existing systems via APIs.
• Consultancy to optimise system performance.
• Development of features for mobile App.
• QA/QC and testing services.

Outcome:

• Successful development and delivery of a range of features for back-office applications and the mobile App.
• Creation and maintenance of process documentation relative to the financial banking domain.

Technology:

• Java, Spring Boot, Kafka
• Platform:
  • Backbase, Microservices, Kubernetes.
• Mobile App:
  • iOS & Android.

Brand/Sector:

Construction

Project:

Workpack is an integrated software platform that provides solutions for project management in the engineering and construction industry. To design and develop a new system with features to allow businesses and general contractors to manage and enhance complex, large scale construction projects.

Solution:

Delivery of the Team Insight platform which includes various modules including Workpack Manage, Workpack Planner and Workpack Assemble which enables contractors to manage and plan all aspects of a project in accordance with international standards.

Outcome:

Easy to use, feature-rich system providing tools to enable users effective project management.

Technology:

• Frontend – ReactJS, HTML5, CSS3, MaterialUI, Redux.
• Backend – Java, NodeJS, Apache Spark.
• Big data – DeltaLake, DeltaBrock, Neo-Flow
• Database – MySQL, PostgresSQL

Brand/Sector:

UK Resort & Spa

Project:

3 concurrent projects augmenting the Spa’s web app:

• Moving blog content from old CMS to Contentful CMS
• Building My Account system
• Improving SEO and Conversion Rate Optimisation

Solution:

Our team of developers worked alongside the client to:

• Scope, design and build a new Blog site migrating from WordPress CMS to Contentful CMS.
• Design and build of the My Account function to enable users to manage their bookings and profile.
• QA and Software testing.

Outcome:

• Successful delivery of upgraded systems within agreed timescale.
• Additional projects on-going.

Technology:

Ruby, HTML, CSS, Contentful CMS, WordPress CMS, ReactJS (NextJS).

Brand/Sector:

Pacific Health Corporation

Project:

To design an web application and develop a data processing tool to extract and collect multi-source data from sales and marketing teams (sources – API, website, excel files, social networks etc.). To standardise data and provide analytic reports to business management to optimise sales/marketing performance.

Solution:

• Drawing data from wide range of sources and allowing users to import huge files.
• Creation of bespoke dashboard to visualise reports from database. Creation of web interface to add or modify database records.

Outcome:

• Clear sales and marketing performance tracking.
• Ability to analyse factors affecting performance including market and customer behaviour.

Technology:

.NET, Python, Angular, MS Factory, Power BI.