AI Powered Test Automation Helps Retailer Increase Test Coverage to 95% and Double Speed to Market

The Vermont Country Store takes pride in being the ‘Purveyor of the Practical and Hard-to-Find’. From its founding in 1946 by Vrest and Mildred Ellen Orton, to the present day when the company is run by 4th and 5th generation of the Orton family, merchants at Vermont Country Store source goods globally that fill the needs of its customers. Over the decades, these needs have changed, and the customer base has grown, but one thing has remained constant – Vermont Country Store’s commitment to customer satisfaction.

We were able to increase our test coverage to 95% of the browsers and devices our customers use, while simultaneously speeding up our testing 2X with eureQa
— Russell Glitman, eCommerce Manager, The Vermont Country Store

This focus on the customer and the quality of customer experience is well represented in The Vermont Country Store’s Customer Bill of Rights, which is prominently displayed on their Web site. When The Vermont Country Store was looking for a solution to ensure the quality of customer experience on their online storefront, they chose eureQa.

As The Vermont Country Store adopted the Oracle Commerce Cloud (OCC) platform they wanted to make sure their customers’ online shopping experience was maintained at the same high levels that generations of in-store or catalog shoppers have been experiencing. The OCC platform made it easy for The Vermont Country Store to make changes to the online store providing a rich shopping experience to its customers, and to add new features and services through integrations to external systems.  These changes however required a great deal of testing, often requiring a ‘all-hands-on-deck’ approach, tapping into customer support staff to augment the QA team to complete the testing in a timely manner.

 
The Oracle Commerce Cloud platform made it easy to push new features, but testing them took a lot of time and effort.

Challenge

With the frequency and pace of changes to its Oracle Commerce Cloud eCommerce site, guaranteeing a great customer experience meant that the frequency and volume of testing had to be significantly increased:

  • Testing OCC platform upgrades could take over a week and require additional staff loaned from customer service to complete.
  • Regression testing in DEV and TEST environments to test feature updates placed significant demands on the QA team, and were a constraint on the pace of development.
  • Validating the customer’s experience on mobile devices and the primary browsers used by customers required a test lab of browsers and devices, which added to the cost of testing.
  • As The Vermont Country Store prepared for the holiday season, it also had to make sure that the online storefront could hold up to the increased holiday traffic without long page load times, which could drive online shoppers away.

Solution

The Vermont Country Store chose eureQa to automate its tests, and run them to ensure changes made to the online store worked correctly and did not affect user experience after updates are made to the live site. To accomplish this, eureQa helped The Vermont Country Store:

  • Automate its smoke and regression tests to quickly assess impact of changes made to the online store.
  • Run tests every time a new release is deployed to the TEST environment or a platform upgrade is applied, to identify and fix issues before going live.
  • Run automated tests on the primary desktop and mobile browsers used by its customers to guarantee a consistent and seamless shopping experience no matter how a customer chooses to shop
  • Simulate 2x the user load expected during the peak holiday shopping season, and assess the eCommerce platform performance to identify and fix any performance bottlenecks. eureQa generates load from Web browsers instead of at the protocol-level to measure performance of rich UI elements on the online storefront providing a true measure of customer experience.
 
The Vermont Country Store was able to build and run AI powered automated functional regression tests, cross browser/device tests, and performance tests on the Cloud with eureQa . The AI features built into the eureQa platform helped accelerate automation development, and reduced false positives thus increasing the reliability of tests.
 
eureQa helped us reduce the number of customer support staff involved in testing by half, so that they could spend more time helping our customers.
— Russell Glitman, eCommerce Manager, The Vermont Country Store

Benefits

eureQa helped Vermont Country Store make sure that its customers had a great shopping experience online, thus staying true to its focus on customer satisfaction through quality that has been the hallmark of its business since its founding.

  • Vermont Country Store was able to reduce the time taken to fully test its online store from more than a week to less than three days, a 2x savings in time. More importantly, it was able to reduce the number of customer support staff involved in testing by half so that they could spend more time helping their customers.
  • By testing on the most popular browsers and devices used by about 95% of its customers, Vermont Country Store was able to make sure that its customers had the best shopping experience.
  • By running performance tests on its online store prior to the holiday season, Vermont Country Store was able to start the peak shopping season with the confidence that its eCommerce store could hold up to the expected customer traffic. This confidence was borne-out as Vermont Country Store was able to get through a holiday season that saw an increase in sales, without a performance glitch.
  • By performing all these tests on a single platform, Vermont Country Store was able to reduce QA costs while continuing to ensure top-flight site quality.

Accelerate eCommerce Website updates

Bare Necessities is the largest online specialty retailer of women’s & men’s branded and designer intimate apparel in the USA. Bare Necessities seeks to differentiate itself by providing an easy, secure and enjoyable online shopping experience; in a market dominated by the largest retailer of lingerie - a company with greater name recall and deeper pockets. The key is a continually rich and engaging digital shopping experience that provides expert guidance as good as, or better than that available in-store.  The Bare Necessities website is updated daily with the latest product catalog, special sale and discount pricing to help achieve its unique selling proposition of a seamless online experience for its customers across touch points. 

At Bare Necessities, we use eureQa to build comprehensive test automation to run tests on a myriad of browser/device combinations. The results from these tests are available within hours instead of days as it was with manual testing. Using eureQa, we are finding, validating and fixing a greater number of critical issues than we were before, and we are able to address these critical issues earlier in the project cycle. eureQa is key to our ability to hit our dates and launch with confidence, knowing we are stable.
— William Saccone, Sr. Director, Quality Assurance
 
 

Challenge

As a leading online provider of designer intimate apparel, Bare Necessities had to make sure that its Web site delivered a consistent user experience across all browsers and devices. To do this Bare Necessities had to:

  • Test its Web site on the top 10 browser/devices used by its customers
  • Test various combinations of promotion codes and discounts on a daily basis
  • Make sure that its testing could keep up with the daily changes to product catalog, pricing and promotions in addition to code changes
 

Solution

Bare Necessities used the eureQa Testing Platform to automate its regression tests used to test the impact of all changes to its eCommerce Web site. It was able to:

  • Run its automated tests in parallel on the top 10 browser/devices used by its customers to access its site
  • Use eureQa’s support for data driven testing to test promotions and discounts in its My Bag test suite
  • Track progress of its tests and trends in its Web site quality using the analytics and reports in the eureQa Testing Platform
 

Benefits

Using the eureQa Testing Platform Bare Necessities was able to:

  • Automate its cross browser and cross device smoke tests in under 3 weeks
  • Stay ahead by consistently testing against the latest browsers and devices used by its customers
  • Avoid hiring additional resources to perform the large amount of manual testing required to ensure that all features of its Web site worked consistently across all browsers and devices
 

TargetX reduces regression testing time by 85% with eureQa

TargetX is a leading CRM Company in higher education, reimagining the student experience through campus wide solutions that help colleges and universities build lifelong relationships with prospective students, current students and alumni. Built on the Salesforce.com platform, TargetX’s solutions can be customized for each department or school within a College/ University. TargetX wanted to incorporate testing into its agile development process and automate regression testing of UI and UX changes, across the several different browsers, devices and OS combinations.

Our solution is built on top of Salesforce.com. Our testing was good but we needed to do more. We included eureQa in our agile development process and completed our tests in hours rather than in days or weeks. eureQa allows TargetX to focus on test content rather than on tools
— Peter Schiano, VP Research & Development
 
 

Challenge

TargetX had adopted an agile development process for its Salesforce.com based CRM product for colleges and universities. Regression testing however, was still a challenge.

  • Manual execution of regression tests were time consuming
  • Automation of Visualforce pages with Custom Selenium Frameworks and HP-UFT was a challenge
  • TargetX wanted to accomplish automation with existing testing QA team members that were not programmers
 

Solution

TargetX used the eureQa Testing Platform to automate its regression tests within an agile development process. TargetX was specifically able to

  • Build test automation using its existing QA team of non-programmers
  • Automate Cross Browser Testing to ensure that its CRM system worked consistently across multiple browsers
  • Implement an agile testing process where tests were run automatically after a build was deployed using integration with CI tools
 

Benefits

eureQa provided TargetX with a number of immediate and tangible benefits. TargetX was able to

  • Reduce regression testing time by 85%
  • Run automated smoke tests as part of agile development process to identify bugs earlier in the development cycle
  • Test its online application product on multiple browsers daily to guarantee a great user experience for its customers
 

Healthcare IT company accelerates test automation 3x with eureQa

RightCare is a pioneering medical technology company whose proprietary technology (D2S2) was developed through a decade of research conducted at the University of Pennsylvania School of Nursing. RightCares evidence-based technology empowers nurses, discharge planners, and case managers to revolutionize the care transitions process at leading hospitals.  Customizing its technology to meet the needs of the hospitals it is being implemented at required rapid changes to be made to the software platform.

eureQa makes it easy to automate regression tests so we can continue to focus our resources on innovation and delivering a quality product, not manually testing or building our own test automation framework. We would never have been able to take on test automation in such a short time and with such little investment without eureQa
— Mrinal Bhaskar, CTO, Rightcare Solutions
 
 

Challenge

RightCare Solutions’ SaaS application suite was changing at a rapid pace adding new functionality and customizations for new customers (hospitals). To keep up with this change RightCare had to:

  • Make sure that the new changes and customizations did not break existing functionality for its customers
  • Test code changes in an agile environment without increasing its QA team size
  • Leverage its existing manual testers to build test automation
 

Solution

RightCare used the eureQa Testing Platform to perform regression testing on its SaaS product suite.It was able to:

  • Use its existing manual testers to build and maintain test automation without programming
  • Incorporate regression testing within its agile development process
  • Run automated tests daily to identify regression errors
 

Benefits

Using the eureQa Testing Platform RightCare Solutions was able to

  • Build test automation three times faster than programming with Selenium
  • Avoid having to hire expensive programmers to automate tests
  • Identify regression issues on a daily basis and improve team velocity
 

Accelerate Testing of Contract Management Software

iContracts is a leading provider of contract, compliance, and revenue management solutions for the Life Sciences, Healthcare, and Education industries. Contracts handling is a complex process involving compliance with ever changing government regulations, changing contracting models and handling multiple vendors.  iContracts’ software solutions tackle these complexities through multiple logical and lengthy workflows 

eureQa has enabled iContracts to automate our regression test plans and execute them more frequently across the multiple browsers and versions that we support. The immediate feedback from execution of our test plans has helped our team identify and fix any issues faster. eureQa has improved our team productivity and helped us roll our product features faster, and with greater quality
— Erik Sieverding, CTO, iContracts
 
 

Challenge

iContracts has development teams in multiple countries working on its products building code multiple times a day

  • Regression tests had to be run after each build to ensure that the no existing features were broken by the new code
  • The frequency of builds made it impossible to perform manual regression testing
  • A custom automation framework (based on Selenium) was expensive to build and keep up to date
 

Solution

iContracts used the eureQa Testing Platform to automate its regression tests:

  • Run the automated tests after each build, multiple times a day
  • Identify issues early to eliminate bugs creeping into production code and causing rework
  • Integrate test execution with DevOps tools to implement an agile testing process
 

Benefits

Using the eureQa Testing Platform iContracts was able to:

  • Test its software 10x faster than manual testing
  • Improved development team productivity by reducing rework caused by broken builds
  • Improved customer satisfaction by eliminating bugs creeping into production deployments
 

Reduce production downtime window for updates by 75%

Weight Watchers International is a US company, that offers various science-driven products and services to clients and assists in weight loss and maintenance, by helping them form helpful habits, eat smarter, get more exercise and provide support. It operates is 30 countries, and provides clients customized plans, with diet charts and exercise regimes that meet their specific lifestyle needs. It regularly updates localized content on 15 language sites and pushes localized content that is accessed by customers based on their geographic location or zip code.

eureQa has worked out very well for us at Weightwatchers. eureQa has allowed us to quickly create automated test scripts and deploy them in Dev, QA and Production environments, across a dozen markets (A single script in effect is used to test 60 different contexts). eureQa has enabled us cut the time taken for post-deployment verification from 2 hours down to 25 minutes. We are saving time and money across multiple teams as we compress the windows needed for deployments.
— Sridhar Narayanan, Director QA & Digital Product Engineering
 
 

Challenge

Weightwatchers publishes new and targeted content daily in over a dozen different language sites all over the world. With frequent changes to these sites Weightwatchers has to ensure that this content update shows up correctly. Weightwatchers had to:

  • Verify that the changes to the multi-language sites showed up correctly on all sites
  • Content was rendered correctly on all browsers and mobile devices
  • Verify impact of changes across all sites, browsers and devices within minutes of the change being made
 

Solution

Weightwatchers used the eureQa Testing Platform to test the content changes on its foreign language sites. It was able to:

  • Build test automation that could be run across all its foreign language sites immediately after the content was updated
  • Use data driven testing to test for updated content in multiple languages
  • Run tests in parallel across all browsers and devices for all foreign language sites
 

Benefits

Using the eureQa Testing Platform Weightwatchers was able to:

  • Reduce the amount of manual testing that was required in the middle of the night when the content was updated
  • Reduce the time required to test content updates
  • Increase accuracy of content updates and improve use experience