Dear Customer, Quality is YOUR problem too!

I usually tell this to my customers at the beginning of an engagement when we discuss product quality. After an initial silence, the response I usually get is, “What do you mean! I am paying you to make sure that my software is built right and just the way I want it. It’s your job to make sure that it works correctly. Why is it my problem!."

I then explain why it is their problem and what they can do about it!

Ensuring software quality has long been seen as the sole responsibility of the product development team - software developers, the QA team, infrastructure and network teams and the project management team. The product development team (whether it is in-house or outsourced) is tasked with understanding the customer’s needs and building a software product to meet these needs.

The challenge however has always been in understanding the customer’s needsand meeting those needs. An exercise that involves getting objective results (meeting those needs) to a subjective problem (understanding customer’s needs) has the odds stacked against it from the beginning. Meanwhile, Qualityas perceived by the customer is often a measure of how well those needs are met.

SDLC processes have been developed and have evolved over decades with the goal to help address this problem. Reducing subjectivity in defining customer needs has been a huge component of the ‘requirements definition’ phase of all SDLCs. Detailed requirements documents, functional and technical specifications, use cases, epics and stories are just a few of the tools that have been used to add objectivity to the customer’s needs. Agile methodologies have gone a step further and included the customer as a key participant in the development process in an effort to improve the understanding of customer’s needs.

While customers are willing to participate in helping with understanding needs, they also must be involved in determining what it means to meet these needs. Establishing Acceptance Criteria and techniques like Behavior Driven Development (BDD) and Test Driven Development (TDD) build rubrics, which can be used to drive tests to assess quality. These help, but don’t solve the problem completely.

The issue here is not with the process, but usually in the unwillingness or inability of the customer (or appropriate business stakeholder) in defining these rubrics with the level of detail that is actually meaningful in assessing quality. Another oft ignored component in this process is the test data. Using the right test data also goes a long way in establishing quality (meeting needs). This is an area where the customer’s participation is critical because they know their data the best. This is especially true for most current software systems where quality depends on a lot more than just source code. It depends on data, configurations, content, rules and usage patterns.

Development teams can do a lot to ensure that the software product they put out is of high quality, but they cannot do it alone. Customer Participation and more importantly Customer Accountability is critical to achieving high quality.

Have you talked to your customer’s about their role in software quality? How have these conversations gone? Would you like to share any of your stories?

A Walk in your Customer's Shoes

The Customer Journey on an eCommerce Site receives a lot of attention. Every site wants to understand what a customer wants or prefers and builds the site to suit that preference. The hope is that this will result in a purchase by the customer and an increase in revenue for the site.

Tools to develop customer journey maps and A/B testing tools to understand customer preferences are numerous. These help define the journey that the customer will make through the site and make sure that the path trodden is filled with objects of interest to the customer and devoid of experiences that will cause them to abandon the journey. They attempt to attract the right customer and interact with them in an appropriate manner to ensure that the customer will transact with the site.

In spite of doing all this, sites often find that they don’t see the results they expect.

So what are they missing?

They are missing the recognition that a customer’s journey includes a key aspect of any journey – the experience. Understanding the customer’s experience is important to ensuring a successful outcome of the journey, i.e., a purchase transaction.

There is an old saying: "You can’t really understand a person until you have walked a mile in their shoes”. If you really want to understand your customer’s experience on your eCommerce site, you just have to get in their shoes and make the journey through your Web site.

This journey will show you what the customer sees and experiences on your site. The personalized content that is presented, the navigation choices available, the promotions that are presented and the product choices that are suggested based on customer targeting rules or past purchases. The experience is also likely to vary based on the means (browser/device etc.) that the customer chooses to access your site. Given that the path taken by different customers through your site can vary significantly (in terms of content, personalization, access method etc.), so will their experience.

Understanding the variety of experiences on your site will help you close the loop between designing the customer journey and validating the impact of that journey on your customer to ensure a successful outcome. This understanding can be gained by testing the eCommerce site with a simulated user (synthetic testing) in a manner that captures the user journey through your site and your site’s response to that journey. Testing every possible variation (different content, personalization and navigation paths) of the customer journey and evaluating the response of the different systems/components that make up your site for all these variations requires what I call Deep Synthetic Testing. The depth in this testing is critical to gathering adequate and relevant data on the customer experience. Data that can then be worked upon by analytics and reporting tools to provide the necessary understanding of the customer experience and validate the journey.

What have you done to understand your customer’s experience on your site? What has worked for you and what has not? Would you be willing to share this?

The Wait is Over! Data Driven Testing is Here

We are extremely thrilled to announce the support for Data Driven Testing and Keyword Driven Testing in eureQa®.

You can now run your test scenarios in eureQa® against large sets of test data directly from the eureQa® Cloud. You don’t have to build any new tests. You can use the tests you currently have and enable them for Data Driven Testing.

You can do this without programming!

With this release we have also made a number of changes to the eureQa® workflows and UI with the goal of making it more user friendly and improving user productivity. You can learn more about the new features added on our Data Driven Testing page.

If you have not already signed up for eureQa®, sign up for a Free Trial today.