Mastering Risk in Quality Engineering: From Detection to Management

Quality Engineering is the activity of uncovering and investigating risks we often call bugs or defects. Our role as Quality Engineers requires communicating our discoveries to those needing these details to make informed decisions.

Everyday life involves revealing and investigating risks. We, too, make decisions based on our findings. What we learn and move on can benefit us or ignore it, leading to disastrous results.

I want to take us on a 360-degree journey into the world of risks. My goal is to sharpen your awareness of genuine risks and enlighten you about areas in which we unknowingly add risk.

Shift-Left Risks

Let’s begin with a risk context most of us are already familiar with. Often referred to as “Shift-Left,” these risks are introduced early in the product’s design phase. We can encounter these risks as early as a whiteboard session or even a discussion over lunch where someone dreams up a new product or feature on a paper napkin.

To master identifying shift-left risks, look for opportunities to embed yourself early into conversations. Pursue genuine friendships with the dreamers of your organization, those who influence new features and products.

Let’s address a potential “elephant” in the room reality. Some of you are anything but extroverts, so having this type of conversation may seem scary, maybe even terrifying. For a moment, step back and consider the following. Being in the right place and hearing a new feature brainstormed could allow you to virtually test the concept before a single line of code is written. The brain and logical reasoning of great Quality Engineers enable a sixth-sense insight into where defects lurk before most even know it’s there.

So, if you’re intimidated by developers, product managers, or even your mother-in-law, now’s the time to figure out how to overcome this challenge. Communication is an essential skill for everyone, but it’s a requirement for any great Test Engineer. Programs like Toastmasters or a public speaking class would be worthwhile. Improving this skill alone can lead to higher compensation and more influence in your company and the products built.

Shift-Right Risks

Have you ever considered that risk can enter the process after (to the right of) development? A “Shift-Right” defect can exist within the build or pipeline process. An issue here may delay or lead to the deployment process being flaky.

Many Quality Engineers lack understanding of what happens once the product moves on from their desks. For example, did you know it’s common for unit tests to be executed before code is compiled and sent to production deployment?

Shift-right defects can often be related to flaky build processes that experience unexpected synchronization issues or uncaught exceptions. Software dependencies may be missing, or incorrect versions could be at the root. Minimal or missing unit tests may give the impression that everything is a green light, ready to push to production. Yet, unit test coverage might be so minimal that it can’t identify genuine issues within the code.

Quality Engineers, consider reviewing the unit test and code coverage that will get kicked off during your build process. Use your Test Engineering skills to review Positive, Negative, and Boundary unit test coverage, giving insights into coverage gaps. Review unit tests, especially around exception handling, to ensure invalid input data is captured and handled well by method calls.

Another common shift-right defect could relate to infrastructure scaling issues. The product may work well in lower environments but lacks the necessary scaling for customer demand of special events such as Black Friday, one of the biggest online shopping days of the year. Ask your architects and infrastructure team how the system will handle higher loads at peak times and how these features were tested.

Shift-Down Risks

Shift-down risks are foundational problems that can impact individual team members and even an entire Quality Team. These risks come from the individuals (ahem) responsible for reviewing the product. Yes, that means Test Engineers (I’m talking to you and I) have the potential to be risks to the product if the team lacks capable members and its management lacks the capabilities to nurture and grow the team well.

Shift-down risks start with those responsible for overseeing and guiding the Quality Team. There is a great chasm between managers and leaders, which can be disastrous for product development, the company brand, and the end user of all our efforts. An individual could be tasked with overseeing the Quality Engineering Team out of sheer desperation to have someone do it. Hopefully, they were carefully interviewed with consideration for firsthand experience and qualifications. This individual will continue laying the foundation for hiring more Test Engineers to execute the work. If they lack good interviewing skills or have no experience in the craft of Quality Engineering, they could bring on team members incapable of good Quality Engineering practices.

It’s not uncommon for Quality Teams to have their roots in the Customer Support team. An organization grows enough that it becomes apparent someone should do some testing before a product is shipped. Individuals of all backgrounds find themselves with the title of Quality Manager or QA Engineer, yet may have little to no real-world experience in quality engineering.

Management leading a Quality Team that lacks real-world experience needs the training required to fulfill the unique requirements of Quality Engineering. Based on my experience, I’ve found managers of quality organizations must have more than a casual interest in the craft of quality. Because of the high integrity requirement necessary for Quality Engineers, if their managers are simply “overseeing people,” they might not pick up on the reality that good and bad testing look the same. Managers may set the precedent that good testing is getting done when minimal risk is actually getting assessed.

An unfortunate shift-down defect is a disregard for ongoing training of the Quality Engineers. The Quality Engineer and the Quality Team managers should own team growth. I enjoy finding those Test Engineers hungry for the craft of testing and encourage them to share what they’re learning with the rest of the team. The engineers best suited for this task possess the humility to share their experience, even when they miss something. Never underestimate the value of transparency in helping a Quality Team grow in maturity.

A specific shift-down defect that exists when it comes to Test Automation Engineers is the hiring of developers lacking test engineering experience. This costly mistake often leads to developing “Passing Tests,” not “Testing Tests.” Inexperienced Quality Managers often think that developing automated testing is the same as developing application software. A Test Automation Engineer with experience in software testing will include the necessary validations and insights to identify risks in the software being tested. Most developers lacking software testing experience will follow a basic path through an application (often called the happy path) that tests very little. Remember that good and bad testing look the same, and this pertains to automated testing as well.

Shift-Up Risks

Shift-up risks come from the top of the organization and can be some of the most expensive risks a company experiences. Shift-up risks stem from management and leaders lacking real-world domain experience at scale to guide healthy product and process development, refinement, and growth.

As a company grows, it needs experienced talent to address the new challenges of scaling, logistics, and a healthy culture. Individuals can get moved into a role they’re not qualified for, lacking the support and coaching necessary for success. When this happens, the initial shift-up risk now compounds itself as lesser-skilled talent influences even lesser-skilled talent. This problem creates a weakened structure from executives to directors and managers the further it descends.

Identifying shift-up risk before joining an organization is advantageous, preferably during your interview process. Ask questions regarding new growth and what new challenges are surfacing. Listen for a balance between internal promotions and the wisdom to hire externally where more experience is necessary.

Be on the lookout for management bragging about their accomplishments and the previous companies they’ve turned around. I encountered this scenario once and quickly realized the individual lacked humility and essential technical competencies, which plagued the organization’s technical success for years. I find shift-up defects require intuition that something is not quite right. You’re probably staring a shift-up risk in the face if you see overconfidence coupled with ego, pride, bragging, control, or fear.

Remember, shift-up defects are best to avoid if possible. You’re unlikely to be able to influence shift-up risks unless you can gain trust and influence with the individual. Those with the best chance to reduce shift-up risk are peers or mentors who can guide them toward humility, professional growth, and maturity.

Reflective Risks

This list wouldn’t be complete if we didn’t look honestly at ourselves and evaluate what risk we bring to the table.

Let’s discuss some specific reflective risks we may personally be introducing to our work.

Lucky Break: Did you get a lucky break from a friend or acquaintance that opened the door for your role in QA? While this may have led to employment, it also means you might have minimal experience in Quality Engineering. Consider getting your Certified Tester Foundational Level (CTFL) certification from ISTQB or ASTQB. Ask your manager if they will cover the cost of your certification test.

Knowledge Gap:

Are you moving into a new testing area like Mobile or API testing? Maybe you have worked in web testing but never tested other technologies. This knowledge gap should be something you address. If you’re moving into testing mobile apps, consider learning the basics of mobile development. Several years ago, I had the opportunity to move into API testing, so I wrote an API to understand the nuances. Recognize gaps and how you can better understand the workings from the inside out.

Imposter Syndrome:

Imposter syndrome is the belief that “If others only knew I was not an expert at…(fill in the blank). they would call me out for it.” Imposter syndrome is such a common phenomenon that almost everyone has experienced it at some point. It can lead us to not operate in the skills we bring to the table for fear of someone figuring out we are not an expert. An excellent way to combat imposter syndrome is to use a mind hack called “grounding.” Grounding is simply stepping back and being objective about the situation. For example, say you are struggling with using all the features of a spreadsheet application. You might ask yourself, “This spreadsheet application contains hundreds of features. Is it reasonable that I don’t know how to perform all of them?” Of course, you’ll not likely know every feature, so this grounding technique can help reduce and hopefully remove imposter syndrome from affecting you.

Confirmation Bias:

An issue common to all humans is confirmation bias. The basis of this problem is that we tend to be biased that our opinion is correct. Particularly for quality professionals, the ability to see beyond your confirmation bias is a worthy pursuit. When considering something from someone else’s perspective, you can objectively see real risks you might otherwise be blind to. Regularly look at an issue from someone else’s perspective to train yourself to overcome confirmation bias. You may disagree with the viewpoint, but you can understand it better by looking at it from their perspective.

Anxiety:

A growing problem in our world is anxiety. It has roots in fear and worry that we may be unable to fulfill someone else’s expectations. Our imaginations can make us think of the worst possible outcome if we don’t perform perfectly, finding all the bugs. Using grounding techniques, we can combat feelings of anxiousness by realizing very few things are perfect in our world, especially in technology and software testing. Anxiety can be encouraged by controlling and insecure individuals. I remember a specific time when a manager shared that I would be required to go through a test if I wanted to switch to another team. If you’re experiencing anxiety in your workplace, consider where that might be coming from. Use grounding techniques to diffuse the anxiety related to unrealistic expectations. Consider getting into a healthier work environment if your current one is associated with toxic and unhealthy individuals.

Codependency:

You may be asking, what’s a topic related to mental health doing in an article for Quality Engineers? This topic seemed appropriate because I have witnessed codependency surface in the workplace yet go unrecognized for what it was. We may be bringing this reflective risk into our work and be completely unaware of it.

When wrestling with codependency, we typically have a great need to be accepted by others. While we all like recognition for our efforts, someone wrestling with codependency may constantly be on the lookout for opportunities for acceptance and praise, no matter what it requires.

Codependency often results in ignoring coaching and correction because the desire to be liked and valued is so important. The good news is this condition can be balanced with a trained professional’s help.

Codependency often shows up as an inability to say “no” to requests or to recognize another’s boundaries. This risk can lead to accumulating a myriad of one-off undertakings and unnecessary work done with the conscious and unconscious longings to receive approval and recognition from others.

To recognize if you may be wrestling with codependency, consider if you’re having trouble saying no, even when you don’t have the time, experience, or responsibility to perform a task.

Seek out professional help if you have codependency traits. The results will help you better serve your work and team and greatly benefit your personal life.

Conclusion

While we’ve covered multiple risk areas, many others exist. Take this article as a starting point to consider other dimensions of risks. Please reach out if you find the 360-degree approach to risk useful in your quality and testing efforts.

Sincerely,
Greg Paskal