Software Testing/Quality related Magazines I used (or at least want) to read

I used (or at least want) to read the following testing related magazines whenever i get time to update my knowledge and to know about various perceptions of a concept/technique related to software testing & quality from various authors belonging to different parts of this universe.

The list is
Testing Experience
Testing Circus
The Testing Planet
Professional tester
Quality Matters

TEST

ST&QA

Please share me if/when you come to know about any such useful software testing/quality related magazines apart from the above ones.

Learning - Latent Defect & Patent Defect

Today's web surfing resulted me in learning these two terms-latent defect and patent defect. Here you go with the definitons/meaning of those.





Latent defect is a hidden defect which could not be uncovered during a routine inspection process.

The opposite of a latent defect is a patent defect.

Patent defect is a defect which is visible during a successfully conducted inspection.

Vedic Literature & Waterfall Model Literature

"As the food,so the mind;as the mind, so the thought;as the thought so the act" -Vedic Literature

"As the requirements,so the design;as the design,so the coding;as the coding so the testing" -Waterfall Model Literature

MUST? or SHOULD? or COULD?

We recently got a requirements document from a new customer where the requirements priority ratings were mentioned as follows



S.No Req Tag Req Description Req Priority Rating
1 R1 AA…AA.AA.AA.AAAA 9
2 R2 BB..BBB.BBB.BBBBB.. 7
3 R3 CC.CCCCCC..CCCC 3
4 R4 DDD…DDDD…DDDDD 3
5 R5 EEEE…..EEEE….EEEEEEE. 7
6 R6 FFF.F…FFFF..FFFFFFF..FF 9



The legend for requirement priority ratings was given as follows
9 :( MUST)A requirement/feature that is fundamental to the project or product
7 :( SHOULD)A requirement/feature that is strongly recommended for the product but is not critical
3 :( COULD)A requirement/feature that is nice to have in the project or product.

R1 and R6 are MUST class requirements/features; R2 and R5 are SHOULD class requirements/features and R3 and R4 are COULD class requirements/features.

We have derived a work plan and schedule and communicated the customers that R1, R6 will be delivered on so and so date first and then R2 and R5 features and then R3 and R4 features. Next day we had received a reply from customer that we want R2 and R5 first and then R1 and R6 for their planning purposes so as to get the things delivered to a third party organization to do some work on their side.

Now we are confused with the question that what is the meaning of requirements priority rating as mentioned in the above table? Then we have decided to reframe as Requirements criticality rating instead of requirements priority rating. This was communicated to the customer and then the above table got modified from the customers as follows with the explanation of the requirements release priority rating as follows
9: Release 1
7: Release 2
3: Release 3

S.No Req Tag Req Description Req criticality Rating Req Release Priority Rating
1 R1 AA…AA.AA.AA.AAAA 9 7
2 R2 BB..BBB.BBB.BBBBB.. 7 9
3 R3 CC.CCCCCC..CCCC 3 3
4 R4 DDD…DDDD…DDDDD 3 3
5 R5 EEEE…..EEEE….EEEEEEE. 7 9
6 R6 FFF.F…FFFF..FFFFFFF..FF 9 7


Lessons Learnt are:
1.Look for the words MUST, SHOULD and COULD
2.Don’t get confused with critical and Priority requirements.

Three Wise Monkeys (wise enough for testing?)

All we are well aware of three wise monkeys concept which tells the world the following principle: Hear No Evil... See No Evil... Speak No Evil. The monkey covering his ears hears no evil; the monkey covering his eyes sees no evil; the monkey covering his mouth speaks no evil.

Recently in my city there were some advertisement posters (as shown above) pasted in all public transport vehicles with a quote like LISTEN, LOOK and CALL. That advertisement is from a flat promoter organization seeking public to book the flats.

That poster reminds me Andy Glover‘s Whats-best-testing-tool-ever blog post where he listed out all human organs as the best testing tools.

If testers exhibits these behaviors of not hearing (listening skills), not seeing (being more observant) and not speaking (talking to the stakeholders and talking about the defects), are those are eligible for testing assignments? It’s absolutely no.

TEST IT LIKE YOU OWN IT

Chevrolet has been advertising the “Beat” with a new caption these days: "TEST IT LIKE YOU OWN IT” Today I had a chance to see one such car with that caption printed on its outer surface.

Took a snapshot of that car and went to my team who were as usual busy in their testing activities. Once they are done with their work I called two of my testers and then showed this picture to them and asked a question to them and that question is “What do you think about this caption- test it like you own it”? and told them that assume you are going to test drive this car.

Here are those answers give by those two different testers:

Tester 1: since this is a demo car just for test drive purposes, the person taking a test drive should be cautious that he/she would not make any damages/accidents while doing the test drive.

Tester 2: This is a demo car for test drive and the person trying test drive can enjoy the freedom of driving this car as per his wish. It may be something like he/she can try driving the car with highest speed, high speed and sudden breaking, zigzag turning etc.

On analyzing those two different answers it looks like tester 1 is constructive and tester 2 is destructive.

The same caption is being understood/taken in a different way by different people but it is not sure what Chevrolet’s real intention of using this caption. They want the test drivers to be constructive or to be destructive?

Do you blindly use this controversial caption “TEST IT LIKE YOU OWN IT” for your testing projects? Your views and comments are welcome.

Driver -A Tester?

I , playing the role of Test lead was speaking about testing progress rate and defect details to my testers group and they were giving me the details and as a result of this discussion it is found that defects count already crossed the earlier predicted threshold limit and still software testers are coming out with lot of defects .On further detailed discussion with my team I had concluded myself that if this trend continues then it’s very tough to complete the testing activity on time since the release date being just two days away from today.

5.00 pm at office

With all the necessary details and the data went to the development team lead and started explaining the current bad quality of the product and listened to him. He replied me like there is no need to worry about this current trend on the testing status and defects count and he said like we will release the product on the planned release date with whatever bugs could be fixed and verified since for him the release date is important and I got shocked on hearing this. For the testing group quality is the important aspect and at the same time it is a reality that release date is also an equal aspect to be considered as quality.

The discussion went for a long time …………..

Now the controversy between testing and development team is: which one is higher priority: Quality or milestone date (release date)?

5.30 pm at office

This is the daily office closing time and I got ready to pack up for the day and started catching the office bus which will be leaving the office premises by 5.40 pm and it will take almost an hour to reach my destination stop to reach my home. The bus as usual started leaving the office by 5.40 pm and I (sitting in one of the very front seats) am inside the moving bus with some other colleagues.

As usual there was considerable amount of traffic on the road and at some places little heavy traffic and I was realizing that the bus will not reach my bus stop by 6.40 pm.

6.30 pm inside bus

Suddenly a lady colleague (who used to get down at the bus stop where I will be getting down) inside the bus stood up from her seat and went towards the driver seat and started speaking with the driver and that discussion went as follows.

Lady: Hey Driver why you are so slow at your driving. Hurry up. Go-fast. I need to reach my home at least by7 pm today. Now itself its 6.40 pm. By this time we should have reached our bus stop.

Driver: Excuse me Madam I am driving as usual but today there is unexpected traffic here and there on the road.

Lady: No. you need to travel still faster so that I can reach my destination by 7 p.m.

Driver: Sorry Madam, There are chances that we may end up in accidents if we are too fast. It is my duty to safely land all the passengers including you on your destination stops and to me today even its ok to land you people at your destination stops with little delay.

At 7.00 pm

Bus reached our destination stop at 7.00 pm where I and that lady colleague safely stepped down from the bus and we started walking towards our home. The bus started moving away from our stop to drop other passengers at their destination stops


On my way of walking towards my home I started thinking about my bus driver and that lady colleague. The point came to my mind was though the bus was delayed by 20 minutes (reached the stop by 7.00 pm instead of 6.40 pm) we were very safely landed. As the driver told to the lady colleague we could have met with some accidents if the driver would have driving too fast.

To me it looks like bus driver is like a tester where the driver’s responsibility is to safely land the people at the destination bus stops on or before the scheduled time and the tester’s responsibility is to deliver a quality product on or before the milestone date.

Though delayed by 20 minutes I and that lady colleague reached our destination safely. I need to wait for 2 more days to see how my product is going to get released…with better quality on time/with some delay? or with not so good quality but on time? Keeping my fingers crossed!

Suggestion to get suggestions to improve current software testing practices

Are you moved to a new testing team within the same organization or to a new organization recently?If yes then I am sure you may be starting exploring or learning the current team/organization’s testing process. And by this time you would have noticed the usage of various software testing related jargon/terms which may be different from your past team/organization.Slowly you would have started learning the new team/organization’s testing terms and processes.


As a test lead or test manager you will be in a position to find out various hidden and open problems related to software testing terms, processes and other stuffs. I was also in that same situation. As a new person there in the team it was too early to create the personal rapport with the members of the testing group.Slowly I had started interacting in person with the people and tried collecting their view points and inputs on the testing process they were following. But people were little reluctant in answering my queries (may be due to fear/hesitation with a new person in place?)


Nothing worked out for me but my main task at that moment was to find out the inconveniences being experienced in the testing processes followed by the testing group.When called for the meeting for an hour and asked about their openings people shared only about their career related issues and not the real improvements needed with the current software testing process.


Then I decided to change my strategy to get suggestions from people in a different way as follows: Circulated a email to the entire group stating like a suggestion box will be kept between so and so dates at a common place where the testing group members can post their suggestions in writing on a paper and we highlighted in bold letters that there is no need to mention their name or employee no or project name or domain name in that suggestion paper.

As communicated the suggestion box was placed for a week and at the end of the week we have opened the suggestion box found that there were around 35 suggestion papers placed by the people.



The further course of actions we had done were self analysis of those suggestions; filtering out a few invalid ones; forwarding the valid ones to the QA group(Process group) for their analysis and to hear their decision to include those valid suggestions into day to day testing practice.


This way of getting suggestions by keeping suggestion box helped me a lot in my environment. This way may work out to you as well when you try but the results may not be guaranteed since it depends mostly on your people and environment.

Red-Green-Yellow –Status/Rating/Decision Indicators

We, software testers are the factual information providers to the decision makers.
Though the decision is going to be made by some other entity or department like project/product management group of your organization, we the testers have a very high responsibility there in helping them to arrive at a decision by providing our valuable inputs and feedback on the quality of the product.

Let us take a look at our day to day life on the road. To control the traffic flow we have traffic light signal system as follows.

Red-A signal to stop
Green -A signal to proceed
Yellow- A signal to proceed/stop with caution (get ready to start/stop)

The above RGY concept is being used in different organizations for different purposes and three such practices what we are following in the testing perspective are as follows.

Case 1: Status Indicator

Status---> testing progress status

Red- Progress is bad; high improvement is needed.
Green –Progress is good
Yellow- Progress is not good as well as not bad; Slight/little improvement needed

Case 2: Rating Indicator

Rating---> rating the inputs/deliverables delivered by the development team to testing team

Red- Things went bad.
Green –Things went good.
Yellow- Things went ok; not good as well as not bad.

Case 3: Decision Indicator

Decision---> may be product release decision or decision to move from an earlier phase to the next phase.

Red- No-Go
Green –Go
Yellow- Conditional Go

No-Go means not allowed proceeding, conditional go means fix some issues and then proceed with release.

If you have some other different purposes of using this RGY model/concept, Please share your thoughts and experiences.

A book titled “Managing the Test People: A guide to Practical Technical Management”

• One of the big mistakes: You’ve brought a problem to your manager without proposing a solution


• When you are looking at old performance review you don’t know who wrote it, so take it as input but not as the authoritative word


• Don’t exclude good candidates who might have similar experience but not direct experience with your product


• The test engineer position is not an analyst or test position. Engineers are expected to architect and write code.


• A person who doesn’t proofread their cover letter and resume isn’t going to proofread bug reports or test plans or test reports


• Job classification -->bit of information necessary to classify the job such as job title, job grade/number/salary range/experience. Job description-->this explains the roles and responsibilities and qualifications


• A lead is a person who has project and personnel responsibility. In some organizations leads never have personnel management roles, they have only project responsibility


• It’s not fair to add management responsibilities on top of someone’s technical duties without offloading some of their work.


• You can learn a lot from a bad manager about what you should not do!


• Time spent in meetings means time spent away from your “real” work.


• Everyone interprets information in terms of how it affects them


• Don’t take ownership of a problem that isn’t yours.


• Tester’s job is to provide factual information to the decision makers.


• Create an open and strong working relationship with your development peers: They can be your strongest allies or your worst enemies.


• Don’t break your own rules


• Keep meetings to an hour or less and surely not more than that


• Ten minutes should be enough time spent on a daily/weekly status report


• A natural leader will already be leading before the title change


• You are responsible for your people and their actions


• Project problems are the easiest to fix-personnel problem are the hardest


• Quitting and leaving a department you have built is one of the most difficult things you’ll do in your career
  

The list continues…..

The above points (a valuable lesson is the right word I can say) were taken from the book “Managing the Test People: Judy McKay: A guide to Practical Technical Management”.

In my 7.5 years of software testing experience I had learnt only a few of them mentioned above but so many of them were new since everyone’s career is an ongoing learning. Judy explained a lot of them with her 20 years of technical and practical expertise. Once started reading these points I got happy that I had learnt so many things with the expense of just 40 USD (Cost of the book) and a few weeks of time taken to read this book.

Thanks a lot Judy McKay

This is not only a guide to practical technical management. This is a guide to practical technical as well as people management. This is a very good book I can recommend to each and every tester, test lead and test manager in the industry.

Best Practices - Dialogue between Test Analyst (TA) and Quality Analyst (QA)

Here goes the conversation happened today between a test analyst and quality analyst regarding “Best Practices”

TA: Hi, Good Morning and I have news for you and it’s that I have identified a best Practice for our testing project we are currently working on.

QA: Had you started implementing that practice or just identified.

TA: Just identified and we are planning to implement.

QA: Oh No! Then you cannot call/claim it as a best practice.

TA: Why is it so?

QA: You had just identified a practice that was not yet implemented in a project and you had not experienced its good results of following it and hence you cannot call it as a Best practice.
Also is that practice you are going to implement a step or part of our organization’s QMS definitions/processes/steps/practices mentioned.

TA: No I had checked and it’s a new one.

QA: That’s nice. Should not have been already mentioned in our organization QMS.

TA: OK.Is there any other constraints?

QA: Yes. A new practice that is applicable only to a specific project team and could not be applied across the organization, is not a best practice

TA: Okay and Thanks for your nice explanation. I will come back to you after implementing this practice in my project with the evidences of its good results. Thank you and bye for now.

After few hours of this conversation I had asked myself what the lessons are learnt by me from this discussion and the answers are

1. Some organizations have best practices (can we say best practices or conditions/guidelines?) to claim some practices as best or not.
2. Without implementing and having the results, a practice merely cannot be claimed as a best practice. This is obviously true.
3. The best practices of a specific project do not work out for other projects within the organization since the context of projects differ. Then how this QA is claiming “A new practice that is applicable only to a specific project team and could not be applied across the organization, is not a best practice”. I really disagree with this point.

Best practices to claim practices as best—a good research topic for any IT guy to research further?

Verification and Validation (V&V) - Different views

Hereby I have collected a bunch of different views/ideas from various software testing related books (I hereby owe credits to the respective authors) and my industry colleagues/peers about the verification and validation concepts being followed in almost all software development/testing/quality departments of various organizations.Welcoming everyone in the software qulaity and testing industry to share your own perception on these terms verification and validation (V&V) so that we all in the industry can have common understanding of these terms/concepts.

Verification:

1. Verification is a Static testing technique/white box testing technique as the workproduct undergoes a review.
2. Verification is the review of documents/code etc (i.e. no real execution happens)
3. Verification is all about "Are we building the product in right way". (Checking whether a right process is followed or not during the development life cycle)
4. Verification is some sort of QA activity (i.e. Process oriented).It talks about processes,standards and guidelines.
5. Verification is "Defect Prevention" based.It checks the process attributes.
6. Verification is the comparison of specified and actual characteristics
7. Verification is the process of confirming that software meets its specifications
8. Verification is the Process of evaluating a system or component to determine whether the products of the given development phase satisfy the conditions imposed at the start of that phase .(Is the system correct to the specification?)

Validation:

1. Validation is a dynamic testing technique/black box testing technique as the work product is executed.
2. Validation is the real execution (give input and check the output so that real execution happens)
3. Validation is all about "Are we building the right product" (Checking whether a right product is made or not as per customer's need)
4. Validation is some sort of QC activity (i.e. Product oriented).It talks about the product.
5. Validation is Defect correction based (detection and control by correction).It checks the product attributes.
6. Validation is the comparison of expected and actual characteristics
7. Validation is the process of confirming that software meets user/customer's requirements
8. Validation is the determination of correctness of the products of software development with respect to the user needs and requirements. (Is this the right specification?)

Testers – Most disliked as well as most liked people in the organization?

In many of the organizations, testers are in fact disliked. This is because developers do not like their mistakes being pointed out by someone else. Tester becomes a bad guy once he/she points out a problem. So most of the development and testing teams fight with each other either on the screen or behind the screen.

Almost in all industries this is termed as “US” vs. “THEM” mentality. Software industry is not an exception to this concept.

Though it looks like the development team dislikes the testing team but internally they are in a position to like the testing team since they need someone to test the product what they have been developed. Without the assurance given by the testers they cannot release the product to the outside world.

At your work place even you can see some development team leads used to fight with you (the testing group) would approach you personally saying like “Hey please test so and so part in the product rigorously”. If he is very much confident in the development work done by his team then what is the necessity to approach the test team seeking to test it rigorously (even without his request testers do their job for what they are being paid)? This clearly reveals that though they dislikes at some point of time they will surely like the test team on few occasions when they are in need.


But as a tester, despite of liking or disliking by the development group, one should deliver his/her testing responsibilities to the best to uncover the defects.

According to me “US” vs. “THEM” Mentality should be present in tester’s mind but with a positive approach and not with an offensive approach against development group i.e. testers should work for the goodness of the project and not for the goodness of the developers.

Smoke Testing - Past and Future

The term “Smoke testing” is familiar to all software testing professionals. All software testing professionals should know the history how the term smoke testing coined. It is the term actually used by the hardware designing/testing professionals. Once they assemble the electronic circuit elements in a board named PCB, they will connect the power supply to the board and switch on the power supply to check whether the circuit works fine at least at a very basic level or the circuit burns out emitting smoke due to some issues. If smoke is not seen then the smoke test is passed.Similarly in software testing smoke testing is meant for checking whether the software is working correct at its basic level.

Smoke testing is normally done by the testing team to take a firm decision on whether the build can be taken for further detailed testing or can be rejected back to the development team with valid reasons and data.

Smoke testing is also known as Build Verification Test (BVT) or Build Acceptance Test (please note that user acceptance testing is different from build acceptance testing) or intake testing.

Now it’s my turn to share my own perceptions on smoke testing. Testing team do the smoke testing to validate the minimum stability of the software. I would like to have development team also to do a smoke test kind of stuff before they deliver the build to the testing team. This may look like a foolish thought as of now but there are chances in near or far future that someone will follow this practice quoting like its development team’s duty also to ensure a minimum stable enough build have to go to testing team for testing. At that time even these practices may get new terms like “Developer’s smoke test” and “Tester’s smoke test”.(or like Outgoing smoke testing and incoming smoke testing.Hmmmm..How addicted we are with this communication/mobile technology!)

Also on the lighter side, in hardware testing one can see the real smoke when smoke test fails but in software testing the smoke cannot be seen but can be realized at development team’s desk like these poor testing guys rejected our build in their smoke testing.

What is software testing?-Birth of a New definition?

Software Testing- A great term as well as a good profession for most of the young talented minds especially in India. So what is really meant by software testing? To this question there may be so many answers like finding bugs, indicating and hence improving the product/service quality, reducing the risks etc. Even someone will tell like Software testing is an art. Some others will tell like it is a science. So there are so many perceptions floating around about software testing. Why it is so and what is really software testing?

Here are some common answers we used to hear from people

1. Software testing is to find the defects in the product/application.
2. Software testing is meant to improve the quality of the product/application.
3. Software testing is to reduce the risks associated with the product.
4. Software testing is to reduce the risks associated with the business.

Now it’s the time to validate each of the answers mentioned above.

1. Software testing is to find the defects in the product/application.

So just finding the defects is enough to ship the product to the end customers/users? What has to be done once defect(s) are found out? Will you buy a product of a company whose advertisement quotes like "In our product "x" no of defects found out? Definitely not right? So then how can a person claim like software testing is just finding the defects. This may be true in the perspective of a third party testing company whose responsibility lies only with respect to finding the defects and reporting them to their client companies who will take decisions further on those defects reported. Even some junior testers in some companies claim like just finding the defects is the responsibility of them and further decision has to be taken by the team lead/manager/upper management.

2. Software testing is meant to improve the quality of the product/application.

If software testing is meant to improve the quality of the product then what would be the responsibility of the development group-Just coding with enough time they have and push to testing team creating tight deadlines? According to me testing group will indicate the current quality of the product so that development group or management can take further decision on shipping the product to the market. So responsibility lies with development group for product quality improvement. But testing group has to ensure that correct inputs are given to the development group on what ways product quality has to be improved further. So testers are quality indicators and not quality improvement responsibility holders.

3. Software testing is to reduce the risks associated with the product.

Software testing being the last part of the SDLC is seen as a phase where testers can take responsibility to reduce the risks of the failure chances of the product failures. Software testing itself is seen as a risk reducing tool/method in the software development life cycle. This is how we used to see from the past of software testing history. But in my perspective we as software testers can assess the risks/chances of risks of the product failures the customer place and will indicate the level/severity of the risks identified or expected. Again it is some other department/group who will take real responsibility in reducing those risks identified to the maximum extent possible. So software testing is just an enabler to reduce the risks of the product failures.

4. Software testing is to reduce the risks associated with the business.

Agreed that whatever business we are doing it has to succeed. In software business product has to be delivered with acceptable quality to the end customers who are paying for it. But how software testing alone can reduce the risks associated with the business. There are other entities like marketing, sales, finance, strategic/product portfolio teams associated with the business. Even if one or more entities fail to deliver their duties/responsibilities then the business of the organization may fail. So according to me software testing is just one of the enablers to reduce the risks associated with the business of the organization.

Then what would be the definition of software testing? According to me, Software testing is defined as “A method/tool to indicate the quality of the software by detecting and reporting the defects in the software application/product to the development/engineering group for the purpose of taking sincere and correct actions to fix those defects with the intention to reduce the risks of the product failures and hence to reduce the risks associated with the product business”.

So is this the birth of a new definition for software testing?
Finally, one can say like testing is not merely finding defects,its about mitigating risks.

Note: Erik Van Veenendaal in his book "The Testing Practitioner" describes testing as " THE PLANNING, PREPARATION AND EXECUTION OF TASKS TO ESTABLISH THE CHARACTERISTICS OF A SOFTWARE PRODUCT AND TO DETERMINE THE DIFFERENCE BETWEEN THE ACTUAL AND REQUIRED STATUS, IN ORDER TO MEET THE QUALITY REQUIREMENTS OF THE CUSTOMER AND TO MITIGATE RISK".