looking for Python automation engineer, job description can be found at pico.net, I can be contacted to discuss the position at ed.cook@pico.net

But, Robin, we can't see *you*

somewhat effective - measure by the number of escapes to the customer.

middling effective: better in the process of development than after release, due to agile communic

Testing is a cost

Depends on the measure..

management wants to be sure they are getting their money's worth for paying testers

If testing productivity is used to evaluate individual testers, especially for performance reviews, I can definitely see why they would resist!

I know why testers resist: they think measures are crude and undiscriminating

I think testers resist being measured by irrelevant metrics

how many bugs are found is not a good measure unless you know how many bugs there actually are in the code

If testing productivity is used to evaluate individual testers, especially for performance reviews, I can definitely see why they would resist!

Many organizations/mangers are not clear how to monitor testing effectiveness

irrelevant metrics would be things like number of defects found, number of tests executed

not all bugs are equal and not all areas of code have the same number of bugs to be found

Management wants to test more with less time and resources, means invested in automation

Whether testing can be effective depends on the clarity of the feature requirements and specifications. Majority stakeholders need to be on the same page prior to the implementation in the development process.

10 or 20 times as much work? How does one measure this? :-)

Leader provide the what. Management provide the how

Managing is doing things right. Leadership is ensuring one we get the right thing done.

Code coverage analysis is one way of measuring test effectiveness. When it is done well, it helps testers focus on creating automation for integration level kind of verification.

PASS or FAIL

someone already suggested number of defects ESCAPED

turn around time..time to fix the defects and restest..

Code coverage, as suggested

Number of test passed

Number of tests failed (that how one finds defects!)

Security compliance?

Impact of defect. Phase defect detected. Phase defect created.

Definition of aaaaaaStory Done == Test Passed

Definition of Story Done == Test Passed

I like David's "you make it, we'll break it" challenge attitude

how many bugs exist

the quality of the code written

Effective: Was the test actually executed successfully, and provide results either way.

Most of security scans and performance tests have revealed deep problems in my experience.

Measure everything

But test case designs ent thru technical review to assure it was good, adequate test.

went

In Agile world: If you have a comprehensive set of test cases that were added based on each user story completion, you will not run into surprising defects after merging to the main line,

defects per lines of code is not a good metric...adding more lines of needless code will decrease defect-density.

Sorry, I have to leave now as I have to eat dinner and work again from 9 PM (prod deployment testing)..thanks..see you in the next meeting..Happy Thanksgiving to all!

which is why people tried to invent things like function points

( to Kenneth's point, not Sanjay's :)

🤣

the part missing is the cost to fix the bugs the customer sees which it significantly more

time spent fixing defects is time not spent developing new features

it’s not realistic to say that the number of defects missed is 0 at the final stage of testing

agreed

according to the table, in both cases, the users receive product with 0 defects

Ed, Robin may mean by "in Production" the defects caught by end users

(in which case, BTW, any defect not felt by any user may not really count as a defect)

well if that’s true, then this makes more sense

the effectiveness is really only known after the customer is using the product and having it meet their needs

Thanks Jim, that does make sense

We assume customer had input into what he/she wanted at the beginning of the project. So we test to that expectation instead of doing no testing and wait till he/she gets it to test use it.

more of them sooner

various studies have concluded that 60

5-80% of defects have a root-cause in requirements

great presentation Robin...thank you

Thanks Robin

When can you share the recording?

Thank you Robin, great topic, makes you think.

requirements should include performance

security should also be part of requirements