…doing my tests in a focused chunk, then never again!
Back in the dark ages, with scrum, we would do the bulk of our testing in a development environment. At some point near the end of the iteration, we would wrap everything up in a build, deploy it to a QA environment, and test some of the items again…enough to make sure they were deployed properly and played nicely with other critical features. Once in QA, we had to dig up previously executed tests, set then up again, and try to remember what we had previously learned about our tests weeks earlier.
With Kanban, we complete our testing in focused chunks. We still do the bulk of our testing in the development environment. But when we’re done with that feature, we deploy it (that same day) to a QA environment, and do any additional testing immediately. This is soooooooo much easier because the tests are fresh in our minds, our SQL scripts are probably still open, and other recent tools are all prepped and ready to go. When we’re done, we deploy to production and those tests can leave our minds (sometimes forever) to make room for the next Feature.
You said... "Once in QA, we had to dig up previously executed tests, set then up again, and try to remember what we had previously learned about our tests weeks earlier."
ReplyDeleteTo me, this means you are NOT documenting your test cases properly. You should not be trying to "remember' what you learned. Your tests should state exactly what actions to perform and what outputs are expected. Any change should be added to the test instructions. Any other knowledge gained should also be added to the test or new tests created. You cannot reliably duplicate your testing efforts if you are constantly trying to "remember" them.
But these are mostly throw away tests. That's an awful lot of waste to document all the details when it's easy to just do it in one chunk. That's the beauty of kanban. You don't have to document this detail or remember it.
DeleteFriends, Kanban has nothing to do with documenting test cases or not; in any case both of you're right. In Agile testing, you don't need to go full detail of the testing designing test cases, as in traditional methodologies, but it's required at least a certain level of documentation, good enough to meet some basic criteria such as repeatability, audit, and evaluation of testing.
ReplyDeleteFor this happen, you should at least run exploratory testing approach, in which you have the possibility to use test charters as a testing documentacional. This way, you can include certain relavantes testing information, indicating your testing mission (scope with main features and secundary features), relevant data set, restrictions and risk, issues investigation, impact analysis and every thing you could consider while you're learning, designing and executing your testing.
Additionaly, you can take important metrics for Agile Testing, as % Effort (Preparing Environments),% Effort (Design and Implementation of Testing),% Effort (Bug Investigation), and % Opportunities.
These metrics, among others, are useful to assess "how we're doing" in relation to the effort of testing / development. You can use SBTM as conceptual base to understand what we are talking about when I say, "Audit", "Repeat", "evaluate", but if these three conditions are not manifested, your testing is in trouble, or at least you're doing testing ad-Hoc, which is not very good to say.
Javo, great comment! Great advice! Yes, SBTM is cool. When done to the extent you describe it's still a bit heavy for me.
ReplyDeleteYes, we still document "test ideas" in our Kanban project. These "test ideas" are normally what I'm calling "throw away" tests, because we don't need them after the initial testing.
Regardless of the test detail one documents (e.g., detailed scripts with data, test fragments, charters), I believe there is still a lot of test-stuff temporarily stored in our brains, that never gets documented. This test-stuff may be a memory of performance, a notion of usability, a piece of data that was interesting in passing.
Kanban allows us to use this temporary undocumented test-stuff in one chunk. And I believe it leads to better testing.
The whole purpose of documenting testcases or test ideas were to not having to remember all of the functionalities and tests to be able to run by testers who dont have required knowledge of the functionality. Also in that case, do you all still maintain regression tests? as they are extremely useful. Incase one is not documenting the very small functionalities, at what stage do we have to create regression test plans? or we dont use regression tests in Kanban ? Please throw some light on these questions.
ReplyDeleteI wish I had read this thread in April, but I will make a couple of quick comments. First making comments as "anonymous" is lame. Identify yourself so we can have a real dialogue about testing. Second, reading between the lines of anonymous I would guess they believe in complete testing and executing every test case ever written (unicorns & Sasquatches). In a lean and agile world I would argue this is an incorrect paradigm. In a lean and agile world regression is executing the tests that are within context and intended to minimize risk. Third point is depending on the tool, kanban boards can be daisy chained together. The testing phase can be its own kanban board. Here is where you pull in the tests that you think are critical within the context of the portion of the system under test. Once the chosen test cases (ideas) move across the testing board, then the story moves to the next phase of the process, perhaps deployment. Sorry but one other point, documenting test cases can be as simple as abbreviated sentences in a mind map. Keep up the GREAT posts Eric!
ReplyDelete