As I struggle to find a tool that is current enough to automate tests for a Silverlight 4.0 app, my former QA manager and thoughtful testing buddy, keeps spouting the same message to me. Alex Kell keeps saying my first mistake was to let the programmers pick SL4 in the first place. We have tools that can automate SL2 and even SL3. Alex says, why not convince the programmers to use the older versions of SL.
His point being, if testing is really so important, then why shouldn’t we select our product’s coding technologies based on the ease of testing them? It’s that thing we always hear about called, “testability”.
When the initial SL4 decision was made, I sheepishly took Alex’s advice. I asked the lead architect if he could use SL3 instead. I explained how easy it would be to convert my existing automated test framework to drive SL3. He paused for two seconds, then said “no”, the SL4 bindings were superior and more favorable for our project. Can you blame him? I can’t. Of course, any team starting a greenfield project would want to begin with the latest supported code version.
So now I'm getting busy rewriting my framework using Microsoft’s CodedUI platform, which has SL4 support and is used elsewhere on my team.
In the meantime I’m haunted by Alex’s advice and wonder how hard I should have pushed. On the other hand, there is merit in adaptability. To be able to support your team with the best testing possible, based on their design choices…
…well that is something too.
What do you think?
8 comments:
Subscribe to:
Post Comments (Atom)
RSS
The ultimate goal at the end create a product that users find value under the constraints of time, money, and effort, where the value generated is greater than the three factors above. If not then there's no point in doing the project.
I would think the dev effort time/money/effort savings with the new SL4 would be a lot more than the time/money/effort savings from keeping it in SL3 just to run your old test apps. On the plus side you also get to learn and expand your skill sets with MS CodedUI.
... He paused for two seconds, then said “no” ...
I think he paused too long as I would've said "no" in a 1 / pi of a second :)
Ack! i'm the naysayer in this posting, and while I did say everything attributed to meal, I was speaking specifically of my experience with SL (any version) and responding to Eric's dismay (and also being a little glib).
If overall product value is increased by the decision, that's absolutely fine -- but it's hugely important that testers explain the costs of the decision so that the overall value of the product is accurately determined.
I think the right course of action was to point: "Using SL 4 would cost the QA team X days and would delay the project by Y days". With this input, your claim would have been more concrete and people would have more real data to work with.
Older "tried and true" technologies will always result in faster time-to-market and lower cost. However, obsolecence issues will crop up sooner, and enhancements will take longer, and follow on project are less likely to reuse the code base.
You also have to think of your software teams long term career and company. Frequently upgrading technologies (at least one every project) keeps the team learning, thinking critically, taking advantages of "best practices", and prepares them for a possibly unknowable future (resume enhancement).
While you may not want to be at the "bleeding edge" of technology, staying at the "cutting edge" benefits every aspect in the long run.
Choice of technology should be a business decision, guided by all the stakeholders - not just development, not just test.
Testability is important, and other non-testing needs are important as well.
My experience with trying to automate our Silverlight application was a nightmare because QTP (even with the patch) doesn’t work well with Silverlight at all.
While Silverlight was great for some things, our dev team found it lacking in many other areas. Luckily for me because QTP does work very HTML 5 and AJAX, which is what the developers decided to port the old code to.
So while the decision to move off of Silverlight had nothing to do with testing, to Alex point, my team will enjoy the benefits of time and cost savings attributed to my new ease in automating.
Omri,
"Using SL 4 would cost the QA team X days and would delay the project by Y days"
Yeah, I agree. I didn't make any statements like that because there were too many unknowns for me to determine X or Y with any confidence. But, you're right. I didn't really give them much to base their decision on.
Testing is important but to me the lack of SL4 test automation tools purely means you can't use those tools to do what you might've with SL3.
So as a tester if you have an existing test suite automated then it's your job to report this to stakeholders as to the expected impact to testing should you go to SL4, but if the stakeholder decides they still wish to go that way (SL4 offers various advantages to coders SL3 doesn't so can have valid technical basis, ie. out of browser support ala AIR) then it's your job as testers to accept this and just adapt.