Friday, May 22, 2009

Testing Einstein !

Celebrating a landmark experiment

http://tinyurl.com/qxm8bl14

"What was arguably the most important scientific experiment of the 20th Century, happened in a slave plantation on a remote African island."

"In 1919, the Royal Astronomical Society (RAS) launched an expedition to observe a total solar eclipse from Principe, to prove or disprove Einstein's theory of general relativity."

So even Einstein's theories needed independent testing.

While we have not yet been asked to write a methodology to test a claim as groundbreaking as the theory of general relativity we are often asked to test some strange products and concepts, from one-way encryption devices, to crayons, international passports and cranes.

The best projects we work on start with the query ; "this may sound like a strange request but do you test .... ?"

Matt Tett
Enex TestLab
http://www.enextestlab.com/

Monday, May 18, 2009

Court Ordered - Code Reviews

Buggy breathalyzer code reflects importance of source review

http://tinyurl.com/raaklg

"Court-ordered source code reviews of the software that is used to power a breathalyzer found bugs and significant technical deficiencies. The results raise serious questions about the reliability of such devices and demonstrate the clear need for source code review."

"A court-ordered audit of the source code that powers a breathalyzer machine has uncovered serious bugs and technical deficiencies. The professional code reviewers contend that the software is far below industry standards for quality and that it contains programming errors. The results of this review have raised serious questions about the viability of such devices as a law enforcement tool."

Interesting to note that the plaintiff and defendent both commissioned seperate test labs to perform independent audits/review of the code used in this device and the results were quite varied. Apart from the individual labs being contracted by seperate sides in the case it appears that the brief/scope each was presented with differed. This demonstrates the importance of getting the product brief/scope correct via thorough requirements discovery. This is something Enex TestLab regularly runs across in many of our clients testing projects. Clients simply have not spent enough time or effort seeking the knowledge and capturing their requirements before proceeding to development. This causes confusion for their developers or integrators and ultimately angst for the client when it appears the project is going in the wrong direction.

At the end of the day the owner of the product under development needs to interact with their ultimate end-user of the technology, be it a website or a router!

Then to ensure that they are on track they need to have an independent party testing through each stage of the development lifecycle to ensure that the project is going down the right path and that the product will fit with the end-user when it goes to market.

The moral to this particular story is; If sufficient project planning and expenditure had been budgeted to allow for independent testing and code review prior to the device being released to market, and the advice contained in the test labs report heeded and acted upon, it would have saved to business significant future cost/expense in relation to the legal action which ultimately culminated in the need for the court to order this code review! Not to mention loss of goodwill and damage to the brand that inevitably resulted.

Matt Tett
Enex TestLab
http://www.enextestlab.com/