Global Quality Journal

Integrated Quality, Where Do We Start?

David Alves:

So, Integrated Quality™, where do we start. Well, despite approaching Agile practice from a QA Engineers perspective, we want to first say that we are big believers in Agile, and of a lot of the groundwork our Agile guru forefathers have laid down for us. This is a broad statement, as Agile can mean many things to many people. Of course we cannot talk about all approaches, all tools, and the associated domain nomenclature for each. So, for this journey, we have chosen to focus on what we consider the most popular or widely used paradigms. If we do not use your particular form of Agile, or your specific test tools, etc., it truly does not matter, as the concepts will be universal. While we may not provide step-by-step examples for your particular environment to follow, with a little bit of effort and learning on your part, you should be able to apply the concepts in any environment. If you really get stuck, drop us a quick email, we've worked with most tools and environments. If time permits, we will do our best to give you a quick, helping hand.

So, to be more specific in this endeavor, let me outline some examples of choices we've made for this discussion series. We will use "SCRUM" for our Agile model. For an IDE, we will use Eclipse(though personally I love NetBeans. In the future we may substitute the Spring Tool Suite for Eclipse as appropriate). Our build automation tool will be Maven. We will use Jenkins for Continuous Integration (though I am very fond of Bamboo). Java as our primary programming language (Greg might occasionally delve into Groovy, he loves it). Cucumber as our primary BDD/TDD framework, amongst other things, and finally, we will use Selenium for our GUI, cross browser testing. In fact, in the next few days to a week, I will put out an in-depth, step-by-step video that will show you how to setup all of these components, while I explain their use for those not familiar with them, and eventually show you how you can create automated acceptance tests that can be run in your Continuous Integration Environment, using Selenium IDE (no programming; maybe a little tweaking here or there). A few days after this, I will introduce a tool I have created (free of charge), that will allow you to manage these test cases you have created, and easily allow you or others to quickly create new test cases, using common, natural language constructs. Somewhere around this same time, my colleague, Greg Margolin will show you how you can write JMeter Performance Tests using natural language constructs, and incorporate these into your Continuous Integration Environment. So why are we starting with Automated Acceptance Tests? Quite frankly, this is the place where most QA Engineers are most comfortable when it comes to Automation. Automating at the GUI level. Therefore, we figured we'd start at a place that we are all familiar with, and use it as a platform to introduce many other tools that you will need to become familiar with (if you are not already), if you are to truly participate in the Integrated Quality ™ the Agile process.

Also, during the course of this journey, you will see us use a lot of Open Source. We love Open Source. However, there is always a place for commercial tools as well, and we have worked with most of the big names. So, as we progress on our journey, we will let you know why we chose certain tools, when and why we felt they were appropriate, and when and under what conditions certain commercial tools might be a better choice, and hopefully we will be able to demo these for you as well (assuming we can do so without violating any license agreements).

Getting back to our Agile forefathers. You will be hearing from us again very shortly, as mentioned above, so stay tuned. In the meantime, a great book that we reference regularly, and agree with and follow many of its tenants, is "Succeeding with Agile" by Mike Cohn. If you have time, and wish to pick it up between now and our next newsletter blast, read chapter 16 on Quality. More important however, pay particular attention to the part that speaks of the "Automation Pyramid". You can also find more on the "Automation Pyramid" here, by Martin Fowler. Contrast this with what we see much more often in practice, the "Automation Ice Cream Cone Anti-Pattern". Which pattern does your organization more closely follow?

(David Alves is CTO of Global Quality Partners)

posted by: David Alves

Is Software Quality Assurance Dead?

Greg Margolin:

Stormy Seas Brighton Beach by  Les Chatfield 

No, it is not. But it is in an exciting, promising, and potentially fateful transition. In this post I will try to outline the scope of both the problem and the opportunity, and will start charting the way (as we, at GQP, LLC see it) for navigating the uncertain waters of today’s software quality environment.
One can say that this is the worst of times and the best of times for software quality assurance, both as a functional discipline and as a profession. On the one hand, as IT and the software that powers it moves to the core of every business, the importance and the attention given to the quality of the systems has never been higher. As reported, President Obama spent many a meeting asking and worrying about the quality of Healthcare.gov. That is, our Commander-In-Chief, spending time from his busy schedule on the issue of software quality. Outages, security breaks, malfunctions and such, all that popular media is charitably calling software glitches, makes the front pages of papers and web sites. At the same time, business and the craft of software development is going through its own revolution -- agile development and devops. In the process, great new technologies, methods, and practices have emerged. To name just a few: agile development, agile testing, continuous integration, devops and continuous delivery. Great open source testing tools and testing frameworks like JMeter, Cucumber, Jenkins and many others were developed, improved, and made popular. Automated testing and continuous integration became part and parcel of the software development life cycle. Then what is lacking in this paradise?
To begin with with we are very far from a paradise. The fact that the Chief Executive of the country is spending time and knocking on wood while listening to the reports of key IT systems, or a major retailer changing its CEO over a cyber security breach, shows that with all the advances in software development, IT systems remain very vulnerable and often fragile. It is no secret that for years the main focus of rapid software development was a race to deliver to end users new and exciting features. Agile techniques turned out to be the right fit for a quick turn-around in producing new features. Product owners and developers that adapted agile could release new products really fast. Continuous integration and devops made frequent updates possible. But all these advances came at a price. Simplistic application of the agile methodology brought many places to quick death marches and mini-waterfalls cycles instead of real agile sprints. Core system improvements that would compete for resources with new features would invariably lose in priority, and be relegated into the “we gotta do it one day” category, thus contributing to ever growing technical debt.
The move to a new QA organizational culture -- away from standard, centralized groups and toward Google like "developer-in-test" came at a price. There is no doubt, that the move to automated testing as part of continuous integration is a great practice. Likewise, the slogan that quality should be everybody’s business is also very true. Nevertheless there is a lot to be said for great domain knowledge, the QA mindset, and a broad view of the system under test that was, and still is the best value contributed by the best traditional QA groups. Sometimes it seems that in a rush to make software quality everybody’s business, the industry is about to make quality everybody’s business in general, and nobody’s in particular. Getting rid of Software Quality Assurance as such, could lead to getting rid of software quality. This could be compared to a health care system getting rid of primary care physicians and entrusting the care of the patient exclusively to specialists. In such a hypothetical case, with many great doctors treating specific ailments with great attention but without coordinating with each other, you might end up with lots of success for specialists and a dead patient as a result. The matter of fact remains, that many automated test suites that are executed as part of integrated testing are not effective. Martin Fowler calls such tests “non-deterministic” and calls for regular work on cleaning and updating these tests. Alas, most teams do not follow such practice as part of regular testing. So called non-functional requirements -- like performance, security, and scalability are tested for, in most places, as a reaction to a crisis, outage, breach, data theft, of impending government regulation. While fire-fighting is unavoidable, it is not prudent or productive to rely on such measures as a matter of doing business.
Hence, the brittle, and unsatisfactory state of software quality today. With all the great progress, software today is neither reliable, nor secure. The Healthcare.gov web site could not be launched on time, cars are routinely recalled due to software defects, and massive data breaches threaten to take down whole industries out. Government shows its concern by issuing edicts, but compliance has a tendency to live in a separate world, mostly slowing the process and not really contributing to quality, security, and robustness of the systems. In other words to borrow from a question asked in a different context: if everything is so good, why is everything so bad?
In our analysis, there is a lot of promise and room for concrete practical improvement in the way we build and test our systems. Five years ago, we at GQP (Global Quality Partners) developed the concept of SIO -- Systems Investment Optimization. This approach was based on the notion that IT systems are among the most vital assets of a modern enterprise and as such, the need to be accounted for, developed, and protected in an effective and efficient manner. We saw the key in an integrated approach -- where quality of user experience -- the most visible and appreciated of system quality attributes, would be coordinated and integrated with other less visible, but no less vital attributes of the system, such as robustness, scalability, security, endurance etc.
The good news is that today with great advancement in tools, testing frameworks and techniques, an integrated quality approach could be implemented in an effective and efficient, practical manner. Such an approach that we call Integrated Quality™, and which we embark to discuss and develop on these pages, would focus on the best ways to integrate software quality assurance across teams and would recommend the best complementing methods, tools, and practices. The emphasis is on practical integration that would allow the industry to produce robust, and secure systems in an efficient and effective manner. In the course of this discussion we will explore questions about how BDD could help integrate teams, where QA fits in the Devops movement, how security and privacy, if designed from the ground up, could become an advantage instead of a cost, what are the best ways to incorporate performance testing, and how to do agile testing right and integrate it in an organization of any size and maturity. We see this as an exciting journey. We are inviting our readers, business associates, and colleagues to join us.

(Greg Margolin is Managing Partner of Global Quality Partners)

posted by: Greg Margolin

Angry Bar Candidates Sue Over Exam Software Problem

It looks like bar exam companies are doing a good job preparing lawyers for future software liability laws advocated by Dan Geer.

Nationallawjournal.com:

Less than a week after a software glitch sent bar-examination takers scrambling to upload their answers, disgruntled law graduates have filed two class actions against the company that provided the testing software.

Phillip Litchfield, a May graduate of Chicago-Kent College of Law, filed his claim against ExamSoft Worldwide Inc. on Aug. 4 in U.S. District Court for the Northern District of Illinois, seeking more than $5 million on behalf of bar-exam takers.

A day later, Wake Forest University School of Law alumnus Catherine Booher and Gonzaga University School of Law graduate Christopher Davis became name plaintiffs in a class action against ExamSoft in U.S. District Court for the Eastern District of Washington. They, too, seek more than $5 million in damages for the class.

“On the long list of things about which exam takers should be worried, wondering whether they will be able to turn in their exams for grading should be at the very bottom,” the Washington state complaint says. “It is hard to imagine anything more basic in an exam than being able to turn it in for grading.” (read full article)

posted by: gqjournal

Dead Men Drafted

It is never too late to serve your country. Or is it?

Cnet.com:

 The Y2K apocalypse may be hitting 14 and a half years late. And, a small town in Pennsylvania appears to be getting the first blow with demands for 14,215 deceased men to join the country's military draft.

A glitch in the Selective Service System sent out letters to the families of thousands of Pennsylvania men born between 1893 and 1897, according to the Associated Press. The letters ordered the men to register to join the US military or face a "fine and imprisonment."

That's a hard thing to do considering these men are no longer alive.

One of the people to get a letter was Chuck Huey, 73. The notice was addressed to Huey's late grandfather Bert Huey, who was a World War I veteran born in 1894 and died in 1995 at age 100, according to the Associated Press.(read full article)

posted by: gqjournal

Honda Recalls 175,000 Hybrids in Japan Due to Software Glitch

If you like high speed...

Autonews.com:

TOKYO (Reuters) -- Honda Motor Co. is recalling 175,356 gasoline-electric hybrid vehicles in Japan, including the Fit subcompact, over a software glitch in the engine control unit.

Honda is recalling all of the third-generation Fit hybrid as well as the new Vezel subcompact SUV hybrid, both of which went on sale last year and are sold only in Japan, the automaker said.

This is the fourth time that Honda, which is also embroiled in a series of recalls over airbags made by Takata Corp., is recalling the hybrid version of the third-generation Fit since it went on sale last September in Japan.

Due to a software programming glitch in the engine control unit, the vehicles are at risk of moving or speeding abruptly, Honda said. Eleven accidents were reported but there have been no reports of injuries or deaths, it said.(read full article)

posted by: gqjournal

About a Cat, Internet of Things, and Hacked Wi-Fi Routers

Welcome to the connected home.

Wired.com:

Late last month, a Siamese cat named Coco went wandering in his suburban Washington, DC neighborhood. He spent three hours exploring nearby backyards. He killed a mouse, whose carcass he thoughtfully brought home to his octogenarian owner, Nancy. And while he was out, Coco mapped dozens of his neighbors’ Wi-Fi networks, identifying four routers that used an old, easily-broken form of encryption and another four that were left entirely unprotected.

Unbeknownst to Coco, he’d been fitted with a collar created by Nancy’s granddaughter’s husband, security researcher Gene Bransfield. And Bransfield had built into that collar a Spark Core chip loaded with his custom-coded firmware, a Wi-Fi card, a tiny GPS module and a battery—everything necessary to map all the networks in the neighborhood that would be vulnerable to any intruder or Wi-Fi mooch with, at most, some simple crypto-cracking tools. (read full article)

posted by: gqjournal

Dan Geer: Liability Laws for Software Are Badly Needed

Dan could be very well right, as he often was on so many issues. More than 10 years ago he was not affraid to go against then omnipotent Microsoft in calling out the danger of a monopoly in computer systems. It is scary to think how much more worse off we would be, if the world did not use open source for IT infrastructure.

Darkreading.com:

Dan Geer Touts Liability Policies For Software Vulnerabilities

Vendor beware. At Black Hat, Dan Geer suggests legislation to change product liability and abandonment rules for vulnerable and unsupported software.

BLACK HAT USA  -- Las Vegas -- Software vendors will probably not rejoice in some of the security policy proposals put forth by Dan Geer during his keynote Wednesday morning at the Black Hat USA conference in Las Vegas.

Some of Geer's suggestions -- all reasoned and responsibly sprinkled with caveats -- are for legal measures that would push much of the onus of security onto those who develop vulnerable software; particularly those about source code liability, "abandonment" of software code bases, and vulnerability discovery.

One trouble, Geer says, is that users have no legal recourse if shoddy coding exposes them to undue danger -- making it wholly unlike other product defects. He quoted the Code of Hammurabi, written over 3,700 years ago: "If a builder builds a house for someone, and does not construct it properly, and the house which he built falls in and kills its owner, then the builder shall be put to death."

"Today the relevant legal concept is 'product liability,'" said Geer, "and the fundamental formula is 'If you make money selling something, then you better do it well, or you will be held responsible for the trouble it causes.' For better or poorer, the only two products not covered by product liability today are religion and software, and software should not escape for much longer."  (read full article)

posted by: gqjournal