Global Quality Journal

Introducing The Concept of Operational Quality

Greg Margolin:

Every day we, as a society, depend more and more on information technology to perform our regular activities. From shopping to banking, to access to health care, education, and government, we rely on the smooth and transparent functioning of the underlying information systems. At the same time, the focus of many quality assurance functions remains, almost exclusively, on features and functionality. We, at Global Quality Partners, believe that the time has come to give the proper attention to a comprehensive, holistic view of the quality of systems. We call it operational quality.

So what is Operational Quality?

We use this term as the positive definition of a management discipline that deals with system aspects that historically have been called non-functional requirements, among which are: performance, availability, usability, security, scalability, effectiveness, and efficiency, just to name few.. Operational quality represents the major business value characteristic of an IT platform.  An IT platform with high operational quality is ready to sustain new volumes of business, is amenable to downsizing if necessary, and is able to securely protect the brand equity of the enterprise it is enabling. Such a platform represents a significant business asset.

In this newsletter we will bring you information about how to achieve high standards of operational quality for information technology systems. We will do it by bringing you information about practical applications, tips, techniques, education and the best practices in this area. Of course, we do not have all the answers. This is supposed to be a two-way conversation. We encourage you to share your best practices and favorite tools with the community.

Please send us your input and feedback to: info@gqpartners.com.

In the October 2010 of our newsletter, as the first installment, we will talk about the practical application of operational quality, share a business case of integrating performance testing into continuous development process, and discuss practical approaches to code quality verification.

(Greg Margolin is a Managing Partner of Global Quality Partners)

posted by: Greg Margolin

The Case for Code Quality

David Alves:

What is Code Quality, and how can it be used within an enterprise in ways that justify the expenditure required to procure the necessary tools?

Code Quality tools come in mainly three flavors:  Static Analysis tools, Dynamic Analysis tools, and Monitoring Tools.

Let’s begin by discussing the tactical capabilities of the various flavors of Code Quality tools, and then we will turn our attention to the more strategic, practical applications of these tools.

Static Analysis tools typically have the following tactical capabilities:  automated code reviews, automatic unit test generation, identification of code paths for testing, identification of probable bugs, security violations, performance issues, stability and integrity issues, poor maintainability, and more, all without executing a single line of code.  In addition, Static Analysis tools typically have a means by which to categorize the nature of the issue found and to rate its severity, so that the user can set appropriate priorities in dealing with these issues in accordance with their own business goals.

From a tactical perspective, Dynamic Analysis tools allow you to analyze your software as it executes, enabling you to trace execution of business processes, objectively trace and evaluate test coverage, validate branch, path, and/or multi-conditional code execution for regulatory compliance or high risk applications, trace data migration across code boundaries throughout an application, perform interactive, ad hoc unit test execution, and perform exception and dump file root cause analysis, just to name a few capabilities.

Tactically, Monitoring Tools allow you to do any combination of the same capabilities as Static or Dynamic Analysis Tools, but they add the concept of trending over time.  Often they integrate into code management systems to allow continuous assessment, so that management may view trends in defects, defect severity, defect nature, assess developer aptitude, and much more.  These tools act as a Code Quality dashboard over the life time of a project.

From a strategic standpoint, Code Quality tools can be effective in a myriad of ways:

• Establish Service Level Agreements (SLAs) with Third-Party Development Partners:  Many organizations today subcontract development to one or more third-parties.  How do you know you are getting good code?  Is the quality of code presenting more problems than the benefits of outsourcing?  Are you using more than one vendor, maybe you can streamline the number of vendors and choose only the cream of the crop and save costs?  Code Quality tools can help you answer all of these questions, and allow you to set specific metrics that your vendors must adhere to (i.e. – no violations above a certain severity) as part of an SLA.  Of course these same SLAs could also be established with internal teams as well.
• Re-Engineering/Re-Factoring Optimization:  If you have a large legacy re-engineering project, Code Quality tools can provide metrics that tell you which code is most likely to have defects, which is most likely to break (or you are likely to introduce more defects) if modified, integration details, complexity of Integration, identification of dead and duplicate code, and code dependencies.  In addition, your organization’s most important business processes can be mapped to specific code execution and data flow, thereby allowing you to prioritize and/or plan an iterative move to next generation code.  This means that you can operate with the precision of a surgeon and focus only where you need to, prioritize with intelligence, and optimize/manage the costs associated with such a project.
• Merger & Acquisition Systems Integration Optimization:  When two organizations merge, there are many duplicate business software systems.  Sometimes one is dropped for the other, other times, some parts of one system must be incorporated into the other, or systems must be integrated.  The abilities inherent with Code Quality tools to map business transactions, map data flow though transactions, and identify unnecessary code, can be instrumental in these processes.  In addition, these same tools can be used to verify business transactions and data flow once the code merger is complete.
• Code Support Infrastructure Optimization:  In today’s economic environment, capacity planning and management is more important than ever.  The code we write has a footprint; it requires some amount of processing power to compute.  Many organizations are now moving to virtual machine technology and many other solutions to maximize their existing computing infrastructure, in order to avoid additional costly capital IT expenditures.  Therefore, streamlining necessary applications to only the necessary code to get the job done is crucial, and can help to maximize existing hardware investments.  Code Quality tools can help in this regard as well.
• Targeted Development Staff Professional Development:  While measuring developer aptitude may not be popular among most developers, we are all human, and most of us want to better ourselves.  Code Quality Monitoring solutions, over time, can expose areas where particular developers may need more professional development and allow organizations to target personnel training with the precision of a patriot missile.

So, we started this article with a few questions; what is Code Quality? How can it be used within an enterprise in ways that justify the expenditure required to procure the necessary tools?  We have covered what Code Quality is, but what about the costs and their justification?

The costs of the tools mentioned above, depending upon the type of tool, and the size of the development organization, range from thousands to tens of thousands of dollars (complete solutions rarely exceed $100,000.00 for most organizations).  At these prices, without getting into detailed calculations, it is easy to envision an ample return on investment by any one of the potential practical applications discussed above.  All it takes is to reduce the costs associated with one bad third-party contract, to save time and resources on one re-engineering or code merging project, or to optimize the targeting of dollars to enhance team productivity and professional development.  If you simply think strategically about the practical application of such tools, especially over time, it is very easy to see that the justification of the costs associated with these tools is an investment with perpetual returns.

Code Quality Solutions offered by Global Quality Partners:

There are many tools in the Code Quality space.  There are a few best of breed tools that GQP works with regularly however.  These tools are:

• McCabe IQ (Static, dynamic, and monitoring for just about all new and legacy languages)
• AgitarOne by Agitar Technologies (Automated code reviews and Dynamic analysis in Java, as well as continuous integration unit testing)
• Hawkeye APOD and Hawkeye Monitor by Code Excellence (Static analysis, Dynamic Root Cause analysis, and monitoring in the SAP and Java space)
• dynaTrace by dynaTrace Software (Dynamic Root Cause Analysis for .NET and Java)
• Xceptance Load Test (XLT – Continuous Integration Functional and Performance Regression Testing for Web applications)

With most of these vendors, GQP has negotiated one-time, on-demand, Software-As-A-Service capabilities on a monthly basis.  This can give you a very cost-effective solution if you have a one-time need, or allow for a cost-effective paid for trial before an actual purchase.  If you decide to purchase after a trial, typically the vendors will apply some sort of credit for the paid trial.

Lastly, for those with SAP solutions, GQP is offering a free code or production dump root cause analysis for a limited time.  Please give us a call for further details.

(David Alves is CTO of Global Quality Partners)

posted by: David Alves

QA Process for Apache Java UDDI (JUDDI)

GQJournal:

In 2009 a team of jUDDI (the open source Java implementation of the Universal Description,  Discovery) sponsors -- Global Quality Partners, LLC. and Xceptance, Inc. developed a performance benchmark and excuted a number of stress and load tests, see detailed info --jUDDI-v3.0.0 Holds Its Own Under Load.

Below Kurt Stam, Technical Lead of Apache JUDDI project shares his team's experience of integrating performance testing into an ongoing development cycle.

Kurt Stam:

Short introduction to the jUDDI QA Process

We've recorded a short presentation about the jUDDI QA Process. We believe that there are three levels of testing:

Unit tests

• Functionality
• Regression of components

Integration tests

• System level functionality
• Regression across components

QA tests

• Long running and repetitive, simulating real user load (memory leaks, performance)

For the QA tests we use XLT, and it it's simplest form the XLT agent and reporting engine are run on the same machine as the jUDDI server. However at true performance load testing multiple agents are deployed on dedicated machines.

posted by: gqjournal

Operational Quality in Practice

David Alves:

Imagine if you will the design, manufacture, operation, and the eventual maintenance of two classes of vehicle; a Prius, and a Ferrari. From a functionality and features perspective, both cars have a lot in common, everything from rack and pinion steering to anti-lock brakes, and a whole lot more. Of course the Ferrari is designed for performance, scales to a higher degree of horsepower and torque, and handles much more stress upon its frame than the Prius. The Prius however is an extremely effective mode of transportation.  It is efficient, reliable, resilient, and requires far less maintenance over its lifetime.

Now imagine that before building the first Prius or Ferrari both engineering teams were given an extensive, documented specification of requirements for vehicle features and functionality that each should have. Each architect was told that for the Prius and Ferrari respectively, that one should have great car mileage and low maintenance, while the other should be fast and furious.  Beyond these non-quantitative verbal statements, not much more discussion or specifications of performance, efficiency, reliability, or any other qualities related to the operation of the car where ever made. In fact, imagine they planned to test the performance of the Ferrari before putting the first one on the lot, but due to budget overruns and lack of time, they decided to skip it.

Sounds absurd, doesn’t it?

Yet this is what we do in IT every day. In my 20+ years as a consultant in the field of software quality, this is the norm.

Now let me digress for one moment. I have also learned over the course of my years that WE QA Professionals come in two flavors; “The Academic”, and the “Practical Professional”. I, being self employed and a business owner for the majority of my life understand that business is number one, cash flow is King, and there is no business without revenue.  I also realize that a successful business is a system, or a series of systems that comprise a complete value chain, that requires discipline and process to manage and maintain profitability. Therefore, as a QA Professional, I believe a cross between “The Academic” and the “Practical Professional” is what is needed to produce business software systems that meet their targeted goals. I will even go out on the limb and say that I believe the slant needs to be towards practicality.

In addition to being practical, we need to be “aware”. Aware of the impact that various Operational Quality attributes have upon the systems we build, as they are often at odds with each other.  Being aware of these conflicts, and incorporating best practices for making tradeoffs related to Operational Quality, allows us to make intelligent decisions in line with our organization’s business goals (the sole purpose of our IT projects).  For instance, often if you make a system more secure, you may decrease its performance and user friendliness (usability) in the process.  Other times, it may be budget or time that limits functionality, security, performance, or any and all quality aspirations.  Being aware and practical allows us to make informed decisions about features, functionality, and operational quality in the construction of our IT systems; for the operational benefits we seek over the products lifetime may be realized post deployment, but they are first born in pre-production.

In upcoming articles we will revisit this topic and we hope to have much more detailed discussions related to decision support frameworks which are both practical, and bring awareness to process participants, and allow for informed, quantitative decision making from the boardroom, down to the engineering lab.  In the end, not only do you need to know if you are building a Ferrari or a Prius, you need to do it within the confines of budget, time, and other business factors unique to your organizations climate.

Until next time, Happy Driving!

(David Alves is CTO of Global Quality Partners)

posted by: David Alves

When Bad Downtimes Happen to Good Organizations

GQJournal:

The industry is ready for a comprehensive approach to managing operational quality. Here are a few recent incidents where focused executive thinking about comprehensive aspects of operational quality would have helped.

Digg outgrows its old architecture, reaches its capacitty limits, then pre-maturely transitions to new architecture. Read more on this case in an article at Data Center Knowledge, Digg Downtime Debacle Debated, 9/8/10

Does the recent downtime at social media hub Digg reflect challenges in deploying NoSQL databases like Cassandra? Or is it simply a case of a company launching a new site architecture before it was ready for prime time? [Read more]

Overdependence on an automated configuration verification. Facebook explained the situation in a post in a company blog on 9/23/10:

Early today Facebook was down or unreachable for many of you for approximately 2.5 hours. This is the worst outage we’ve had in over four years, and we wanted to first of all apologize for it. We also wanted to provide much more technical detail on what happened and share one big lesson learned.

The key flaw that caused this outage to be so severe was an unfortunate handling of an error condition. An automated system for verifying configuration values ended up causing much more damage than it fixed. [Read more]

JPMorgan Chase blames three-day outage on a problem with a third-party software component. Computerworld reported on 9/16/10:

Computerworld - JPMorgan Chase is trying to move past three days of problems on its online banking site with an apology and an explanation that seems to put the cause on a third party.

The bank's online site went offline Monday night and remained offline Tuesday. Service appeared restored by Wednesday, although there were some reports by Twitter users of problems.

The bank, in a statement posted online, said it was "sorry for the difficulties" that customers encountered, and said "we apologize for not communicating better with you during this issue."

At first, Chase simply cited a "technical issue" for the problem. It has since provided a little more information.

The bank, the nation's second largest, said in a separate statement that a "third party database company's software caused a corruption of systems information disabling our ability to process customer log-ins to chase.com." It added that the problem "resulted in a long recovery process."

Analyst Curt Monash of Monash Research, commenting on the bank's explanation, said: "Their ambiguity is unhelpful." [Read more]

The stakes in managing operational assets were never higher.

posted by: gqjournal