Greg Margolin:
Politico.com recently reported that ever since HealthCare.gov failed to pass real life scalability test, President Obama took to a habit of knocking on wood every time he hears a bit of a good news about the site. Indeed the term “scalability testing” made it to board room and general media discussions. Shortly after the launch, it was amazing and amusing for software quality assurance veterans to watch media pundits and industry tycoons to finally expand on the need for user acceptance and scalability testing. Aetna’s CEO, Mark Bertolini, plainly stated in a video interview that prior to the HealthCare.gov launch, it was clear that the site was not even ready to start proper scalability testing.
The situation in numbers looked this way — HealthCare.gov was designed with the intention to scale up to 50,000 concurrent visitors. In reality the site could barely handle 10,000 concurrent visitors, while the load on the first day was around 200,000 concurrent visitors. So, what is scalability? The definition states that “Scalability is the ability of a system, network, or process to handle a growing amount of work in a capable manner or its ability to be enlarged to accommodate that growth”. In other words, scalability means the ability of the system (and system design) to handle an ever growing work load by adding appropriate resources. Scalability could be horizontal, as when additional nodes are added to the system, or vertical, as when the central node is being beefed up with additional resources. When the system cannot effectively handle more work, despite available resources, this means that the system hit its scalability ceiling. In software system engineering, scalability belongs to a group of awkwardly named quality attributes known as "non-functional requirements". While this group of requirements is defined by what they are not — akin to "nontechnical staff" or "wireless communication" — it has been proven to be absolutely vital for today’s IT systems. At the time of the HealthCare.gov launch, industry experts stated that problems with the site were not unique, just the opposite, they were all too common. In today's fast pace world, software development teams are all too often solely focused on launching a site by a certain date, and on making sure that core features work. Scalability (and often) security fall by the way side. Then, immediately after launch, a rude reawakening comes to light, when what should have been a welcome success — more concurrent users than originally planned — turns into a disaster capable of killing a company. The story of HealthCare.gov has a happy ending so far. The best engineers from American leading companies descended on the site and feverishly worked on making it scalable — by partitioning the site, moving it to the cloud, prioritizing work load, and testing architectural changes in the process under real life and simulated loads. So, today, six months and one Secretary of Health and Human Services later, HealthCare.gov is holding its own under current load. There are applicable and non-applicable lessons for Main Street businesses in this story. What’s not applicable? Not every Joe can call Google, Amazon, and Verizon, etc. and ask for immediate access to the best industry experts. Not every business can as easily sustain a sharp expansion in the budget allocated to their site the way federal government can. Does it mean that for Mr. and Mrs. Main Street’s e-Commerce web site there is no hope? Not at all. Thanks to open source software, everybody can have access to many of the tools used by industry giants. Google in its performance and scalability testing heavily relies on JMeter (an open source tool) and did a lot to make the tool much more robust and full-featured. The proliferation of cloud environments, pioneered by Amazon, allows for rapid deployment of test tools in the cloud and the creation of test environments on demand. Finally, there are also industry experts such as Global Quality Partners, LLC available to put all the necessary components in a properly packaged, affordable, scalability and performance testing service.
The lessons of HealthCare.gov are there for all of us in plain view. It is for us to learn from them, or to ignore them. Modern eCommerce sites should be designed with scalability in mind. For scalability testing there is JMeter and the cloud. And, of course, there is always knocking on wood. Just ask the President.
(Greg Margolin is a Managing Partner of Global Quality Partners)
posted by: Greg Margolin
Comments