Who's Your Data?

When Yellowstone was commissioned by Ulysses S. Grant in 1872, it was the first wilderness to be set aside as a Natural Preserve anywhere in the world. For the first 30 years or so, it was managed as you might expect. It was left well enough alone, unmanaged, as wilderness generally is. In the dawn of the 20th century however, the park service was formed, and was assigned the task of preservation and management of the environment of Yellowstone.

Within the first ten years of their management of the park, the ecology of the park had been forever changed by the well intentioned yet incorrect assertions of the park service on how best to preserve the land. First they believed that the elk were about to become extinct, so they began the process of shooting and poisoning all of the wolves in the reserve, who were the natural predator of the elks. This resulted in an explosion of the elk population and another totally unexpected result. You see, part of the diet of the elk was in fact the trees that the beavers of the park used to make their dams, and the lack of those trees and the resulting dams threw into chaos the water management of the park. The beavers disappeared, the meadows dried up and the otters and trout vanished.

Common patterns of correction, followed corrections of corrections have followed for the last century at the park. Grizzly Bears have been protected, and then killed off, wolves killed off, then brought back, etc., etc. I find Crichton’s summary of the situation through one of the book’s characters particularly interesting,

Things are constantly in flux. Species are winning, losing, taking over, being pushed back…Ours is a changing world and if you want to preserve a piece of land in a particular state, you have to decide what that state is and then actively, and even aggressively, manage it…Any change hurts some plant or animal. It’s inevitable…There is no free lunch. (Crichton, pg. 534-535)

So what could this possibly have to do with your data and how you manage it? I would argue that an extended metaphor could be drawn from Yellowstone to the modern datacenter, in how the park was managed, and in turn how enterprises protect the intellectual property of their organizations.

Your datacenter is not unlike Yellowstone, being in fact a complicated and intertwined ecology of hardware, applications, and information, all of which is greatly and often inexorably interdependent. Not to mention in most cases, like Yellowstone, the landscape was not preplanned, but rose and evolved through necessity. Also in common with Yellowstone is the fact that changes to individual components have effects to the whole and there is absolutely No Free Lunch.

Let’s say that you want to increase availability in the environment. You’re going to need additional copies of your data, in different places. Whether it be though using tape backup or replication or versioning through snapshots, you’ve got to have full detached offline copies, and you need them in a format that is portable. You’re also going to need additional servers to stand in wait for application failover both on-site, and off-site, waiting and ready with the last good copy of the data to pick up where a failed server left off.

Now that you’ve gone and done all of the above, and have by positively impacting the availability of systems and applications created new issues and negative impacts to both security and performance. Additional copies of vital data in the wild is not an ideal scenario from a security perspective and copies mean making two or more writes of everything, not to mention network impact of data transfer and potential application slowdown from the confirmation wait on synchronous replication.

So now you go and plug those holes. You fix security through tight encryption, and you fix performance by improving your network links, and buying more and faster disks. Then the encryption dings your performance, and the pipes and disk you bought (which yanked money out of the budget you planned to spend for Compliancy Solutions) hasn’t resolved your performance issue. Next step is to then get the application component owners (Web, App, Database, Storage, Network) into a room and throw blame around until you come up with an idea of where the issue might be.

Then you go out and fix that problem and on and on etc.

How do you manage the ecology of the datacenter? You have to actively and aggressively manage it. You need tools that can root out the interdependencies that are not immediately apparent, and get a global perspective both on the individual resources and how they function as a whole, and you need it in real time. Whiteboard sessions, scans and audits won’t get the job done. Synthetic transactions won’t get the job done either, you need to get your hands on real production data.

You also need to simplify the equation by cutting down the total number of variables. The easiest and most economical way of doing this is by removing some of the limitations of the hardware regarding how it handles core infrastructure functions through standardization on one set of tools for key functions. One tool for backup, not one for windows and another for open systems, one tool for clustering, not one per OS, one tool for Application performance, not one per application tier, and one storage management tool, not one per storage vendor.

You also need one set tools for the threat analysis of the environment. Not one vendor to handle security from soup to nuts necessarily, but definitely have a solution in order to correlate the mountain of data generated from your perimeter devices as well as your endpoints in order to have perspective on your organization’s security position at any one point in time.

Finally, you need tools to ensure compliance across the enterprise. Not so much to ensure that you are relieving risk pertaining to costs associated with fines due to non-compliance to federal regulations, but more importantly to enforce the internal policies of the enterprise, and ensure that everyone is singing from the same sheet of paper, instead of just reading the sheet and discarding it.