Over the past 10 years, corporations have experienced a dramatic change in the way they store and process information. In the early 1990s corporate information systems tended to be isolated from one another.

Most data processing operations were performed on slow (by today's standards) mainframes that implemented proprietary technologies insofar as how data is stored, retrieved, and manipulated. Data was almost never exchanged between systems. If corporate computers were connected at all, it was through purchased bandwidth that was expensive, nonreliable, and slow. Computing power rarely stretched throughout the enterprise. In most cases, computing power never even reached out of the computer room. All of this was about to change.

As the years went by, bandwidth increased and got cheaper, computers became ever more powerful, and the Internet evolved into an accepted way to connect computers and computing devices. As the corporate enterprise embraced evolving technologies, the corporate computing infrastructure became ever more connected. Like the phone system a century before, this interconnectivity of computers led businesses to continue to leverage advancements in technology and apply them to core business systems. These advancements, mainly in the interconnectivity of computers, forever changed the way business was conducted

Many challenges are still left unresolved. Information technology managers are asked to reduce development costs, while providing the information infrastructure to enable the company to improve services, provide faster time-to-market, and increase availability of mission-critical information. The financial and technological resources of a company rarely grow at the rate of its computational needs. The demand and workload of core business systems grow, while the capacity to handle them does not. This, of course, leads to a capacity problem. If a company is global, a server hosting an e-commerce site may have the same workload 24 hours a day. On the other hand, a mainframe that handles a "traditional" order entry system may be worked to overcapacity during business hours, then run idle the rest of the time. Clearly, capacity planning presents a challenge to most organizations.

As businesses provide computing resources to handle increasing and more complex needs, often the results are systems that are hopelessly overloaded or barely utilized. Even with load balancing, servers that experience "spikes" in processor usage can barely be used the rest of the time - resulting in a large amount of potential computing capacity remaining untapped. The end result is a system that is both over- and underutilized.

The real question here is the allocation of computing resources. Investment in computing devices needs to be justified. In reality, most organizations have to find ways to do more work with available resources. Still, even if an organization were to invest in new infrastructure, this may lead to the inefficiency of purchasing highly depreciative hardware, often unique to each system.

A more efficient solution would be to find a way to maximize existing investment in existing IT infrastructure without compromising performance. This solution would have to handle dynamically changing computing workloads and immediately direct processing power to where it is needed. In fact, this solution would have to create entire computing environments, then reallocate or destroy them when no longer needed. The solution does not exist in the future - in some galaxy far away. It exists today and is called grid computing - the new frontier for the corporate enterprise. The rest of this article will address the future of grid computing and if it should be adopted in corporate information systems.

Overcapacity
When planning for capacity, many companies use a "peak provisioning" approach. Basically buying the computing power to handle "peak" demand. Most businesses will tell you that their computing systems are overloaded. Often the reverse is true.

At any given time it has been estimated that 75% of most companies' computing infrastructure is doing absolutely nothing. Think about your desktop computer and what it really does. Most of the time a desktop computer does nothing. When not busy performing a spell check, it happily sits idle. Now imagine the untapped computing capacity of 500 desktop computers at 3 a.m. at a corporate headquarters in New York City. Halfway around the world, a network server is swamped with requests. If this business was on a grid, the overwhelmed server ccould unleash the processing power of the sleeping desktops in New York. Grid computing is all about using capacity you never knew you had. The goal is more efficient use of enterprise computing power.

Grid Computing
The concept of grid computing is really simple: take arrays of computing devices and make their computing power available to each other. This design is analogous to how a print server directs documents to a printer that can handle (print) them the quickest. Computing grids work in this fashion. Jobs requiring CPU cycles are directed to the device(s) that is the best able to handle it. A grid is not limited to servers. Any networked resource that contains computational power, such as scientific instruments and other embedded devices, can be included. All devices work as a single resource, combining massive computing power whenever and wherever needed. Hardly any processing power is wasted. Grids can exist across an organization or across the planet. They act as seamless computing systems for the people who use them. Most users don't know (or care) which device or group of devices is performing the processing. As long as the job is processed quickly and correctly, it doesn't matter if the processing power came from Boston or Bangalore. Although not a totally new concept - they have been used in research applications for years - grids are quickly evolving into the business world. After all, if grids can be used to predict weather patterns, why can't they be used to predict the movement of equity markets? Or determine the best design for a car airbag? How about a new and improved George Foreman grill? Actually, they can.

Grids in the Business World
To the surprise of few, businesses are looking at how computing grids can help the bottom line. CIOs are salivating over the delicious prospect of exponentially increasing computing power while actually reducing overall TCO for all enterprise systems. As the total cost of ownership falls, it does not take as long to see ROI. In many cases this has been realized.

For instance, the insurance division of the Royal Bank of Canada has used computing grids to solve complex probability algorithms. The power of grid computing reduced a 2.5-hour routine job to just under 10 minutes. According to Royal Bank, the investment in grid technology was only a fraction of the (would be) investment in additional hardware capacity. Royal Bank was quick to leverage their newfound grid. In another application, Royal Bank was able to accelerate actuarial calculations from 18 hours to just under 32 minutes. A French bank, Société General, has embarked on a grid project of its own. Its commodity division hopes to speed up the pricing of financial instruments such as foreign currencies and options. These pricing algorithms are as lengthy as they are complex. The result is that pricing data would be generated for the market one hour ago. Société General will spread the job to over 500 of its computers across the world with the hope that its worldwide staff will get data to the desktop in almost real time.

Closer to home, agricultural kingpin Monsanto's Gene Discovery division has used grid computing to process up to 50% more genetic data per day. Monsanto sources estimate that having access to idle computational power costs only 10% of the investment of more traditional hardware. Monsanto adds that grid computing has cut new hardware purchases by 90%.

Current Grid Developments
In the business world, Ford Motor Company deployed a grid system employing 1,000 CPUs. The grid is being evaluated to determine if it can accelerate complex engineering algorithms. Likewise, IBM and Butterfly.net have embarked on a grid-based hosting service. This hosting grid will distribute the processing power of video games over a network of "server farms."

Most of the current developments and deployments of the grid are, not surprisingly, government or academic projects. The United States Department of Energy is working with IBM to connect over 3,000 processors, which would make this grid one of the most powerful in the world. Across the pond, Britain has started a similar project: linking super computers across the commonwealth to create a single shared computing resource. Back in the United States, The National Science Foundation has started a project named the "TeraGrid". This behemoth of a project will use its over 13.6 trillion calculations (per second) to model the Earth's climate and study life sciences. In medicine, the University of Pennsylvania is building a grid that focuses its collected power on digitally storing mammograms. By accessing resources on the grid, medical professionals can model disease patterns across the entire population.

Vendors have been busy getting ready for demand. Database giant Oracle is expected to ship its first grid-capable database and application server (both named 10g) soon. The application server and development IDE is patterned after its successful product JDeveloper and will feature Java as a development language. Earlier, Hewlett-Packard announced that it has developed software that can link grids within its own computing architecture. IBM, although it claims no commercial grid software or clients, boasts about its research and involvement in the OGSA open standard.

Clearly, vendors are ramping up their knowledge of grid computing with expectations of the demand to come. With high-profile vendors jumping on the grid bandwagon, get ready for the marketing hype to follow. Yet again, prepare to separate the substance from the spin; the marketing jingles from reality.