By: Richard Pastore

February 1, 2003, CIO Magazine

Enterprise Value Award winners must run a review board gauntlet and pass muster with five picky CIOs before they can claim the enterprise value mantel.

The Enterprise Value Awards judging process is a demanding one. More than 80 organizations entered their systems in the 11th annual CIO Enterprise Value Awards competition by the May 15, 2002, deadline. Beginning in June, CIO editors screened the applicants. To make it to the level of finalist, the systems had to have been deployed for at least two years, and the applicants had to satisfactorily demonstrate the benefits of those systems for the entire enterprise. Out went the applications that simply ported activities over to the Web or reduced inefficiency by automating manual labor. By September, the field had been narrowed to 11 finalists. Then things got tough. The finalists had to face the review board; each company was visited by a member of this group to examine the system firsthand, to validate the data and to probe. "The review board members are trained researchers, have seen quite a few systems and applications, and as a result, they ask the tough questions to really find out about the enterprise value the applicant is getting from the system," says Richard W. Swanborg, president and founder of research group ICEX and Enterprise Value Awards cochair (with CIO Editor in Chief Abbie Lundberg). A review board member interviews the business sponsor and the IT sponsor, talks to users, and contacts customers and users outside the organization.

"What impresses me is, of course, the business value, but also the passion the people I'm meeting with have for the system, and their ability to articulate the business value," says review board member Madeline Weiss, president of Weiss Associates, a management consultancy in Bethesda, Md.

"The critical thing I look for when I'm doing a site review is the quality of the partnership that exists between the folks in information technology and the business," says Jim McGee, a consultant and clinical professor of technology and electronic commerce at Northwestern's Kellogg School of Management. "[In contrast,] I spend very little time worrying about the details of the technology."

Judgment Day

On Sept. 17, sequestered in a Boston hotel conference room for a day that stretched from 8 a.m. to 4 p.m., five CIO judges listened to six review board members present the details, strengths and weaknesses of the finalists. Cochairs Lundberg and Swanborg facilitated, clarified questions of criteria and made sure the judges observed the ground rules. For example, the enterprise value needed to be in sync with the core business of the organization, not something tangential. And the judges were counseled that when assessing each finalist, they should not rank any particular form of value -- be it financial, operational, transformational or societal -- higher than any other. "The finalists either have to be good at delivering several [types of value] or exceptionally good at one to win the award," says Swanborg. "That's what the judges have to determine."

Pushing Back

Judges pushed and probed, challenged each others' assumptions and asked the review board for clarification. "It's a solid B2B application, but so what?" asked one judge about a finalist's online customer service system. "They're treading water in a competitive industry," said another judge. "This is just ensuring survival." Added another: "If there was more growth that resulted from superior customer service, they could claim that as enterprise value. But there's no evidence." The vote was thumbs down for this one.

Eventually, the judges agreed on five winners plus two honorable mentions that were too strong to pass up. The judges were deliberately tough on the finalists and each other for the betterment of the award. "If one emerges as a winner, obviously, it's a remarkable accomplishment, and the organization that does win should be quite proud," says John Glaser, vice president and CIO of Partners HealthCare System in Boston, a veteran Enterprise Value Awards judge and former award honoree himself. "For the rest of the industry, the winners are people we can learn from and emulate. They represent why we're all here, which is to advance the ability of our organizations to be as effective as they can possibly be."

Application forms for the 2004 CIO Enterprise Value Awards will be available April 1 on our website at www.cio.com/eva. The deadline for entry is May 30, 2003. For more information, contact Special Projects Manager Lynne Rigolini at lrigolini@cio.com.far left Awards cochair Richard

Swanborg facilitates the judging process. left The five CIO judges listened to six review board

members present the details of the strengths and weaknesses of the finalists' systems, then debated the outcomes, in a meeting that stretched from 8 a.m. to

4 p.m. Right Judge John Glaser argues the merits of a finalist.


The Judges

Carolyn T. Purcell
CIO
State of Texas
Austin, Texas

John Glaser
Vice President and CIO
Partners HealthCare System
Boston

Doug Barker
CEO
Barker & Scott Consulting
Washington, D.C.

Former CIO, The Nature Conservancy
Rebecca Rhoads
Vice President and CIO
Raytheon
Lexington, Mass

Review Board

Kathleen F. Curley
Research Professor, Information Systems Department
Boston University School of Management Boston

Susan H. Cramm
President
Valuedance San Clemente, Calif.

Jim McGee
Clinical Professor of Technology and Electronic Commerce
Kellogg School of Management
Northwestern University Evanston, Ill.

Madeline Weiss
President
Weiss Associates Bethesda, Md.

Bob Reck
President and Cofounder
Kendall Consulting Group
Sarasota, Fla.

Stephanie Watts
Assistant Professor, Information Systems Department
Boston University School of Management Boston