With the release of the first beta copies of Windows Vista in midsummer, Microsoft Corp. began ramping up the testing process for the next version of its client operating system. But long before that, a special group of corporate testers had already started to help shape the new software.
Thirteen companies, including eight with at least 15,000 PCs each, began working with Microsoft on the architecture and design of their Windows Vista environments about 18 months ago, when the product was code-named Longhorn. In April, they got their hands on alpha code with a Network Access Protection feature they had agreed to test. Another 35 users joined them last May and June as part of Microsoft's Technology Adoption Program for validating the features, functionality and product-readiness of Vista, said Linda Apsley, a group program manager for Windows TAPs.
The TAP process differs from an ordinary testing program in that it requires participants to make a commitment to run software in live production environments prior to a product's final release. Microsoft runs three different types of TAPs across its product line: one for product validation that was formerly known as the Joint Development Program, a rapid deployment TAP that was previously called the Rapid Adoption Program, and a less rigorous product evaluation program that's more like an advanced beta for IT pros.
The software vendor used to have to do heavy recruiting to get companies to participate in the Windows TAPs, which started with Windows 2000. Now, Apsley said, there are eight to 10 applications for each open slot in the Windows Product Validation TAPs, and many participants ask to continue from program to program.
"Our budget is extremely tight, and we're always looking for ways to stretch our dollar," said Robert Taylor, CIO for Fulton County in Georgia. "Agreeing to [take part in] TAP meant we could leverage our work on testing to get more benefits out of our budget and be a better custodian of public funds."
The county's IT department joined the TAP process about two years ago when Windows XP Service Pack 2 and Windows Server 2003 SP1 were being developed. It's currently involved in TAPs for Windows Vista and Longhorn Server, plus Version 4 of Microsoft's Systems Management Server (SMS) software, Taylor said.
TAP participation means that some of the county's best IT employees get a chance to interact with Microsoft's "A-team" engineers, he noted. The county also gains access to Microsoft consultants, sometimes on-site, and the highest tier of product support the vendor has to offer. For problems that couldn't be resolved by telephone, Fulton County staffers have at times boxed up PCs and shipped them to the development team at Microsoft.
"We feel like they will not let us fail," Taylor said. But he added that he's still second-guessed by CIOs about his decision to get involved in a program requiring him to use software still being developed.
Microsoft recognizes that companies "take a risk to be in our program," Apsley said, adding that the company matches the risks posed by TAP participation with top-tier support -- responding within 15 minutes if a user experiences a serious problem. "My team is measured on how well they keep these customers' businesses in good shape," she said.
The IT shops that participate in the Product Validation and Rapid Deployment TAPs do have to pay a price by committing the time and resources necessary to test and deploy products on a schedule they agree to with Microsoft.
"It is a two-way street," said Les McCarter, director of IT infrastructure and operations at Hawaiian Electric Co. "With each new release of the beta software, we had to make sure we had it installed on x number of machines -- not just test machines, but machines that were running in a live environment."
The Honolulu-based power company started deploying Windows XP SP2 within its IT department as part of the TAP process and later extended the test code to users of a representative sample of its applications, as well to other end users who were willing to try beta software.
McCarter said that despite some concerns about running beta code on live systems, Hawaiian Electric was anxious to get the security-focused SP2 release, which it wanted to deploy simultaneously with Office 2003 to its 1,500 end users. With each new release of the SP2 beta code, the company retested its applications to make sure they remained compatible. At one point, when an application didn't run well with the firewall built into Windows XP, Microsoft postponed the release of a public beta so it could fix the problem, McCarter said.
"My technical staff was amazed that our reported problem contributed to a publicly announced postponement," he said. "We felt we made a mark on the final XP release."
McCarter said the effort was worth it for this particular scenario, but he's not sure if the utility will sign up for another TAP. "We look back and debate whether we would do it again, because it did require a fair amount of effort on our side," McCarter said. "If you commit to this, it forces you to dig deep into the technology. It's not something to take on lightly."
For some, the extra assistance is tough to resist. Gunnar Thaden, CIO at Tuev Nord Group in Hannover, Germany, said he was astonished to see Microsoft essentially copy his company's 40-server installation at its Redmond, Wash., campus for testing of SMS 2003.
During a week's stay there as part of the SMS TAP, Thaden worked with the vendor's engineers and SMS developers from 8:30 a.m. to 10 p.m. and saw numerous Microsoft employees work one night from 10 p.m. to 4 a.m. to fix a particularly nagging problem.
Tuev Nord, a technical services provider with a workforce of more than 7,000, is participating in TAPs for Windows Vista, SQL Server 2005, Office 12 and the Network Access Protection technology. Thaden estimated that his IT staffers spend an average of 15 hours per week on TAP activities. But he added that on some days, it might be just an hour, while on others, they might devote the entire workday to the testing program, he said.
Access to Labs
Siemens AG has been involved in more than 80 TAPs during the past seven years and is currently participating in a dozen. The electrical and electronic products manufacturer wants to make sure that any products it deploys will scale for its nearly 400,000 PCs worldwide, said John Minnick, an Alpharetta, Ga.-based enterprise architect at Siemens.
Siemens also finds it important to have access to Microsoft's labs, where it can bring together a global team in one spot, Minnick added. "We're driving the leading edge, not just following it. That's the benefit of TAP," he said.
Minnick noted that Siemens has submitted more than 520 design-change requests and bug reports to Microsoft as part of the TAP process. In addition, the company's participation has helped it make decisions about product deployments. For example, Minnick said, Siemens didn't deploy Exchange Server 2000 because of the software's lack of distributed administration capabilities. It also had scalability issues with Windows 2000.
"If it wasn't for the participation in TAP programs, we would not have been able to explain why it was a showstopper when the product was released," Minnick said.
Denver Health and Hospital Authority Inc. couldn't get a scripting tool to work during a four-week TAP that included migrating its Microsoft SNA gateway servers to the vendor's Virtual Server software. Microsoft and its consulting partner, Interlink Group Inc. in Englewood, Colo., couldn't get the tool to work either, according to Michael Brown, a former support services manager at Denver Health who recently left to work for Sun Microsystems Inc.
The Microsoft utility would have eliminated the need for the not-for-profit health care provider to manually build virtual servers on a one-by-one basis. Brown said Denver Health's IT department ultimately opted for a tool from Toronto-based PlateSpin Ltd.
Because Brown viewed the scripting tool only as "the cherry on top," he still considered the Virtual Server TAP to be a success, as Denver Health consolidated 14 physical servers to four. The company has also participated in TAPs for Exchange Server 2000 and Active Directory as well as SMS 2003, he said.
But users shouldn't sign up for TAPs without giving it some thought, Brown cautioned. "You have to make sure a program like this is going to meet your expectations," he said. "You have to go in with a very specific reason."