In-House Innovation

Given the direction things are going, you might think that the internally developed application is an endangered species. A lot of technology is fast becoming a commodity, with low-cost computers and off-the-shelf software making it possible to buy an end-to-end IT infrastructure. When software development is required, a growing number of companies turn to offshore contractors. And if a company does put software developers to work, many of them aren't using state-of-the-art tools. "Your average developer spends less money on tools than they do on lattes," laments Java creator James Gosling, a VP and Sun Fellow at Sun Microsystems.

Building applications without programming is a hotbed of activity, Sun Microsystems VP Gosling saysThose trends, however, are misleading. Custom code is alive and well and serving a vital role at many companies. Proprietary software, crafted by engineers who are close to a company's inner workings and systems, continues to be the secret sauce that gives many businesses an edge. Whether it's to make business processes faster and more efficient, deliver new services, improve customer support, or innovate in other ways, the software that makes big things possible is often unique, and intentionally so. "The stuff that gives you competitive advantage--that's the stuff you need to own and develop in-house," says Jeff Brandmaier, senior VP and CIO at H&R Block Inc.

Best known for its tax-preparation services, H&R Block would like more customers to use its mortgage-financing and brokerage services. To facilitate that, the company developed middleware that integrates its online and walk-in financial services in new ways. Internally developed software, for example, simplifies the process of creating a brokerage account for a customer of H&R Block's tax service and can then direct an electronic tax refund into the new account. H&R Block is looking at utility computing--an example of the commoditization trend--as a way of adding flexibility to its IT infrastructure. But it doesn't plan to reduce internal application development. If anything, "we probably have opportunities to get much better" at it, Brandmaier says.

IT budget cuts, server consolidation, and layoffs not withstanding, software development still serves a strategic role at many companies. The projects range from a handful of Java developers writing applications on goals, assists, and points for the National Hockey League to Dell's homegrown just-in-time ordering system. Morgan Stanley is developing "systems that increase client connectivity, improve business intelligence, or otherwise provide a business advantage," says Guy Chiarello, chief technology officer of Morgan Stanley and CIO of the company's Institutional Securities business. The Wall Street firm buys more commercial applications than it did in the past--a result of the maturing software industry--but that lets it focus internal resources on innovative, customer-facing apps, he says.

The do-it-yourself approach isn't cheap. The median salary for a software developer is $70,000 this year, while median pay for an application-development manager is $93,000, according to InformationWeek Research's National IT Salary Survey. Multiply that by dozens or hundreds of personnel and the costs can quickly rise into the millions of dollars for months-long projects. But it's work that, in many cases, simply couldn't be purchased anywhere else.

At Fannie Mae, the nation's second-largest corporation based on assets and the largest source of home-mortgage financing, software engineers are in the midst of a multimillion-dollar development project that promises to transform the way home lenders interact with the big financial company. Fannie Mae is building the applications rather than buying them because "infrastructure sets the speed limit for our new product development," says Julie St. John, executive VP and chief technology officer.

Fannie Mae's so-called "core" infrastructure project involves writing in Java new applications that are intended to be more flexible than the 20-year-old mainframe apps currently in use, which were written when 30-year, fixed-rate loans were the norm in home financing. In addition to providing a more-flexible platform for offering different kinds of home loans, the custom applications will support the acquisition of mortgages from Fannie Mae's lending partners and the delivery of mortgage-based securities to investors--in short, the entire life cycle of a home loan.

The project is being done with the help of IBM and American Management Systems Inc., which together employ about 60% of the developers involved. One of the first achievements of the years-long initiative is an online system launched last year. Called eCommitting, it lets mortgage lenders commit to selling loans faster and with greater efficiency. Applications that support mortgage acquisitions and service reporting are scheduled for deployment over the next two years.

Fannie Mae has sequestered the development team working on the applications, which are being built using Java 2 Enterprise Edition components, in a building separate from its Washington headquarters. "There has to be a total focus on architecting it correctly," St. John says. The languages, tools, and methodologies behind in-house development are evolving, with reusable components and Web-services standards (XML; Simple Object Access Protocol; Universal Description, Discovery, and Integration; and Web Services Description Language) increasingly being used within services-based architectures. The programming environments generally fall into two camps: Sun's J2EE or Microsoft's .Net. While there are strong opinions about the advantages of each, the choice doesn't always boil down to one or the other. "We believe organizations, more and more, will do hybrid projects," says Ted Shelton, senior VP and chief strategy officer at Borland Software Corp., which has been selling development tools for 20 years.

Either way, Shelton says, the trend among corporate development teams is to use tools such as Borland's JBuilder, StarTeam, and Together to treat their work less like a craft and more like a "practice." That means being more attentive to the way software-development teams collaborate with users and business managers to determine the requirements of applications and then being more methodical in the design, development, and testing of those apps. "Software developers today have to be very familiar with the full application life cycle," Shelton says.

AXA Financial Inc., a diversified financial-services company, already expects 10% productivity gains each year from its staff of more than 300 developers. Even so, there's opportunity for greater improvement through more carefully coordinated development activities, says David Wollin, managing director of E-business and emerging technology. "That's a big and important thing," he says. "You can improve the way you collect and organize business requirements. That's got to save us a ton of time."

With an established software infrastructure that includes PeopleSoft Inc.'s general-ledger and human-resources applications, the emphasis now is on layering AXA Financial's systems with new, internally developed capabilities, Wollin says. One project under way will let customers view quarterly investment statements online. The company's programmers write in C++, Cobol, Java, and other languages. Tools in use include IBM's Rational Rose for business modeling, Borland's JBuilder for code generation, and Mercury Interactive's testing software for quality control. Wollin plans to use Microsoft's Project 2003 to improve the way he tracks the progress and costs of development projects.

Key to making it all work, Wollin says, is regular interaction with the business units that will use the new capabilities. In fact, usability, a project's timing, and business-process alignment all need to be determined by close collaboration with business units, experts say. "The hard part of the exercise is mapping your business process to the technology," says Eric Rudder, senior VP of Microsoft's tools and platforms division. "None of these technologies is a substitute for good thinking."

But technological advances certainly help. The London Stock Exchange is increasingly using Microsoft's C# language in lieu of Cobol, a decades-old language still widely used on mainframes. Ian Homan, head of technology for the exchange, says the shift is possible because the choice in application-development tools can now be made independently of hardware platform and operating system. In the past, he says, "the hardware would choose the operating system, the operating system would choose the development language, and I would have no choice whatsoever."

Languages such as C#, Java, and Microsoft's Visual Basic .Net are faster, easier, and safer to use than older languages, Homan says. That's because the more contemporary languages avoid or mask some of the complexities (such as pointers, threads, and extended template libraries) of earlier languages. "The testing is a lot easier, and if the applications do start to go wrong, you get a lot more predictable failure," he says.

The cost benefits are also compelling. Homan estimates programmers can generate code using C# at half the cost of Cobol. And, because newer languages unhitch programmers from mainframes and other proprietary platforms, money saved on hardware gets factored into total cost of ownership. Moving from a proprietary hardware platform that requires an older programming language to Intel-based servers that allow for newer development tools can pay for itself in three years, Homan estimates. The exchange is assessing whether the economics justify moving its trading operations from the Hewlett-Packard Himalaya systems that require Cobol programming.

Meanwhile, the London Stock Exchange is writing most new applications using Microsoft's Visual Studio .Net 2003 development system, the C# language, and the .Net Framework in Windows Server 2003. In April, after 10 months of design, development, and testing, the exchange used that combination of Microsoft technologies to launch an application that delivers real-time market information to traders and other customers. The exchange's programming work is done by 100 to 150 engineers who are employees of Accenture; the number fluctuates depending on need.

On this side of the Atlantic, a smaller team of developers is using a different approach to build data-rich applications of another kind. The NHL's staff of five programmers uses open-source tools and Java to write applications that let various user constituencies--fans, sports writers, broadcasters, team executives--slice and dice hockey statistics. Grant Nodine, senior director of Web operations for the NHL, says the decision to use open-source tools such as the NetBeans development environment and Cayenne data-modeling framework was only partly a cost-saving move. "I personally have found the open-source tools tend to be better documented and more up-to-date than their commercial counterparts," Nodine says.

Still, the NHL's programmers are under pressure to control costs and increase efficiency. One way they've done that is by creating a common framework of methods and functions that sits between newly developed number-crunching applications and the back-end Oracle database that houses player and game statistics. "We're developing applications in such a way that it significantly reduces the amount of time necessary to manage them," Nodine says.

Detroit Edison, a subsidiary of DTE Energy Co., is taking a different path to software-development efficiency. A few months ago, the utility signed up for a service from TopCoder Inc. that lets it submit work requests to the outsourcing company, which uses thousands of freelance developers located in more than 100 countries. Detroit Edison also gains access to growing catalogs of J2EE and .Net infrastructure components developed by TopCoder programmers. "Prebuilt, pretested components shorten your development time," says Rod Davenport, technology strategist with Detroit Edison, which has its own staff of 300 developers. "This gives us another way to have a greater variety of components and leverage developers at pretty low cost."

In addition to building custom applications, in-house programmers stay busy fine-tuning the commercial enterprise-resource-planning applications used by their companies, integrating it all, and, more recently, introducing Web services to those software environments. When it comes to Web services, Microsoft's Rudder says development managers face a decision: whether to build the Web-services layer themselves or rely on commercial software vendors. Regardless of how they do it, Web services should eventually free software engineers from some of the grunt work and give them more time to spend on business processes.

Such advances are making development staffs at many U.S. companies more productive than ever. New graphical development environments such as Visual Studio .Net and Sun's forthcoming Rave tools also help by automating the way software is developed, creating lines of code with the click of a mouse. "Building applications without actually programming" is one of the hotbeds of activity in development organizations, Sun's Gosling says (see story, Tool Time: Features Boost Developers' Productivity).

But practitioners say there's plenty of room for improvement. "There's still a lot of manually intensive work that goes on in the development process," says AXA Financial's Wollin. Gosling says he encounters Emacs, a 25-year-old bare-bones coding tool, "a lot more than I think is reasonable."

The question of programmer productivity--and the cost of that work--is getting closer scrutiny with the increasing interest in offshore outsourcing. While new languages and tools keep raising the output of U.S. developers, the same products are available to engineers in India, Russia, and other overseas software centers, where upstart companies promise quality code at lower costs. Local developers do have some advantages in working for U.S. companies--day-to-day proximity to the business counts for a lot--but there's no escaping the fact that a growing number of CIOs look at offshore development as a way of lowering costs for at least some projects.

Where does that leave the large base of U.S. developers? For many, the skills required of them may change. Borland's Shelton says U.S. software engineers will have to manage more, develop less. "They're going to be coordinating and controlling quality and making sure requirements are fulfilled rather than writing lines of code," he says. "The lines of code are going to be written somewhere else."

In addition, Gosling says, U.S. developers need to work even harder at understanding user needs, translating business requirements into software solutions, and fitting into the culture of the companies they work for. "You have to be really close in touch with what it is that people want from you," he says.

For developers willing to change with the times, there should be plenty to do. Businesses show no signs of pulling back on the custom software projects that set them apart.

Originally appeared in InformationWeek, Sept. 15, 2003.