7. Quicksand

The story of the end of the university's preeminence in the free software world is a tale of greed and corporate power. While many saw an unhappy ending coming for many years, few could do much to stop the inevitable collision between the University of California at Berkeley and its former patron, AT&T.

The lawsuit between AT&T and the University of California at Berkeley had its roots in what marriage counselors love to call a “poorly conceived relationship.” By the end of the 1980s, the computer science department at Berkeley had a problem. They had been collaborating with AT&T on the UNIX system from the beginning. They had written some nice code, including some of the crucial software that formed the foundation of the Internet. Students, professors, scientists, and even Wall Street traders loved the power and flexibility of UNIX. Everyone wanted UNIX.

The problem was that not everyone could get UNIX. AT&T, which had sponsored much of the research at Berkeley, kept an iron hand on its invention. If you wanted to run UNIX, then you needed to license some essential software from AT&T that sat at the core of the system. They were the supreme ruler of the UNIX domain, and they expected a healthy tithe for the pleasure of living within it.

One of the people who wanted UNIX was the Finnish student Linus Torvalds, who couldn't afford this tithe. He was far from the first one, and the conflict began long before he started to write Linux in 1991.

Toward the end of the 1980s, most people in the computer world were well aware of Stallman's crusade against the corporate dominance of AT&T and UNIX. Most programmers knew that GNU stood for “GNU's Not UNIX.” Stallman was not the only person annoyed by AT&T's attitude toward secrecy and non-disclosure agreements. In fact, his attitude was contagious. Some of the folks at Berkeley looked at the growth of tools emerging from the GNU project and felt a bit used. They had written many pieces of code that found their way into AT&T's version of UNIX. They had contributed many great ideas. Yet AT&T was behaving as if AT&T alone owned it. They gave and gave, while AT&T took.

Stallman got to distribute his source code. Stallman got to share with others. Stallman got to build his reputation. Programmers raved about Stallman's Emacs. People played GNU Chess at their offices. Others were donating their tools to the GNU project. Everyone was getting some attention by sharing except the folks at Berkeley who collaborated with AT&T. This started to rub people the wrong way.

Something had to be done, and the folks at Berkeley started feeling the pressure. Some at Berkeley wondered why the professors had entered into such a Faustian bargain with a big corporation. Was the payoff great enough to surrender their academic souls? Just where did AT&T get off telling us what we could publish?

Others outside of Berkeley looked in and saw a treasure trove of software that was written by academics. Many of them were friends. Some of them had studied at Berkeley. Some had even written some of the UNIX code before they graduated. Some were companies competing with AT&T. All of them figured that they could solve their UNIX problems if they could just get their hands on the source code. There had to be some way to get it released.

Slowly, the two groups began making contact and actively speculating on how to free Berkeley's version of UNIX from AT&T's grip.

7.1. Breaking the Bond

The first move to separate Berkeley's version of UNIX from AT&T's control wasn't really a revolution. No one was starting a civil war by firing shots at Fort Sumter or starting a revolution by dropping tea in the harbor. In fact, it started long before the lawsuit and Linux. In 1989, some people wanted to start hooking their PCs and other devices up to the Internet, and they didn't want to use UNIX.

Berkeley had written some of the software known as TCP/IP that defined how computers on the Internet would communicate and share packets. They wrote the software for UNIX because that was one of the favorite OSs around the labs. Other companies got a copy of the code by buying a source license for UNIX from AT&T. The TCP/IP code was just part of the mix. Some say that the cost of the license reached $250,000 or more and required that the customer pay a per-unit fee for every product that was shipped. Those prices didn't deter the big companies like IBM or DEC. They thought of UNIX as an OS for the hefty workstations and minicomputers sold to businesses and scientists. Those guys had the budget to pay for big hardware, so it was possible to slip the cost of the UNIX OS in with the package.

But the PC world was different. It was filled with guys in garages who wanted to build simple boards that would let a PC communicate on the Internet. These guys were efficient and knew how to scrounge up cheap parts from all over the world. Some of them had gone to Berkeley and learned to program on the VAXes and Sun workstations running Berkeley's version of UNIX. A few of them had even helped write or debug the code. They didn't see why they had to buy such a big license for something that non-AT&T folks had written with the generous help of large government grants. Some even worked for corporations that gave money to support Berkeley's projects. Why couldn't they get at the code they helped pay to develop?

Kirk McKusick, one of the members of the Computer Systems Research Group at the time, remembers, “People came to us and said, 'Look, you wrote TCP/IP. Surely you shouldn't require an AT&T license for that?' These seemed like reasonable requests. We decided to start with something that was clearly not part of the UNIX we got from AT&T. It seemed very clear that we could pull out the TCP/IP stack and distribute that without running afoul of AT&T's license.”

So the Berkeley Computer Systems Research Group (CSRG) created what they called Network Release 1 and put it on the market for $1,000 in June 1989. That wasn't really the price because the release came with one of the first versions of what would come to be known as the BSD-style license. Once you paid the $1,000, you could do whatever you wanted with the code, including just putting it up on the Net and giving it away.

“We thought that two or three groups would pay the money and then put the code on the Internet, but in fact, hundreds of sites actually paid the one thousand dollars for it,” says McKusick and adds, “mostly so they could get a piece of paper from the university saying, 'You can do what you want with this.'”

This move worked out well for Berkeley and also for UNIX. The Berkeley TCP/IP stack became the best-known version of the code, and it acted like a reference version for the rest of the Net. If it had a glitch, everyone else had to work around the glitch because it was so prevalent. Even today, companies like Sun like to brag that their TCP/IP forms the backbone of the Net, and this is one of the reasons to buy a Sun instead of an NT workstation. Of course, the code in Sun's OS has a rich, Berkeley-based heritage, and it may still contain some of the original BSD code for controlling the net.

7.2. In for a Penny, in for a Pound

In time, more and more companies started forming in the Bay Area and more and more realized that Berkeley's version of UNIX was the reference for the Internet. They started asking for this bit or that bit.

Keith Bostic heard these requests and decided that the Berkeley CSRG needed to free up as much of the source code as possible. Everyone agreed it was a utopian idea, but only Bostic thought it was possible to accomplish. McKusick writes, in his history of BSD, “Mike Karels [a fellow software developer] and I pointed out that releasing large parts of the system was a huge task, but we agreed that if he could sort out how to deal with re-implementing the hundreds of utilities and the massive C library, then we would tackle the kernel. Privately, Karels and I thought that would be the end of the discussion.”

Dave Hitz, a good friend of Bostic's, remembers the time. “Bostic was more of a commanding type. He just rounded up all of his friends to finish up the code. You would go over to his house for dinner and he would say, 'I've got a list. What do you want to do?' I think I did the cp command and maybe the look command.” Hitz, of course, is happy that he took part in the project. He recently founded Network Appliance, a company that packages a stripped-down version of BSD into a file server that is supposed to be a fairly bulletproof appliance for customers. Network Appliance didn't need to do much software engineering when they began. They just grabbed the free version of BSD and hooked it up.

Bostic pursued people far and wide to accomplish the task. He gave them the published description of the utility or the part of the library from the documentation and then asked them to reimplement it without looking at the source code. This cloning operation is known as a cleanroom operation because it is entirely legal if it takes place inside a metaphorical room where the engineers inside don't have any information about how the AT&T engineers built UNIX.

This was not an easy job, but Bostic was quite devoted and pursued people everywhere. He roped everyone who could code into the project and often spent time fixing things afterward. The task took 18 months and included more than 400 people who received just notoriety and some thanks afterward. The 400-plus names are printed in the book he wrote with McKusick and Karels in 1996.

When Bostic came close to finishing, he stopped by McKusick's office and asked how the kernel was coming along. This called McKusick and Karels's bluff and forced them to do some hard engineering work. In some respects, Bostic had the easier job. Writing small utility programs that his team used was hard work, but it was essentially preorganized and segmented. Many folks over the years created manual files that documented exactly what the programs were supposed to do. Each program could be assigned separately and people didn't need to coordinate their work too much. These were just dishes for a potluck supper.

Cleaning up the kernel, however, was a different matter. It was much larger than many of the smaller utilities and was filled with more complicated code that formed a tightly coordinated mechanism. Sloppy work in one of the utility files would probably affect only that one utility, but a glitch in the kernel would routinely bring down the entire system. If Bostic was coordinating a potluck supper, McKusick and Karels had to find a way to create an entire restaurant that served thousands of meals a day to thousands of customers. Every detail needed to work together smoothly.

To make matters more complicated, Berkeley's contributions to the kernel were mixed in with AT&T's contributions. Both had added on parts, glued in new features, and created new powers over the years. They were de facto partners on the project. Back in the good old days, they had both shared their source code without any long-term considerations or cares. But now that AT&T claimed ownership of it all, they had to find a way to unwind all of the changes and figure out who wrote what.

McKusick says, “We built a converted database up line by line. We took every line of code and inserted it into the database. You end up finding pretty quickly where the code migrated to and then you decide whether it is sufficiently large enough to see if it needed recoding.”

This database made life much easier for them and they were able to plow through the code, quickly recoding islets of AT&T code here and there. They could easily pull up a file filled with source code and let the database mark up the parts that might be owned by AT&T. Some parts went quickly, but other parts dragged on. By late spring of 1991, they had finished all but six files that were just too much work.

It would be nice to report that they bravely struggled onward, forgoing all distractions like movies, coffeehouses, and friends, but that's not true. They punted and tossed everything out the door and called it “Network Release 2.”The name implied that this new version was just a new revision of their earlier product, Network Release 1, and this made life easier with the lawyers. They just grabbed the old, simple license and reused it. It also disguised the fact that this new pile of code was only about six files short of a full-grown OS.

The good news about open source is that projects often succeed even when they initially fail. A commercial product couldn't ship without the complete functionality of the six files. Few would buy it. Plus, no one could come along, get a bug under his bonnet, and patch up the holes. Proprietary source code isn't available and no one wants to help someone else in business without compensation.

The new, almost complete UNIX, however, was something different. It was a university project and so university rules of camaraderie and sharing seemed to apply. Another programmer, Bill Jolitz, picked up Network Release 2 and soon added the code necessary to fill the gap. He became fascinated with getting UNIX up and running on a 386 processor, a task that was sort of like trying to fit the latest traction control hardware and anti-lock brakes on a go-cart. At the time, serious computer scientists worked on serious machines from serious workstation and minicomputer companies. The PC industry was building toys. Of course, there was something macho to the entire project. Back then I remember joking to a friend that we should try to get UNIX running on the new air-conditioning system, just to prove it could be done.

Jolitz's project, of course, found many people on the Net who didn't think it was just a toy. Once he put the source code on the Net, a bloom of enthusiasm spread through the universities and waystations of the world. People wanted to experiment with a high-grade OS and most could only afford relatively cheap hardware like the 386. Sure, places like Berkeley could get the government grant money and the big corporate donations, but 2,000-plus other schools were stuck waiting. Jolitz's version of 386BSD struck a chord.

While news traveled quickly to some corners, it didn't reach Finland. Network Release 2 came in June 1991, right around the same time that Linus Torvalds was poking around looking for a high-grade OS to use in experiments. Jolitz's 386BSD came out about six months later as Torvalds began to dig into creating the OS he would later call Linux. Soon afterward, Jolitz lost interest in the project and let it lie, but others came along. In fact, two groups called NetBSD and FreeBSD sprang up to carry the torch.

Although it may seem strange that three groups building a free operating system could emerge without knowing about each other, it is important to realize that the Internet was a very different world in 1991 and 1992. The World Wide Web was only a gleam in some people's eyes. Only the best universities had general access to the web for its students, and most people didn't understand what an e-mail address was. Only a few computer-related businesses like IBM and Xerox put their researchers on the Net. The community was small and insular.

The main conduits for information were the USENET newsgroups, which were read only by people who could get access through their universities. This technology was an efficient way of sharing information, although quite flawed. Here's how it worked: every so often, each computer would call up its negotiators and swap the latest articles. Information traveled like gossip, which is to say that it traveled quickly but with very uneven distribution. Computers were always breaking down or being upgraded. No one could count on every message getting to every corner of the globe.

The NetBSD and the FreeBSD forks of the BSD kernel continue to exist separately today. The folks who work on NetBSD concentrate on making their code run on all possible machines, and they currently list 21 different platforms that range from the omnipresent Intel 486 to the gone but not forgotten Commodore Amiga.

The FreeBSD team, on the other hand, concentrates on making their product work well on the Intel 386. They added many layers of installation tools to make it easier for the average Joe to use, and now it's the most popular version of BSD code around.

Those two versions used the latest code from Berkeley. Torvalds, on the other hand, didn't know about the 386BSD, FreeBSD, or NetBSD. If he had found out, he says, he probably would have just downloaded the versions and joined one of those teams. Why run off and reinvent the wheel?

7.3. AT&T Notices the Damage

Soon after Network Release 2 hit the world, the real problems began for BSD. While AT&T didn't really notice 386BSD, NetBSD, or FreeBSD, they did notice a company called Berkeley Software Design Incorporated. This corporation created their own OS by taking Network Release 2 and adding their own versions of the missing six files, but they didn't release this for free on the Net. They started putting advertisements in the trade press offering the source code for $995, a price they claimed was a huge discount over AT&T's charge.

The modern, post-Internet reader should find this hilarious. Two to three groups and countless splinter factions were distributing the BSD software over the Internet for free and this didn't seem to catch AT&T's attention, but the emergence of BSDI selling the same product for almost $1,000 rang alarm bells. That was the time, though, before the Internet infrastructure became ubiquitous. In the early 1990s, people only halfjoked that FedEx was the most efficient Internet Service Provider around. It was much faster to copy hundreds of megabytes of data onto a magnetic tape and drop it in FedEx than to actually try to copy it over the Internet. Back then only real nerds were on the Internet. Managers and lawyers wore suits and got their news from the trade press and advertisements.

BSDI's cost-cutting was a major headache for AT&T. This small company was selling a product that AT&T felt it had shepherded, organized, and coordinated over time.

AT&T started off by claiming UNIX as a trademark and threatening BSDI with infringing upon it. BSDI countered by changing the ads to emphasize that BSDI was a separate company that wasn't related to AT&T or the subsidiary AT&T created to market UNIX known as UNIX System Laboratories, or USL.

That didn't work. USL saw its cash cow melting away and assumed folks would jump at the chance to buy a complete OS with all the source code for $995. The price seems outrageously high today, but that's only after the stiff price competition of the 1990s. It was still a good deal at the time. So USL sued BSDI for actually stealing proprietary source code from AT&T.

This argument didn't work, either. BSDI turned around and waved the Network Release 2 license they got from Berkeley. They bought all but six of the files from Berkeley, and Berkeley claimed that all of the source code was theirs to sell. BSDI wrote the missing six files themselves and they were quite sure that they got no help from AT&T or USL. Therefore, BSDI didn't steal anything. If AT&T thought it was stolen, they should take it up with Berkeley. The judge bought BSDI's argument and narrowed the case to focus on the six files.

This was a crucial moment in the development of the free software movement and its various kernels. AT&T found itself cornered. Backing down meant giving up its claim to UNIX and the wonderful stream of license fees that kept pouring in. Pressing ahead meant suing the University of California, its old friend, partner, and author of lots of UNIX code. Eventually, the forces of greed and omnipotent corporate power won out and AT&T's USL filed a lawsuit naming both BSDI and the University of California.

Taking sides in this case was pretty easy for most folks in the academic and free software world. The CSRG at Berkeley did research. They published things. University research was supposed to be open and freely distributed. AT&T was trying to steal the work of hundreds if not thousands of students, researchers, professors, and others. That wasn't fair.

In reality, AT&T did pay something for what they got. They sent their employees to Berkeley to get master's degrees, they shared the original Versions 5, 6, and 7 and 32/V source code, and they even sent some hardware to the computer science department. The original creators of UNIX lived and worked at Bell Labs drawing AT&T paychecks. Berkeley students got summer jobs at AT&T. There wasn't an official quid-pro-quo. It wasn't very well spelled out, but AT&T was paying something.

Some folks on AT&T's side might even want to paint the CSRG at Berkeley as filled with academic freeloaders who worked hard to weasel money out of the big corporations without considering the implications. The folks at Berkeley should have known that AT&T was going to want something for its contributions. There's no such thing as a free lunch.

There's something to this argument because running a high-rent research project at a top-notch school requires a fair amount of guile and marketing sophistication. By the 1990s, the top universities had become very good at making vague, unofficial promises with their pleas for corporate gifts. This sort of coquetry and teasing was bound to land someone in a fight. McKusick, for instance, says that the CSRG designed the BSD license to be very liberal to please the corporate donors. “Hewlett-Packard put in hundreds of thousands of dollars and they were doing so under the understanding that they were going to use the code,” he said. If the BSD hadn't kept releasing code like Network Release 2 in a clear, easy-to-reuse legal form, he says, some of the funding for the group would have dried up.

But there's also a bit of irony here. McKusick points out that AT&T was far from the most generous company to support the CSRG. “In fact, we even had to pay for our license to UNIX,” he says before adding, “although it was only ninety-nine dollars at the time.”

AT&T's support of the department was hardly bountiful. The big checks weren't grants outright. They paid for the out-of-state tuition for AT&T employees who came to Berkeley to receive their master's degrees. While AT&T could have sent their employees elsewhere, there's no doubt that there are more generous ways to send money to researchers.

McKusick also notes that AT&T didn't even send along much hardware. The only hardware he remembers receiving from them were some 5620 terminals and a Datakit circuit-based switch that he says “was a huge headache that really did us very little good.” Berkeley was on the forefront of developing the packet-based standards that would dominate the Internet. If anything, the older circuit-based switch convinced the Berkeley team that basing the Internet on the old phone system would be a major mistake.

To make matters worse, AT&T often wanted the BSD team to include features that would force all the BSD users to buy a newer, more expensive license from AT&T. In addition, license verification was never a quick or easy task. McKusick says, “We had a person whose fulltime job was to keep the AT&T licensing person happy.”

In the end, he concludes, “They paid us next to nothing and got a huge windfall.”

Choosing sides in this battle probably isn't worth the trouble at this point because Berkeley eventually won. The hard work of Bostic's hundreds of volunteers and the careful combing of the kernel by the CSRG paid off. AT&T's case slowly withered away as the University of California was able to show how much of the distribution came from innocent, non-AT&T sources.

Berkeley even landed a few good blows of its own. They found that AT&T had stripped copyrights from Berkeley code that they had imported to System V and had failed to provide due credit to Berkeley. The BSD license is probably one of the least restrictive ones in the world. Companies like Apple use BSD source code all the time. The license has few requirements beyond keeping the copyright notice intact and including some credit for the University of California. AT&T didn't pay attention to this and failed to cite Berkeley's contributions in their releases. Oops. The CSRG countersued claiming that AT&T had violated a license that may be one of the least restrictive in the world.

The battle raged in the courts for more than a year. It moved from federal to California state court. Judges held hearings, lawyers took depositions, clerks read briefs, judges heard arguments presented by briefs written by lawyers who had just held depositions. The burn rate of legal fees was probably larger than most Internet start-ups.

Any grown-up should take one look at this battle and understand just how the free software movement got so far. While the Berkeley folks were meeting with lawyers and worrying about whether the judges were going to choose the right side, Linus Torvalds was creating his own kernel. He started Linux on his own, and that made him a free man.

In the end, the University of California settled the lawsuit after the USL was sold to Novell, a company run by Ray Noorda. McKusick believes that Noorda's embrace of free competition made a big difference, and by January 1994 the legal fight was over. Berkeley celebrated by releasing a completely free and unencumbered 4.4BSD-Lite in June 1994.

The terms of the settlement were pretty minor. Net Release 2 came with about 18,000 files. 4.4BSD-Lite contained all but three of them. Seventy of them included a new, expanded copyright that gave some credit to AT&T and USL, but didn't constrain anyone's right to freely distribute them. McKusick, Bostic, and the hundreds of volunteers did a great job making sure that Net Release 2 was clean. In fact, two people familiar with the settlement talks say that Berkeley just deleted a few files to allow USL's lawyers to save face. We'll never know for sure because the details of the settlement are sealed. McKusick and the others can't talk about the details. That's another great example of how the legal system fails the American people and inadvertently gives the free software world another leg up. There's no information in the record to help historians or give future generations some hints on how to solve similar disputes.