20. T-Shirts

If there's a pantheon for marketing geniuses, then it must include the guy who realized people would pay $1 for several cents' worth of sugar water if it came in a shapely bottle blessed by the brand name CocaCola. It might also include the guy who first figured out that adding new blue crystals to detergent would increase sales. It is a rare breed that understands how to get people to spend money they don't need to spend.

The next induction ceremony for this pantheon should include Robert Young, the CEO of Red Hat Software, who helped the Linux and the open source world immeasurably by finding a way to charge people for something they could get for free. This discovery made the man rich, which isn't exactly what the free software world is supposed to do. But his company also contributed a sense of stability and certainty to the Linux marketplace, and that was sorely needed. Many hard-core programmers, who know enough to get all of the software for free, are willing to pay $70 to Red Hat just because it is easier. While some may be forever jealous of the millions of dollars in Young's pocket, everyone should realize that bringing Linux to a larger world of computer illiterates requires good packaging and hand-holding. Free software wouldn't be anywhere if someone couldn't find a good way to charge for it.

The best way to understand why Young ranks with the folks who discovered how to sell sugar water is to go to a conference like LinuxExpo. In the center of the floor is the booth manned by Red Hat Software, the company Young started in Raleigh, North Carolina, after he got through working in the computer-leasing business. Young is in his fifties now and manages to survive despite the fact that most of his company's devotees are much closer to 13. Red Hat bundles together some of the free software made by the community and distributed over the Net and puts it on one relatively easy-to-use CD-ROM. Anyone who wants to install Linux or some of its packages can simply buy a disk from Red Hat and push a bunch of keys. All of the information is on one CD-ROM, and it's relatively tested and pretty much ready to go. If things go wrong, Red Hat promises to answer questions by e-mail or telephone to help people get the product working.

Red Hat sells their disk at prices that range from $29.95 to $149.95. That buys the user a pretty box, three CD-ROMs including some demonstration versions of other software, all of the source code, a manual, and some telephone or e-mail support. That is pretty much like the same stuff that comes in software boxes from a normal company. The manual isn't as nice as the manuals produced by Apple or Microsoft, but it's not too bad.

The big difference is that no one needs to buy the CD-ROM from Red Hat. Anyone can download all of the software from the Net. A friend of mine, Hal Skinner, did it yesterday and told me, “I just plugged it in and the software downloaded everything from the Net. I got everything in the Red Hat 6.0 distribution, and it didn't cost me anything.”

Of course, Red Hat isn't hurt too much by folks who grab copies without paying for them. In fact, the company maintains a website that makes it relatively easy for people to do just that. Red Hat didn't write most of the code. They also just grabbed it from various authors throughout the Net who published it under the GNU General Public License. They grabbed it without paying for it, so they're not really put out if someone grabs from them.

The ability to snag GPL'ed software from around the Net keeps their development costs much lower than Sun, Apple, or Microsoft. They never paid most of the authors of the code they ship. They just package it up. Anyone else can just go find it on the Net and grab it themselves. This pretty much guarantees that Red Hat will be in a commodity business.

To make matters worse for Red Hat, the potential competitors don't have to go out onto the Net and reassemble the collection of software for themselves. The GPL specifically forbids people from placing limitations on redistributing the source code. That means that a potential competitor doesn't have to do much more than buy a copy of Red Hat's disk and send it off to the CD-ROM pressing plant. People do this all the time. One company at the exposition was selling copies of all the major Linux distributions like Red Hat, Slackware, and OpenBSD for about $3 per disk. If you bought in bulk, you could get 11 disks for $25. Not a bad deal if you're a consumer.

So, on one side of the floor, Young had a flashy booth filled with workers talking about what you could get if you paid $50 or more for Red Hat's version 6.0 with new enhancements like GNOME. Just a few hundred feet away, a company was selling ripoff copies of the same CDs for $3. Any company that is able to stay in business in a climate like that must be doing something right.

It's not much different from the supermarket. Someone can pay $1 or more for two liters of Coca-Cola, or they can walk over a few aisles and buy Kool-Aid and raw sugar. It may be much cheaper to buy the raw ingredients, but many people don't.

Young is also smart enough to use the competition from other cheap disk vendors to his advantage. He can't do anything about the GPL restrictions that force him to share with knockoff competitors, so he makes the best of them. “When people complain about how much we're charging for free software, I tell them to just go to CheapBytes.com,” he said at the Expo. This is just one of the companies that regularly duplicates the CDs of Red Hat and resells them. Red Hat often gets some heat from people saying that the company is merely profiting off the hard work of others who've shared their software with the GPL. What gives them the right to charge so much for other people's software? But Young points out that people can get the software for $3. There must be a rational reason why they're buying Red Hat.

Of course, the two packages aren't exactly equal. Both the original and the knockoff CD-ROM may have exactly the same contents, but the extras are different. The Red Hat package comes with “support,” a rather amorphous concept in the software business. In theory, Red Hat has a team of people sitting around their offices diligently waiting to answer the questions of customers who can't get Red Hat software to do the right thing.

In practice, the questions are often so hard or nebulous that even the support team can't answer them. When I first tried to get Red Hat to run on an old PC, the support team could only tell me that they never promised that their package would run on my funky, slightly obscure Cyrix MediaGX chip. That wasn't much help. Others probably have had better luck because they were using a more standard computer. Of course, I had no trouble installing Red Hat on my latest machine, and I didn't even need to contact tech support.

The Red Hat packages also come with a book that tries to answer some of the questions in advance. This manual describes the basic installation procedure, but it doesn't go into any detail about the software included in the distribution. If you want to know how to run the database package, you need to dig into the online support provided by the database's developers.

Many people enjoy buying these extra packages like the manual and the support, even if they never use them. Red Hat has blossomed by providing some hand-holding. Sure, some programmers could download the software from the Internet on their own, but most people don't want to spend the time needed to develop the expertise.

When I say “Red Hat software,” I really mean free source software that Red Hat picked up from the Net and knit into a coherent set of packages that should be, in theory, pretty bug free, tested, and ready for use. Red Hat is selling some hand-holding and filtering for the average user who doesn't want to spend time poking around the Net, checking out the different versions of the software, and ensuring that they work well together. Red Hat programmers have spent some time examining the software on the CD-ROM. They've tested it and occasionally improved it by adding new code to make it run better.

Red Hat also added a custom installation utility to make life easier for people who want to add Red Hat to their computer. ¹² They could have made this package installation tool proprietary. After all, Red Hat programmers wrote the tool on company time. But Young released it with the GNU General Public License, recognizing that the political value of giving something back was worth much more than the price they could charge for the tool.

This is part of a deliberate political strategy to build goodwill among the programmers who distribute their software. Many Linux users compare the different companies putting together free source software CDROMs and test their commitment to the free software ideals. Debian, for instance, is very popular because it is a largely volunteer project that is careful to only include certified free source software on their CD-ROMs. Debian, however, isn't run like a business and it doesn't have the same attitude. This volunteer effort and enlightened pursuit of the essence of free software make the Debian distribution popular among the purists.

Distributors like Caldera, on the other hand, include nonfree software with their disk. You pay $29.95 to $149.95 for a CD-ROM and get some nonfree software like a word processor tossed in as a bonus. This is a great deal if you're only going to install the software once, but the copyright on the nonfree software prevents you from distributing the CD-ROM to friends. Caldera is hoping that the extras it throws in will steer people toward its disk and get them to choose Caldera's version of Linux. Many of the purists, like Richard Stallman, hate this practice and think it is just a not very subtle way to privatize the free software. If the average user isn't free to redistribute all the code, then there's something evil afoot. Of course, Stallman or any of the other software authors can't do anything about this because they made their software freely distributable.

Young is trying to walk the line between these two approaches. Red Hat is very much in the business of selling CD-ROMs. The company has a payroll with more than a handful of programmers who are drawing nonvolunteer salaries to keep the distributions fresh and the code clean. But he's avoided the temptation of adding not-so-free code to his disks. This gives him more credibility with the programmers who create the software and give it away. In theory, Young doesn't need to ingratiate himself to the various authors of GPL-protected software packages. They've already given the code away. Their power is gone. In practice, he gains plenty of political goodwill by playing the game by their rules.

Several companies are already making PCs with Linux software installed at the factory. While they could simply download the software from the Net themselves and create their own package, several have chosen to bundle Red Hat's version with their machines. Sam Ockman, the president of Penguin Computing, runs one of those companies.

Ockman is a recent Stanford graduate in his early twenties and a strong devotee of the Linux and GPL world. He says he started his company to prove that Linux could deliver solid, dependable servers that could compete with the best that Sun and Microsoft have to offer.

Ockman has mixed feelings about life at Stanford. While he fondly remembers the “golf course-like campus,” he says the classes were too easy. He graduated with two majors despite spending plenty of time playing around with the Linux kernel. He says that the computer science department's hobbled curriculum drove him to Linux. “Their whole CS community is using a stupid compiler for C on the Macintosh,” he says.“Why don't they start you off on Linux? By the time you get to [course] 248, you could hack on the Linux kernel or your own replacement kernel. It's just a tragedy that you're sitting there writing virtual kernels on a Sun system that you're not allowed to reboot.”

In essence, the computer science department was keeping their kids penned up in the shallow end of the pool instead of taking them out into the ocean. Ockman found the ocean on his own time and started writing GPL-protected code and contributing to the political emergence of free software.

When Ockman had to choose a version of Linux for his Penguin computers, he chose Red Hat. Bob Young's company made the sale because it was playing by the rules of the game and giving software back with a GPL. Ockman says, “We actually buy the box set for every single one. Partially because the customers like to get the books, but also to support Red Hat. That's also why we picked Red Hat. They're the most free of all of the distributions.”

Debian, Ockman concedes, is also very free and politically interesting, but says that his company is too small to support multiple distributions. “We only do Red Hat. That was a very strategic decision on our part. All of the distributions are pretty much the same, but there are slight differences in this and that. We could have a twelve-person Debian group, but it would just be a nightmare for us to support all of these different versions of Linux.”

Of course, Penguin Computing could have just bought one Red Hat CD-ROM and installed their software on all of the machines going out the door. That would have let them cut their costs by about $50. The GPL lets anyone install the software as often as they wish. But this wouldn't be pure savings because Ockman is also offloading some of his own work when he bundles a Red Hat package with his computers. He adds, “Technically the box set I include allows customers to call Red Hat, but no one ever does, nor do we expect them or want them to call anyone but us.” In essence, his company is adding some extra support with the Red Hat box.

The support is an important add-on that Young is selling, but he realized long ago that much more was on sale. Red Hat was selling an image, the sense of belonging, and the indeterminant essence of cool. Soda manufacturers realized that anyone could put sugar and water in a bottle, but only the best could rise above the humdrum nature of life by employing the best artists in the land to give their sugar water the right hip feeling. So Young invested in image. His T-shirts and packages have always been some of the most graphically sophisticated on the market. While some folks would get girlfriends or neighbors to draw the images that covered their books and CDs, Red Hat used a talented team to develop their packaging.

Young jokes about this. He said he was at a trade show talking to a small software company that was trying to give him one of their free promotional T-shirts. He said, “Why don't you try giving away the source code and selling the T-shirts?”

At the LinuxExpo, Red Hat was selling T-shirts, too. One slick number retailing for $19 just said “The Revolution of Choice” in Red Hat's signature old typewriter font. Others for sale at the company's site routinely run for $15 or more. They sucked me in. When I ordered my first Red Hat disk from them, I bought an extra T-shirt to go with the mix.

The technology folks at Red Hat may be working with some cuttingedge software that makes the software easy to install, but the marketing group was stealing its plays from Nike, Pepsi, and Disney. They weren't selling running shoes, sugar water, or a ride on a roller coaster--they were selling an experience. Red Hat wasn't repackaging some hacker's science project from the Net, it was offering folks a ticket to a revolution. If the old 1960s radicals had realized this, they might have been able to fund their movement without borrowing money from their square parents. Selling enough groovy, tie-died T-shirts would have been enough. ¹³

Many of the other groups are part of the game. The OpenBSD project sold out of their very fashionable T-shirts with wireframe versions of its little daemon logo soon after the beginning of the LinuxExpo. They continue to sell more T-shirts from their website. Users can also buy CD-ROMs from OpenBSD.

Several attendees wear yellow copyleft shirts that hold an upsidedown copyright logo <:=Copyleft> arranged so the open side points to the left.

The most expensive T-shirt at the show came with a logo that imitated one of the early marketing images of the first Star Wars movie. The shirt showed Torvalds and Stallman instead of Han Solo and Luke Skywalker under a banner headline of “OS Wars.” The shirt cost only $100, but “came with free admission to the upcoming Linux convention in Atlanta.”

The corporate suits, of course, have adjusted as best they can. The IBM folks at the show wore identical khaki outfits with nicely cut and relatively expensive polo shirts with IBM logos. A regular suit would probably stick out less than the crisp, clean attempt to split the difference between casual cool and button-down business droid.

Of course, the T-shirts weren't just about pretty packaging and slick images. The shirts also conveyed some information about someone's political affiliations in the community and showed something about the person's technical tastes. Sure, someone could wear an OpenBSD shirt because they liked the cute little daemon logo, but also because they wanted to show that they cared about security. The OpenBSD project began because some users wanted to build a version of UNIX that was much more secure. The group prides itself on fixing bugs early and well. Wearing an OpenBSD shirt proclaims a certain alliance with this team's commitment to security. After all, some of the profits from the shirts went to pay for the development of the software. Wearing the right T-shirt meant choosing an alliance. It meant joining a tribe.

Young is keenly aware that much of his target market is 13-year-old boys who are flexing their minds and independence for the first time. The same images of rebellion that brought James Dean his stardom are painted on the T-shirts. Some wear shirts proclaiming TOTAL WORLD DOMINATION SOON. Raging against Microsoft is a cliché that is avoided as much as it is still used. The shirts are a mixture of parody, bluster, wit, and confidence. Of course, they're usually black. Everyone wears black.

Ockman looks at this market competition for T-shirts and sees a genius. He says, "I think Bob Young's absolutely brilliant. Suddenly he realized that there's no future in releasing mainframes. He made a jump after finding college kids in Carolina [using Linux]. For him to make that jump is just amazing. He's a marketing guy. He sat down and figured it out.

“Every time I hear him talk,” Ockman says about Young, “he tells a different story about ketchup. If you take people who've never had ketchup before in their life and you blindly feed them ketchup, they have no taste for ketchup. They don't like it.” If you feed them ketchup over time, they begin to demand it on their hamburgers.

“No one who's never had Coca-Cola before would like it,” Ockman continues. “These things are purely a branding issue. It has to be branded for cool in order for people to sit down and learn everything they have to know.”

In essence, Young looked around and saw that a bunch of scruffy kids were creating an OS that was just as good, if not better, than the major OSs costing major sums of money. This OS was, best of all, free for all comers. The OS had a problem, though. The scruffy kids never marketed their software. The deeply intelligent, free-thinking hackers picked up on how cool it was, but the rest of society couldn't make the jump. The scruffy kids didn't bother to try to market it to the rest of society. They were artists.

Most people who looked at such a situation would have concluded that this strange clan of techno-outsiders was doomed to inhabit the periphery of society forever. There was no marketing of the product because there was no money in the budget and there would never be money in the budget because the software was free. Young recognized that you could still market the software without owning it. You could still slap on a veneer of cool without writing the code yourself. Sugar water costs practically nothing, too.

Young's plan to brand the OS with a veneer of cool produced more success than anyone could imagine. Red Hat is by far the market leader in providing Linux to the masses, despite the fact that many can and do “steal” a low-cost version. Of course, “steal” isn't the right word, because Red Hat did the same thing. “Borrow” isn't right, “grab” is a bit casual, and “join in everlasting communion with the great free software continuum” is just too enthusiastic to be cool.

In August 1999, Red Hat completed an initial public offering of the shares of its stock, the common benchmark for success in the cash-driven world of Silicon Valley. Many of the principals at Red Hat got rich when the stock opened at $14 a share on August 11 and closed the day at $52. Bob Young, the CEO of Red Hat, started the day with a bit more than 9 million shares or 15 percent of the company. Technically, not all of this was his because he had distributed some (3,222,746 shares, to be exact) to his wife, Nancy, and put some more (1,418,160) in various trusts for his children. Still, this cut adds up to about $468 million. Marc Ewing, executive vice president and chief technology officer, also ended up with a similar amount of money divided between trusts and his own pocket. Matthew Sulzik, the president, who joined in November 1998, got a bit less (2,736,248 shares) in his pot, but he was a relative newcomer. The big investors, Greylock IX Limited Partnership, Benchmark Capital Partners II, and Intel, split up the big part of the rest of the shares.

Now, what happened to the boys who wrote the code? Did Richard Stallman get any of it? Did Linus Torvalds? Some of the major developers like Alan Cox and David Miller already work for Red Hat, so they probably drew shares out of the employee pool. There are thousands of names, however, who aren't on anyone's radar screen. They've written many lines of code for naught.

Red Hat tried to alleviate some of the trouble by allocating 800,000 shares to “directors, officers and employees of Red Hat and to open source software developers and other persons that Red Hat believes have contributed to the success of the open source software community and the growth of Red Hat.” This group, occasionally known as the “friends and family,” was a way to reward buddies. Red Hat drew up a list of major contributors to the open source distribution and sent out invitations.

“Dear open source community member,” began the e-mail letter that Red Hat sent to about 1,000 people.

In appreciation of your contribution to the open source community, Red Hat is pleased to offer you this personal, non-transferable, opportunity. . . . Red Hat couldn't have grown this far without the ongoing help and support of the open source community, therefore, we have reserved a portion of the stock in our offering for distribution online to certain members of the open source community. We invite you to participate.

Many programmers and developers were touched by the thoughtfulness. The list probably wasn't long enough or inclusive enough to pull everyone into the circle, but it did do a good job of spreading the wealth around. The plan began to backfire, however, when E*Trade began to parcel out the shares. Everyone who made it onto the list filled out a form listing their net worth, and E*Trade attempted to decide who was a sophisticated investor and who wasn't. Some folks who had little money (perhaps because they spent too much time writing free software) were locked out.

One contributor, C. Scott Ananian, wrote about his rejection in Salon magazine, “I filled out the eligibility questionnaire myself. I knew they were trying to weed out inexperienced investors, so on every question that related to experience, I asserted the maximum possible. I knew what I was doing. And it was my money, anyway--I had a God-given right to risk it on as foolhardy a venture as I liked.”

The article drew plenty of flack and murmurs of a class action lawsuit from the disenfranchised. A discussion broke out on Slashdot, the hardcore site for nerds. Some defended E*Trade and pointed out that a Red Hat IPO was not a lock or a guarantee of wealth. Too many grandmothers had been burned by slick-talking stock salesmen in the past. E*Trade had to block out the little guys for their own protection. Stock can go down as well as up.

Steve Gilliard, a “media operative” at the website NetSlaves, wrote, “If the Red Hat friends and family group were judged by normal standards, there is no brokerage in the U.S. which would let many of them buy into an IPO. In many cases, they would be denied a brokerage account. Poor people are usually encouraged to make other investments, like paying off Visa and Master Card.”

Others saw it as a trick to weed out the pool and make sure that E*Trade could allocate the shares to its buddies. The more the small guys were excluded, the more the big guys would get for their funds. In the end, the complaints reached some ears. More people were able to sneak in, but the circle was never big enough for all.

20.1. World Domination Pretty Soon?

Red Hat's big pool of money created more than jealousy in the hearts and minds of the open source world. Jealousy was an emotional response. Fear of a new Microsoft was the rational response that came from the mind. Red Hat's pool of cash was unprecedented in the open source community. People saw what the pile of money and the stock options did to Bill Gates. Everyone began to wonder if the same would happen to Red Hat.

On the face of it, most open source developers have little to worry about. All the code on the Red Hat disk is covered with a General Protection License and isn't going to become proprietary. Robert Young has been very open about his promise to make sure that everything Red Hat ships falls under the GPL. That includes the distribution tools it writes in-house.

The GPL is a powerful force that prevents Red Hat from making many unilateral decisions. There are plenty of distributions that would like to take over the mantle of the most popular version of Linux. It's not hard. The source code is all there.

But more savvy insiders whisper about a velvet-gloved version of Microsoft's “embrace and extend.” The company first gains control by stroking the egos and padding the wallets of the most important developers.

In time, other Red Hat employees will gradually become the most important developers. They're paid to work on open source projects all day. They'll gradually supplant the people who have day jobs. They'll pick up mindshare. Such a silent coup could guarantee that Red Hat will continue to receive large influxes of cash from people who buy the CD-ROMs.

There are parts of this conspiracy theory that are already true. Red Hat does dominate the United States market for Linux and it controls a great deal of the mindshare. Their careful growth supported by an influx of cash ensured a strong position in the marketplace.

In November 1999, Red Hat purchased Cygnus Solutions, the other major commercial developer of GPL-protected software, which specialized in maintaining and extending the compiler, GCC. Red Hat had 235 employees at the time and Cygnus Solutions had 181. That's a huge fraction of the open source developers under one roof. The Cygnus press release came with the headline, RED HAT TO ACQUIRE CYGNUS AND CREATE OPEN SOURCE POWERHOUSE.

To make matters worse, one of the founders of Cygnus, Michael Tiemann, likes to brag that the open source software prevents competitors from rising up to threaten Cygnus. The GPL guarantees that the competitors will also have to publish their source, giving Cygnus a chance to stay ahead. In this model, any company with the money and stamina to achieve market dominance isn't going to be knocked down by some kids in a garage.

Those are scary confluences. Let's imagine that the conspiracy theory is completely borne out. Let's imagine that all of the other distributions wither away as corporate and consumer clients rush head over heels to put Red Hat on their machines. Red Hat becomes the default in much the same way that Microsoft is the default today. Will Red Hat have the power that Microsoft has today?

Will they be able to force everyone to have a Red Hat Network logon button on their desktop? Perhaps. Many people are going to trust Red Hat to create a good default installation. Getting software to be loaded by default will give them some power.

Can they squeeze their partners by charging different rates for Linux? Microsoft is known to offer lower Windows prices to their friends. This is unlikely. Anyone can just buy a single Red Hat CDROM from a duplicator like CheapBytes. This power play won't work.

Can they duplicate the code of a rival and give it away in much the same way that Microsoft created Internet Explorer and “integrated” it into their browser? You bet they can. They're going to take the best ideas they can get. If they're open source, they'll get sucked into the Red Hat orbit. If they're not, then they'll get someone to clone them.

Can they force people to pay a “Red Hat tax” just to upgrade to the latest software? Not likely. Red Hat is going to be a service company, and they're going to compete on having the best service for their customers. Their real competitor will be companies that sell support contracts like LinuxCare. Service industries are hard work. Every customer needs perfect care or they'll go somewhere else next time. Red Hat's honeymoon with the IPO cash will only last so long. Eventually, they're going to have to earn the money to get a return on the investment. They're going to be answering a lot of phone calls and e-mails.