This history of interactive media is extremely rich and can inform us today. The development of the mouse and the graphical user interfaces that we use today can inform us of where we need to go next in interaction design and development.

The Mother of All Demos

In a huge conference room in San Francisco, Doug Engelbart walked out in front of a huge crowd, alone, and sat down in front of a console in the middle of a stage with a twenty-foot screen behind him. He turned to the audience and asked a simple simple question: “[what] if, in your office, you as a intellectual worker were supplied with a computer display, backed up by a computer that was alive for you all day and was instantly responsive to every action you had, how much value could you derive from that?” He asked this question in 1968. A year after the first Super Bowl and the first heart transplant. Man was still a year away from walking on the moon. The idea that anyone would be supplied with a computer that was constantly on and was instantly responsive was unheard of. He answered his hypothetical question posed to the Fall Joint Computer Conference with an hour and a half presentation that has been dubbed The Mother of all Demos (which you can view here: http://www.youtube.com/watch?v=X4kp9Ciy1nE). In it, he introduced the world to video conferencing, word processing, long distance collaborative working and hyperlinks. Driving the entire demo was Engelbart’s pioneering input device: the mouse.

Engelbart had been inspired by an article published in Life magazine in 1945 by Vannevar Bush titled As We May Think. In it, Bush imagines the “memex”, a desk-like device (fig 1) where users could riffle through a library of articles through a series of switches. While entirely mechanical, Bush describes a device that features hyperlinked text, aggregated notes and bookmarks all extending human capacity to research and process information.

Figure 1: The Memex

The article inspired Engelbart to quit his job and attend graduate school at UC Berkeley (Moggridge 2007, 30-31). He wrote a paper that was published in 1962 titled Augmenting the Human Intellect: A Conceptual Framework. In it, Engelbart fleshed out his ideas, greatly influenced by Bush’s memex concept. He envisioned a world where “a trained human being” would use computers and highly integrated systems to access a wealth of information through a visual display device (Engelbart 1962). He wrote that the system would consist of:

“…a trained human being, together with his artifacts, language and methodology. The explicit new system we contemplate will involve as artifacts computers and computer-controlled information storage, information handling and information display devices…to make significant use of such equipment in an integrated system.” (emphasis added)

This paper was pivotal in that it cemented Engelbart’s vision and he used it to join the Defense Advanced Research Projects Agency (DARPA) where he got to work building his system. When considering what sort of artifacts his system could leverage, he recalled a sketch for a device based on an engineer’s planimeter. The device would consist of two perpendicular wheels which one could use as a pointing device on a computer screen. One wheel would control the y-axis of the cursor, the other the x-axis. He brought out this old notebook when discussing the idea with his colleague Bill English at ARC. A hand crafted, wooden prototype was put together and the first mouse was born.

Figure 2 First Mouse in Hand, 1963-64

The mouse became one of the primary devices in the system they developed. It used a complex sequence of nouns and verbs. There was an assumption that the user would be trained in this language before they got started. This enabled for the display to do without any real user interface. The user would rarely take their hands off the keyset and mouse to use the keyboard or take their eyes off the screen.

However, this efficiency had a cost that would doom the initial implementation: it had a very high learning curve.

Figure 3 Workstation with Mouse and Keyset, 1966

The team decided to show off their project and presented at the Fall Joint Computer Conference in 1968 (fig 4). It was Vannevar Bush’s vision realized in the flesh and the team was extremely proud.

Figure 4 Misc. Images from Demo at Joint Computer Conference, 1968

Unfortunately, the team stagnated and number of key ARC team members to look for other research positions. They were lured away to the Xerox PARC.

Xerox had established a research facility in facility in Palo Alto, a suburb of San Francisco. This facility developed a number of huge innovations we still use today including laser printers and Ethernet (which is part of the backbone of the Internet).

The transition from the grant-funded research institution ARC to the commercial research facility Xerox PARC forced the engineers working at ARC to think differently about their invention. Rather thank thinking of it as the most efficient way to augment human capacity, they were forced to think about how it would work in the “real world”. User research proved that the mouse couple be simplified and the system could use it as the primary interface rather than the complex system Englebart had developed. However, the team had a problem. What was the dominant metaphor? Users understood how to drag objects from one place to another and how to select things user the mouse, but they had no metaphor to frame the interactions.

Figure 5 Tim Mott’s reconstruction of his sketch

One evening the team came up with an office metaphor. Mott writes:

“I was just obsessed with this design [problem] at the time; I was just consumed by it. I was thinking about what happens in an office. Someone’s got a document and they want to file it, so they walk over to the file cabinet and put it in the file cabinet; or if they want to make a copy of it, they walk over to the copier and make a copy of it; or they want to throw it away, so they reach under their desk and throw it in the trash can…the metaphor was that entire documents could be grabbed by the mouse and moved around on the screen…they could be dropped into a file cabinet, or they could be dropped onto a printer, or they could be dropped into a trashcan” (Moggridge 2007, 53).

They doodled their concept on a napkin (fig 5) and brought it in to the office. In many ways, they were conceiving the first real screen-based graphical user interface (GUI) for a computer, well beyond programming-like text displays that predated it.

Figure 6 Zerox Alto Interface

This concept was extremely well received, both by Tesler’s test users as well as by his management. He and Mott guided development of Xerox PARC’s user interface for the Alto (fig 6), released in 1973. The Alto was innovative in a number of key ways, but the bitmap technology of the screen enabled graphics and text to interplay in windows for the first time. This technology, combined with the office metaphor and the mouse created a relatively easy to use machine. Unfortunately, it cost over $18,000 to produce one (Ceruzzi 2003)!

Two years later, Xerox introduced a commercial version, the 8010 Star Information System (Ceruzzi, 2003). While it was released with a marketing blitz, the Star never took off. Xerox had made two critical mistakes:

1. It was exceedingly expensive.
2. They marketed their new product to executives rather than actual office workers. Executives would never see the relative advantage that this new way of doing business would have. They also would have no way of knowing how it was compatible with the existing workflows in an office. At this point, the mouse and the user interface that it drove seemed dead in the water.

Apple’s Lisa

The computer was rapidly becoming an extremely disruptive technology and established companies like Xerox had a problem. They had existing processes and systems that spelled trouble. Xerox perceived itself as a copy machine manufacturer and Xerox was struggling to maintain dominance in the copier market because of increased competition. They also had expensive union contracts that drove up production cost. The Star simply could not have the revenue that Xerox was used to.

Apple, on the other hand, was a small startup. They had tremendous success with their low-cost Apple ][ consumer product. They were manufacturing and selling computers for a $800, dramatically less than what Xerox was selling because they had relatively low overhead, innovative engineers who were able to do more with less, were comfortable with lower margins and where not hamstrung by Xerox’s existing production systems.

Apple was heading towards IPO and many companies wanted to invest. Xerox decided to negotiate with Apple to let Apple use their technology (including the graphical interface and the mouse) in return for Xerox being able to invest $1 million in Apple. Key figures from Xerox PARC began to migrate to Apple and begin work on their Lisa product and Apple employees were given rapid tours of the PARC and Apple picked up where the PARC team left off. First, they reworked the internal mechanics of the mouse to reduce the cost of the mouse from nearly $400 to only $25.

The interface was reinterpreted by the Lisa team as well. Fleeting glances at the Star while visiting PARC were really all the Lisa team had to go on. They quickly set about reworking the office metaphor and created many truly innovative interaction elements that continue to today: overlapping windows, the “desktop” and pull-down windows from items along the top of the screen with fly-out sub-menus (fig 7).

Figure 7,  Polaroids of the Lisa interface

 

However, when it was released in 1983, like the Star, the Lisa was expensive ($10,000) when compared to the less expensive IBM PCs that were being marketed to the same customers. Accordingly, it ran in to similar problems that the Star had. Decision makers for office purchased were not the same folks using them and those decision makers saw no reason to spend more money just for the mouse and the interface. It was simply too expensive and it failed in the market.

Again, it looked like the graphical interface driven by the mouse would not catch on.

Apple’s Macintosh

Partway through the development of the Lisa, Raskin had moved on from the Lisa to start what he called the Macintosh. Raskin realized the Lisa was too expensive and in the wrong market. He proposed something relatively radical: repurpose the graphical interface concepts and mouse from the Lisa, but develop hardware that could sell at a price that Apple’s existing Apple ][ customers could afford. His goal was $1500, a fraction of Lisa’s $10,000 asking price. At the time, this was unheard of. Raskin convinced a Burrell Smith, a boy-genius hardware designer, to join his team. Smith developed boards that could mimic the high-powered hardware behind the Lisa at a fraction of the price. About halfway through the development of the Lisa, Steve Jobs, co-founder of Apple, saw Smith’s boards and left the Lisa project to join Raskin (Hertzfeld 2011). Jobs managed to keep it independent of the established Apple organization.

Figure 8 Macintosh Control Panel interface

 

The Macintosh team continued to refine the product, paying particular attention to the graphical user interface. They simplified the Star and Lisa’s office metaphors trying to make it palatable to their consumer and hobbyist audience. Just from something as simple as the control panel (fig 8), it is clear that the interface was playful and accessible rather than cold and sterile like its predecessors.

In many ways, one merely needs to look at the marketing to see the fundamental difference between the Lisa and the Macintosh. The Lisa was clearly marketed to office executives while the Macintosh to consumers. The Lisa was a serious machine. The Macintosh was playful (fig 9). The Lisa’s mouse and graphical user interface was presented as yet another way to navigate spreadsheets and databases. The Macintosh showed a huge “hello” drawn by hand using the mouse.

Figure 9 Apple Mac and Lisa, from 1984 brochures

Early reviews simply did not understand. They were used to seeing computers look like children of mainframe computers and office copy machines. The Macintosh was something different. The interface was even compared to a “crib toy” by an early reviewer. Famed technology journalist John C. Dvorak wrote: “The Macintosh uses an experimental pointing device called a ‘mouse’. There is no evidence that people want to use these things.” Thankfully, the reviewers were wrong. The Macintosh was a resounding success and the mouse and graphical user interface were solidified as components of computing.

The graphical user interface of windows, icons, metaphors (like files and folders) and the mouse pointer became standard elements in computing. Interaction designers call this model WIMP (Windows, Icons, Metaphors, Pointers). Microsoft aped this interface for Windows and worked with hardware suppliers to dramatically undercut the Macintosh on price. After enjoying a handful of years of no real competition, the Macintosh lost its market share when Microsoft released Windows 3.1. It wasn’t until Steve Jobs returned to Apple (he was forced out of his own company only a few years after the release of the Macintosh) in the late nineties that Apple found its footing and recovered.

Today

The development of the mouse and graphical user interface is a fascinating look into how innovation occurs. In it, we see that innovation frequently occurs where different disciplines collide and through unyielding iteration towards solving real problems for real customers. It is this same approach that successful companies are using as we move away from WIMP interfaces towards Post-WIMP interfaces like the gestural controls in the XBOX Kinect and touch-based interfaces in Apple’s iOS devices (iPhone, iPod Touch and iPad), Android and Microsofts’ next iteration of Windows.

It’s an exciting time to be an interaction designer, developer and strategist. At AIMS, we’re preparing students by working as cross-disciplinary teams working to solve real problems for real customers.

---

References:

Bower, J.L. and Christensen, C.M. 1994. Disruptive Technologies: Catching the Wave. Harvard Business Review January-February 1995: 1-11.

Bush, V. 1945. As We May Think. LIFE 19 (11): 112-124.

Ceruzzi, P.E. 2003. A History of Modern Computing. 2^nd ed. Cambridge, MA: The MIT Press.

Crawford, C. M. 1983. New Product Management. Homewood, IL: Richard D. Irwin, Inc.

Doug Englebart 1968 Demo 2005. Online video clip. Google Video http://www.youtube.com/watch?v=X4kp9Ciy1nE  (accessed October 14, 2011)

Englebart, D. 1962. Augmenting Human Intellect: A Conceptual Framework. http://www.liquidinformation.org/engelbart/62_frset-1.html (accessed October 14, 2011).

Hertzfeld, A 2011. Revolution in the Valley: The Insanely Great Story of How the Mac Was Made. San Francisco, CA: O’Reilly Media.

History in Pix n.d. Doug Engelbart Institutue. www.dougengelbart.org (accessed October 14, 2011).

Licklider, J.C.R. 1960. Man-Computer Symbiosis. IRE Transactions on Human Factors in Electronics volume HFE-1: 4-11.

Linzmayer, O.W. 2004. Apple Confidential 2.0: The Definitive History of the World’s Most Colorful Company. San Francisco, CA: No Starch Press.

Moggridge, B. 2007, Designing Interactions. Cambridge, MA: The MIT Press.

Rogers, E.M. 2003. Diffusion of Innovations. 5^th ed. New York, NY: Free Press.

 

 

 

You can leave a response, or trackback from your own site.