Design and the Limitations of the Computer Screen

There's an interesting quotation Tufte offers in one of his side notes on page 50 regarding the design limitations of computers:

"In user interfaces for computers, a problem undermining information exchange between human and software is 'constant context switches. By this we mean that the user is not presented with one basic display format and one uniform style of interaction, but instead, with frequent changes: A scatterplot is present; it goes away, and is replaced by a menu; the menu goes away, and is replaced by the scatterplot; and so on [...] This means that users constantly have to adjust to a changing visual environment rather than focusing on the data. The user is also forced to remember things seen in one view so that he or she can use the other view effectively. This means that the user's shortterm memory is occupied with the incidentals rather than with the significant issues of analysis.'"

Which of us hasn't faced this limitation in designing texts on a computer screen, which forces to pen and paper, or a combination of paper and screen, to plan conceptually complex compositions. This quotation doesn't even get at the non-intuitiveness of using a mouse and pointing with an arrow to complete tasks on the screen, forcing one, among other things, to manipulate their environment with one hand only.

But this problem may be waning.

To actively isolate and display varying pieces of election information, CNN is using a large touchscreen that involved intuitive hand movements to manipulate the screen environment. If the reporter wants to zoom in on a map, she pinpoints a area on the screen and pulls her two index fingers outward, as if opening window shutters. If he wants to zoom out, he places his fingers widely apart and pulls them together. To isolate particular states, she just points on the map.

Microsoft has designed a concept computer that is designed as a coffee table. The surface of the coffee table is itself the entire screen. The table offers intuitive touchscreen manipulation of digital objects. Want to upload images from your digital camera? Just set the camera on the table (no plugs) and your pictures automatically spill out onto the coffee table, as if pouring out a shoebox of snapshots. From there, you can sort and stack the pictures with the movement of your fingers of both of your hands, a pile on the left with your left hand, a pile on the right with your right hand. Want to crop a photo, slide across the table surface a straight edge tool that you align with your hands. With the enlarged surface, that is similar to a surface that we commonly use for physical projects, you have spaces available for multiple points of reference, without needing to close a particular menu or particular scatterplot.

Imagine, on such a surface, seeing the whole of an 8 1/2" by 11" page image while you are typing on it, without the need to scroll up or down. Or being able to spread those pages across a surface to visually see the entirety of your composition. Or breaking the boundaries of the sheet of paper entirely and still being able to see, as a whole, your composition while manipulating it.

This, among others, is why I see Kress' notion of the "dominance" of image or screen to be so shortsighted. We will quickly go, instead of screen, toward this intuitive surfaces, which we manipulate with both hands, as if in 3-D.