The point is that almost every piece of information we collect has a trail to other bits of information. Right now we can’t see those trails. And so, our data spaces, be they our Gmail archives, or our hard drives, or our Delicious tags, etc., are really more like old attics with years of junk covered in thick dust. Modern software technology can and should do better.

• Production
• Creation
• Relaxation

The “real world” analogues to these categories are art, craft, and (for lack of a better word, since I don’t want to just reuse “relaxation”) entertainment. All life pursuits have elements of these categories. Art is driven by imagination. Craft implies more hard work and focus on technique. Entertainment is of psychological value, as a de-stressing agent, and can fuel both art and craft.

So, on your computer. Writing an email to your mom? Creation and relaxation. Looking up a recipe? All three. Doing your taxes? Production. Editing a home video? All three. Making graphics for the website your boss needs? Creation and production. Playing a videogame? Relaxation. Making weekend plans with a friend? Production and relaxation.

What does this mean for interaction design? How can programs understand what activity you’re engaging in and optimize its suitability for that activity? This gets back to context, which I’ve written about before and will write about again. If I’ve been chatting with a friend and then switch to my web browser and look up movie times, my computer should know that I’m in relaxation mode with a hint of the production, and make subtle changes to how it reacts to my activities: if I then start to look for someone’s email address, it should know to suggest friends, and preferably local ones, before coworkers. If I’ve been writing work-related code for a solid hour and then switch to my IM buddy list, it should know that I probably want to talk to my coworkers about something and, perhaps, enlarge their names in the list slightly. These are subtle touches, but easily doable, and if enough of them are consistently added across your entire computing platform, the overall enhancement to usability and simplicity could be tremendous.

Obviously this requires a level of computer “understanding” of your data and actions that hasn’t been reached yet, but there are strides in that direction. Social networking tools and community websites are making your relationship data available for programmatic access, so determining who’s a friend and who’s a coworker is becoming easier. By watching your calendar and work-related activities throughout each day, your computer can learn when you’re generally working, and take that into account when making schedule-related suggestions. By checking the location of your geotagged photos and matching the photo’s descriptive terms (tags, title, etc.) with other items in your data-profile, software can guess where certain people live or where certain activities take place.

This is all part of your computer being context-aware. Once the intercommunication between applications and (secure) exposure of your data to approved applications is comprehensive, your computer will be able to use its knowledge to make whatever aspect of computing you’re engaging in — creation, production, or relaxation — simpler, faster, and better.

Score isn’t everything

When I look at the TV and a game’s on, there are a handful of key data points I want to gather as quickly as possible so that I’m up to speed with the game and can pay more attention to the action than to the numbers. That’s the supposed purpose of the infographics: to provide a quick snapshot that I can instantly parse into a game state, to mentally construct a symbolic understanding of the game so far, which I will then use to fully understand the meaning of the teams’ current actions in the larger context of the entire game. These are the data points that I believe any football fan needs to build an accurate mental snapshot:

• Quarter: how far along is the game?
• Time left: are we nearing a break?
• Score: who’s winning, and by how wide a margin?
• Possession: which team has the ball?
• Down and Yardage: is the offense running out of chances to make progress?
• Ball position: how close is the offense to scoring?

Additionally, the graphic needs to be designed to take the following accessibility factors into consideration:

• Obtrusiveness: is there a good balance between readability and screen coverage?
• Contrast: is the graphic readable by viewers with poor vision?

How they do it

The three major networks that broadcast NFL games—Fox, CBS, and NBC—each have their own style of infographic, although Fox and NBC use a similar design, and while theirs are better than CBS’s, they’re still not great. Here’s why.

CBS

CBS chose to go with a shape-shifting graphic that hovers in the upper corner of the screen. Its default layout shows the Quarter, the Time left, the Scores of the two teams (using abbreviated team locations), and Possession is indicated by a tiny yellow dot. Occasionally, a black bar appears at the bottom of the graphic that shows the Down and Yardage, but this information comes and goes seemingly at whim. At other times, a yellow “Flag” indicator appears at the bottom, and sometimes other information about drive length, possession time, individual player statistics and such appear at the bottom or at the end of the “arm” of the graphic. Ball position is never shown. Occasionally, the information-rich arm disappears, leaving only the “NFL CBS” logo. Woe to the viewer who tunes in when that’s the only thing displayed on-screen!

The CBS graphic is somewhat Obtrusive, since it hovers in the corner of the screen, set well away from the edges, obscuring a portion of the camera’s view of the field. However, it does have good Contrast: the text is all white on black or black on white.

One nice subtle touch is that the arm switches direction to indicate the direction of the current offensive drive; unfortunately, doing this requires the graphic to swap from the top left to the top right of the screen, an inconsistency that I think is harmful; consistency is key.

NBC

NBC’s graphic lies Unobtrusively along the bottom edge of the screen. It reads naturally left-to-right in a single line, and you can immediately figure out the Score and the Quarter and Time Left. The section containing the Down and Yardage, however, is dynamic. Sometimes it shows nothing, sometimes the Down and Yardage, sometimes a yellow “Flag”. Nowhere is Possession indicated, which strikes me as a major oversight. The text has good Contrast, however, and the inclusion of the team logos is a nice addition.

Fox

Fox takes a similar approach to NBC, only their graphic stays almost at the top of the screen, and is narrower than NBC’s, making it very Unobtrusive. Contrast is decent, although the small fonts hurt readability. Scores, Quarter, and Time left are all constantly displayed and easy to find, but once again, the Down and Yardage displayed here is in a section of the graphic that is constantly in flux. The rightmost third of the graphic (showing “Playoffs - Divisional”) is unchanging, and could be put to better use providing more contextual information, such as the Ball position. Possession is indicated with a bold, high-Contrast outline, making it easy to tell which team has the ball.

How it should be done

The (fictional, of course) Yin Yang Network takes a similar approach to that of NBC and Fox, but enhances it by adding all the necessary information that the others leave out. On the left, we have the Scores, including team logos and team location abbreviations. Possession is indicated by a colorful highlight around one team (and on the bar above it), and Ball position is listed next to the team currently in possession. In the center is the necessary network logo. On the right is the Down & Yardage, beneath which can appear a conditional “Flag” marker (but never anything else). To the right of that can occasionally appear the play clock, when it is close to running out. And finally, on the far right, we have the Quarter and Time Left, fulfilling all our informational needs.

And as for accessibility, the graphic is thin, partially translucent, and sits at the very top of the screen, making it as Unobtrusive as possible. The fonts are big, bold, and Contrast well with the background, making them easily readable by even viewers with poor vision or colorblindness.

While I have no illusions about any network adopting my design, I figured it was a good exercise to compare what’s being done and see how it could be improved. What do you think? Is my design better or worse than the others? Have I missed any considerations? What would you have done differently?

(Three of Humanized’s four principals were just hired by Mozilla, which probably means Humanized will shut down, but Mozilla’s interface will probably improve. Congratulations to everyone involved!)

To generate interest and press, Humanized recently developed and released Songza, a simple website that lets you search for music and play it in the browser. It makes use of several Humane principles: transparent zero-interaction “monolog” boxes, pie menus, and text labels instead of icons (mostly). This is great! It’s a simple and useful demonstration of how a Humane interface can be quickly and easily understood by people even when they’re not used to it, and it shows how nicely it works. In this sense, it’s a success.

I consider myself a “power user” of the web. When I find a website that performs some function I enjoy, I try to incorporate it into my life as seamlessly as possible. One thing I use often, for instance, is Firefox’s Smart Keywords feature, which lets me specify a keyword that I can use to quickly send a search to a specific website. I’ve set up dozens and dozens of these, so that finding the information I want is just a few keystrokes away: I can type “google jef raskin” to do a Google search, or “wp jef raskin” to look him up on Wikipedia, or “amazon jef raskin” to find his book on Amazon, or “dict humane” to find a definition, or “wu chicago, il” to find the weather at the Humanized offices. Once I’ve set these up and used them a few times, they become habit, and I barely have to think; looking up the information I want from a site I trust is utterly simple, and the computer has “learned” how to use its smarts to help me achieve my goals more quickly. This is the essence of a Humane interface: turning the user’s interaction with the interface into a benign habit that requires no thought.

To set up Smart Keywords, Firefox has created a simple process: you right-click on a form field, then select “Add a Keyword for this Search…” and then specify the keyword you want to use. It works on any standard web form, and Firefox figures out all the nitpicky details for you. Handy, no? Well, when I tried this with Songza, something went wrong: Firefox didn’t offer me the option of adding a keyword for the Songza search form. Viewing the source showed me the problem: the Songza search form, contrary to standard web design practices, doesn’t specify an action. The whole interface is handled by javascript and Flash: when you push the search button or press enter, the text you’ve entered isn’t sent to the server via HTTP, it’s grabbed by javascript and sent to the server using AJAX in the background. Hence, Firefox has no idea what’s going on, and can’t automate it for me, thereby denying me the expected simplicity of turning Songza into one of my habits.

To further aggravate things, there’s no reason why the site should be designed this way. It would be dead simple, from a programmer’s perspective, to make the form send a GET request back to the main page, and make the main page automatically handle it. Then, voilá, the form works as a standard web form, Smart Keywords work, and people can even link directly to search results. Add an auto-play option, and when I want to hear a song, all I have to do is type “songza perpetuum mobile” and, seconds later, it’ll be playing for me. (”Perpetuum Mobile” as performed by the Penguin Café Orchestra is a beautiful song. If I could provide a link here for you to listen to it directly on Songza, something like http://songza.com/perpetuum%20mobile, I would.)

It frustrates me that Humanized is trying so hard to effect a sea-change in how people interact with computers, but they (at least with Songza) aren’t trying to play nice with standard functionality at the same time. I understand that a paradigm shift often requires cracking a few eggs, so I don’t expect everything to go smoothly as they try to show how the Humane Interface is better than the current accepted computer UI, but I don’t think it’s wrong of me to say that their attempts should try to work with the current system as much as possible. People are going to be more impressed, and more enthusiastic, if your groundbreaking interface also does all the handy little “power user” things they expect, and provides as many avenues to habituation as possible.

Edit: You know what? I feel so strongly that this would improve Songza that, if any of the Humanized guys happen to read this post and want to take me up on it (and give me access to their code, of course), I’ll gladly do all the work necessary to add the above functionality to the site. How about it?

The recipe is simple; it’s basically the filling for a rhubarb pie with the topping from an apple crisp, and it takes mere minutes to throw together. But be warned: both times I’ve made this so far this year, the entire dish has been emptied by the next day.

Creamy Rhubarb Crisp

Cut enough rhubarb into approximately half-inch pieces to fill a 9-by-13 casserole dish. In a bowl, mix the following:

• 1 cup sugar
• 4 tablespoons flour
• 1 teaspoon nutmeg
• 1 tablespoon tapioca flour (or corn starch or arrowroot powder)
• 1 tablespoon butter
• 2 well-beaten eggs

Stir the mixture into the rhubarb until all the rhubarb is coated. Then, in another bowl, mix these:

• ¾ cup brown sugar
• 1 cup rolled oats
• ½ cup flour
• 3 tablespoons butter

Spoon it over the top of the rhubarb, then bake (uncovered) at 350° F for an hour.

Jason Kottke doesn’t think offline mode is that important, which is a view I see often coming from tech-industry people who live in big cities (Kottke is in Manhattan), because most large cities have an abundance of open wifi networks available. For those of us who live in more rural areas, though, finding yourself without a connection to the internet is a commonplace occurrence. Since moving to New Hampshire, we’ve had two long-term power outages (too long for our UPSes to handle), and more than once I’ve sat myself down in a public place only to discover that the wifi network I thought I could connect to was too weak, so I’d either have to leave my comfy spot and try to find a place with a better connection, or just attempt to work offline.

Anyway, I look forward to being able to use Docs offline once Google gets its butt in gear (har) and adds Gears support.

Today, I watched a video of Cory Doctorow giving a talk about these topics (his favorites) at Google. You can watch on YouTube if you’re interested. It’s an hour long including the Q&A session at the end, but Doctorow is a compelling speaker and it’s a great talk if you care about information freedom and the future of technology. The reason I bring it up, though, is that thirty minutes into his talk (at 30:45, actually), he offhandedly mentions that he’s “doing some consulting on a movie” that Kirby Dick, the creator of “This Film is Not Yet Rated,” is doing.

So I think that, given Doctorow’s involvement, maybe Dick is actually in the process of making that copyright/software patents/DRM documentary that I dreamed up. You heard it here first!

I’m sorry Dave, I’m afraid I can’t do that

People think that making our computers into “smarter” tools is a bad idea. They say they don’t want a system that does things they don’t ask it to–as if they have one of those right now. They say you can’t make a smart computer that will actually be smart in a useful, non-aggravating way.

Like the Dread Pirate Roberts says, nonsense. You’re only saying that because no-one ever has.

It’s easy to make a smarter computer. All you need is some simple algorithms that look for repetitive behavior, on any timescale, analyzes it to figure out what’s being done, and then offers to do it automatically. The problem isn’t figuring out how to do it, it’s causing the paradigm shift in software design that will make this sort of analysis a natural and expected part of any application. Smartness needs to be used regularly and shown to be so effective that users begin to demand it. Why use an app that makes you do extra work when this smart app makes everything simpler?

For example, the computer can pay attention to when I power it on: usually around 6:20 a.m. on weekdays and 8 a.m. on weekends. It can notice that on days that my calendar says I’m away from home, the start time is erratic, and on days when I have a morning appointment, I don’t turn it on until afterwards. Then it can ask if I would prefer it to power itself on automatically based on these observations, and if I say yes, most days my computer will be sitting waiting for me, powered up, when I want it to be.

The system can notice what I do with it on a regular basis. If I tend to start a certain set of programs immediately after booting, it can ask if I want it to start those programs automatically after booting, and arrange their windows the way I arrange them. My web browser can notice what sites I usually go to at certain times of day and pre-cache parts of those sites to speed up their loading.

And on a smaller scale, if I’m going through a text document adding the same few characters to every line, the system can see what I’m doing after the first few lines and offer to do it for me, show me what it would do, and allow me to tweak that result if it’s not quite right.

Adding context

The more context the system has, the more it can respond to our needs. A computer with a video camera can tell when someone’s sitting down in front of it to use it, and spin up its hard drives in anticipation. A computer hooked to a weather station can adjust the estimated travel time for a trip if the current weather indicates road conditions are bad. A computer that knows when your alarm clock is set for can learn how soon after waking you tend to start it up, and get itself going for you by that time. A computer that pays attention to your calendar can have driving maps at the ready when you need to go somewhere you’ve never been before. If you want many more interesting ideas about context-awareness, read Bret Victor’s in-depth “Magic Ink” paper. It’s long, but well worth the read if you find this sort of stuff intriguing.

First thing in the morning, a “status report” would be great: something that summarizes useful stuff, like the weather forecast for today (detailed) and the next two days (brief), how many emails I have waiting, how many new items in my newsreader, any important news stories that broke overnight, and other customizable information based on my interests. It should read my calendar remind me of today’s scheduled items; if I’m traveling and have put my destination on the calendar, it should find the correct forecast for my destination. All this is available with a good portal website, but I’d want it to also integrate with the other devices in my life. It could query my DVR and find out what was recorded last night, what was not recorded that I would have liked, and what’s going to be recorded soon. It could find out if I missed any calls while my phone was off during the night, and show me the names (if available) and numbers of the calls, and say whether they left messages. Ideally, it could even query my car (wouldn’t a car interface like this be great?) and find out if there are any issues that need to be addressed. And so on.

Wrapping things up

I don’t think I’m asking for much. I want a computer that is stable enough to survive hardware failures and able to notify me when those failures need to be dealt with, usable enough that I can be very productive without needing to simultaneously have two hands on my keyboard and one on my mouse, simple enough that taking it apart and fixing items is a snap, and smart enough that it anticipates my needs and provides the information and behavior I want before I even realize I want it.

It sounds like a pipe dream, but I don’t think the stuff I’ve asked for in these five articles are out of reach, either from an engineer’s, a programmer’s, or a customer’s standpoint. With effort in the right directions, I believe all of this can be accomplished, and I don’t even think it would take that long once the right people set their minds to it.

I’m one of those people. As a computer programmer, I can start doing my part by attempting to write smart software that is context-aware and tries to learn patterns and anticipate needs. I’m just a small part, but if everyone does their part, it won’t be long before we have stable, usable, simple, smart computers that are more fully extensions of our lives than just dumb tools.

(This post is the fifth and final part of my five-part post series “Reimagining the PC.” You might like to start back at the beginning.)

The Filesystem is a Robot Butler

Picture this: you wrote a paper by hand recently about chimpanzee mating habits and you put it somewhere in your house. A while later, you want to fetch that paper again, and thankfully, you have a robot butler to do the fetching for you, but first you need to tell the robot butler how to find the document. If the robot butler is like the current computer filesystem, you’d have to say “Robot Butler, bring me the document entitled “Hey Good Ooking” that’s in the yellow folder in the center pile on the top shelf of the second shelving unit on the western wall of the small room downstairs.” However, if your Robot Butler was designed for maximum usability, you’d just have to say “Robot Butler, bring me that thing I wrote about chimps,” and the Robot Butler would know just what you were talking about.

I dunno about you, but I’d rather have that second type of Robot Butler. Current computer filesystems are designed to be efficient for the system, and for programmers, to use. Their tree structure is logical and easy for simple algorithms to parse and build. However, to a human, trees like that can become unwieldy quickly. When I want to access something and I can’t remember if it’s in /bin, /sbin, /usr/bin, /usr/sbin, /usr/local/bin, /usr/local/sbin, or /usr/X11R6/bin, that’s a problem. People don’t remember which branches of trees to follow to get to the fruit they’re looking for, they just look up and grab the first fruit they see. A filesystem should allow us to specify what we’re looking for based on metadata–that is, easily-remembered contextual information about the thing we’re looking for. When I want to open that paper I talked about above, I don’t care where in my filesystem tree it’s located. I just want to say “open the document I wrote a few days ago about chimps” and have it appear before me. “Tagging,” which is all the rage on the web these days, is a useful and simple type of metadata that makes categorizing and finding information quicker.

A Humanized Filesystem

To solve this problem, we need a database filesystem designed primarily for human use. It needs to keep track of all the metadata it can, things like when the file was created, who created it, what type of file it is, and keywords it can extract from the file, as well as allowing us to specify our own metadata for it. Files should be associated into projects or packages for easy access. When we feel the need to view our creations, they should be grouped by default into projects and ordered by when we last worked on them, so that our current projects are the first things we see, and finding something we worked on last week is just a matter of scrolling back along the timeline a little ways.

A filesystem like this would require us to ditch the familiar but frustrating “open file” dialog that we’ve all become so used to. Opening a file will no longer be an exercise in digging through tree branches. Instead, the system will just need to prompt us for metadata. The new “open file” dialog will present us with a sensible selection of contextually relevant projects (recent ones at the top of the list, of course), and ask us to type some keywords. If we need to narrow things down further, we can select a timeframe or a filetype for the file we’re looking for.

Efficiency through Organization

Furthermore, we don’t expect one program to handle all our different filetypes for us, so why do we expect one filesystem to be the right choice for every type of file? I visualize four distinct filesystems on a computer: two for the system to use, two for the user.

For the system:

• Application filesystem: small blocks, organized to keep each application’s files clustered. All application settings, etc. stored in the AppFS. Programs bundled into packages so that uninstalling is as simple as deleting the package, and all settings go with it. Shared libraries are also stored in the AppFS.
• Cache filesystem: small blocks, used for high-traffic things like cached web files and temporary files. Transient user-specific application data, basically. When the CacheFS is close to full, the oldest stuff begins getting auto-expired, but until then it’s easily and quickly recalled.

For the user:

• Document filesystem: small blocks, designed to store user’s documents: letters, stories, papers, memos, etc.
• Media filesystem: large blocks, designed to store audio, video, and imagery. A notable difference in this filesystem could be the generation of psychoacoustic and psychovisual metadata for each file, to help keep track of media based on what it looks or sounds like instead of its exact data.

Of course, these different filesystems would be largely transparent to the end user. They wouldn’t need to know or care which filesystem is used to store a specific piece of data, and the operating system and applications should be designed to make it a non-issue. The organization of data among the filesystems is only there to make the system more efficient in regards to drive space and more sensible for programmers and software designers. Additionally, fully encapsulating applications into packages in the AppFS will keep cruft from building up in the system, and restricting most user activity to the DocFS and the MediaFS will help maintain software stability.

In the next and final installment of this series, I will explore how the system can proactively become more than just a glorified calculator by sensing and incorporating context in its algorithms.

(This post is the fourth part of my five-part post series “Reimagining the PC.” You might like to start back at the beginning.)

The keyboard is a fantastic invention: an arrangement of buttons designed to let a person type any word or string of symbols they choose. However, the QWERTY layout (it is often said) was designed to slow typewriter typists down, so that the typebars wouldn’t jam. This is obviously not a concern with modern keyboards, but do we ever see any serious attempts to correct the layout and increase typing speed and productivity? Dvorak has sadly never caught on. And worrying about the layout doesn’t even touch on the fact that, with the addition of the ubiquitous computer mouse, the typical computer interface is optimal for a human being with three hands.

The mouse is a stopgap measure. Originally created to act as a sort of finger-proxy, a way to translate the intuitive human ability to point, push, drag, and tap with the fingers onto non-sensitive screens, it could be entirely superfluous now that touch-sensitive screens are both available and relatively inexpensive. But we’re stuck in our rut, and we don’t want to give up our three-handed interface, so we continue using a system that would be perfect for a mutant but requires wasted effort for most of us.

What to do?

Let’s face it: half-assed measures won’t cut it. If people have the option of sticking with the familiar suboptimal interface, they will. The redesigned computer shouldn’t give people this option. Its redesign must be so compelling, so obviously superior, that people will be willing to make the leap from the comfortable to the new. We need a powerful, intuitive new interface that eliminates the three-hand problem while still allowing us to do everything we already do and more.

Lots of research is being done into new input interfaces for PDAs and mobile phones and small-sized tablet PCs, but very little time and effort is being spent on regular old desktop or laptop input interfaces. I’ve spent hours looking into one-handed keyboards like the Matias Half Keyboard, the Frogpad, and the Twiddler, but they all suffer from one notable problem: typing anything other than regular English text is very difficult, so programmers like me will experience significant slowdown, which can be a stumbling block to staying in the creative zone. I’ve checked out on-screen writing software like Quikwriting and Dasher, but while these would be great for mobile phones or PDAs, they’re far too slow for regular use.

The makers of the Frogpad have begun mentioning that they’re working on a Frogpad for programmers that makes typing punctuation and symbols easier, but until that comes out, I think we’re effectively stuck with a two handed keyboard. So what can we do?

I recently took an informal poll of some friends to find out how they hit the spacebar on their keyboard while typing. Over 4/5ths of them answered that, like me, they always use the same thumb to type a space, leaving the other thumb always hanging there uselessly while they’re typing. An eye-tracking system uses cameras to track where on the screen you’re looking, and requires you to use something to input mouse clicks. Why not use that other, useless thumb? Instead of having a giant spacebar on your keyboard, just cut it down slightly and use the extra space as a “mouse key.” Then, to move the mouse pointer around the screen, you just have to look where you want it, and to click you just tap that key. Sure, you’d only have one button, but as Mac users will tell you, that’s not necessarily a problem.

The drawbacks to this method are that accurate eye-tracking hardware is expensive, cutting down on the spacebar’s size will be uncomfortable to many people (at least at first), and trying to control the mouse via eye-tracking is somewhat difficult, since eyes are prone to drifting and moving somewhat unsteadily. For more information on eye-tracking, see the Stanford HCI EyePoint project and the openEyes resources from Iowa State University. And finally, it leaves users with the suboptimal QWERTY keyboard layout, which most people won’t voluntarily abandon.

The big pro of this method is that it gives you the ability to handle positional input with your eyes as well as use both hands for typing on a full-size keyboard–a truly two-handed interface, as long as the tracking hardware and software can be made usably accurate.

But eye-tracking isn’t the only potential solution. Ever since Apple announced the iPhone, the web has been abuzz with talk of how multi-touch screens will revolutionize computers. Most touch-sensitive screens in the past have only been single-touch: they could only sense where you’re touching with one finger, and if you used more fingers, the interface would either ignore them or get confused. Multi-touch screens allow you to touch the screen with multiple fingers, manipulating objects from various points at the same time.

While people are oohing and aahing over Apple, most of them haven’t heard of an NYU researcher named Jefferson Han who has been working on an intuitive multi-touch interface for years. While his technology isn’t available outside his own lab yet, someday it will be. To get an idea of just how cool a well-designed multi-touch interface can be, try watching this video of Jeff demonstrating his interface at the TED Conference in 2006 or reading this in-depth FastCompany article (and be sure to watch the accompanying video for that, too).

Unfortunately, multi-touch screens don’t necessarily provide any sort of fantastic new method for text input. You can use both hands to manipulate objects on-screen, but when you want to type, you’ll be forced to take both hands off the screen and place them on a keyboard, or maybe just type on a simulated on-screen keyboard. Either way, you can’t effectively do both textual and kinetic input, so while multi-touch screens may provide a more intuitive and thorough way to interact with the computer kinetically, it still leaves us with the original problem.

I envision our redesigned computer using a combination of these two technologies. While typing, you can control the mouse with your eyes and a mouse-button on the keyboard. If you want to manipulate things on-screen more directly, you can reach up and do so via the multi-touch capability. This isn’t ideal, but I think it’s better than what we have now, and it’ll have to do until we get those direct neural interfaces up and running.

Next time, I’ll explore how the core of the computer’s operating system can be better-designed to support a human way of thinking.

(This post is the third part of my five-part post series “Reimagining the PC.” You might like to start back at the beginning.)