Annotate reality with the LiveScribe smartpen

When I first heard of the LiveScribe Pulse smartpen, I wondered:

“OK, it’s a pen that records a lecture. I already carry with me two or three devices that can record audio and video. Why would I want another ?

After trying it out, I can tell you why you might want this. It is a lot more than a recording device. When the Pulse smartpen records audio, it syncs that recording to whatever you are writing or drawing at the time. Then it OCR’s your notes (even if your handwriting is terrible!), and makes them searchable. You can search the lecture for keywords based on your notes. Then copy and paste the text of your notes into other applications.

Lecture capture that is more than just powerpoint. 

The obvious use for the smartpen that comes to mind first is for taking notes at lectures. It captures the notes you take, diagrams and equations and syncs them all to the recorded audio. But once you start thinking about what it really is, there are lots of other possibilities, anywhere you might want to annotate reality, record your drawing, writing, and sound. 

Like: 

• A doctor taking notes as they ask a patient for information about their condition. 
• A reporter annotating an interview or discussion. 
• A video editor making notes over an audio track. 
• A psychiatrist taking notes during a patient session. 
• A reviewer or professor annotating a musical composition.
• A teacher sketching a diagram and verbally annotating it as you sketch.
• Architects or engineers brainstorming a new design on paper and capturing the discussion while drawing.

Your entire session can then be played back, and the diagrams and handwritten notes are re-drawn exactly as you originally drew or wrote them. So it can be used for recorded demos that need to be drawn by hand. This is artist’s screen-capture software.

Now here’s where it gets really cool: the smartpen can understand printed or DRAWN graphics as controls. In other words, the buttons and commands that control or the pen’s various applications are printed on paper, or drawn by the user. The covers of the pen’s notebooks contain printed calculators, including statistical, financial and scientific, as well as a keyboard for “typing” with the pen. 

The “Nav Plus” symbol is what you touch with the pen to navigate up, down, left and right through the pen’s menus. If you DRAW a cross on the page, and click the center, it will recognize that as a navigation symbol. Use the navigation to enter the Piano application, and it will lead you through drawing 8 “keys” after which you can play your little 1-octave synthesizer, complete with multiple instruments and rhythm tracks.

Click anything you have written on the page, and it will play back whatever audio was going on at the moment you wrote that note.  It’s quite fast at jumping from place to place in the audio – you could use it to piece together words or phrases.

Though the pen will write on anything that a ballpoint pen would, the recording features require that you use  “dot paper” with a unique ultra-fine dot pattern in the background. The pattern is so fine that it is difficult to see without magnification, so it will not interfere with your writing. If you have a color laser printer, you can print your own, but the dot paper notebooks aren’t expensive. 

The best thing about the pen is that it has an open API, and active developer support, as well as an app store. There are already lots of applications, including games, language assistance and educational apps.

On their educational blog, elivescribe.com, there are ideas for using it for teaching math, music, for assistive technologies, making anatomy study guides, etc.

Blue sky ideas for the future

Ideally, it would be nice to be freed from the special paper, or perhaps to make the paper act like a monitor in designated areas – designated on the fly, by drawing! Draw a rectangle, start writing a URL, and the rectangle fills with a mobile-sized website. Draw a button, write “Bookmarks”, and open up a list of bookmarks. Write a note to someone, click a button and email it right from the paper. 

Well, you get the idea. Sort of like…

Annotate whatever you hear