The Smartest Pen Ever

I painted my living room and dining room over the winter break.  Both rooms required two coats.  Typically, I listen to audiobooks or This American Life podcasts while I do this kind of work.  I noticed that as I applied the second coat, I could remember vividly what I had been listening to the last time I had painted each part of the wall.  The Pulse Smartpen from Livescribe creates a similar effect for much better results — and no fumes!

The Pulse Smartpen from Livescribe is amazing.

When using the Smartpen, you can record audio and / or the movements of the pen.  The audio can be played back with the pen or uploaded to your computer (where you’ll also find .pdf versions of the notes you wrote!)  When you use the Smartpen on Livescribe’s special dot paper, it indexes the audio you record to what you were writing when you recorded it.

This is a boon when you go to review your notes.  The same way I recalled each of the 20 acts in 60 minutes while rolling on the paint around each faceplate, a student can hear what her professor was saying while she was taking each line of notes.  She can literally click on a line of notes and the pen will play back that part of the audio.  When demonstrated, it almost seems like magic.

Some of the practical uses of this technology are obvious.  John, the undergraduate student who demostrated his Smartpen to me in the Digital Union, said he found it most helpful for taking notes in chemistry, where he was often too busy drawing chemical structures to also note everything the professor said.  (Incidentally, John bought his $200 Smartpen the day after he had to return his demo model.)  Would it work for ESL students?  Perhaps.  Obviously, the combination of input types can really help different learning styles, and being able to review notes in more than one media would be an advantage.

I’ve also been thinking about how this technology relates to my earlier post on captioning digital audio.  With both technologies, they key is indexing the graphic with the auditory because the former is easily searchable.  Once Livescribe adds some kind of optical character recognition, which would make the notes more easily editable, and once searchable captions become a standard component of digital video, we will have finally integrated all of this information in a truly useful way.  Can the Singularity be far behind?

Captioning Digital Video

A screenshot from MP4Box. This is going to take some figuring out.

Got the lowdown on how the Web Accessibility Center at OSU captions digital video at the Digital Media in a Social World conference, which I have been attending today.  MP4Box, which is freely availalble, can be used to import a variety of video and caption formats, which results in a compliant stream.

As I’ve blogged before, this is exciting from an accessibilty perspective, a language learning perspective, and because digital video can become searchable using these transcripts.

From what I’ve heard, this process is a bit geeky.  I’m planning to roll up my sleeves and give it a shot.  I will, of course, report my results here.  If you have experience with this, or similar software, leave a comment.

Searchable Video – Enter the Dragon

I caught the tail end of the monthly Exploring Learning Technologies community meeting recently and became intrigued by the topic: accessibility. (Full disclosure: I’m part of the committee that plans these meetings.)

This is an important topic because, as more and more educational video is put online for class use (lectures, for example) accessibility becomes a greater issue. The more I heard, the more I thought about how processes like captioning video can be helpful with second language learners as well as people with hearing impairments. So, in general, the conclusion was, it’s good to make captioning part of your practice if you post video online. The most difficult part of this process, obviously, is transcribing the text.

There are several ways to do this, but, in general, you will need to pay someone to listen and type. Whether you hire someone yourself or use an online service, the cost of both kinds of service increase with the accuracy and rapidity of the completion of the transcription.

Dragon speech recognition software.

An interesting alternative incorporates Dragon speech recognition software. Unfortunately, you can’t just have this software listen to the video and produce a transcript. Background noise and other issues make this impossible. But you can have someone watch the video and repeat the transcript for the software. In effect, this someone becomes a biological interface between two digital entities! For a moment, I was distracted by images from the Matrix movies, in which machines use humans disposably, but then I started to realize the most useful feature of captioned video: searchability.

If you want to find a phrase in your favorite movie, you likely have to guess where it is and then skip forwards and / or backwards until you find it. This is difficult.  Now imagine looking for the same phrase in a movie you have never seen.  Or, searching a dozen movies. By searching a text transcript which is linked to the timeline of the movie, it would be extremely easy to find the phrase. Looking for “classroom technology?” The phrase is used at 03:58 and again at 17:22.

This process is costly and labor-intensive now, but eventually, whether speech recognition software is able to scan video and automatically transcribe speech accurately, or there is an offshore matrix of borg-like transcribers scouring YouTube, all video will be transcribed in a searchable way. This will make video useful and accessible in the same way that the Internet has made texts useful and accessible. And we’ll look back and say, why did we wait so long to do this?