Kinect-Based IWB

Ever since a $3000 bounty was placed on cracking open Microsoft’s fab new gaming hardware, the motion-sensing Kinect for Xbox, hackers and tinkerers have been putting the open-source drivers to lots of interesting uses on platforms that Microsoft never envisioned.  I’ve written about interesting Kinect hacks before (and before that,) and I’ve written about my experience with the Wii-based $50 Interactive Whiteboard (IWB,) but I haven’t seen a fully-developed Kinect-based Interactive Whiteboard.

Perhaps an Interactive Whiteboard is too narrow a description.  Many of the pieces are in place (see below) to interface with a computer using Kinect.  So, as with the Wii-based IWB, any application you can use on your computer can be controlled by this hardware.  If you connect your computer to a projector, you essentially have an Interactive Whiteboard.

Is the Kinect-based experience different from a Wii-based IWB or a Smartboard?  Almost certainly.  There would be no need to touch the screen at all, but rather to gesture in front of the Kinect to interact with the projection on the screen.  Would this be an improvement?  I’m not sure.  A touch-based IWB is more analogous to traditional whiteboard that uses markers and an eraser.  So, the touchless experience would be quite different.  I need to try it myself to really wrap my head around the opportunities that this motion-sensing interface offers.

I’m not sure if anyone here at Ohio State is working with Kinect as an interface for non-Xbox applications.  But I do know that the Digital Union has a Kinect which could probably be used to see if and how things work.  If anyone else is interested in trying to pull this together, drop me a line or leave a comment.

Multitouch with Kinect

Kinect on a Mac

Multitouch Kinect

Kinect Fingertip Detection

Kinect + PC + Mario = Fun

Kinect Development

Microsoft recently announced plans to release a software development kit (SDK) for the Kinect.  This should allow academics and enthusiasts to find new ways to connect the motion-sensing Xbox hardware to other platforms, such as desktop and laptop computers, much more easily.  In short, there should be many more Kinect hacks to come.

I’m still not sure how this would directly apply to classroom teaching, although it stands to reason that these applications could someday replace physical interactive whiteboards in the same way that Kinect was originally designed to replace physical videogame controllers for the Xbox.

For more, see my previous post on Kinect Hacks and below for some new examples of how Kinect is being used in new and exciting ways.

Control Windows 7

The touchless multitouch is really nice.  Mice are so 2008.

3D Tetris with Face Tracking

As the user moves his head, the perspective on the screen changes to match so that the 3D perspective is constantly updated.

Kinect Lightsaber

A wooden stick becomes a lightsaber in real time.  This would save hours of  frame-by-frame editing.

Balloon Body

After Kinect scans your body, use your scroll wheel to expand or contract the surface.

Christmas Lights

Use Kinect attached to a bunch of dimmers to control Christmas lights for a very nice effect.

Flying Robot

The 3D capability of connect makes it perfect for a robot that navigates three-dimensional space.

Click Don’t Scan

Have you ever taken a picture of the board at the front of your ESL classroom?  It’s actually a pretty good way to capture lots of notes in a hurry, but you won’t be able to edit those notes once the picture is snapped.

Some document scanners have built in text recognition, but it can take a while for the scanner bar to drag across the document.  Sure, it’s only a matter of seconds, but if you have a big stack of documents to put through the scanner one page at a time, it can be a real inconvenience.  In fact, this scanner bar technology (a one-dimensional sensor being dragged across a two-dimensional surface) seems just a bit out of date, doesn’t it?

Enter a new line of scanners described in Popular Science that incorporate digital camera sensors to capture an entire document at one time — no more waiting for the sensor to drag.

But wouldn’t it be nice to snap a picture instead of scanning a document?  Well, it turns out there is an app for that.  Scanner Pro (reviewed by cnet) turns your iPhone into a .pdf-producing document scanner.  Forget trying to find a fax machine when you need to sign a document and send it to someone.  Sign a document, then scan it and email it, all from your phone.  There are other apps available for iPhones and iPods beginning at $0.99 and likely similar options for other flavors of smartphone as well.  The future is here today!

Thanks to the OSU Yammer community for ideas and links used in this post.

Open and Kinect

A few days ago, I wrote about how the new Microsoft Kinect has been hacked so that you don’t need an Xbox to use it.  There are now lots of tinkerers and hackers working with this hardware to see what else might be possible.  Although it’s not as easy to see the immediate applications for Kinect in the language classroom as it was for the Wii-based interactive whiteboard, there are obvious parallels.  And this new gaming hardware is more advanced than the Wiimote, which may offer more possibilities.  I’ve posted some examples of some interesting Kinect-based projects below.

How does it work?

Infrared beams, and lots of them.  Here’s how it looks with an infrared / nightvision camera.

Multitouch IWB

Because Kinect can “see” surfaces in 3D, it can be used to create a multitouch interactive whiteboard on multiple surfaces.

Control your browser

Forget your mouse.  Kinect can see the gestures you make in three-dimensional space.  Use gestures to control your browser and more.

Teach it

Teach it to recognize objects.  Obviously, there is a lot more software in use here, but Kinect provides the interface.

Digital puppets

Who wouldn’t want one of these?

Visual camouflage

In 1987, the movie Predator cost $18M.  A significant portion of what was left over after paying Arnold Schwarzenegger was likely spent on the cool alien light-bending camouflage effects.  Just over 20 years later, you can make the same effects on your computer using the $250 Kinect hardware.

3D video

At first glance, this looks like really poor quality video, but stick with it.  Notice the Kinect camera does not move, but with the flick of a mouse, the point of view can be changed as Kinect extrapolates where everything is in the space based on what it can see from where it is.  The black shadows are where Kinect can’t see.

Using 2 Kinects, most of the shadows are filled in.  The effect is like a translation of the real world into a low resolution Second Life-like environment.

Hacking Kinect

I never really thought much about Microsoft’s Kinect until I saw what hackers were doing with it.  A story in the New York Times outlines how a designer and senior editor at Make magazine posted a $3000 bounty for the first person to post an open-source hack of the Kinect interface.  Huzzah!  In fact, I’m still not that impressed with it — 3D drawings are cool, but will they help me teach English? — but I’m thrilled that hackers big and small are poking around under the hood.

Interestingly, Johnny Chung Lee, who became famous for his TED talk where he described hacking a Wiimote to act like an interactive whiteboard, is involved in the development of Kinect.  Microsoft were so impressed with his skills on the Wii-based IWB and other projects they hired him.  He is reportedly very happy to see hackers taking on Kinect in the way he took on Wii a couple of years ago.  If a hacker can squeeze an interactive whiteboard out of a $40 Wiimote, what will come out of the $150 Kinect system?

Will this technology help us teach ESL and EFL?  It’s not easy to see how, at least not immediately.  But prepare for a giant step forward in how we interface with computers in the next few years.  Interactive whiteboards are just the beginning.  You can always show your students this video and ask them to predict the future (in English).

Computer Games in ESL

Computer games are a medium that has become as popular as Hollywood movies.  It’s not uncommon for teachers to show movies in class, but how can games be incorporated?  This post will discuss these questions and will serve as the handout to my session at the Ohio TESOL Technology Fair 2010.

Games as a source of English

MMORPGs and other complex, multiplayer games can provide a rich source of English (or many other languages) in which students can choose to immerse themselves.  Also, because these games are almost impossible to solve without teamwork, there are typically

World of Warcraft (WoW) – The granddaddy of all MMORPGs and by far the biggest.  Players create avatars that go on quests and have adventures.   In addition to finding potentially complex communication tasks with other players during the game, the WoW wiki is the second largest wiki on the internet after Wikipedia.  This is also potentially a good source of target language input.

Second Life – Though not exactly a game, per se, Second Life is an online 3D virtual world through which players’ avatars can navigate.  There has been much educational interest in Second Life which means there are several “islands” dedicated to language practice for users to explore.

Analysis of Simulations

Distinguishing between games, serious games and simulations is not as important as how we can use them.  Playing a simulation may not be entirely satisfying because a totally accurate simulation of any complex system is extremely difficult to create.  But this creates an opportunity for students to try them and then critique them.  There are lots of examples listed on Historical Simulations.org.  Some of my favorites are below.

Budget Hero – Where would you increase and / or decrease the federal budget and what ramifications would each decision have on the future?  Lots of information in a very accessible format.

Energyville – See if you can meet the energy needs of a city of almost 6 million people.  Do you think cutting all fossil fuels immediately is the answer?  Not in this simulation.  Does knowing that it was created by Chevron make you question anything about this simulation?  Lots to discuss here.

McDonald’s Video Game – What decisions would you make (have to make?) to keep your franchise humming.  Would you cut corners?  What effects would this have?  And, as with Energyville, above, is this an unbiased view or is there an underlying message in this game?

Group Problem Solving

Even the simplest games can generate complex discussion when played in pairs or as a group.  Two students working at one computer need to negotiate everything starting from who gets to use the mouse.  If you have access to an interactive whiteboard, a larger group can work together to play the game or solve the puzzle much more comfortably.

Samorost 2 – This is a visually compelling game that, at first, does not seem to have a point.  During a brief cartoon introduction, a dog is kidnapped from a tiny planet by space aliens and the protagonist begins his pursuit.  Now what?  By clicking on various items on the screen they can be manipulated.  Puzzles can be solved by finding the appropriate series of manipulations.  I’ve had a half-dozen students working on these puzzles on an interactive whiteboard.  I was worried because when working alone on my desktop, I found the puzzles to be quite challenging and I almost gave up on more than one occasion.  But the power of the group was amazing to see as the students moved quickly through several levels, working together and suggesting new ideas to each other as they went.

Grow Cube – I’ve been intrigued by this game since the first time I discovered it.  It is a puzzle in which the player ten turns to select from ten actions that can be performed on a cube.  Each one has a cute animation that interacts with the others.  Most importantly, some actions require several turns to fully develop.  Others must be performed sequentially to work properly (spoiler alert: place the pot on the cube before lighting the fire or the fire will grow too hot and crack the pot.)  The puzzle typically requires one or two attempts to get a feel for the game before players can really begin to notice the effects that each action has on the others.  Even if the puzzle can not be solved, there is a complete walkthrough available for help.  But do yourself a favor and don’t peek until you’ve given it a few tries.

Other Tips & Suggestions

Some of the most complex games available will also be the most expensive.  Even if that hurdle is overcome (possibly by purchasing older versions, for example), there may be a variety of reasons that prohibit the installation of World of Warcraft in the local computer lab (“You want to play games?!?”).  Fortunately there are a number of online games which are freely available and only require an internet connection to play.  Of course, if you routinely battle a firewall for internet access, you may want to test whether you can access them on the computers you intend to use before you plan to use them.

Ask what your students are playing and see if those games might provide a jumping off point.  Are they addicted to Farmville on Facebook?  Bejeweled?  Can they analyze the game critically?  Can they teach someone else the strategy involved?

Most games have wikis which describe all of the parts of the game as well as strategies that can be used to win.  Is there an undocumented way to win?  Have students contribute their ideas to the wiki.

Mischievous Mouses

Unlock your mouse!

I occasionally get emails from people looking for a little exposure for a new product or service.  Excluding the copious spam, I do appreciate the interest in my blog, though most of these messages miss the mark.  I won’t write about something unless I actually see it as useful.  Fortunately, I recently received something useful in my inbox.  From Microsoft.  It’s called Mouse Mischief.

After watching the video on the homepage, I’m intrigued.  Mouse Mischief allows a teacher to connect up to 15 mice to one computer.  If each student has a wireless mouse they can all interact on the screen at the same time.  Each mouse can have a different cursor, so a teacher can ask a question and have students mouse-over the answer.  Students can also perform other mouse functions like drawing and clicking from their seats.  Teachers can even track which students answer correctly as well as how often and how quickly.

I like this idea because it’s a bit like clickers, which give each student the ability to answer a question by pushing a button, and a bit like an interactive whiteboard, which allow students to interact collaboratively with the screen.

Of course, moving a mouse on top of a desk is not the same experience as actually touching a touchscreen.  And wireless mice aren’t free, though Microsoft conveniently links to some which start at about $20 —  plan on about $300 to be fully outfitted, assuming you already have a computer and a projector in your classroom.  Plus the software, which is free to download, only works with certain versions of some of their products (PowerPoint 2007 and 2010, for example).

Microsoft seems to be taking the right path by encouraging teachers to upload their best lesson plans so that teachers can collaborate on the best uses of this technology.  Will it be the next “big thing”?  I’m not sure it will, but it’s an innovative use of an old technology.  And if using one mouse is good, I’d like to see what can be done with 10 or 12 of them.

OutSMARTed

I just got a letter (dated June 14) from the folks at SMART, makers of SMARTboards, etc.  I’ve included the full text of the letter below, in case the link doesn’t work.  It wasn’t sent to me personally, but rather sent out to licensed users of their products.

As I have mentioned before, I really like SMART hardware, but SMARTboards are expensive and not very portable.  I’ve also documented a $50 DIY alternative, based on the Nintendo Wii controller, which has fewer features but can fit in a large pocket.  The SMART software is also very good, but there are several alternatives and workarounds that can accomplish many of the same things.

This puts SMART in a difficult position.  They have been selling their hardware and giving away their software for “free”.  (It’s free the way a drink is free at Taco Bell if you buy a taco.)  Unfortunately for SMART, edupunks who have access to an LCD projector can build the equivalent hardware for $50.  I’ve been told from other people who have built their own interactive whiteboards that the best possible combination is the “free” SMART software on the more portable DIY hardware, which works with any LCD projector.  So, now SMART is trying to clarify that their software is only “free” if you buy or use their hardware.

This seems like a losing battle for SMART.  The RIAA’s approach to penalize consumers who copied music did not make sense but offering reasonably priced songs on sites like iTunes did.  Hollywood was heading the same direction by threatening uploaders of copyrighted material until a compromise was reached that gives the copyright holder a share of revenue generated by ads next to their videos.  Now that is smart.

What is the solution for SMART?  Well, they could start by being clearer about the price of software — it isn’t free.  They could also be clearer about how to purchase SMART software to use on other hardware (so called “Restricted Products,” below).  They seem to be doing this in this letter, but is that it?  I understand SMART’s desire to protect the products they have developed, but treating customers and users of SMART products as scofflaws does not engender much good will.

Our college just got a SMARTboard and two other interactive whiteboards made by SMART’s competitors and put them in the same room so that they can be evaluated head-to-head-to-head.  I’ve suggested setting up a $50 Wii-based version as well.  May the best interactive whiteboard win.

June 14, 2010

To all SMART customers:

SMART Technologies has been investing in SMART Notebook™ collaborative learning software for 15 years. We update and improve our software regularly based on feedback from our users, and we are currently developing version 11. Soon we will release a service pack for version 10 that will be accompanied by a revised licensing agreement, which addresses in more detail the permitted use of SMART Notebook software.

It has come to our attention that misleading or incorrect information about the use of SMART Notebook software is being provided. In advance of the software update, we are writing to confirm the permitted use of SMART Notebook software to help you make informed choices.

When you purchase any of the following eligible “Licensed Product,”
· SMART Board™ interactive whiteboard
· SMART Board interactive display
· SMART Response™ interactive response system
· SMART Podium™ interactive pen display
· SMART Document Camera

a SMART Notebook license is included with the product and you are permitted to use the software on any computer connected to these Licensed Products. SMART Notebook software may also be used on a reasonable number of computers associated with your district or school that are not connected to a pen or touch-enabled devices. This permits teachers to use the software at home to create lessons for use on their Licensed Products in the classroom.

The license agreement does not, however, normally permit the use of SMART Notebook software when a computer is connected to a restricted pen- or touch-enabled device (“Restricted Product”). Restricted Products include, but are not limited to, any touch-enabled or pen-enabled devices that are not on the Licensed Products list above, including the following:
· Interactive whiteboards
· Interactive projector systems
· Display screens
· Screen digitizing devices or slates

To provide options for our customers and enable access to .notebook files by anyone, anywhere, on any device, including Restricted Products, we offer the SMART Notebook Express™ web application, found at express.smarttech.com.

If you have any questions regarding or wish to inquire about use of SMART Notebook software with Restricted Products, please contact SMART at 1.866.518.6791 and follow the voice prompt to press 7 for SMART Notebook license. You can also send us an e-mail at info@smarttech.com with “SMART Notebook license” in the subject line.

Yours truly,
Patric Nagel
Vice President, Sales – Americas

Smartboard Trials and Tribulations

Last quarter I received a Smartboard to use for the quarter on a trial basis via the TELR Smart SEED program at Ohio State. I’ve been really impressed with how smoothly the Smartboard works as an interface.  Compared to the $50 Wii-based version, the Smartboard requires much less calibration and feels more natural without the LED pen.  In short, it’s a really slick product.  However, the sheer size of the portable version has been a big barrier to me.

Unfortunately, as you can see / hear in the above video, the model I was given was a bit awkward to move and we were not able to find a good place to store it aside from leaving it in the classroom, which was not secure.  I did manage to find a couple of rooms that had permanently mounted Smartboards, which were fantastic, but I wasn’t actually able to regularly use the one I was given.

It would be great to have a Smartboard in every room, but this is a huge initial investment.  Before making this committment, we need to decide that this is a worthwhile investment.  To make this decision, we should give teachers experience with them.  I think this catch-22 can be solved by distributing the $50 Wiimote-based version to decide if it’s worth making the larger committment.

There are lots of articles out there that talk about how interactive whiteboards are transformative, but when you read through the examples of how the technology is used, many of the examples do not fully exploit the technology (i.e. writing can be done on a traditional whiteboard, movies can be projected on a traditional screen).

So, I’m going back to encouraging teachers to build and use $50 interactive whiteboard, but I’m also going to encourage them to use Smartboards, if they are available.  (Another alternative would be to combine the Smartboard software with the Wii-mote-based hardware, but the Smarttech requires a separate license to do this.)

I’m compiling uses for interactive whiteboards by tagging them with me Delicious account (see them at http://delicious.com/eslchill/iwb).  If you have ideas for innovative uses, please share them.  The technology can be transformational, but I’m not sure we’ve found the “killer app” yet.

Interface with the Future

I’ve been thinking about interfaces for a while. Interactive whiteboards (IWBs) probably got be thinking in this direction, and the recent announcement of the motion sensing Project Natal for the Xbox has finally pushed me to write about them.

Note the fine print: actual features and function may vary.

As computers have evolved, so have the interfaces we use to interact with them.  To give you an idea of my own personal timeline, I have no experience programming on punchcards, but I do remember the pre-GUI days.  The mouse has become ubiquitous, but what comes next?  Here are some possible answers:

Interactive Whiteboards: These are pretty heavily documented on this blog and elsewhere.  Here, the user can interact with the display, making the mouse and, to an extent, the keyboard unnecessary.

Slap widgets: Physical tools to be used with IWBs and the like, further blurring the line between the physical and the virtual.  (Watch the video!)

Siftables: An interesting miniaturization of an IWB-type interface with the added bonus that each mini screen can interact with the others.  If each Siftable had a letter or a word, how could they be used to teach English?

MIT’s Sixth Sense: This video went viral a few months ago, but keeps popping up.  Why can’t your computer interface with the physical world?  It soon may.

Wii & Natal: Nintendo has enjoyed blockbuster sales of it’s Wii gaming system which features motion-sensitive controllers.  Now, Microsoft has unveiled the same thing with Natal, but no controllers are needed.

Do it yourself interfaces: In addition to making your own IWB using a Wiimote, other examples include the Beatbearing, in which ball-bearings are used to sequence electronic drum beats, a modified Nintendo Powerglove, which upgrades late 1980s virtual reality technology, and a giant Katamari ball controller, which is a controller designed and built specifically for the game Katamari, in which players roll a ball through an environment which attracts more stuff (litter, park benches, people, their pets, etc.).  The makers of each of these interfaces have posted detailed plans and instructions online so that you can build and modify your own.

Move over visual learners, kinesthetic learners are about to have their day.  And why not?  Why should  simply watch something when we can interact with it in more natural ways?  The next generation of computer interfaces promises to expand our idea of how we relate to computers.