The H2O Story, as told by Hal Roberts

Live notes from the TIE seminar, April 16, 2003, 12:00 – 2:00 p.m.:

Hal, who’s a wonderful presenter, is describing why at the Berkman Center we’re writing code — open source educational code, to be precise, free and open and hosted at the Berkman Center, for starters.  How did we get there?  Mainly, the story is the story of Jonathan Zittrain’s teaching and scholarship.  Many good people have worked on the project, for close to five years — some of the best of the Berkman Center’s team, in fact: Wendy S., Alex M., Becca N.  We’ve long wondered what to do with all those ethernet jacks at every desk in HLS classrooms (today’s wondering has to do with the WiFi that spills over into the classrooms from the common spaces).  We’ve long been wondering how to improve the limitations of traditional online discussions.  We’ve decided to answer with H2O’s rotisserie.  It breaks things up into semi-synchronous rounds.  We’re trying to get the best of both asynchronous and synchronous discussion modes.  People tend to post quickly prior to the deadline but not publish it (the system only publishes at the deadline), then mull it over and get back to it before the deadline.  It encourages democratic participation: everyone gets equal voice.  Routing defaults to randomness: first post is randomly cranked around to someone else in the class.  But we now have bounce-back and poll-based routing.  Another problem we’re trying to solve: balkanization.  We let people set up “projects,” which are equivalent to courses or similar structured discussions.  The project is meant to support teaching in the classroom, but also the outside world; that’s why they’re called projects and not courses, so that people will be encouraged to use it outside of the formal academic setting, as in with the Iraq War project or our Internet & Society projects that have been created within H2O.  Anyone from anywhere can log in and create a project or course.  You can also link courses and projects, by inviting people from another project to participate.  A question asked of one group of people can be asked also of a second group of people, and foster discussion across communities.  One could envision people studying at separate universities but studying similar topics joining their discussions.  We’ve just demonstrated how one could link up the two discussions.

The questions from the class:

Questioner: Does everyone get one question to respond to?  Hal: Yes, everyone gets one question sent to them from the rotisserie.  But if not everyone answered the question in the first instance, more than one person may get the question to reply to. 

Questioner: How do ratings work?  Hal: It used to be that every student writes 500 word responses (or whatever the length might be) and posted to the system.  It would to hours and hours to read the archives.  They’re listed in alphabetical or random or some other order.  The archives are essentially useless.  You have to pull out the interesting stuff manually.  So we’ve added a rating system to try to sort out the most interesting threads.  That way you won’t have to work so hard to sort the information.  We’re trying to make the system as simple as possible, so that people can use it easily (teachers and students) but also for developers who might want to take the code and host the program elsewhere, possibly with their own features added to the code.  (Side note: iCommons, at Harvard at large, is set up to offer great academic software and provide support for it — something we’d love to have happen with H2O.)  We present only the cumulative ratings on the thread. 

Question: What are the criteria for the ratings?  Hal: We don’t set criteria for making your ratings.  Individual teachers or project leaders set the criteria (mostly, people say: “how interesting is this thread?”).

Question: Grading systems: can the grading be blind?  Could you show the post but not who’s posting it?  Can assignments be randomized in terms of who has to respond to whom?  Hal: the default mode is random cranks of the rotisserie.  Follow-up: the assignment in random order has a democratizing effect.  It’s nice if you could add anonymity of the original posters, so that you’re just responding to the idea, not the person.  Another commenter: anonymity can cut two ways.  Other issues arise, if you allow for anonymous posts. 

Question: What access to other posts do you have?  Hal: We’ve added features to the system to allow people to react to responses that they’ve not been assigned, or to reply to people’s replies.  “Ad hoc” comments, or thread comments, work more like a traditional discussion board.  You can add a comment to the thread at any time.  We are discouraging people from using them, so as to keep from becoming a discussion board.  Timng of the publication of the comments is used to discourage its use.

Question: How was it used in classrooms?  Me: Worked well in our Harvard Extension School class [now I have to talk, not blog!].

Question: What size groups are targeted? Hal: We have mostly 20 to 100 per group or class, but it can scale — quite well — to say 1,000 person groups.

Question: Design process: who came up with the ideas about how to design the system?  Right now, we’re seeking to build and online community and have design issues.  Hal: It’s a difficult issue and a good question.  We’ve done it three ways.  The ideas came mostly from JZ and Charlie, who are the visionaries.  JZ continues to be the lead.  First, we’ve worked with law students and summer interns, who are good and cheap but tend not to be the best coders.  We produced an in-house prototype, that worked pretty well, and laid a critical baseline for the rest of the project.  Second, we hired outside consultants to build it out.  We spent lots of money and it was not any good.  Third, we built a team of developers in-house, which has been much the most successful approach.  This third mode has developed the current product.  Important mode point: this is also an open source project, which we hope will result in the development of other new and interesting features.  Follow-up: intense collaboration between the visionary and the coders.  Hal: Yup.  But also this is different from a more process-intensive approach that includes developing and reviewing lots of documentation, like functional specs, etc.

Question: Anybody running it out in the wild?  Anybody added to it?  Hal: Very little use out there in the wild.  But this is an experiment in open source development, and we hope to see this lead to great things.

Question: Is the code meant to be modifiable?  Hal: Yes, absolutely, and it’s partly why it’s good to have professional coders doing the work.  The best way to do that is to provide really good, clean code.

Comment: US AID and World Bank are thinking about using tools like these abroad.  They like to gather people’s ideas and responses all at once across broad swaths.

Hal: OK, let’s switch to the future of the application.  We’re thinking about building out into a total course management tool.  We need also to allow people to use things they want, like threaded discussion boards.  We’re also adding in the Idea Exchange.  It’s a way for people to develop syllabuses and to share content.  We want to continue to address the balkanization issue.  We want to let the teacher to develop their syllabus easily.  Teachers should be able to ask the system: “I’m teaching on World War II history.  Who else is teaching about this?  And when are they teaching it, so that we might collaborate?”  Then the system would connect you to people teaching on similar topics, or to people who have previously taught in this area and allow you to use some of the content that they’ve used in their classes.  [Hal shows a demo of how the Idea Exchange would allow a teacher to develop a syllabus, culling assignments and ideas from other teachers’ syllabuses.  It’ll help show relevant content, topics, etc. to those things that you’re trying to add to your syllabus.  Idea Exchange will be partly, initially developed by mid-summer.]

Comment: This Idea Exchange will depend on a tipping point.  People will need to provide lots of information in order to make it useful.  Hal: Yup.  This is an important problem.  We have a series of ideas, but we don’t have it answered well.