Joel Thierstein Interview Notes - August 19, 2009

Conducted with Erhardt Graeff and Carolina Rossini via telephone on August 19, 2009, metrics for content generation and project success in OER communities and in comparison to traditional educational materials forms.

Interviewee

• Joel Thierstein
  • Executive Director, Connexions
  • Email: joel [dot] thierstein [at] cnx [dot] org

Notes

Difficulty of Comparing different forms of EM and OER

• I don't know if you can even evaluate traditional publishing now versus 20 years ago because the role of educational materials is changing
  • Sales or adoptions are tougher to measure in aggregate
  • Traditional professors + trad. textbooks = easy
  • Hybrid EM curricula complicates this
• This is a “transitional time” in educational material usage (a difficult transition)
  • Teachers can take two paragraphs from a module and use that which is completely a new practice; OER provision allows them to
  • PREDICTION: there is going to be an end to this transitional period because people are going to start charging for this content (hopefully not OERs) and people NEED to measure this stuff

Connexions Proposed to study Monetizing OER Content with UK group

• IDEA: Can we design a model for a monetary infrastructure where content gets paid for?
  • Content producers were reimbursed (UK Gov could fund this)
  • Focus on life-long learning; content would be for anyone
  • Learner would report back with what learning materials they would use when they were being tested - reimburse producers of content that was reported to be used
  • Proposal never reached granularity of pay by clicks/views
• Study never went anywhere because economy collapsed
• KEY: System of payments similar to copyright revenues, only driven by demand (conflict w/ OER); OER could be reimbursed even though content is open

Measuring Adoption/Usage

PREDICTION: Measurement is going to start to flesh itself out so that effectiveness of materials can be tracked

Note: Trying to measure the use of a module in a traditional classroom environment is way beyond apples and pears

How Does Connexions Measure?

• Webalyzer statistics
• Google analytics attached to certain modules (attached by module owners) like for the Collaborative Statistics textbook
• Measuring amount of content (14,000 modules from 3,000 (3 years ago))
• GOAL: Answer “Who is using the materials?”, with “We had a textbook published; it was adopted; and we can figure out what its success rate was.”
• Right now Connexions has learners all of the world saying I want module #444444.
  • Users can go to each module and ask how many people are using this content: all related to views on the page
  • Downloads is only in the server logs
  • Externally: Collaborative Statistics textbook adoption is being monitored by CCCOER's Open Textbook Project
  • Participation statistics
    • Modules have individual contributions per version
    • Author accounts have list of contributions

      But contributions might be typos or whole contents but not necessarily done through system - might just be e-mails by author: PROBLEM

    • Also tracks how many times a module has been “forked” one book to new book

Language for Representing Success

• How do you explain to someone what a learning object is? A picture? A book? A sentence?
• Also hard for educators to define this idea?
• PREDICTION: I don't think the statistics are going to be sorted for another 5 to 10 years
• QUESTION: How will the money and sustainability play out?
  • Instructors slowly experiment with self-strung-together sets of modules
  • Only works if Rice University continues to exist and give instructors time to think about it?
  • If Kaplan / proprietary online systems rule the space, INSTEAD, then instructors don't have time to put together their own curriculum THEN this isn't going to take off.

LEARNING ASSESSMENT

• QUESTION: In Brazil, there is an example encouraging teachers to share how people are assembling materials together; is Connexions or anyone capturing these exchanges?

  Connexions wrote NSF (or NIH?) grant that didn't get funded to do that

• QUESTION: How is this module helping my students to learn the material?
  • We have 200 third grade classes, only 5% of students are learning this math component then it is a materials problem
  • The opposite statistic might suggest it is a teacher problem - we don't have these statistics
• KEY ISSUE: OER looks great but how do we know if it will work, we don't even know if our current textbooks work?
  • Need to focus our efforts on this front then this will help us to track usage (needs to be tied to some measurable standard of assessment).
  • CONTACT: Kathi Fletcher asked if she could work on assessment... she has been looking at this for the last year and half

Justifying Obama's OER Investment

Carol: If we don't have good numbers on this, how does Obama justify his $500M investment in OER?

Joel: Cost savings are more than just the book is free; the content is also in module form and we only have to update individual modules; not an entire book

• Lowers long-term cost of EM
• Flexibility of material allows for opening new markets in education and make it more accessible - better education in the United States

Key Takeaways

• SIDE NOTE by JOEL: “We are shifting to the content model
• If we can say there are no clear measures, our research project needs to make a strong case that there MUST to be work on this front!

Navigation

Back to Contacts for EM
Back to Educational Materials