PROSE Workshop Report
Last week we held the PROSE Organizing Workshop at the University of South Alabama. We had 30 people in person and several more attended virtually. The attendees included seasoned veterans from the PreTeXt and Runestone projects as well as some people completely new to the project. One virtual attendee had literally learned of the project just days before the start of the workshop. We had a great time and got a lot of work done. We have a lot of work to do still, but we are off to a great start. In this post I will summarize the takeaways from the workshop and the next steps for the project.
What is the Scope of the Ecosystem?
One key question that needed to be answered was What and/or who is the PreTeXt / Runestone ecosystem? After lots of good discussion we decided that the ecosystem is best described by a set of principles, anyone or any project that shares those principles may be part of the ecosystem. The principles are:
Our Open Source Ecosystem is a community of students, teachers, authors, researchers, and developers committed to the following principles:
- We exist to:
Help students learn more efficiently
Help teachers teach more effectively
Help authors write better materials
Enable researchers to develop more effective tools and pedagogies
We are committed to ensuring access to the best open source learning environment to all of our stakeholders, but especially to the students who need it the most.
We embrace the open web and open standards.
We welcome new partners who share our values.
We will be flexible in creating a platform that incorporates new innovations.
We recognize and celebrate all contributors and innovations that move our ecosystem forward.
In an open source ecosystem we are all “standing on the shoulders of giants,” there are countless projects that we rely on to make our software work, however we want to recognize some of the initial projects that are core to our ecosystem:
PreTeXt - An XML vocabulary for authoring and publishing high quality print and digital textbooks
Runestone Interactive - A Learning Engineering and Analytics Portal (LEAP) for hosting textbooks, assignments, and interactive learning activities.
WebWork - An open source online homework system for math and science courses
Doenet - An open source platform for creating interactive learning activities
It also includes many textbook authors, instructors, and researchers who are using these tools to create and deliver high quality learning experiences for their students. Research is ongoing at several universities including the University of Michigan, the University of South Alabama, and the University of Illinois, UT Austin.
Leadership
We have a long term vision for the leadership of the ecosystem that may become a formal entity like the Apache foundation or an academic society. For our initial work, we have created an advisory council of eight people who represent many different aspects of our ecosystem of Open Educational Resources (OER) and embrace our guiding principles.
The initial leadership council includes the following people, all of whom have committed to an initial two year term.
Rob Beezer - Professor Emeritus of Mathematics at the University of Puget Sound, Founder of the PreTeXt project, author of the open source textbook A First Course in Linear Algebra.
Matt Boelkins - Professor of Mathematics at Grand Valley State University, author of the open source textbook Active Calculus.
Steven Clontz - Associate Professor of Mathematics at the University of South Alabama, maintainer of tbil.org resource library, and co-author of the PreTeXt-cli authoring toolkit.
Barbara Ericson - Assistant professor of Information at the University of Michigan, and co-author of the open source textbook CSAwesome.
Danny Glin - The University of Calgary, and member of the WebWork Executive committee.
Brad Miller - Professor Emeritus of Computer Science at Luther College, founder of Runestone Academy LTD, and co-author of the open source textbook Problem Solving with Algorithms and Data Structures using Python.
Jennifer Nordstrom - Professor of Mathematics at Linfield University, and author of the open source textbook Game Theory: a Discovery Approach.
Duane Nykamp Associate Professor and Director of Strategic Initiatives at the University of Minnesota, and founder of the Doenet project.
This group will guide us through the initial stages of the project and help us to create a more formal governance structure as the project matures.
As part of the next steps we will be creating additional leadership opportunities in the form of Fellowships for people who are working to grow our ecosystem. We will also be funding mini grants to inspire and reward contributors to the ecosystem.
Next Steps
There were so many great ideas that came out of the workshop, I can’t possibly list them all here. But we did identify some concrete actions that we need to get busy on in the coming months in order to make this project a success. Here are some of the next steps that we identified at the workshop.
Write a Phase II grant proposal for the NSF to fund the next stage of the project.
Find a name for the project that is more descriptive of the ecosystem, and will allow us to create a website that will be the hub of the ecosystem.
Work on ways to keep the community engaged and build on the momentum from the workshop.
Develop a plan for making sure that we are defending the intellectual property of the ecosystem.
Continue our discussions and planning to create a sustainable funding model for the project beyond the initial NSF grant.
Continue the technical discussions that were started at the workshop. These include a working group for improving the user experience on Runestone Academy; a working group for improving the authoring experience for PreTeXt; continue the the work on integrating Doenet and WebWork with PreTeXt and Runestone.
Develop a plan to be more inclusive and diverse in our ecosystem.
PROSE Workshop Day 1
Yesterday 30 of us convened at the University of South Alabama for the first day of the PROSE workshop. Steven kicked us off reminding us that the main goal for the week was to prepare for writing our Phase II proposal. Rob and I both gave a little talk on the state of PreTeXt and Runestone.
Here is a link to Rob’s State of PreTeXt
Here is a link to my State of the Stone
In the afternoon we did an exercise to get everyone thinking about the different groups of stakeholders in the ecosystem and how we might serve them better. We will hear more results from the that tomorrow.
In the evening, we convened a hackathon back at the hotel. It was a noisy affair and we brought the hotel wifi to its knees. We did manage to get a few things done though.
Important Privacy Developments
For years, student data privacy has been a hot topic in education. In 2014, the U.S. Department of Education issued guidance on the Family Educational Rights and Privacy Act (FERPA) and the Protection of Pupil Rights Amendment (PPRA) to help schools and districts understand how to protect student data. In 2015, the White House released a Student Privacy Bill of Rights to help ensure that student data is used only for educational purposes. And in 2016, the U.S. Department of Education released a new Privacy Technical Assistance Center (PTAC) to help schools and districts understand how to protect student data.
The way all of this has impacted Runestone Academy is that many school districts are now requiring that we sign a Data Privacy Agreement with them before they will allow their students to use Runestone Academy. In the beginning this was a giant pain because each district had their own agreement, usually thrust upon them by their state department of education. There was little chance to negotiate and many of the agreements contained clauses that were not acceptable to us. For example, some of the agreements required that we delete all student data at the end of the semester. This is not acceptable to us because we want to be able to provide students with a record of their work in the course. (We do automatically remove data after 2 years) Other agreements wanted us to certify that we were HIPPA compliant, but since we don’t collect any health information this was not something we could do. Most problematic was that many agreements had a clause that obliged us to reimburse the school district if we ever had a data breach. Although highly unlikely, the costs associated with recovering from a data breach could be very high and we could not afford to take on that risk. Thankfully after a long search, we were able to find a company that would provide us with reasonably priced insurance against a data breach and we are now comfortable signing these agreements.
In 2020 we signalled our strong support for student privacy by signing the Student Privacy Pledge. You can read the pledge here: https://studentprivacypledge.org/ I also bothered countless people with emails complaining that these data privacy agreements were killing open source projects like Runestone because we don’t have the resources or knowledge to negotiate these agreements. I am happy to say that things are getting WAY better.
Introducing the National Data Privacy Agreement (NDPA) which is a standard agreement that schools and vendors can use to protect student data. This agreement (fostered by the Student Data Privacy Consortium) was developed jointly by schools and vendors and creates a set of common expectations between schools and providers like Runestone. We have already signed agreements with several schools that use the NDPA. You can read more about it here: https://privacy.a4l.org/national-dpa/ We are happy to sign this agreement with any school that requires it. Even better, if your school is a part of a consortium where another member has already signed the agreement, then all your school has to do is sign Exhibit E and send us a copy. These consortia are often run at the state level, but in some cases multiple states have banded together. The map below shows the states that have adopted the NPDA and have consortia in place. In addition most of the state consortia have a database of vendors so you can do a quick search to find out if Runestone Academy has already signed with that state. If we have, then all you need to do is sign Exhibit E and send it to us. If we have not, then you can send us the agreement and we will sign it.
The development of the NDPA is such a positive step forward for student privacy and it is a huge reduction in the amount of work that I have to do to sign agreements with schools. I am very grateful to the Student Data Privacy Consortium for their work on this. If your school is not a part of a consortium, please encourage them to join one. It will make it much easier for them to sign agreements with vendors like us. If your school is developing their own agreement, please encourage them to use the NDPA rather than creating their own. It will save them a lot of time and effort and will make it easier for them to sign agreements with other vendors.
Colophon
I have been enjoying using Github copilot with VSCode for both Python programming and writing blog posts. Copilot is based on OpenAI’s GPT-3 language model. In fact the first paragraph of this post was written by Copilot. I wrote the title and the first few words of the first sentence and it wrote the rest. I did have to edit it a bit, but it was a great starting point. It also had some good and not so good suggestions for other parts of this post, including making up urls to non existent pages. Its really great at taking some of the drudgery out of programming in that it can write a lot of the boring code very quickly leaving me more time to work on the hard stuff. I am looking forward to seeing how this technology develops.
Top 10 Reasons to Host your Book on Runestone Academy
After four years of collaboration, Runestone and PreTeXt are ready to encourage hosting PreTeXt books on Runestone Academy servers as a routine matter. We’ve got several months before fall classes, so now is a good time.
The process is quite simple: put your PreTeXt source on GitHub if it is not there already, and provide a publisher file that specifies that you want a build specifically for Runestone servers. If you are using the command line tools / Codespaces with a project file then you are nearly ready. Let Runestone know by creating an issue in the Runestone repository. Your book will be made available to thousands of instructors and will be re-built automatically every Saturday.
Here’s our Top 10 list for why you would want to consider this, numbered for mathematicians and for computer scientists.
Free. In every sense of the word. Open-source software and texts, hosting free to instructors and students.
Hosting is low-friction for instructors, no local IT required. Fantastic up-time.
Weekly updating with no hassle. Or more frequently with the author interface.
Runestone Academy had over 115 million page views last year and has good search-engine optimization to increase the exposure of your book.
Instructors get a detailed dashboard of students’ work and progress.
Runestone Academy is used by over 10,000 instructors–who already use OER and are happy to spread the word.
Authors get gross analytics across all institutions. Which of your exercises work, and which don’t?
Authors get accurate data on adoptions and readership. Impress your Dean.
You maintain complete ownership and control of your materials.
Instructors get autograding and autorecording. Get a spreadsheet at any time or link it to your LMS through LTI.
PreTeXt and Runestone are actively promoting books at both Mathematics and Computer Science conferences.
To get started, read the chapter on Runestone in the Publisher’s part of the Guide, especially the second section. It has pointers to other relevant parts of the documentation.
Chapter 32: Conversion to Runestone
We would encourage you to look at the annotated sample book, especially the Runestone chapter to see the interactive elements that you can begin using in your books, written natively in PreTeXt but using Javascript developed by Runestone over the last 12 years.
There is a fast-developing converter from Runestone restructuredText syntax to PreTeXt, which will ease the transition for computer science textbooks to PreTeXt.
We would be remiss to not acknowledge the trailblazers who have helped test this out as it has been developed.
Native PreTeXt:
Oscar Levin; Discrete Mathematics, An Open Introduction
Sean Fitzpatrick; APEX Calculus
Mike May; Business Calculus with Excel
Alex Jordan; Open Resources for Community College Algebra
Kathy Yoshiwara; Intermediate Algebra: Functions and Graphs
Tom Judson; Abstract Algebra, Theory and Applications
Tom Judson; The Ordinary Differential Equations Project
Alex Jordan; WeBWorK integration (!)
Matt Boelkins, Mitch Keller, Chrissy Safranski; Active Calculus
Steven Clontz, Drew Lewis; Linear Algebra for Team-Based Inquiry Learning
Rob Beezer; A First Course in Linear Algebra
Conversions from Native Runestone:
Brad Miller; How to Think Like a Computer Scientist
Brad Miller; Problem Solving with Algorithms and Data Structures using Python
Brad Miller, Barb Ericson, J. David Eisenberg; Python for Everybody
Beryl Hoffman; Java, Java, Java (from LaTeX!)
Jan Pearce; Problem Solving with Data Structures and Algorithms using C++
Jan Pearce; C++ for Python Programmers
Jan Pearce; On Complexity
Runestone at SIGCSE 2023
I am pleased to announce that our NSF grant provides us with funds to be an exhibitor at SIGCSE this year. Please stop by our booth and say hello. If you don’t know anything about Runestone we would love to introduce you. If you are a long time user we would be happy to hear your feedback and ideas. Also, we have stickers.
Do commercial publishers have a place in the open textbooks ecosystem?
Suppose a textbook was used at more than 100 universities, with many websites advertising the book and showing off its numerous high-quality features. Suppose many experts in the subject have produce supporting materials: videos, worksheets, sample exams, sample syllabi, advice to the instructor, and suppose those materials were already available for free – making it very attractive for the textbook to be adopted by new institutions.
And suppose a commercial publisher was given the opportunity to produce and distribute the book. Would that be a good business opportunity? Of course it would! A textbook publisher should jump at the chance to bring out a new book with an already well-established market!
Except, it isn’t actually a new book. And unlike a traditionally published book, it isn’t the sole and exclusive property of the author, because the book has been released with a license that allows anyone to modify it and make their new version available. And everything in the printed book already exists on free websites, and for free as PDF, EPUB, or on a kindle. The printed book has to compete against itself in other formats.
Is the open nature of the book a deal-breaker for publishing commercially? I say No. Many open textbooks are already being published in print form. Students buy them: many want an inexpensive printed copy because the online versions are great in some ways but fall short in others. Many open textbook authors go to the effort to produce a print version; I speculate that many would be happy to hand that work off to a commercial publisher.
Issues, from the perspective of the publisher
By far the biggest issue is dealing with an unfamiliar situation.
Until recently, there were not a lot of excellent open textbooks. Now there are. PreTeXt (https://pretextbook.org/) and Runestone deserve a lot of credit for facilitating authors to create high-quality books with many features that enhance teaching and learning.
So, it will take vision, and a tolerance for the unknown, for a commercial publisher to dip their foot in this pond.
For the sake of argument, consider the situation of an open textbook which is being used at several universities, with free HTML version available online (possibly at more than one website), with a print version from the author (or someone they designated) available for $20-25.
The publisher probably would like the author to stop publishing the print version. It is safe to assume the author would agree, provided that the price did not increase significantly.
The publisher would probably want the existing print version (which probably is print-on-demand) removed from the marketplaces where it appears. Again, the author should agree.
What does the publisher think about the author’s website hosting the HTML version? How does that relate to the home page of the book on the publisher’s website? Presumably those pages should link to each other, because both parties benefit from the success of the other.
How about the PDF version the author currently posts? Probably that is a non-issue, because for PreTeXt books you can print from the HTML, and for many purposes it does what one wanted from the PDF. If there is a reason the author would object to taking down the PDF, I would like to hear it.
EPUB is another issue. Many publishers make money by selling the EPUB: that revenue stream would be diminished if the author posted it for free. The EPUB version is recognized as a valuable resource for people in prison, because it does not require an Internet connection and it does not take up space. This is important to people involved with the Prison Education Project and related endeavors. Both sides should be aware of these issues during the negotiations.
If the above are the main concerns of the publisher, then they have missed some essential facts about the world of open textbooks.
Some realities which have to be discussed and understood
As was already mentioned, the book isn’t the sole and exclusive property of the author.
If the source is available with an open license, anyone can make a PDF version, or produce a print-on-demand version and sell it at whatever price they want.
Anyone can also add a new chapter or section and publish it however they want. Forbidding the author from using that book as the basis for a competing product, cannot stop someone else from doing the exact same thing. And the original author’s name is also on the new book! (How authorship is credited in these situations is untested waters. Open textbooks are touted for their ability to be modified and customized, but in practice this rarely happens and the community has not had to deal with complicated situations.)
The publisher has to accept that they do not have a monopoly on the book: they will be the only print publisher if the author agrees that it is better for the publisher to handle the print version, and nobody else decides that they can do it better (where “better” involves some combination of price, quality, and features).
A clause in a contract may forbid the author from taking certain actions, but if the book has an open license, that contract cannot prevent someone else from performing those exact same actions.
The general point has been made, but I’ll repeat it for this specific case: nobody can stop someone else from adding a small amount of material and publishing it as the “new edition” of the book.
Here is an issue which seems silly but is not: movie rights. The excellent open textbook APEX Calculus has a version with more than 600 videos by Sean Fitzpatrick (not the author of the book, and that version is not hosted by the author) with a cumulative duration of more than 75 hours, explaining concepts and working through numerous examples. Stacking those videos creates a multi-part epic movie version of the book. Whose rights are infringed by that cinematic extravaganza?
Most commercial publishers have a fundamentally misguided policy concerning Braille versions. Each institution which needs a Braille copy has to go to the effort, typically taking months and costing $20,000 or more, of having an expert Braille transcriber convert the book. They are not allowed to then share it with another university! Think about it: not only does the publisher not provide a Braille version, they make it as hard as possible for other people to do so. Textbooks written in PreTeXt automatically convert to Braille, at minimal marginal cost. That fact should have an impact on the publishing contract.
A modest proposal
One possible way to address some of these issues is for the publisher to commission a new edition of the book, and for the author to not release that edition with a fully open license. This would not change the fact that the previous edition was still available open source and with a permissive license. But the publisher’s concerns about competing products would be mitigated by the fact that the anyone other than the author would be several steps away from replicating the published book.
The “edition trick” is not a magic wand
The modest proposal would involve the author abandoning their previous vision for the open source ecosystem of their book. Would they release the new edition with a more restrictive open license, such as CC-Attribution-ShareAlike-Noncommercial, or would they keep the source private and reserve all rights? Would the restrictions apply only to print publishing, or would other online versions also be prohibited?
Is this modest proposal the author’s first step on the slippery slope of producing a commercial book that is exactly what they were trying to avoid when the decided to write an open textbook?
Keeping the fully-functional online HTML version available for free seems like an absolute requirement for the author. Is the author assigning the copyright to the publisher and the publisher granting rights for the online version, or is the author retaining copyright and granting the publisher the right to produce and sell a print version? Or does the book have an open license and the publisher has to accept the fact that keeping the author happy is their main protection against competing print versions?
If the book incorporates material under an open license from another author, putting a more restrictive license on the next edition might not even be an option.
Here’s an ironic fact. Suppose the author decides not to go with a commercial publisher and just releases the new edition with the same open license. Then the publisher can just produce and sell the print version anyway! And they won’t have to pay any royalties!
If I come out with a book that has 80% overlap with an existing book, I will be sued by the publisher, and I will lose. But for an open textbook, even with the “edition trick” I have been given the rights to produce such a similar book based on the previous edition. No agreement between the author and publisher can change that. And the author cannot remove the rights which I have previously been given.
Maybe all of these concerns are non-issues?
Perhaps, with or without the “edition trick”, the online versions of the new edition can function exactly like the previous edition, with the print version now coming from a commercial publisher who provides a modestly priced copy and still makes some money, and nobody is motivated to duplicate either the author’s or the publisher’s work?
There would be very little risk for an established publisher to produce and distribute the print version of an existing open textbook, on just a handshake agreement with the author. The author would agree to not publish a print edition, and would make a reasonable effort to promote the publisher’s print version and discourage others from making a competing print version. The publisher would agree to make their print version available at a price comparable to what the author could do on their own. The publisher would advertise the book in ways beyond what the author could do. Every free online view of the book would be an effective advertisement for the print version. The publisher makes money. The author probably makes a bit less per print copy, but they are spared the work of being their own print publisher and end up making more in the long run because more copies are sold.
The publisher’s lawyers might be unhappy with a handshake agreement, so a contract could codify the points in the previous paragraph. If the contract were for one year, renewable and properly synchronized with the academic year, then it could be workable for one party to handle the print version and traditional advertising, and the other party to manage the online versions and provide support and outreach to instructors using the book.
Are there commercial publishers who can appreciate this business opportunity? Do most open source textbook authors understand that they are not giving up anything if they enter the type of agreement I propose?
I await the publisher and the authors who fully understand the open source ecosystem and its relationship to commercial publishing, and see an opportunity for harmonious collaboration.
Year in Review 2022
With the year 2022 in the rear view mirror I wanted to take some time to update you all on what we have accomplished in the last year, give you some fun facts, a financial update, and talk about the community in general. So lets get to it!
Fun Facts
During 2022 we welcomed 74,000 registered students and instructors to the community! That is a lot of new people and a lot of money saved on textbooks! During an average school day we have around 1000 students active during a given five minute period. I think that is something we can leverage moving forward. The traffic on Runestone clearly shows that we have a “homework bump” that is once the traffic slows down at the end of the school day across North America, we have an evening spike that lasts three to four hours. We have no such spike on Friday nights. Also, students do not do homework on Saturdays. In 2022 we had 115,100,000 page views! That is nearly double the number of page views from 2021. About 60% of the page views are from registered users while the other 40% come from searches through Google, Bing, and others.
What makes Runestone Academy Tick?
People, 100’s of people who donate their time in many ways.
Authors – between PreTeXt and Runestone we probably have around 60 different authors represented on Runestone Academy.
Developers – Over the years around 100 developers have contributed at least one line of code to Runestone, and around 50 developers have contributed at least one line of code to PreTeXt. Over the years this job has become more difficult as the complexity of the whole system increases. I am aware of this and actively trying to find ways to make it easier for anyone to help with development.
Dev Ops - Monitoring the servers, building, testing and deploying updates, to our servers, building, updating and deploying updates to the textbooks, are all things that happen weekly. It takes time, although a lot of that time is just keeping an eye on things while they run in the background. For example it takes about 4 hours to rebuild and deploy all of the books on Runestone Academy. Its mostly automated but you have to keep an eye on it for those few times when something goes wrong. It takes another 30 minutes or so to rebuild and deploy all of our servers.
Bug Finders - Reporting problems to us is equally as important as writing code! I am always surprised by the number of people who just live with a bug instead of reporting it to us on github. PreTeXt is tracking 1242 issues while Runestone is tracking 2216. Of course those numbers are obsolete the moment I type them. Issues come in all forms and we welcome reports of typos as well as serious bugs in the Runestone servers.
Exercise Creators - Over the years the question bank for Runestone has grown by leaps and bounds - over 14,000 questions not including those that are part of a book. This is a great way to learn a little about restructuredText or soon PreTeXt and contribute to the whole. If you have a good exercise please feel free to add it through the web interface. You can keep it private if you wish or you can share it with all the other instructors out there looking for new homework problems.
Exercise Editors - With the huge growth in the size of the question bank, we have created quite a problem for ourselves. Many of these questions are just dummy questions that instructors use to learn and should be weeded out. Anyone can help with this problem by flagging a bad question. This puts it in the queue for an editor to take a look at and either delete or update. We need loads more exercise editors!
Community Builders - Do you know what makes for the most successful textbooks on Runestone? It is an active and supportive community of teachers using that book. The best case in point for that is CSAwesome. It truly is awesome the way teachers help and support each other on the mailing list. But that takes some effort and leadership to nurture Beryl Hoffman and the people at the Center for Computer Science Education do a wonderful job. We need more of that for all of our books!
Documentation Writers and Trainers - Have you gotten pretty good at some aspect of Runestone? How about taking a look at the documentation and bringing it up to date, or making it easier to understand? By now I’ve lost all perspective of what it is like to be a newcomer to Runestone, but many of you have not. Would you like to host a workshop to train people how to use Runestone? We would be happy to support that!
Word Spreaders - If you like Runestone or any of its books please help us spread the word. We would love to have everyone recommend Runestone and PreTeXt to two colleagues. We would love to have everyone mention Runestone and PreTeXt on social media. Do you Write a blog? Toot on Mastadon? Post on Facebook? Tweet on twitter? How about a shout out? I have left both Facebook and Twitter but if someone would like to take over the FB page or the iRunestone account on twitter let me know! Anybody can help with this and the more we grow our community the more good ideas we get.
Accomplishments in 2022
New Server Architecture
Looking back at it now, I have to agree that 2022 started off a little bit rocky. We moved to a brand new server architecture, and a brand new server to handle the majority of the book serving traffic. After a month or so of headaches things calmed down, and I’m super pleased with the new architecture. It allows us to scale horizontally as our traffic ebbs and flows over the course of the year. There is much more to be done in this area, but we will do our best to make it as seamless as possible.
The PreTeXt Partnership
During the Summer of 2022 a lot of work that has been going on since just before the global pandemic took hold really began to pay off. The partnership with the PreTeXt project allows me to focus more on the server and interactive widgets for learning, while freeing me from maintaining and improving an authoring language. We converted our first books from restructuredText to PreTeXt and in the words of Billy Crystal “they look mahhhhhvelous!” My hope is that we can transition the rest of the books during 2023. As we master the conversion process we will also be able to make PDF, Braille, ePub and other formats of our interactive books available. Though of course they will not be interactive.
I’m also really excited to see that a number of the Mathematics authors have embraced the interactive elements that Runestone brings to the table in writing their textbooks. Parson’s problems for doing proofs or derivations anyone?
The Launch of Peer Instruction
During the winter of 2022 we piloted our new peer instruction system, this was followed by a workshop attended by about 20 instructors who used peer instruction in their classroom during the Fall of 2022. This is a very exciting new feature that you can use to replace clickers in your classroom and which also works well in the virtual classroom. This work was funded by the NSF and led by Dr. Barbara Ericson at the University of Michigan.
Micro-Parsons
Formerly known as horizontal parsons, this new interactive question type allows you to create a parsons problem for an individual expression or statement. This is great for teaching SQL or regular expressions, or even helping students work through some trickier kinds of expressions that you might write in Java or Python. The new Micro parsons are also leading the way in terms of allowing parsons problems to be graded by unit tests in addition to simple block order.
Joint Runestone - PreTeXt Workshop
In June of 2022 we held our first joint workshop, where we brought together about 20 authors and developers from the PreTeXt and Runestone projects. The workshop was held in Ann Arbor on the University of Michigan campus and it was an amazing week of learning together, sharing research, and brainstorming ideas for the future. I’m so excited about where this could all lead.
NSF POSE Grant
We were honored to receive a grant for the project we call **PROSE** - PreTeXt Runestone Open Ecosystem. This NSF grant is designed to help projects like our become a sustainable ecosystem. For us that means many things: recruiting community managers, developers, authors, editors, question bank maintainers, and more. It also means reaching out other projects to see how we can work together more effectively and efficiently. There are so many projects that are part of the “Ed Tech” world and there is so much needless duplication of effort. The marriage of PreTeXt and Runestone demonstrates that two projects can come together and be more efficient and stronger when we work together, we want to spread that message and find other projects who share our vision and are willing to join us. Another big part of this grant is to help us figure out how to organize and maintain all that we do going forward. In a world where anything can happen, we need procedures and people in place if something horrible were to happen.
Finances
In late 2021 Runestone Academy LTD was formed as a new corporation to supplant Runestone Interactive. We are operating Runestone Academy as a non-profit. Although the conversion process took a lot longer than expected Runestone Academy has also filed our paperwork to get IRS approval as a 501(c)(3) which will give us official non-profit status for tax purposes. We have also named a board of directors to help us think strategically about how we ensure a strong future for Runestone. I am honored to have this group of forward thinking people help guide the way: Paul Resnick, Assoc. Dean of the University of Michigan School of Information, David Farmer of the American Institute of Mathematics, Susan Rodger, former President of SIGCSE and of Duke University, Danny Fenjves former founder and CEO of Upperline code now leads the LEARN team at Giant Machines and Jane Miller our CFO and Secretary/Treasurer who has been a partner in my crazy endeavors for the majority of my life.
The financial details of any transition year can be confusing, especially when trying to compare to a past year, so lets just focus on 2022 and mostly not try to compare to last year.
These are unofficial numbers but they should be pretty accurate. We have several different sources of income including students, schools, ads, partner support from people who use Runestone for their own purposes but want to support our work, Grant money, and consulting work on grants. Here is how that all breaks down.
Source |
Amount |
|---|---|
Direct support from students |
$10,000 |
Support from institutions (departments, colleges, schools) |
$17,000 |
Advertising Income |
$16,500 |
Support from Grants |
$11,250 |
Support from partners |
$2,400 |
Webwork income |
$23,300 |
Other |
$3,000 |
Total |
$83,450 |
Expense |
Amount |
|---|---|
Hosting and server costs |
$8,800 |
Webwork expenses |
$15,500 |
Legal fees |
$9,725 |
Credit card and Paypal fees |
$900.00 |
Travel |
$1,800 |
People costs (healthcare, salary, etc) |
$44,600 |
Total |
$81,325 |
So, you might think that we have a net profit of about $2,000 which would be great for a non-profit, everything balances. But let me point out a few problems with that conclusion.
People costs – $44,607 includes 1 full time person and 1 part time person. Thats not a very good salary.
Support from institutions is what we have Invoiced for $17,195 Unfortunately we have been paid only $12,165 of that. People will often ask for an invoice and then completely ignore the invoice and our followup emails. This makes it hard for us to plan and count on that money.
In 2023 we have and will institute a couple of changes.
If you school wants Runestone Academy to sign a contract or data privacy agreement we will invoice the school for $100.
During the first part of the year we will be working on determining a small set of features will will require payment. None of the paid for features would prevent you from using Runestone for free. These are all features that are nice to have, but provide good backup for why you could pay to support Runestone. I know that many public schools have policies that don’t allow you to “donate” so by providing a few features that you can only use if you pay turns that donation into a fee for service.
Runestone Academy is not a big corporation. The two employees mentioned are myself and my wife. As I made clear above, Runestone is powered by volunteers. The Runestone community has accomplished a LOT on very little over the last 10 years. The challenge is to sustain that progress and spirit in an ever changing world.
Conclusions
This past fall I ran a survey for the instructors who use Runestone Academy, I learned a lot from the survey that will help us set a course to smooth out some of the rough edges of the Runestone platform. But even more than that was the outpouring of support for the mission. So many people have come to rely on using Runestone Academy in their classrooms every day, and so many people said that love it! That kind of feedback is really energizing and makes me want to work hard to keep building and improving things.
Our mission is expanding, Democratizing textbooks for the 21st century is only part of what we do now. An updated mission statement might read as follows:
>To enable authors and researchers to maximize the accessibility of open educational resources for learners, and advance the teaching of STEM fields.
As with any mission statement it needs a bit of unpacking. Accessibility is the practice of making information, activities, and/or environments sensible, meaningful, and usable for as many people as possible. And the goal is to create a virtuous circle of feedback that drives progress for our four major stakeholders: learners, researchers, authors, and teachers.
It is worth repeating… we can use your help! If you or your school can join the project by supporting us financially that would be awesome. But beyond that we want YOU to become an active participant in our community in any of the ways I mentioned above.
An Open Letter to Future Authors
Dear Future Textbook Author Who Knows LaTeX,
LaTeX is a markup language which is designed to help you produce nice PDF. PreTeXt is a markup language which does everything LaTeX can do, and in addition provides multiple output formats, interactivity, and accessibility. If you are writing a textbook, you will find it well worth the effort to learn PreTeXt. And it is better to start with PreTeXt instead of writing in LaTeX and then converting.
A major attraction of PreTeXt is that your book will be available in multiple formats, including highly accessible HTML. Formats are only part of the story: if your book is merely the book you would have written in LaTeX, translated into PreTeXt, the result will fail to achieve its potential. What will be missing are the features of PreTeXt which support teaching and learning.
But first, what is a textbook? In the past, the textbook was the source of all material being taught in a course and also the source of all examples, exercises, etc. These days it is common for a course to call on multiple resources. PreTeXt makes it possible to bring back the time when the textbook contained everything – with PreTeXt the textbook can be the portal to everything needed by the instructor and the student.
And second, what is your vision for your textbook? Some textbooks are a monograph: one person’s articulation of a particular subject – a beautiful and coherent exposition on a topic. A monograph, even if it contains many exercises, is no longer sufficient to be the sole resource for teaching or learning a subject. A modern textbook can be crafted to enhance teaching and learning, supporting good pedagogy and recognizing the realities that face both students and teachers.
If you are writing a monograph, some PreTeXt features will not be of use to you. But you will find that PreTeXt helps you think about the structure of your document, resulting in better exposition. The extra features are available if you later decide to use them.
If you are trying to actively support teaching and learning, then PreTeXt has many features which will help you create a textbook that improves learning and enhances teaching.
For example, some PreTeXt books have preview activities at the start of each section. A preview activity gives the student a few tasks to perform – tasks which they are capable of doing with only their current knowledge and which introduce (without using any terminology) the main concepts of that section. This supports good pedagogy, because
definition → example
works fine for research and graduate level exposition, but in most of the undergraduate curriculum
examples → terminology
works better.
Other PreTeXt books have reading questions at the end of each section. An ideal reading question is easy for a student to answer if they have understood the material in the section, and difficult if they have not. Reading questions are graded on effort.
Your book can be available in an online system (such as Runestone) which gives the instructor easy access to the students’ answers to preview activities or reading questions. A quick glance at those answers immediately before class will allow the instructor to productively modify their plans for the classroom, allowing more time for those topics where the students need it most.
So: think about how instructors might use your book. Then create a book which will help them be better instructors.
Let’s think about the students. What is the single most significant cause, throughout the history of STEM education, for students having difficulty learning the material? A good case can be made for: the long delay between when they answer a question and when they receive feedback on their answer. Some STEM fields, notably physics and computer science, have recognized this issue and taken effect steps to address it. Numerous types of problems have been created, where students can automatedly receive instant feedback on their work. Misconceptions and common errors are caught quickly, and the result is better learning.
PreTeXt supports many types of interactive problems. Multiple choice and short answer, of course. Also Parsons Problems (multi-step answers where the steps are given, to be put in the correct order), Matching (match Column A with Column B), Clickable Area, and more. Not every type of problem is suitable for instant feedback, but surprisingly many are.
PreTeXt also supports multiple types of interactive demonstrations, such as GeoGebra, Desmos, Sage cells, CalcPlot3d, etc. Students can explore an example as much as they wish, and the demonstration can be reset just by reloading the page.
So: think about how students might use your book. Then craft a book which will help them to learn more: more than they would just by reading clear explanations, seeing illustrative examples, and turning in their work to be graded later.
All of the interactive examples and exercises can be right there in the book: there is no need to cobble together and keep track of a variety of resources. If the book is hosted in Runestone, the instructor has direct access to submitted student work and a dashboard of learning analytics.
Some books have introductory videos in each section and some have videos showing worked examples. These are appreciated by those students who prefer learning by watching and listening. It is better for your book to include quality material that you have selected, instead of having your users wade through the swampy haystack of the Internet.
It takes a while to start thinking holistically about how your book will serve as the interface which connects the material you are passionate about to the students learning the material and the instructors helping the students to learn. The ability to properly support all of that activity, by yourself as an OER author, is a new idea which is now made possible by the combination of PreTeXt and Runestone. But you are not by yourself: there is an active community of PreTeXt and Runestone users and developers, and there is a growing list of textbooks which illustrate what is now possible.
A good textbook might present the material in a clear and coherent manner, but a great textbook actively supports teaching and learning.
PreTeXt and Runestone are free and open source tools which can help you craft a great textbook.
Regards,
Instructor Survey 2022
Let me start out with a big thank you to everyone who responded to the Instructor Survey over the last few weeks! About 200 of you took the time to answer the questions. The results as I watched them come in were remarkably consistent. I would like to break down the results into four sections: Features you value, Features you might be willing to pay for, Contracts and Privacy, and Thoughts on the future.
Features you Value
The following graph summarizes the responses to the first four questions. Recall that 1 is least important and 5 is most important.
I was a little surprised by how many instructors said that LTI was really not important, nearly 50% of respondents rated the importance of LTI a 1. Assignments and grading, practice, and tracking student progress were rated a 4 or 5 by 60% That reaffirms that these are our core features that you rely on. Although reading through the open text response it is clear that we can make the assignment and grading pages much better. I’ve started a redesign of those in my head, but I would love to hear from you on what exactly the pain points for you are. I heard words like clunky, cumbersome, difficult, and those are somewhat helpful but some detail would sure help.
The thing that made me the most happy was the overall satisfaction score with over 90% of the responses rating us a 4 or 5! There were so many nice comments in the text section that I will read again and again on the days when I am slogging through the code trying to fix some obscure bug!
Features you Might be Willing to Pay for
I know the question was a big vague, and I think think that it led to some unnecessary panic by some of the respondents. What is clear to me is that there is a section of our user base that values Runestone Academy and would like support us. There is also a sizable number that come from school districts that would not have the budget to support us even though you might want to. This latter group is why Runestone exists, we exist to serve you and we want to be here for you. Recall that 1 is least likely to pay and 5 is most likely. Here is the summary graph:
A couple of things people are clearly not interested in paying for: early access to features, dedicated email support, LTI. But about 20% of respondents would be willing/able to pay for a support fee, access to the student progress features, the practice feature, assignments and grading, PDF copies of the books, and even LTI. Because I’m kind of a nerd I made a correlation matrix:
This does show some interesting correlations that may suggest a way forward. First there is a 0.5 correlation between those who value the practice feature and those that would be willing to pay for it. This is also true of LTI at 0.42. Assignments and Grading and Student progress are at around 0.3. I think it is kind of cool that satisfaction is not correlated with anything. One might hypothesize that the more satisfied you are the more you are willing to pay. But it is probably just be that because 90% of the responses were very satisfied that there is not much to correlate with.
Contracts and Privacy
I have written about this a lot in the past. Processing these contracts that schools want me to sign is a major time sync and a terrible use of my time. Nearly all of them require Runestone to promise to reimburse the school for costs associated with a data breach! For a service they are not paying for! Is it reasonable to charge schools a $100 fee to process these? As the following graph shows the overwhelming majority 85% feel it is fair.
I would love to hear the reasoning of why 5% of the responses do not think a $100 fee is reasonable. I suppose you might think that I should trust your school. They are not out to get me, just to protect their students. But that is no way to run a business. More and more I have become convinced that Runestone Academy needs to have a Cyber insurance policy. I am happy to say that after an extensive search and a LOT of back and forth with underwriters we have finally obtained a policy. But, guess what… They don’t give those policies away for free, they charge a substantial premium that adds to the cost of running Runestone.
Thoughts on the Future
I want to start this section by reminding everyone of why Runestone Academy exists. We exist so that we can make high quality, accessible textbooks available to students no matter their economic situation or physical limitation. Too many students do poorly because they cannot afford (and therefore do not buy) the textbooks they need for class. We exist to provide students engaging books that help them learn more effectively than paper books. We exist to help teachers teach more effectively by providing them feedback about their students. We exist to help authors and researchers do their jobs more effectively by providing a system where they can experiment with new pedagogical techniques and new technology for textbooks. You will notice that not once in that list of reasons we exist did I mention the word profit or money. Making money is not why we exist. The materials we provide will always be free to students.
Yet the hard facts of the world are that nothing is for free. We have hosting fees and legal fees and credit card fees and paypal fees in 2021 we had about $28,000 in fees. That is before I even try to pay myself a salary for working on this full time. What happens if I disappear? Is there someone else willing to devote full time to keeping Runestone running for little or no pay? I haven’t met anyone yet, which means we have to get to the point where we can pay someone and keep Runestone sustainable.
There are a few immediate decisions from the results of the survey.
Runestone books will continue to be free to all students. We will not charge the students for access to our books.
For all schools with courses beginning after January 1 2023 that want Runestone to sign a contract we will require a $100 contract fee. If your school cannot afford that fee we have options for you to continue using Runestone without a contract.
I think that there is room for a “freemium model” to work. We will work on defining a set of features that will remain free to everyone, for example assignments, grading, and student progress remain free but features like LTI integration, practice and peer instruction may cost a small fee. I am being transparent about this because
I don’t want people to panic, I want to open a dialog to get your feedback and I want to reassure everyone that if we do have to charge for some features you will get plenty of notice. I also want to reassure you that we will do everything we can to find grant money so that everyone who wants to use any book on Runestone Academy can continue to do so regardless of the financial position of the school.
Pathways to enable Open Source Ecosystems
Early this year my friend, colleague, and Runestone board member Paul Resnick forwarded me a copy of a new NSF program called POSE, the solicitation said: “The purpose of the program is to harness the power of open-source development for the creation of new technology solutions to problems of national and societal importance.” The solicitation continues: “The POSE program aims to support managing organizations that will facilitate the creation and growth of sustainable high-impact OSEs around already-developed open-source research products.” Cool!
This is not a solution to all of our funding needs as the solicitation further explains: “Importantly, the POSE program is not intended to fund the development of open-source research products, including tools and artifacts. The POSE program is also not intended to fund existing well-resourced open-source communities and ecosystems. Instead, the program aims to fund new managing organizations that catalyze community-driven development and growth of the subject OSEs.”
Well, I thought to myself if that isn’t a program specifically designed to help Runestone and PreTeXt in their joint mission I don’t know what is. On September 9th we learned that indeed we had been funded as a Phase I project! The work begins immediately!
We say PROSE
Our proposal which we called the PreTeXt – Runestone Open Source Ecosystem (PROSE) says the following:
We propose an Open-Source Ecosystem (OSE) which will support such a community. This is timely because software for creating OER is just reaching a mature state, in particular, PreTeXt for authoring, and Runestone for publishing. The governing organization for this OSE, which we will call the PreTeXt-Runestone OSE (PROSE), will be built upon the following principles.
Students and instructors should have no-cost access to high-quality OER that supports STEM learning without the equity issues inherent in for-profit educational publishing.
Authors should have a no-cost mechanism for creating high-quality and accessible OER that is not locked into one specific format or vendor.
Authors should have a no-cost mechanism for disseminating their OER as part of fully- featured courseware that provides rich usage and learning analytics for students and instructors.
Researchers, authors, and instructors should have access to high-quality student data to inform research and development efforts related to OER, and improve student learning.
OER must be accessible to everyone, regardless of background, demographic, or disability.
I like to think of these things in terms of a virtuous circle of improvement. Educational Research makes good Open Educational Resources better, which in turn helps teachers teach more effectively, which improves student learning, which drives even more interesting educational research…
This way of thinking is part of the DNA of both the PreTeXt and Runestone projects since their earliest days. Every time we add a new feature and study how it effects students and teachers we learn something new! New knowledge inspires new ideas for new features. My TODO list has never run low in over a decade of working on the project, and in fact it is longer today than ever. Recently I’ve had to admit to myself that the list is too long. There is too much to do and Runestone Academy has grown so much that it is more than one person can manage. More than that, is the feeling that we are on the cusp of something disruptive, transformative, and sustainable. That is why this grant is so important and timely.
How you can help?
Over the next months we will be hosting a series of online discussions around areas where we need help. Everything from cybersecurity and dev-ops to community building and training. We invite you to participate in these discussions to share your ideas. We also invite you to consider if there is an area of interest where you can actively participate and contribute some time to the project. Both Runestone and PreTeXt have operated on a BDFL model. That is Rob and I act as Benevolent Dictators for Life. In order to grow and ensure that these projects are sustainable we must move beyond the BDFL modus operandi to become community driven. Check out the calendar of events and sign up on our new PROSE Website
We have many questions to answer in the coming months:
What will the PreTeXt / Runestone ecosystem look like in 5 years? 10 years?
How do we transition from our current organization(s) to a community driven model?
Is our goal to be the preeminent publisher of online textbooks for STEM or are we more focused on being a catalyst for change by serving the research community or both?
How do we fund an organization like ours to provide stability for the future? How do we retain the ability to support teachers and students yet pay for support staff and servers?
What organizations can we partner with to facilitate future growth and provide mutual benefits?
How can we recruit new authors? Particularly authors in STEM fields outside of mathematics and computer science?
At the end of all this process we will have the opportunity to submit a Phase II grant to help us achieve the goals we will have set. The POSE program is a wonderful opportunity for a project like ours, but it will only achieve its full potential if we can convince people like you to join us in making it a success.
PROSE is supported by NSF Award #2230153.