Runestone Academy Democratizing textbooks for the 21st century en-us Tue, 11 Jun 2024 00:00:00 -0500 <![CDATA[Collaborative NSF Grants Awarded to Runestone Academy, Michigan, Duke, Virginia Tech, and Berea and American Institute of Mathematics]]> Collaborative NSF Grants Awarded to Runestone Academy, Michigan, Duke, Virginia Tech, and Berea and American Institute of Mathematics

NSF Logo Runestone Academy LTD got good news this week. We received our first collaborative NSF Grant! The grant (#2336758) Testing the Effects of Peer Discussion is a collaborative research project with the University of Michigan, Duke University, Virginia Tech, and Berea College! The grant will allow us to study the effects of different modes of discussion during peer instruction sessions on student learning in computer science. As I was writing this post I got a second email from the NSF indicating that a second proposal had also been awarded!! (#2334768) Collaborative Research: PreTeXt-Runestone: Open Textbooks Engaging Undergraduates in STEM which is a Collaboration with Mathematics educators at the University of Michigan and the American Institute of Mathematics.

We are excited to be working with two amazing teams of researchers and educators. We are also excited to be able to provide a platform for this research.

Peer Instruction

The first grant will allow us to improve the Peer+ platform on Runestone Academy. The grant will fund the following activities:

  • Improve the A/B testing mechanism on Runestone to better compare synchronous text-chat versus verbal discussion in the context of Peer Instruction (PI).

  • Add hundreds of existing questions for CS2 (in Java and C++) and discrete math to Peer+ on Runestone Academy.

  • Test the use of Large Language Models (LLMs) to improve the interactivity of the asynchronous text-chat.

  • Test Peer+ in CS1, CS2, and discrete math courses at the four institutions mentioned earlier.

  • Improve the instructor interface to reduce the time to find existing PI questions.

  • Improve the instructor interface for authoring new PI questions.

  • Conduct an instructor workshop to improve knowledge of PI and best practices.

  • Add additional instructor support to help writing PI questions for ebooks.

PreTeXt and Runestone for STEM

The second grant will advance our knowledge of the process of developing content for three novel interactive questions for four PROTEUS textbooks (PreTeXt- authored textbooks hosted in HTML format on the learning analytics platform Runestone Academy) and their use in classrooms. The project seeks to answer the following questions: How do groups of textbook users (students, teachers, authors, and researchers) work together to write interactive questions designed to elicit multiple ways of students’ thinking in classrooms? and How do students and instructors use those interactive textbook questions in their classrooms? The interactive questions are (i) Short Answer free response, (ii) Matching Exercises, and (iii) Parsons Problems. These will be embedded in textbooks for calculus and linear algebra (Active Calculus, Boelkins, 2022; APEX Calculus HTML Edition; Hartman et al, 2022; First Course in Linear Algebra, Beezer, 2022; and Understanding Linear Algebra; Austin, 2022), chosen because they vary in their support for active teaching, use of visualizations, and emphasis on proof. This grant also provides support for Runestone Academy to make a new hire in the area of development operations and technical support! Yahoo!!


The process of getting these grants was no small feat! Writing the grants was the easy part. Then, back in December, we learned that both had been recommended for funding. But that was just the beginning. In order for Runestone Academy to be awarded the money, we had to go through a rigorous review process to demonstrate that we were capable of managing the grant. This forced us to “grow up” a bit and add in more processes and procedures to our small operation. It’s all about checks and balances in the accounting world! To be clear, Runestone has been a part of several NSF grants in the past but as a contractor, not as a collaborating Principal Investigator.

Once we passed the “New Performer” process then we just had to wait. As you may know the NSF budget was held up in congress for a while, and funding is tight. We were not sure if we would get the grant or not. But this week we got the good news! We are excited to be able to start the project and to be able to work with such an amazing team of researchers and educators. We are looking forward to the results of the study and to sharing them with the community. This is a real boost for Runestone Academy and all of our students, teachers, and collaborators! Hopefully this will help demonstrate our legitimacy and help us to secure more funding in the future.

Tue, 11 Jun 2024 00:00:00 -0500 <![CDATA[SIGCSE 2024: How do you do it?]]> SIGCSE 2024: How do you do it? Runestone Booth at SIGCSE 2024

A second SIGCSE is in the bag for Runestone as an exhibitor. It was a good year. It was amazing to see so many people who have been using Runestone for years and to hear their stories. The number of people who stopped by the booth to just say thank you was really overwhelming! I hope you all know that your gesture will keep me going for months! It was also great to see so many new faces who were just learning about Runestone. I am always amazed at the number of people who have never heard of Runestone before. I guess that is a good thing, it means there are still lots of people to reach.

It was great to connect with so many colleagues from years past. I was able to catch up with people I have not seen in years. I was also able to meet some new people who I have only known through email, Zoom, or Discord. It was great to put real live faces to names.

It was so interesting to hear the questions that people asked. This year felt different than years past. The top questions were “How do you do it?” “How do you keep Runestone going?” “How do you keep it free?” “Are you really free?”” And, “do you have a book that covers X?” But the question I want to address in this post is how do we keep it going? I understand that in this day and age the assumption is that “if it’s free online then the user is the product.” But that is not how we roll, that is not our mission, that is not our model. We are a non-profit and we are here to serve the community. We are not here to make money off of you or your students. We are here to help you teach and learn computer science and math. Our North Star is to make computer science and mathematics education accessible to everyone.

We had answered that question so many times that we determined that we needed to create a new table banner for next time. We also decided that we could have a tongue in cheek answer for the rest of the conference, so Rob created a tip jar. We had a lot of fun with that! Some people were so amused that they threw in a dollar! We are not going to get rich off of that, but it was fun.

Tip Jar

Here is the answer to the question “How do you do it?” We are a non-profit. We are funded by donations, grants, memberships, and ads. We have a small team of volunteer developers who put their effort into Runestone. We have a medium community of authors who help us keep Runestone up to date. We have a large community of students and teachers who help us find bugs and suggest improvements. Yes, professional copyeditors are great, but the masses of students and teachers are even better.

A second part of the solution is to serve ads through the Ethical Ad Network. We don’t like ads in general but the Ethical Ad Network is great because they don’t do any tracking! Also, we don’t show ads to students who are registered for a course.

Finally, we do have a Patreon account, and a Github supporter account, and of course you can donate directly We would love to have your support! We are always looking for new ways to fund Runestone, if you are independently wealthy and want to fund us, we would love to talk to you! If you have a contact at a foundation that might be interested in funding us, we would love to talk to them! If you have a contact at a company that might be interested in funding us, we would love to talk to them! We are always looking for new ways to fund Runestone.

Over the long term we are looking for solutions that will enable us to remain true to our mission of making computer science education accessible to everyone. We know there are many commercial offerings that have been free and then pulled the rug out from under all of the schools that could not pay. That is not who we are. We are committed to keeping the content and as much of the instructor interface free as we can, while planning for the day that Rob and Brad decide to step aside and really retire. It might mean a freemium model, it might mean a subscription model, it might mean a different model. We are open to ideas and suggestions as long as we can remain true to our mission.

In the meantime, we are always looking for new contributors. If you are interested in helping out, please let us know. We are always looking for people to help us with content, with the platform, with the community, with the business side of things. We are always looking for new ideas and new ways to make Runestone better.

Sun, 24 Mar 2024 00:00:00 -0500 <![CDATA[Roadmap 2024]]> Roadmap 2024

A lot of the podcasts I listen to, and quite a few of the blogs I read, encourage you to pick a touchstone word for the year. I’ve never done that before, but I’m going to try it this year. My word for 2024 is “Simplicity”. I’m going to try to use this word to guide my decisions and actions for the year, both in my personal life and in my professional life.

My decision to choose simplicity was reinforced by a recent blog post by Matt Mullenweg, the CEO of Automattic, the company behind WordPress. And a second by Terry Sutton, a front end developer. In Terry’s post he says:

Good open source software is / does / can (roughly in order):

  1. easy to install and setup, for free, both locally and on my own server

  2. powerful, capable

  3. make easy things easy

  4. make hard things possible

  5. opinionated, and so you get:

  6. a lot out of the box, so you don’t have to:

  7. constantly reinvent the wheel to do simple or common things

  8. interoperable, flexible, extensible

  9. Has a core user story in mind (ie: it appeals extremely well to a couple of use cases, while enabling many others)

  10. continuously improved and updated

I think that’s a great list, and I think that the Runestone software ticks a lot of those boxes, but not the first one, and I’ve heard from many of you, not not the third one. It certainly is the fifth one, and I think sixth through tenth follow. The good news is that principles 1 and 3 are closely related to my touchstone word for the year.

Runestone has been around since 2011, and it has grown organically over the years. Myself, students, other contributors added features as I’ve needed them, and as others have asked for them. I’ve tried to keep the code clean, and the architecture flexible, but I think it’s time to take a step back and look at the big picture. I think it’s time to simplify.

Moving to the monorepo was a first step in that direction. I think it’s time to take a second step. I’m going to try to simplify the user experience, particularly for instructors. I’m going to try to make it easier to install and setup, and I’m going to try to make it easier to use. I’m going to try to make it easier to contribute to, and I’m going to try to make it easier to extend.

Here are some some of the things I’m targeting especially for the first half of 2024:

  • Create a quick programming project interface to serve as a replacement for instructors who used

  • Simplify the experience for creating and grading assignments

  • Simplify the installation process

  • Simplify the configuration process

  • Make the developer experience better for contributors

  • We are going to have to bite the bullet on LTI 1.3 support soon! People are deprecating 1.1 as we speak.

Please note, these are Goals not guarantees. I had many features I wanted to implement in the Fall of 2023, but most of my time was eaten up answering questions from users, and dealing with growth and scaling issues. I’m hoping that by setting these goals, and by sharing them with you, I’ll be able to stay focused on them.

I’m not going to go into details here, but I’ll post about progress periodically. If you want to get into the details, it is all on Github in the form of six projects (Sort them by name). Under the hood the main goal is to get all of the instructor facing pages ported to the same underlying technology we use for the assignment and book serving pages. This will make it easier to maintain, and easier to extend. It will also make it easier to add new features, and to make the user experience more consistent.

Sun, 07 Jan 2024 00:00:00 -0600 <![CDATA[Problem Solving with Algorithms and Data Structures using (Python|C++|Java)]]> Problem Solving with Algorithms and Data Structures using (Python|C++|Java)

A triumph of open source and open content

In 2005 I was in my second year of teaching computer science at Luther College. I had enough experience to adamantly oppose Java as our introductory language, and I had learned about Python at the 2004 SIGCSE conference. I was excited about Python and had been playing with it for a year. There was an excellent introductory textbook by John Zelle who was at a nearby college. In our departmental discussions we thought that Python would be a good introductory language, but we didn’t want to give students one semester of Python and then a second semester in Java or C++. We wanted to stick with Python. So I said to my colleague David Ranum, “lets write our own data structures book!” And so we did. We contacted Zelle’s publisher who was very interested in our idea. We wrote the book in 2005 and it was published in 2007. It has been used at several universities as the primary textbook for the data structures course taken by second year computer science majors, as well as a supplementary text for students in other disciplines. The book has been translated into Spanish as well as Chinese.

The goals for the book were: * To have a strong focus on problem solving, that introduces the students to fundamental data structures and algorithms. * To introduce algorithm analysis in terms of Big O notation. * To use Python as the programming language. * To illustrate that we are all “standing on the shoulders of giants.” In computer science we are building on the work of those who came before us.

I loved teaching from this book. The students seemed to enjoy the problems we introduced and learned a lot. By the end of the course they were all much more fluent in Python and ready to tackle other languages and more advanced topics. They all had good “gut level” understanding of the major data structures and algorithms. I was very proud of the book and the course.

I’m also proud that in the 17 years since the first edition was published we have only had two more editions. There is no need to gouge students every two years with a trivial update. The book is still relevant and useful today.

Fast forward to 2011, I was on sabbatical and was working on the second edition of this book. But I had a serious case of “writers block” my main problem was that I just hated the idea of a paper textbook for computer science in the 21st century. Books should be interactive, you should be able to run these algorithms right in the book! That was the genesis of Runestone Academy. The next step was to convince our publisher that we should make the book freely available online. I’ll never forget the response of our publisher, Jim Leisy, “Lets go change the world!” With that, we were able to make the book available under a Creative Commons license. The book is still available in print, but the online version is the one that is being actively maintained, and through our strong partnership with the PreTeXt project we can write once in PreTeXt and publish both online and in print.

The Creative Commons license is a great example of the “shoulders of giants” lesson we wanted to teach our students as in 2018, Jan Pearce of Berea College in Berea Kentucky took our book and created a version for her students that used C++ as the primary programming language! She kept the Python as “pseudocode,” and added C++ examples along side. The content was what she was looking for but the programming language was wrong. So she changed it! She also added some new content and examples. She then shared her changes back with us, and we were able to incorporate the book into our collection. The C++ edition of the book has been used by over 3,000 students in over 223 courses around the world.

In the Fall of 2023 J. David Eisenberg of Evergreen Valley College in San Jose, CA, took the book and created a version for his students that used Java as the primary programming language! The content was what he was looking for but the programming language was wrong. So he changed it! This book does not have Python as pseudocode, but has a lot more interactive exercises and more of the examples can be run in the book than in the Python edition. He also added some new content and examples.

I can’t tell you how proud I am of these two projects. Even though I was personally opposed to Java (and C++) as an introductory language, we don’t all need to agree. And I know I’ll never convince the college board they are wrong. These two books are a great example of the power of open source and open content. I hope that you will consider using this book in your own courses, and if you do, please consider sharing your ideas or changes back with us so that we can all benefit from your work. I would suggest that the Java book is very accessible to high school students who have had the AP Computer Science A course. The Python edition is very accessible to students who have had a semester of programming in any language. If you are willing to help with any of the editions, but especially the Python edition I would love to hear from you. I would really like to make the Python edition much more interactive.

Thu, 04 Jan 2024 00:00:00 -0600 <![CDATA[PROSE Workshop Report]]> PROSE Workshop Report

Last week we held the PROSE Organizing Workshop at the University of South Alabama. We had 30 people in person and several more attended virtually. The attendees included seasoned veterans from the PreTeXt and Runestone projects as well as some people completely new to the project. One virtual attendee had literally learned of the project just days before the start of the workshop. We had a great time and got a lot of work done. We have a lot of work to do still, but we are off to a great start. In this post I will summarize the takeaways from the workshop and the next steps for the project.

What is the Scope of the Ecosystem?

One key question that needed to be answered was What and/or who is the PreTeXt / Runestone ecosystem? After lots of good discussion we decided that the ecosystem is best described by a set of principles, anyone or any project that shares those principles may be part of the ecosystem. The principles are:

Our Open Source Ecosystem is a community of students, teachers, authors, researchers, and developers committed to the following principles:

  1. We exist to:
    • Help students learn more efficiently

    • Help teachers teach more effectively

    • Help authors write better materials

    • Enable researchers to develop more effective tools and pedagogies

  2. We are committed to ensuring access to the best open source learning environment to all of our stakeholders, but especially to the students who need it the most.

  3. We embrace the open web and open standards.

  4. We welcome new partners who share our values.

  5. We will be flexible in creating a platform that incorporates new innovations.

  6. We recognize and celebrate all contributors and innovations that move our ecosystem forward.

In an open source ecosystem we are all “standing on the shoulders of giants,” there are countless projects that we rely on to make our software work, however we want to recognize some of the initial projects that are core to our ecosystem:

  • PreTeXt - An XML vocabulary for authoring and publishing high quality print and digital textbooks

  • Runestone Interactive - A Learning Engineering and Analytics Portal (LEAP) for hosting textbooks, assignments, and interactive learning activities.

  • WebWork - An open source online homework system for math and science courses

  • Doenet - An open source platform for creating interactive learning activities

It also includes many textbook authors, instructors, and researchers who are using these tools to create and deliver high quality learning experiences for their students. Research is ongoing at several universities including the University of Michigan, the University of South Alabama, and the University of Illinois, UT Austin.


We have a long term vision for the leadership of the ecosystem that may become a formal entity like the Apache foundation or an academic society. For our initial work, we have created an advisory council of eight people who represent many different aspects of our ecosystem of Open Educational Resources (OER) and embrace our guiding principles.

The initial leadership council includes the following people, all of whom have committed to an initial two year term.

  • Rob Beezer - Professor Emeritus of Mathematics at the University of Puget Sound, Founder of the PreTeXt project, author of the open source textbook A First Course in Linear Algebra.

  • Matt Boelkins - Professor of Mathematics at Grand Valley State University, author of the open source textbook Active Calculus.

  • Steven Clontz - Associate Professor of Mathematics at the University of South Alabama, maintainer of resource library, and co-author of the PreTeXt-cli authoring toolkit.

  • Barbara Ericson - Assistant professor of Information at the University of Michigan, and co-author of the open source textbook CSAwesome.

  • Danny Glin - The University of Calgary, and member of the WebWork Executive committee.

  • Brad Miller - Professor Emeritus of Computer Science at Luther College, founder of Runestone Academy LTD, and co-author of the open source textbook Problem Solving with Algorithms and Data Structures using Python.

  • Jennifer Nordstrom - Professor of Mathematics at Linfield University, and author of the open source textbook Game Theory: a Discovery Approach.

  • Duane Nykamp Associate Professor and Director of Strategic Initiatives at the University of Minnesota, and founder of the Doenet project.

This group will guide us through the initial stages of the project and help us to create a more formal governance structure as the project matures.

As part of the next steps we will be creating additional leadership opportunities in the form of Fellowships for people who are working to grow our ecosystem. We will also be funding mini grants to inspire and reward contributors to the ecosystem.

Workshop Attendees

Next Steps

There were so many great ideas that came out of the workshop, I can’t possibly list them all here. But we did identify some concrete actions that we need to get busy on in the coming months in order to make this project a success. Here are some of the next steps that we identified at the workshop.

  1. Write a Phase II grant proposal for the NSF to fund the next stage of the project.

  2. Find a name for the project that is more descriptive of the ecosystem, and will allow us to create a website that will be the hub of the ecosystem.

  3. Work on ways to keep the community engaged and build on the momentum from the workshop.

  4. Develop a plan for making sure that we are defending the intellectual property of the ecosystem.

  5. Continue our discussions and planning to create a sustainable funding model for the project beyond the initial NSF grant.

  6. Continue the technical discussions that were started at the workshop. These include a working group for improving the user experience on Runestone Academy; a working group for improving the authoring experience for PreTeXt; continue the the work on integrating Doenet and WebWork with PreTeXt and Runestone.

  7. Develop a plan to be more inclusive and diverse in our ecosystem.

Sun, 18 Jun 2023 00:00:00 -0500 <![CDATA[PROSE Workshop Day 1]]> PROSE Workshop Day 1

Yesterday 30 of us convened at the University of South Alabama for the first day of the PROSE workshop. Steven kicked us off reminding us that the main goal for the week was to prepare for writing our Phase II proposal. Rob and I both gave a little talk on the state of PreTeXt and Runestone.

In the afternoon we did an exercise to get everyone thinking about the different groups of stakeholders in the ecosystem and how we might serve them better. We will hear more results from the that tomorrow.

In the evening, we convened a hackathon back at the hotel. It was a noisy affair and we brought the hotel wifi to its knees. We did manage to get a few things done though.

Tue, 13 Jun 2023 00:00:00 -0500 <![CDATA[Important Privacy Developments]]> Important Privacy Developments

For years, student data privacy has been a hot topic in education. In 2014, the U.S. Department of Education issued guidance on the Family Educational Rights and Privacy Act (FERPA) and the Protection of Pupil Rights Amendment (PPRA) to help schools and districts understand how to protect student data. In 2015, the White House released a Student Privacy Bill of Rights to help ensure that student data is used only for educational purposes. And in 2016, the U.S. Department of Education released a new Privacy Technical Assistance Center (PTAC) to help schools and districts understand how to protect student data.

The way all of this has impacted Runestone Academy is that many school districts are now requiring that we sign a Data Privacy Agreement with them before they will allow their students to use Runestone Academy. In the beginning this was a giant pain because each district had their own agreement, usually thrust upon them by their state department of education. There was little chance to negotiate and many of the agreements contained clauses that were not acceptable to us. For example, some of the agreements required that we delete all student data at the end of the semester. This is not acceptable to us because we want to be able to provide students with a record of their work in the course. (We do automatically remove data after 2 years) Other agreements wanted us to certify that we were HIPPA compliant, but since we don’t collect any health information this was not something we could do. Most problematic was that many agreements had a clause that obliged us to reimburse the school district if we ever had a data breach. Although highly unlikely, the costs associated with recovering from a data breach could be very high and we could not afford to take on that risk. Thankfully after a long search, we were able to find a company that would provide us with reasonably priced insurance against a data breach and we are now comfortable signing these agreements.

In 2020 we signalled our strong support for student privacy by signing the Student Privacy Pledge. You can read the pledge here: I also bothered countless people with emails complaining that these data privacy agreements were killing open source projects like Runestone because we don’t have the resources or knowledge to negotiate these agreements. I am happy to say that things are getting WAY better.

Introducing the National Data Privacy Agreement (NDPA) which is a standard agreement that schools and vendors can use to protect student data. This agreement (fostered by the Student Data Privacy Consortium) was developed jointly by schools and vendors and creates a set of common expectations between schools and providers like Runestone. We have already signed agreements with several schools that use the NDPA. You can read more about it here: We are happy to sign this agreement with any school that requires it. Even better, if your school is a part of a consortium where another member has already signed the agreement, then all your school has to do is sign Exhibit E and send us a copy. These consortia are often run at the state level, but in some cases multiple states have banded together. The map below shows the states that have adopted the NPDA and have consortia in place. In addition most of the state consortia have a database of vendors so you can do a quick search to find out if Runestone Academy has already signed with that state. If we have, then all you need to do is sign Exhibit E and send it to us. If we have not, then you can send us the agreement and we will sign it.

Map of states with SDPC consortia

The development of the NDPA is such a positive step forward for student privacy and it is a huge reduction in the amount of work that I have to do to sign agreements with schools. I am very grateful to the Student Data Privacy Consortium for their work on this. If your school is not a part of a consortium, please encourage them to join one. It will make it much easier for them to sign agreements with vendors like us. If your school is developing their own agreement, please encourage them to use the NDPA rather than creating their own. It will save them a lot of time and effort and will make it easier for them to sign agreements with other vendors.


I have been enjoying using Github copilot with VSCode for both Python programming and writing blog posts. Copilot is based on OpenAI’s GPT-3 language model. In fact the first paragraph of this post was written by Copilot. I wrote the title and the first few words of the first sentence and it wrote the rest. I did have to edit it a bit, but it was a great starting point. It also had some good and not so good suggestions for other parts of this post, including making up urls to non existent pages. Its really great at taking some of the drudgery out of programming in that it can write a lot of the boring code very quickly leaving me more time to work on the hard stuff. I am looking forward to seeing how this technology develops.

Thu, 25 May 2023 00:00:00 -0500 <![CDATA[Top 10 Reasons to Host your Book on Runestone Academy]]> Top 10 Reasons to Host your Book on Runestone Academy

After four years of collaboration, Runestone and PreTeXt are ready to encourage hosting PreTeXt books on Runestone Academy servers as a routine matter. We’ve got several months before fall classes, so now is a good time.

The process is quite simple: put your PreTeXt source on GitHub if it is not there already, and provide a publisher file that specifies that you want a build specifically for Runestone servers. If you are using the command line tools / Codespaces with a project file then you are nearly ready. Let Runestone know by creating an issue in the Runestone repository. Your book will be made available to thousands of instructors and will be re-built automatically every Saturday.

Here’s our Top 10 list for why you would want to consider this, numbered for mathematicians and for computer scientists.

  1. Free. In every sense of the word. Open-source software and texts, hosting free to instructors and students.

  2. Hosting is low-friction for instructors, no local IT required. Fantastic up-time.

  3. Weekly updating with no hassle. Or more frequently with the author interface.

  4. Runestone Academy had over 115 million page views last year and has good search-engine optimization to increase the exposure of your book.

  5. Instructors get a detailed dashboard of students’ work and progress.

  6. Runestone Academy is used by over 10,000 instructors–who already use OER and are happy to spread the word.

  7. Authors get gross analytics across all institutions. Which of your exercises work, and which don’t?

  8. Authors get accurate data on adoptions and readership. Impress your Dean.

  9. You maintain complete ownership and control of your materials.

  10. Instructors get autograding and autorecording. Get a spreadsheet at any time or link it to your LMS through LTI.

  11. PreTeXt and Runestone are actively promoting books at both Mathematics and Computer Science conferences.

To get started, read the chapter on Runestone in the Publisher’s part of the Guide, especially the second section. It has pointers to other relevant parts of the documentation.

Chapter 32: Conversion to Runestone

We would encourage you to look at the annotated sample book, especially the Runestone chapter to see the interactive elements that you can begin using in your books, written natively in PreTeXt but using Javascript developed by Runestone over the last 12 years.

There is a fast-developing converter from Runestone restructuredText syntax to PreTeXt, which will ease the transition for computer science textbooks to PreTeXt.

We would be remiss to not acknowledge the trailblazers who have helped test this out as it has been developed.

Native PreTeXt:

  • Oscar Levin; Discrete Mathematics, An Open Introduction

  • Sean Fitzpatrick; APEX Calculus

  • Mike May; Business Calculus with Excel

  • Alex Jordan; Open Resources for Community College Algebra

  • Kathy Yoshiwara; Intermediate Algebra: Functions and Graphs

  • Tom Judson; Abstract Algebra, Theory and Applications

  • Tom Judson; The Ordinary Differential Equations Project

  • Alex Jordan; WeBWorK integration (!)

  • Matt Boelkins, Mitch Keller, Chrissy Safranski; Active Calculus

  • Steven Clontz, Drew Lewis; Linear Algebra for Team-Based Inquiry Learning

  • Rob Beezer; A First Course in Linear Algebra

Conversions from Native Runestone:

  • Brad Miller; How to Think Like a Computer Scientist

  • Brad Miller; Problem Solving with Algorithms and Data Structures using Python

  • Brad Miller, Barb Ericson, J. David Eisenberg; Python for Everybody

  • Beryl Hoffman; Java, Java, Java (from LaTeX!)

  • Jan Pearce; Problem Solving with Data Structures and Algorithms using C++

  • Jan Pearce; C++ for Python Programmers

  • Jan Pearce; On Complexity

Mon, 22 May 2023 00:00:00 -0500 <![CDATA[Runestone at SIGCSE 2023]]> Runestone at SIGCSE 2023

I am pleased to announce that our NSF grant provides us with funds to be an exhibitor at SIGCSE this year. Please stop by our booth and say hello. If you don’t know anything about Runestone we would love to introduce you. If you are a long time user we would be happy to hear your feedback and ideas. Also, we have stickers.

../../../_images/booth_banner.png ]]>
Sun, 12 Mar 2023 00:00:00 -0600 <![CDATA[Do commercial publishers have a place in the open textbooks ecosystem?]]> Do commercial publishers have a place in the open textbooks ecosystem?

Suppose a textbook was used at more than 100 universities, with many websites advertising the book and showing off its numerous high-quality features. Suppose many experts in the subject have produce supporting materials: videos, worksheets, sample exams, sample syllabi, advice to the instructor, and suppose those materials were already available for free – making it very attractive for the textbook to be adopted by new institutions.

And suppose a commercial publisher was given the opportunity to produce and distribute the book. Would that be a good business opportunity? Of course it would! A textbook publisher should jump at the chance to bring out a new book with an already well-established market!

Except, it isn’t actually a new book. And unlike a traditionally published book, it isn’t the sole and exclusive property of the author, because the book has been released with a license that allows anyone to modify it and make their new version available. And everything in the printed book already exists on free websites, and for free as PDF, EPUB, or on a kindle. The printed book has to compete against itself in other formats.

Is the open nature of the book a deal-breaker for publishing commercially? I say No. Many open textbooks are already being published in print form. Students buy them: many want an inexpensive printed copy because the online versions are great in some ways but fall short in others. Many open textbook authors go to the effort to produce a print version; I speculate that many would be happy to hand that work off to a commercial publisher.

Issues, from the perspective of the author

We already mentioned an attraction for the author: they probably don’t want to do the work of preparing and distributing the print edition.

A commercial publisher would also provide effective advertising – another aspect of self-publishing where many authors fall short.

Also, it is still an unfortunate (and misguided!) fact that open/self-published textbooks are viewed with suspicion by textbook adoption committees. The stamp of approval from a commercial publisher carries a lot of weight.

Primary concerns for the author are price and availability.

The author may have been motivated by the seemingly high price of some commercial textbooks. If they have been self-publishing the print version, it probably sells for around $20. An excessively high price on the commercial print version could be a deal breaker.

The author probably has been making a high-quality HTML version available for free, and probably also a PDF version, and maybe also EPUB. It is unlikely the author would be willing to give up the free HTML version, hosted on a website that they control. It is unclear whether the author would have strong opinions about the EPUB and free online PDF.

There are other author issues, which do not seem specific to open textbooks, such as duration of the agreement with the publisher, what happens if the book goes out-of-print, expectations about subsequent editions, non-compete clauses about writing similar books, royalty rate, etc. As we will see below, some of those issues are in fact different for open textbooks.

Issues, from the perspective of the publisher

By far the biggest issue is dealing with an unfamiliar situation.

Until recently, there were not a lot of excellent open textbooks. Now there are. PreTeXt ( and Runestone deserve a lot of credit for facilitating authors to create high-quality books with many features that enhance teaching and learning.

So, it will take vision, and a tolerance for the unknown, for a commercial publisher to dip their foot in this pond.

For the sake of argument, consider the situation of an open textbook which is being used at several universities, with free HTML version available online (possibly at more than one website), with a print version from the author (or someone they designated) available for $20-25.

The publisher probably would like the author to stop publishing the print version. It is safe to assume the author would agree, provided that the price did not increase significantly.

The publisher would probably want the existing print version (which probably is print-on-demand) removed from the marketplaces where it appears. Again, the author should agree.

What does the publisher think about the author’s website hosting the HTML version? How does that relate to the home page of the book on the publisher’s website? Presumably those pages should link to each other, because both parties benefit from the success of the other.

How about the PDF version the author currently posts? Probably that is a non-issue, because for PreTeXt books you can print from the HTML, and for many purposes it does what one wanted from the PDF. If there is a reason the author would object to taking down the PDF, I would like to hear it.

EPUB is another issue. Many publishers make money by selling the EPUB: that revenue stream would be diminished if the author posted it for free. The EPUB version is recognized as a valuable resource for people in prison, because it does not require an Internet connection and it does not take up space. This is important to people involved with the Prison Education Project and related endeavors. Both sides should be aware of these issues during the negotiations.

If the above are the main concerns of the publisher, then they have missed some essential facts about the world of open textbooks.

Some realities which have to be discussed and understood

As was already mentioned, the book isn’t the sole and exclusive property of the author.

If the source is available with an open license, anyone can make a PDF version, or produce a print-on-demand version and sell it at whatever price they want.

Anyone can also add a new chapter or section and publish it however they want. Forbidding the author from using that book as the basis for a competing product, cannot stop someone else from doing the exact same thing. And the original author’s name is also on the new book! (How authorship is credited in these situations is untested waters. Open textbooks are touted for their ability to be modified and customized, but in practice this rarely happens and the community has not had to deal with complicated situations.)

The publisher has to accept that they do not have a monopoly on the book: they will be the only print publisher if the author agrees that it is better for the publisher to handle the print version, and nobody else decides that they can do it better (where “better” involves some combination of price, quality, and features).

A clause in a contract may forbid the author from taking certain actions, but if the book has an open license, that contract cannot prevent someone else from performing those exact same actions.

The general point has been made, but I’ll repeat it for this specific case: nobody can stop someone else from adding a small amount of material and publishing it as the “new edition” of the book.

Here is an issue which seems silly but is not: movie rights. The excellent open textbook APEX Calculus has a version with more than 600 videos by Sean Fitzpatrick (not the author of the book, and that version is not hosted by the author) with a cumulative duration of more than 75 hours, explaining concepts and working through numerous examples. Stacking those videos creates a multi-part epic movie version of the book. Whose rights are infringed by that cinematic extravaganza?

Most commercial publishers have a fundamentally misguided policy concerning Braille versions. Each institution which needs a Braille copy has to go to the effort, typically taking months and costing $20,000 or more, of having an expert Braille transcriber convert the book. They are not allowed to then share it with another university! Think about it: not only does the publisher not provide a Braille version, they make it as hard as possible for other people to do so. Textbooks written in PreTeXt automatically convert to Braille, at minimal marginal cost. That fact should have an impact on the publishing contract.

A modest proposal

One possible way to address some of these issues is for the publisher to commission a new edition of the book, and for the author to not release that edition with a fully open license. This would not change the fact that the previous edition was still available open source and with a permissive license. But the publisher’s concerns about competing products would be mitigated by the fact that the anyone other than the author would be several steps away from replicating the published book.

The “edition trick” is not a magic wand

The modest proposal would involve the author abandoning their previous vision for the open source ecosystem of their book. Would they release the new edition with a more restrictive open license, such as CC-Attribution-ShareAlike-Noncommercial, or would they keep the source private and reserve all rights? Would the restrictions apply only to print publishing, or would other online versions also be prohibited?

Is this modest proposal the author’s first step on the slippery slope of producing a commercial book that is exactly what they were trying to avoid when the decided to write an open textbook?

Keeping the fully-functional online HTML version available for free seems like an absolute requirement for the author. Is the author assigning the copyright to the publisher and the publisher granting rights for the online version, or is the author retaining copyright and granting the publisher the right to produce and sell a print version? Or does the book have an open license and the publisher has to accept the fact that keeping the author happy is their main protection against competing print versions?

If the book incorporates material under an open license from another author, putting a more restrictive license on the next edition might not even be an option.

Here’s an ironic fact. Suppose the author decides not to go with a commercial publisher and just releases the new edition with the same open license. Then the publisher can just produce and sell the print version anyway! And they won’t have to pay any royalties!

If I come out with a book that has 80% overlap with an existing book, I will be sued by the publisher, and I will lose. But for an open textbook, even with the “edition trick” I have been given the rights to produce such a similar book based on the previous edition. No agreement between the author and publisher can change that. And the author cannot remove the rights which I have previously been given.

Maybe all of these concerns are non-issues?

Perhaps, with or without the “edition trick”, the online versions of the new edition can function exactly like the previous edition, with the print version now coming from a commercial publisher who provides a modestly priced copy and still makes some money, and nobody is motivated to duplicate either the author’s or the publisher’s work?

There would be very little risk for an established publisher to produce and distribute the print version of an existing open textbook, on just a handshake agreement with the author. The author would agree to not publish a print edition, and would make a reasonable effort to promote the publisher’s print version and discourage others from making a competing print version. The publisher would agree to make their print version available at a price comparable to what the author could do on their own. The publisher would advertise the book in ways beyond what the author could do. Every free online view of the book would be an effective advertisement for the print version. The publisher makes money. The author probably makes a bit less per print copy, but they are spared the work of being their own print publisher and end up making more in the long run because more copies are sold.

The publisher’s lawyers might be unhappy with a handshake agreement, so a contract could codify the points in the previous paragraph. If the contract were for one year, renewable and properly synchronized with the academic year, then it could be workable for one party to handle the print version and traditional advertising, and the other party to manage the online versions and provide support and outreach to instructors using the book.

Are there commercial publishers who can appreciate this business opportunity? Do most open source textbook authors understand that they are not giving up anything if they enter the type of agreement I propose?

I await the publisher and the authors who fully understand the open source ecosystem and its relationship to commercial publishing, and see an opportunity for harmonious collaboration.

Wed, 08 Mar 2023 00:00:00 -0600 <![CDATA[Year in Review 2022]]> Year in Review 2022

With the year 2022 in the rear view mirror I wanted to take some time to update you all on what we have accomplished in the last year, give you some fun facts, a financial update, and talk about the community in general. So lets get to it!

Fun Facts

During 2022 we welcomed 74,000 registered students and instructors to the community! That is a lot of new people and a lot of money saved on textbooks! During an average school day we have around 1000 students active during a given five minute period. I think that is something we can leverage moving forward. The traffic on Runestone clearly shows that we have a “homework bump” that is once the traffic slows down at the end of the school day across North America, we have an evening spike that lasts three to four hours. We have no such spike on Friday nights. Also, students do not do homework on Saturdays. In 2022 we had 115,100,000 page views! That is nearly double the number of page views from 2021. About 60% of the page views are from registered users while the other 40% come from searches through Google, Bing, and others.

What makes Runestone Academy Tick?

People, 100’s of people who donate their time in many ways.

  • Authors – between PreTeXt and Runestone we probably have around 60 different authors represented on Runestone Academy.

  • Developers – Over the years around 100 developers have contributed at least one line of code to Runestone, and around 50 developers have contributed at least one line of code to PreTeXt. Over the years this job has become more difficult as the complexity of the whole system increases. I am aware of this and actively trying to find ways to make it easier for anyone to help with development.

  • Dev Ops - Monitoring the servers, building, testing and deploying updates, to our servers, building, updating and deploying updates to the textbooks, are all things that happen weekly. It takes time, although a lot of that time is just keeping an eye on things while they run in the background. For example it takes about 4 hours to rebuild and deploy all of the books on Runestone Academy. Its mostly automated but you have to keep an eye on it for those few times when something goes wrong. It takes another 30 minutes or so to rebuild and deploy all of our servers.

  • Bug Finders - Reporting problems to us is equally as important as writing code! I am always surprised by the number of people who just live with a bug instead of reporting it to us on github. PreTeXt is tracking 1242 issues while Runestone is tracking 2216. Of course those numbers are obsolete the moment I type them. Issues come in all forms and we welcome reports of typos as well as serious bugs in the Runestone servers.

  • Exercise Creators - Over the years the question bank for Runestone has grown by leaps and bounds - over 14,000 questions not including those that are part of a book. This is a great way to learn a little about restructuredText or soon PreTeXt and contribute to the whole. If you have a good exercise please feel free to add it through the web interface. You can keep it private if you wish or you can share it with all the other instructors out there looking for new homework problems.

  • Exercise Editors - With the huge growth in the size of the question bank, we have created quite a problem for ourselves. Many of these questions are just dummy questions that instructors use to learn and should be weeded out. Anyone can help with this problem by flagging a bad question. This puts it in the queue for an editor to take a look at and either delete or update. We need loads more exercise editors!

  • Community Builders - Do you know what makes for the most successful textbooks on Runestone? It is an active and supportive community of teachers using that book. The best case in point for that is CSAwesome. It truly is awesome the way teachers help and support each other on the mailing list. But that takes some effort and leadership to nurture Beryl Hoffman and the people at the Center for Computer Science Education do a wonderful job. We need more of that for all of our books!

  • Documentation Writers and Trainers - Have you gotten pretty good at some aspect of Runestone? How about taking a look at the documentation and bringing it up to date, or making it easier to understand? By now I’ve lost all perspective of what it is like to be a newcomer to Runestone, but many of you have not. Would you like to host a workshop to train people how to use Runestone? We would be happy to support that!

  • Word Spreaders - If you like Runestone or any of its books please help us spread the word. We would love to have everyone recommend Runestone and PreTeXt to two colleagues. We would love to have everyone mention Runestone and PreTeXt on social media. Do you Write a blog? Toot on Mastadon? Post on Facebook? Tweet on twitter? How about a shout out? I have left both Facebook and Twitter but if someone would like to take over the FB page or the iRunestone account on twitter let me know! Anybody can help with this and the more we grow our community the more good ideas we get.

Accomplishments in 2022

New Server Architecture

Looking back at it now, I have to agree that 2022 started off a little bit rocky. We moved to a brand new server architecture, and a brand new server to handle the majority of the book serving traffic. After a month or so of headaches things calmed down, and I’m super pleased with the new architecture. It allows us to scale horizontally as our traffic ebbs and flows over the course of the year. There is much more to be done in this area, but we will do our best to make it as seamless as possible.

The PreTeXt Partnership

During the Summer of 2022 a lot of work that has been going on since just before the global pandemic took hold really began to pay off. The partnership with the PreTeXt project allows me to focus more on the server and interactive widgets for learning, while freeing me from maintaining and improving an authoring language. We converted our first books from restructuredText to PreTeXt and in the words of Billy Crystal “they look mahhhhhvelous!” My hope is that we can transition the rest of the books during 2023. As we master the conversion process we will also be able to make PDF, Braille, ePub and other formats of our interactive books available. Though of course they will not be interactive.

I’m also really excited to see that a number of the Mathematics authors have embraced the interactive elements that Runestone brings to the table in writing their textbooks. Parson’s problems for doing proofs or derivations anyone?

The Launch of Peer Instruction

During the winter of 2022 we piloted our new peer instruction system, this was followed by a workshop attended by about 20 instructors who used peer instruction in their classroom during the Fall of 2022. This is a very exciting new feature that you can use to replace clickers in your classroom and which also works well in the virtual classroom. This work was funded by the NSF and led by Dr. Barbara Ericson at the University of Michigan.

The Launch of the Author Server

Every year I hear from a handful or more of instructors that want to customize one of our books for the classroom. While making that easy for everyone is still a ways off we took our first steps in enabling that by creating a server just for authors. The server allows you to start with a brand new book, or with the fork of an existing book on github and gives you control over when the book is updated and published either publicly or privately. A few brave authors have been using it for the 2022 fall term and it has worked quite well. This server also has the beginnings of what I am calling an impact report, that is, as an author you will be able to track the usage of your book, and understand what parts of your book are used the most. We look forward to working with authors to see how we can expand this out to provide much finer grained information on individual questions and other features.


Formerly known as horizontal parsons, this new interactive question type allows you to create a parsons problem for an individual expression or statement. This is great for teaching SQL or regular expressions, or even helping students work through some trickier kinds of expressions that you might write in Java or Python. The new Micro parsons are also leading the way in terms of allowing parsons problems to be graded by unit tests in addition to simple block order.

Joint Runestone - PreTeXt Workshop

In June of 2022 we held our first joint workshop, where we brought together about 20 authors and developers from the PreTeXt and Runestone projects. The workshop was held in Ann Arbor on the University of Michigan campus and it was an amazing week of learning together, sharing research, and brainstorming ideas for the future. I’m so excited about where this could all lead.


We were honored to receive a grant for the project we call **PROSE** - PreTeXt Runestone Open Ecosystem. This NSF grant is designed to help projects like our become a sustainable ecosystem. For us that means many things: recruiting community managers, developers, authors, editors, question bank maintainers, and more. It also means reaching out other projects to see how we can work together more effectively and efficiently. There are so many projects that are part of the “Ed Tech” world and there is so much needless duplication of effort. The marriage of PreTeXt and Runestone demonstrates that two projects can come together and be more efficient and stronger when we work together, we want to spread that message and find other projects who share our vision and are willing to join us. Another big part of this grant is to help us figure out how to organize and maintain all that we do going forward. In a world where anything can happen, we need procedures and people in place if something horrible were to happen.


In late 2021 Runestone Academy LTD was formed as a new corporation to supplant Runestone Interactive. We are operating Runestone Academy as a non-profit. Although the conversion process took a lot longer than expected Runestone Academy has also filed our paperwork to get IRS approval as a 501(c)(3) which will give us official non-profit status for tax purposes. We have also named a board of directors to help us think strategically about how we ensure a strong future for Runestone. I am honored to have this group of forward thinking people help guide the way: Paul Resnick, Assoc. Dean of the University of Michigan School of Information, David Farmer of the American Institute of Mathematics, Susan Rodger, former President of SIGCSE and of Duke University, Danny Fenjves former founder and CEO of Upperline code now leads the LEARN team at Giant Machines and Jane Miller our CFO and Secretary/Treasurer who has been a partner in my crazy endeavors for the majority of my life.

The financial details of any transition year can be confusing, especially when trying to compare to a past year, so lets just focus on 2022 and mostly not try to compare to last year.

These are unofficial numbers but they should be pretty accurate. We have several different sources of income including students, schools, ads, partner support from people who use Runestone for their own purposes but want to support our work, Grant money, and consulting work on grants. Here is how that all breaks down.




Direct support from students


Support from institutions (departments, colleges, schools)


Advertising Income


Support from Grants


Support from partners


Webwork income









Hosting and server costs


Webwork expenses


Legal fees


Credit card and Paypal fees




People costs (healthcare, salary, etc)




So, you might think that we have a net profit of about $2,000 which would be great for a non-profit, everything balances. But let me point out a few problems with that conclusion.

  1. People costs – $44,607 includes 1 full time person and 1 part time person. Thats not a very good salary.

  2. Support from institutions is what we have Invoiced for $17,195 Unfortunately we have been paid only $12,165 of that. People will often ask for an invoice and then completely ignore the invoice and our followup emails. This makes it hard for us to plan and count on that money.

In 2023 we have and will institute a couple of changes.

  1. If you school wants Runestone Academy to sign a contract or data privacy agreement we will invoice the school for $100.

  2. During the first part of the year we will be working on determining a small set of features will will require payment. None of the paid for features would prevent you from using Runestone for free. These are all features that are nice to have, but provide good backup for why you could pay to support Runestone. I know that many public schools have policies that don’t allow you to “donate” so by providing a few features that you can only use if you pay turns that donation into a fee for service.

Runestone Academy is not a big corporation. The two employees mentioned are myself and my wife. As I made clear above, Runestone is powered by volunteers. The Runestone community has accomplished a LOT on very little over the last 10 years. The challenge is to sustain that progress and spirit in an ever changing world.


This past fall I ran a survey for the instructors who use Runestone Academy, I learned a lot from the survey that will help us set a course to smooth out some of the rough edges of the Runestone platform. But even more than that was the outpouring of support for the mission. So many people have come to rely on using Runestone Academy in their classrooms every day, and so many people said that love it! That kind of feedback is really energizing and makes me want to work hard to keep building and improving things.

Our mission is expanding, Democratizing textbooks for the 21st century is only part of what we do now. An updated mission statement might read as follows:

>To enable authors and researchers to maximize the accessibility of open educational resources for learners, and advance the teaching of STEM fields.

As with any mission statement it needs a bit of unpacking. Accessibility is the practice of making information, activities, and/or environments sensible, meaningful, and usable for as many people as possible. And the goal is to create a virtuous circle of feedback that drives progress for our four major stakeholders: learners, researchers, authors, and teachers.

It is worth repeating… we can use your help! If you or your school can join the project by supporting us financially that would be awesome. But beyond that we want YOU to become an active participant in our community in any of the ways I mentioned above.

Thu, 05 Jan 2023 00:00:00 -0600 <![CDATA[An Open Letter to Future Authors]]> An Open Letter to Future Authors

Dear Future Textbook Author Who Knows LaTeX,

LaTeX is a markup language which is designed to help you produce nice PDF. PreTeXt is a markup language which does everything LaTeX can do, and in addition provides multiple output formats, interactivity, and accessibility. If you are writing a textbook, you will find it well worth the effort to learn PreTeXt. And it is better to start with PreTeXt instead of writing in LaTeX and then converting.

A major attraction of PreTeXt is that your book will be available in multiple formats, including highly accessible HTML. Formats are only part of the story: if your book is merely the book you would have written in LaTeX, translated into PreTeXt, the result will fail to achieve its potential. What will be missing are the features of PreTeXt which support teaching and learning.

But first, what is a textbook? In the past, the textbook was the source of all material being taught in a course and also the source of all examples, exercises, etc. These days it is common for a course to call on multiple resources. PreTeXt makes it possible to bring back the time when the textbook contained everything – with PreTeXt the textbook can be the portal to everything needed by the instructor and the student.

And second, what is your vision for your textbook? Some textbooks are a monograph: one person’s articulation of a particular subject – a beautiful and coherent exposition on a topic. A monograph, even if it contains many exercises, is no longer sufficient to be the sole resource for teaching or learning a subject. A modern textbook can be crafted to enhance teaching and learning, supporting good pedagogy and recognizing the realities that face both students and teachers.

If you are writing a monograph, some PreTeXt features will not be of use to you. But you will find that PreTeXt helps you think about the structure of your document, resulting in better exposition. The extra features are available if you later decide to use them.

If you are trying to actively support teaching and learning, then PreTeXt has many features which will help you create a textbook that improves learning and enhances teaching.

For example, some PreTeXt books have preview activities at the start of each section. A preview activity gives the student a few tasks to perform – tasks which they are capable of doing with only their current knowledge and which introduce (without using any terminology) the main concepts of that section. This supports good pedagogy, because

definition example

works fine for research and graduate level exposition, but in most of the undergraduate curriculum

examples terminology

works better.

Other PreTeXt books have reading questions at the end of each section. An ideal reading question is easy for a student to answer if they have understood the material in the section, and difficult if they have not. Reading questions are graded on effort.

Your book can be available in an online system (such as Runestone) which gives the instructor easy access to the students’ answers to preview activities or reading questions. A quick glance at those answers immediately before class will allow the instructor to productively modify their plans for the classroom, allowing more time for those topics where the students need it most.

So: think about how instructors might use your book. Then create a book which will help them be better instructors.

Let’s think about the students. What is the single most significant cause, throughout the history of STEM education, for students having difficulty learning the material? A good case can be made for: the long delay between when they answer a question and when they receive feedback on their answer. Some STEM fields, notably physics and computer science, have recognized this issue and taken effect steps to address it. Numerous types of problems have been created, where students can automatedly receive instant feedback on their work. Misconceptions and common errors are caught quickly, and the result is better learning.

PreTeXt supports many types of interactive problems. Multiple choice and short answer, of course. Also Parsons Problems (multi-step answers where the steps are given, to be put in the correct order), Matching (match Column A with Column B), Clickable Area, and more. Not every type of problem is suitable for instant feedback, but surprisingly many are.

PreTeXt also supports multiple types of interactive demonstrations, such as GeoGebra, Desmos, Sage cells, CalcPlot3d, etc. Students can explore an example as much as they wish, and the demonstration can be reset just by reloading the page.

So: think about how students might use your book. Then craft a book which will help them to learn more: more than they would just by reading clear explanations, seeing illustrative examples, and turning in their work to be graded later.

All of the interactive examples and exercises can be right there in the book: there is no need to cobble together and keep track of a variety of resources. If the book is hosted in Runestone, the instructor has direct access to submitted student work and a dashboard of learning analytics.

Some books have introductory videos in each section and some have videos showing worked examples. These are appreciated by those students who prefer learning by watching and listening. It is better for your book to include quality material that you have selected, instead of having your users wade through the swampy haystack of the Internet.

It takes a while to start thinking holistically about how your book will serve as the interface which connects the material you are passionate about to the students learning the material and the instructors helping the students to learn. The ability to properly support all of that activity, by yourself as an OER author, is a new idea which is now made possible by the combination of PreTeXt and Runestone. But you are not by yourself: there is an active community of PreTeXt and Runestone users and developers, and there is a growing list of textbooks which illustrate what is now possible.

A good textbook might present the material in a clear and coherent manner, but a great textbook actively supports teaching and learning.

PreTeXt and Runestone are free and open source tools which can help you craft a great textbook.


David Farmer
Director of Programs
Mon, 28 Nov 2022 00:00:00 -0600 <![CDATA[Instructor Survey 2022]]> Instructor Survey 2022

Let me start out with a big thank you to everyone who responded to the Instructor Survey over the last few weeks! About 200 of you took the time to answer the questions. The results as I watched them come in were remarkably consistent. I would like to break down the results into four sections: Features you value, Features you might be willing to pay for, Contracts and Privacy, and Thoughts on the future.

Features you Value

The following graph summarizes the responses to the first four questions. Recall that 1 is least important and 5 is most important.


I was a little surprised by how many instructors said that LTI was really not important, nearly 50% of respondents rated the importance of LTI a 1. Assignments and grading, practice, and tracking student progress were rated a 4 or 5 by 60% That reaffirms that these are our core features that you rely on. Although reading through the open text response it is clear that we can make the assignment and grading pages much better. I’ve started a redesign of those in my head, but I would love to hear from you on what exactly the pain points for you are. I heard words like clunky, cumbersome, difficult, and those are somewhat helpful but some detail would sure help.

The thing that made me the most happy was the overall satisfaction score with over 90% of the responses rating us a 4 or 5! There were so many nice comments in the text section that I will read again and again on the days when I am slogging through the code trying to fix some obscure bug!

Features you Might be Willing to Pay for

I know the question was a big vague, and I think think that it led to some unnecessary panic by some of the respondents. What is clear to me is that there is a section of our user base that values Runestone Academy and would like support us. There is also a sizable number that come from school districts that would not have the budget to support us even though you might want to. This latter group is why Runestone exists, we exist to serve you and we want to be here for you. Recall that 1 is least likely to pay and 5 is most likely. Here is the summary graph:


A couple of things people are clearly not interested in paying for: early access to features, dedicated email support, LTI. But about 20% of respondents would be willing/able to pay for a support fee, access to the student progress features, the practice feature, assignments and grading, PDF copies of the books, and even LTI. Because I’m kind of a nerd I made a correlation matrix:


This does show some interesting correlations that may suggest a way forward. First there is a 0.5 correlation between those who value the practice feature and those that would be willing to pay for it. This is also true of LTI at 0.42. Assignments and Grading and Student progress are at around 0.3. I think it is kind of cool that satisfaction is not correlated with anything. One might hypothesize that the more satisfied you are the more you are willing to pay. But it is probably just be that because 90% of the responses were very satisfied that there is not much to correlate with.

Contracts and Privacy

I have written about this a lot in the past. Processing these contracts that schools want me to sign is a major time sync and a terrible use of my time. Nearly all of them require Runestone to promise to reimburse the school for costs associated with a data breach! For a service they are not paying for! Is it reasonable to charge schools a $100 fee to process these? As the following graph shows the overwhelming majority 85% feel it is fair.


I would love to hear the reasoning of why 5% of the responses do not think a $100 fee is reasonable. I suppose you might think that I should trust your school. They are not out to get me, just to protect their students. But that is no way to run a business. More and more I have become convinced that Runestone Academy needs to have a Cyber insurance policy. I am happy to say that after an extensive search and a LOT of back and forth with underwriters we have finally obtained a policy. But, guess what… They don’t give those policies away for free, they charge a substantial premium that adds to the cost of running Runestone.

Thoughts on the Future

I want to start this section by reminding everyone of why Runestone Academy exists. We exist so that we can make high quality, accessible textbooks available to students no matter their economic situation or physical limitation. Too many students do poorly because they cannot afford (and therefore do not buy) the textbooks they need for class. We exist to provide students engaging books that help them learn more effectively than paper books. We exist to help teachers teach more effectively by providing them feedback about their students. We exist to help authors and researchers do their jobs more effectively by providing a system where they can experiment with new pedagogical techniques and new technology for textbooks. You will notice that not once in that list of reasons we exist did I mention the word profit or money. Making money is not why we exist. The materials we provide will always be free to students.

Yet the hard facts of the world are that nothing is for free. We have hosting fees and legal fees and credit card fees and paypal fees in 2021 we had about $28,000 in fees. That is before I even try to pay myself a salary for working on this full time. What happens if I disappear? Is there someone else willing to devote full time to keeping Runestone running for little or no pay? I haven’t met anyone yet, which means we have to get to the point where we can pay someone and keep Runestone sustainable.

There are a few immediate decisions from the results of the survey.

  1. Runestone books will continue to be free to all students. We will not charge the students for access to our books.

  2. For all schools with courses beginning after January 1 2023 that want Runestone to sign a contract we will require a $100 contract fee. If your school cannot afford that fee we have options for you to continue using Runestone without a contract.

  3. I think that there is room for a “freemium model” to work. We will work on defining a set of features that will remain free to everyone, for example assignments, grading, and student progress remain free but features like LTI integration, practice and peer instruction may cost a small fee. I am being transparent about this because donotpanic I don’t want people to panic, I want to open a dialog to get your feedback and I want to reassure everyone that if we do have to charge for some features you will get plenty of notice. I also want to reassure you that we will do everything we can to find grant money so that everyone who wants to use any book on Runestone Academy can continue to do so regardless of the financial position of the school.

Tue, 25 Oct 2022 00:00:00 -0500 <![CDATA[Pathways to enable Open Source Ecosystems]]> Pathways to enable Open Source Ecosystems

Early this year my friend, colleague, and Runestone board member Paul Resnick forwarded me a copy of a new NSF program called POSE, the solicitation said: “The purpose of the program is to harness the power of open-source development for the creation of new technology solutions to problems of national and societal importance.” The solicitation continues: “The POSE program aims to support managing organizations that will facilitate the creation and growth of sustainable high-impact OSEs around already-developed open-source research products.” Cool!

This is not a solution to all of our funding needs as the solicitation further explains: “Importantly, the POSE program is not intended to fund the development of open-source research products, including tools and artifacts. The POSE program is also not intended to fund existing well-resourced open-source communities and ecosystems. Instead, the program aims to fund new managing organizations that catalyze community-driven development and growth of the subject OSEs.”

Well, I thought to myself if that isn’t a program specifically designed to help Runestone and PreTeXt in their joint mission I don’t know what is. On September 9th we learned that indeed we had been funded as a Phase I project! The work begins immediately!

We say PROSE

Our proposal which we called the PreTeXt – Runestone Open Source Ecosystem (PROSE) says the following:

We propose an Open-Source Ecosystem (OSE) which will support such a community. This is timely because software for creating OER is just reaching a mature state, in particular, PreTeXt for authoring, and Runestone for publishing. The governing organization for this OSE, which we will call the PreTeXt-Runestone OSE (PROSE), will be built upon the following principles.

  1. Students and instructors should have no-cost access to high-quality OER that supports STEM learning without the equity issues inherent in for-profit educational publishing.

  2. Authors should have a no-cost mechanism for creating high-quality and accessible OER that is not locked into one specific format or vendor.

  3. Authors should have a no-cost mechanism for disseminating their OER as part of fully- featured courseware that provides rich usage and learning analytics for students and instructors.

  4. Researchers, authors, and instructors should have access to high-quality student data to inform research and development efforts related to OER, and improve student learning.

  5. OER must be accessible to everyone, regardless of background, demographic, or disability.

I like to think of these things in terms of a virtuous circle of improvement. Educational Research makes good Open Educational Resources better, which in turn helps teachers teach more effectively, which improves student learning, which drives even more interesting educational research…


This way of thinking is part of the DNA of both the PreTeXt and Runestone projects since their earliest days. Every time we add a new feature and study how it effects students and teachers we learn something new! New knowledge inspires new ideas for new features. My TODO list has never run low in over a decade of working on the project, and in fact it is longer today than ever. Recently I’ve had to admit to myself that the list is too long. There is too much to do and Runestone Academy has grown so much that it is more than one person can manage. More than that, is the feeling that we are on the cusp of something disruptive, transformative, and sustainable. That is why this grant is so important and timely.

How you can help?

Over the next months we will be hosting a series of online discussions around areas where we need help. Everything from cybersecurity and dev-ops to community building and training. We invite you to participate in these discussions to share your ideas. We also invite you to consider if there is an area of interest where you can actively participate and contribute some time to the project. Both Runestone and PreTeXt have operated on a BDFL model. That is Rob and I act as Benevolent Dictators for Life. In order to grow and ensure that these projects are sustainable we must move beyond the BDFL modus operandi to become community driven. Check out the calendar of events and sign up on our new PROSE Website

We have many questions to answer in the coming months:

  1. What will the PreTeXt / Runestone ecosystem look like in 5 years? 10 years?

  2. How do we transition from our current organization(s) to a community driven model?

  3. Is our goal to be the preeminent publisher of online textbooks for STEM or are we more focused on being a catalyst for change by serving the research community or both?

  4. How do we fund an organization like ours to provide stability for the future? How do we retain the ability to support teachers and students yet pay for support staff and servers?

  5. What organizations can we partner with to facilitate future growth and provide mutual benefits?

  6. How can we recruit new authors? Particularly authors in STEM fields outside of mathematics and computer science?

At the end of all this process we will have the opportunity to submit a Phase II grant to help us achieve the goals we will have set. The POSE program is a wonderful opportunity for a project like ours, but it will only achieve its full potential if we can convince people like you to join us in making it a success.

NSF PROSE is supported by NSF Award #2230153.

Mon, 12 Sep 2022 00:00:00 -0500 <![CDATA[A Practical Introduction to Databases]]> A Practical Introduction to Databases

I am pleased to announce the availability of the new textbook A Practical Introduction to Databases on Runestone Academy (book id: practical_db).  The purpose of this book is to provide a free, open textbook on material of practical interest for students going into computing professions.  The text at present is suitable as the core of an introductory course on databases, and may also find use as a gentle introduction to the topic in more advanced courses on databases, web applications, or software engineering.

In this first release, the book covers three core topics: SQL, data modeling, and relational database theory.


The chapters on SQL (part 1 of the book) take full advantage of the Runestone platform to provide live, interactive examples with a real database.  Students can see the examples in action, test themselves on the challenge questions, and experiment with querying the provided tables.  As the database is an in-memory copy that is replaced whenever the browser page refreshes, students can freely try out data modification and DDL statements.

For students or instructors who wish to use something other than the SQLite engine built into Runestone (perhaps wanting more persistence), scripts are provided to create the book’s database in PostgreSQL, MySQL, Oracle, and SQL Server.  The text also gives appropriate guidance for these systems where they differ from the SQL used in the book’s examples; an appendix provides a SQL reference that similarly supports these systems.

SQL coverage includes: basic SELECT, INSERT, UPDATE, DELETE; joins (both explicit and implicit); data types and DDL (CREATE TABLE); keys & constraints; subqueries; grouping and aggregation; and set operations.  The chapter on joins includes coverage of one-to-one, one-to-many, and many-to-many relationships and their typical implementations (e.g., cross-reference tables).  There is also a very brief explanation of views, CTEs, and window functions.

Data modeling and theory

Part 2 of the book covers ERD (Chen notation) and translation of the ERD into a relational database, with a complete worked example.  Crow’s foot notation and other alternatives are also covered.

The third part of the book covers the relational model, relational algebra, and normalization (through 4NF), and has a short chapter on the differences between the relational model in theory and as implemented in SQL.

Future work

Future releases are planned, covering software programming with databases, NoSQL, and other topics of practical interest, and some expansion of existing topics.  The book is licensed under a Creative Commons (CC BY-SA 4.0) license.  The github repo is public (, please feel free to fork and adapt as per the license terms.  Suggestions, corrections, and pull requests are all welcome!

Christopher Painter-Wakefield, Colorado School of Mines

Tue, 02 Aug 2022 00:00:00 -0500 <![CDATA[WeBWorK Hosting Fees]]> WeBWorK Hosting Fees

In case you missed it, check out the annual WeBWorK report from last December.

Starting Fall 2022, we are raising the fee for WeBWorK hosting for most (but not all) institutions. The goal is to bring a little more support to WeBWorK and Runestone developers who still do the majority of code development as volunteer workers. All of the additional revenue will go to those nonprofit organizations (once the additional taxes are accounted for). For example, I’m not going to paid more starting in the Fall. In fact, we’ve streamlined some things and will be reducing the per course alottment that I am paid.

Who is not affected?

Secondary schools will continue to pay $100 per course. We appreciate that it is generally more difficult for secondary school faculty to get expenses approved.

Also, if your institution has many courses each term/semester, the hosting fees are effectively not going up.

OK, what is the new fee structure?

  • Secondary (and primary) institutions: $100 per course

  • Higher ed: $120 per course. But every 6th course is free. For example, 13 courses would come to $1320.

The 6th course discount for higher ed applies at the time of invoicing. So for example there is no discount for ordering 4 courses in the Fall and then 4 more in the Spring. But you are welcome to preorder courses for a future term/semester, as long as it’s not more than 8 months in advance. Since this hosting service is only promised annually, we can’t accept orders too far into the future.

How long will it be until it could go up again?

We will stick with this through the 22/23 and 23/24 academic years and reassess at that time. Who knows, if there are more subscriptions or public funding, we could even reduce the cost.


Sat, 02 Jul 2022 00:00:00 -0500 <![CDATA[PreTeXt Runestone Workshop]]> PreTeXt Runestone Workshop

Happy July 4th Weekend! I am glad to be home after almost two weeks in Ann Arbor Michigan. But it was a great two weeks. Here are a few of the highlights.


The week of June 20 – 24 was the first joint workshop with the PreTeXt team. We had 28 attendees including authors, developers, educational researchers, and instructors representing the Runestone and PreTeXt projects. One of the main goals for the week was to give everyone a chance to get to know each other and learn more about the two projects.

The PreTeXt Runestone Partnership

After spending an entire global pandemic working on integration Rob Beezer and Brad Miller presented the integration of Runestone and PreTeXt. PreTeXt now has the ability to natively author most of the Runestone Interactive exercises including multiple choice, Parsons problems, Matching, clickable area, short answer, fill in the blank, activecode, and CodeLens! Rob has built a very nice annotated sample book (alternative link without source annotations) that illustrates these questions in action. Runestone Academy can now host books written in PreTeXt and fully supports using questions written in PreTeXt in our analytics and assignments.

Going forward PreTeXt will be the primary authoring language for books hosted on Runestone Academy. We will continue to research and develop new interactive components but we will not have to worry about inventing markup. If you are a current Runestone author I think you will really like the consistency and expressiveness of PreTeXt. It may take a bit to get used to the more verbose XML style of writing, but is well worth learning for future development. There are many advantages to using PreTeXt as an authoring language.

  • PreTeXt books are highly accessible

  • PreTeXt books that are interactive on the web can also be rendered in various static forms including pdf, ePub, Kindle, Braille, and more. We make two types of PDF, one is meant specifically for print/print-on-demand. If you want a bound copy of a book, you can get it!

  • PreTeXt books are highly structured, which will allow for more customization in the future. We spent a lot of time during the workshop talking about various ways we can make the textbooks more customizable.

One big topic of discussion at this workshop is how to grow the PreTeXt Runestone ecosystem. With our combined forces we need to do more in the following areas:

  • Developing champions for each textbook to develop community around each textbook.

  • Developing instructor materials for each book

  • Improving videos

  • Doing more blogging

  • Creating a new home page for Runestone Academy that is more informative about what we are.

  • Being visible at conferences and workshops.

New Titles Coming to Runestone Academy

Starting this summer we are converting many of our current books to use the PreTeXt markup language. The first of these is also the first textbook to be converted to Runestone. How to Think like a Computer Scientist: Interactive edition. We are finalizing the conversion, but I encourage you to take a look at what we have done so far on our development server I am super excited to see this all come together and you will see many other textbooks undergoing the conversion soon. In addition we will have a lot of other new titles available for the Fall.

  • Ordinary Differential Equations

  • Discrete Math: an Open Introduction

  • Active Calculus

  • Apex Calculus

  • Elementary Algebra

  • Intermediate Algebra: Functions and Graphs

  • Trigonometry

I am also pleased to announce that the classic Java Java Java: Object Oriented Problem Solving textbook is now available. Although this textbook was originally authored in LaTeX we have the scripts to convert to PreTeXt. If you have a open source textbook in LaTeX or practically any other format we can convert it to PreTeXt just like this one. Of course the conversion is only part of the work. What we talked a lot about at the workshop is how interactive textbooks that support active learning are structured quite differently than a traditional book. Our research shows that students jump straight to the questions or activities and only come back to read the text when they don’t understand the material. This makes for much more engaged readers.

New Features

One of the features of the workshop was a sharing of a lot of research done by the PreTeXt team as part of the UTMOST project sponsored by NSF. We also got a demo of a new feature of Runestone Academy that allows you to do Peer Instruction in your classroom. This peer instruction feature was the focus of the second workshop the week of June 27.

For Runestone authors it is also exciting to know that Sage and the R language are both supported natively by PreTeXt.

Starting a new book

It is easy to get started writing a book with PreTeXt using the PreTeXt command line tool. This tool allows you to quickly start a book, build books, and see updates to your book as you write. In addition there is also a great extension for Visual Studio Code!

We are working on an editor designed specifically for PreTeXt that will make it super easy for anyone to edit a PreTeXt textbook, more on that another day.

The future

One of the biggest goals for both PreTeXt and Runestone is to ensure that these projects thrive for many years into the future. This means growing the combined team, finding funding and many other things. If you are interested in getting involved in any way we would love to have you join the team. You don’t need to be an expert coder, you don’t even need to know how to code at all. We need willing proof readers, editors, testers of new features, community builders, bloggers, and authors. You don’t need to be a mathematician or a computer scientist either, great open source textbooks are a benefit to students studying any topic!

Sat, 02 Jul 2022 00:00:00 -0500 <![CDATA[Summer 2022]]> Summer 2022

Summer time is often when instructors of all kinds can help out with Runestone Development. Since June marks the beginning of summer I thought I would highlight what is on my list for development list for the next few months. If something sounds interesting to you then please get in touch!


PreTeXt is now the official authoring language for Runestone. There are a host of reasons why this makes sense, and I’ve written about them before. It has taken some time but we have finally arrived in a place where it is a reality. I have converted our first Runestone book ever How to Think Like a Computer Scientist to PreTeXt and Rob Beezer has added markup to support our interactive elements. This is a huge step forward and we now begin a process of converting our current books. Like any conversion process it is not 100% so each book is going to take some manual work to perfect the conversion.

  • select questions

  • toggle questions + A/B testing
    • I think the correct way to do this moving forward is to make toggles tasks and have Runestone show the tasks as dropdown

  • Convert select books to PreTeXt for Fall 2022

  • Build APEX Calculus for Runestone

  • Build ORCCA for Runestone

  • Build Yoshiwara’s Algebra book for Runestone

  • Finish up the SAGE Cell integration

  • Jaas - Jing as a service - Jing is a validator for PreTeXt

  • Make a PR to store traces with a hash of the input.

Peer + Grant Work

This work is funded by an NSF Grant for Barb Ericson at the University of Michigan.

  • Grading - 2 interaction Qs plus chat (maybe) - update autograder

  • Add async peer instruction page

  • Get tags working with question bank searching - what we really want is prereqs and reqs - add to web UI and make use of some taxonomy - someone please give me a taxonomy!

Automatically Create Parsons problems

This work is also funded by an NSF Grant for Barb Ericson!

  • Get Overcode working in python3 - this may also be a good task for Celery

  • Generate parson’s problems from the selected

Github Integration for Custom books

This work is funded by a grant from Elliot Houser and UT Austin.

  • Add simple interface to connect a Repository

  • Add simple interface to publish draft/published

  • Integrate the web hooks to auto build when pushed - this may be a separate server

Runestone Internal Projects

  • Updates to student progress page - replacing d3 stuff with pandas

  • Allow instructors to download LARGE classes on their own - using a celery task with an email followup - maybe create a special role for trusted teachers / researchers to allow them access
    • celery could also be used to do build on demand of Runestone / PreTeXt books

  • Finish redoing the login process

  • prototype an author/researcher dashboard

  • Convert other books

  • Does PreTeXt support hidden pages? Somehow we will need a workaround for how barb has been adding problems to the database but not on a page that is linked to a TOC

  • Create a bibtex file with references and generate a web page of references from that.

Sat, 04 Jun 2022 00:00:00 -0500 <![CDATA[Peer+ Webinar]]> Peer+ Webinar

Today, Barb Ericson and I hosted a webinar on Peer+. Peer+ is an NSF funded research project on using Peer Instruction in a Runestone eBook. Click the link to see a recording of the 1 hour zoom or feel free to just look at the slides

I won’t try to repeat the information here, but just emphasize that Peer Instruction is a well studied way to increase your students engagement and success in your class. Please check it out, and don’t hesitate to give us feedback on this exciting new feature.

Mon, 23 May 2022 00:00:00 -0500 <![CDATA[The New New Academy]]> The New New Academy

This is a big announcement that changes some things very little and some things a lot!

We have incorporated Runestone Academy and applied for 501(c)(3) status. Creating the new corporation is done and official. Eventually, Runestone Interactive LLC will cease to exist as a corporate entity. As of January 1, 2022 the server that you have come to know as is operating under Runestone Academy LTD.

Why the change?

The reason for the change is really quite simple. We want to keep Runestone free for all students. Period, end of story.

However, the reality is a bit different. In an ideal world every school that used Runestone would send us $10 per student. This would allow us to hire support staff and incentivise new authors to add more books to runestone for very little cost to the school and no cost to the students. But that does not happen. I know that many school districts do not have big budgets, and my goal has never been to “get rich from Runestone.” Don’t even get me started about the legal fees we would have to pay to have the “contracts” from school districts reviewed!!!

But We have big goals! I would love to see Runestone helping two million students a day instead of fifty thousand. To do that we need more authors, more books, more servers, more support people, and more programmers! I am greatful to all of the schools and teachers that send us support! Although I feel bad about teachers that send me money from their own pocket! I am from and education family and I know how little teachers at public schools and small colleges make!

So, what to do?

The more I researched this problem the more convinced I became that Runestone had to become a non-proft. This opens the door for us at many corporate and family foundations that care deeply about STEM education. We have several grant applications outstanding, but like everything else, this is a process. New groups have to prove themsevles, and that takes time. In the meantime we will be rolling out new programs that make it possible for individuals or schools to help us financially. More about that soon.

Sat, 22 Jan 2022 00:00:00 -0600 <![CDATA[Academy Update]]> Academy Update

Thanks to everyone for your patience over the last three weeks! As a recap here is what has transpired.

  1. I took Runestone Academy offline on Christmas day and started up my script to migrate the database to a new machine dedicated to running Postgresql. That script took just over 25 hours to complete. It modernizes the database and adds many new constraints to make sure that when we store something in the database it is good and consistent and valid.

  2. On January 27 I brought up two “worker bee” servers and our load balancer. This allows me to add as many “worker bees” as we may need to serve the traffic going to runestone. I changed our DNS entry to point “” at the new load balancer. Unfortunately I did not delete a previous DNS entry so now some people were getting the old address and some the new. Chaos ensued for a few days.

  3. By Monday January 3rd I was patting myself on the back and thinking “this is going better than I thought” LOL – Many problems – to numerous to count ensued as soon as classes actually started again.

  4. LTI - or what I call the “community supported feature” was just supposed to work. But did not. Thankfully I think that as of yesterday it is now working great!

Today was the first day since December 27 that I have not had a new bug report. I know there are still problems out there, but at this point they are outliers not the norm. That does not mean I don’t care, I do, but it does mean that I need a lot of detailed information to try to figure out what is wrong. Major widespread problems are easy to figure out because I hear from hundreds of people. I’m not hearing from 100’s anymore!! So details are important.

If your students are having trouble

  1. Logout - clear browsing history, and log in.

  2. Try incogito mode

  3. Open up the Javascript console on their browser and send me everything! No detail is too small. And Its not your fault! at this point I am trying my best to debug the odd cases. I am even willing to do a zoom with you and your students to debug the problem.

I cannot emphasize how useful it is to do #3! So much of what happens in Runestone is delegated to the browser. If we did not delegate this work to the browser we could never scale to as many students as we have!

Thanks for sticking with us through this transition. Major shifts like this are always painful, and I know we are not through yet. But this new architecture sets us up for a very bright future!

Wed, 19 Jan 2022 00:00:00 -0600 <![CDATA[Academy Downtime]]> Academy Downtime

Runestone Academy is undergoing some major upgrades this holiday season! Unfortunately these changes will result in some downtime. Here is our schedule of events so you know what to expect in the coming weeks.

  1. December 25 - 26 Runestone will be offline as we move our database server to its own machine. In order to ensure that we don’t lose any data we need to take all of the books offline while we move the database. This is also a great time for us to do some maintainance on our data in preparation for our new servers.

  2. December 26-27 Bring back online. In this initial period the old servers will come back up but use the database in its own location. There may be some unexpected crashes during this period as we have added a number of protections to the database to keep future data problems.

  3. December 27–January 1 Migrate away from the old server and onto the new servers. If all goes well this is mostly just some DNS changes, but it will still likely cause the appearance of some downtime as you may not reach the new server and / or the old server may look like its certificate is expired.

  4. January 3 and onward Back to business as usual. Although there are likely to be some hiccups as traffic picks up with students and teachers getting back to school.

A Few Details

For those that are interested, this change is pretty big. Here is what Runestone has looked like in the past:

../../../_images/oldserver.jpg runs the web2py application server as well as the database server (postgresql). This server has been in service since 2017. I haven’t had to reboot it for 695 days as of this writing! It is definitely showing its age, and can definitely not keep up with all of the traffic. Our stopgap measure to alleviate the pressure on this machine was to add and route all traffic for the csawesome book to that server. It still used the database on academy. And it used most of the cpu power of the five jobe servers that compile over 100,000 Java programs each day. If it looks like this is a “Rube Goldberg” contraption its because it is.


For years we have wanted to move to a new architecture. And this change is our first step in that direction. This new architecture is all based around Docker containers and gives us the ability to add more capacity in just minutes. It will also let us scale back during the less busy summer months, thus saving us money as well.

All traffic will now go to It will be routed to one of N docker containers that run the old server which is now just for the instructor dashboard. Page requests for books will be handled by our brand new bookserver. This new server is built using a very modern web framework and will have a ton of benefits for developing new features going forward. Each container also houses a JOBE server for handling java and other compiled languages not supported in the browser.


Any big change like this is bound to come with some setbacks. I’m sure there are things around the edges and in the corners of our new server that will break over the first couple of months. Please be patient, and please please report problems on our github issues page Don’t assume that someone else will report a problem or that it is your own fault.

It will also take us some time to tweak the performance of this new setup. We have had almost five years to squeeze as much performance out of the old system as we could. It will take some time to get the most out of this new one as well. So again please be patient and let us know of any particular bottlenecks.

Please Support Runestone

As long as I have your attention, and we are at that end of the year time when we think about supporting the organizations that matter to us, please consider a small donation to keep runestone solvent. You can donate directly on PayPal Thanks!

Enjoy the holidays!


Tue, 21 Dec 2021 00:00:00 -0600 <![CDATA[Annual WeBWorK report]]> Annual WeBWorK report

As we near the end of the year, it feels like a good time to report on Runestone’s foray into WeBWorK hosting. In Fall 2020, Runestone began this service. Three factors contributed to this:

  • Runestone had just set up its own WeBWorK server to support PreTeXt books that use WeBWorK.

  • The MAA hosting service was coming to a close, and many users would soon have no hosting service to turn to.

  • The COVID pandemic was on the upswing, and institutions everywhere were looking for new (to them) online tools to support remote learning.

So it seemed like a useful idea for Runestone to offer a WeBWorK hosting service. For that first Fall 2020 semester, hosting was free while we assessed the cloud computing and customer service demands. Since January 2021, hosting incurs a $100 fee per course.

Facts and Figures

Over the terms and semesters of 2021, the service has supported:

  • 62 institutions in 8 countries (55 higher ed and 7 secondary)

  • 117 individual sections

Our revenue from this service was $10,800. For transparency, here is where this money has gone. These numbers are preliminary, as the year is not yet over. But I expect they won’t differ by a significant amount. - $5400 to Alex for work hours - $1493 operational expenses - $1076 Runestone paying taxes - $1365 transferred to The WeBWorK Project - $1365 retained by Runestone

The last two items are particularly important. These amounts may be small, but they contribute to a future of sustainable support for WeBWorK and Runestone. Each project is an open source software development, and relies on “tens of thousands of dollars” in annual development. That is in quotes because for the most part, the work is done by unpaid volunteers. That’s not a sustainable practice, and we lose good contributors to burnout. Real funding will help the developers feel appreciated, and in some cases buy out a chunk of their day job duties (say, one course release) and provide real time for their efforts.

Moving Forward

My goal is to grow each of those last two numbers up to $4000 for next year’s report. I think this can be accomplished considering:

  1. Growth in the number of subscriptions. Already from Spring 2021 to Fall 2021 there was a lot of growth. So even if there is no more growth, 2022 will have more revenue than 2021. But it is reasonable to expect continued modest growth.

  2. Raise the average cost per course to around $120. The additional money would go towards WeBWorK and Runestone, not labor costs or operational costs. My hope is that with transparency like this report, a price increase like this is not ill-received by subscribers. The increase is to support the software itself.

  3. Offer and highlight other avenues for contributing. For example, a higher tier course subscription option.

There are other factors to consider for a new pricing scheme.

  • Should secondary schools pay the same as higher ed institutions? It is typically more difficult for a secondary school instructor to have an expense like this approved. In general, the budget is tighter and there is more red tape.

  • Some institutions have a dozen or more courses, and some have only one. Should there be some sort of departmental discount?

  • Some courses have a dozen students doing 5 exercises once per week. Some courses have several hundred students doing 30 exercises twice per week. Should a pricing scheme address that discrepancy? Certainly the high volume courses are the ones responsible when we need to upgrade our cloud computing resources.

The Runestone Team will study the options soon and publish a new pricing scheme that will come into effect Fall 2022. (There will be no change in the $100 fee per course through summer 2022.)

Other Significance

In the course of managing a WeBWorK server across multiple institutions, a few WeBWorK bugs were naturally exposed and have been addressed. Typically, something to do with a course’s time zone differing from the server’s time zone. Also, with so many users having one clear contact to report a perceived bug, this has been a good feedback collection device. And then simply to support multiple institutions using the same server, a number of new features were added. Overall I credit around two dozen improvements to WeBWorK 2.16 and (upcoming) 2.17 to this Runestone hosting project.

Thank you to all of our subscribers. You are supporting the continued improvement of two significant open-source apps for higher ed. May these projects continue to grow and serve you better.

Alex Jordan

Mon, 13 Dec 2021 00:00:00 -0600 <![CDATA[On Data Privacy Agreements]]> On Data Privacy Agreements

An Open Letter

Rant Alert

Before I get into the heart of this I will apologize in advance if this sounds a little like a rant. I started this post on a day when I got three different reqeusts to sign contracts! I am trying to clarify my position, open a dialog, and provide you some alternatives for using Runestone without needing a long contract.

Dear School Districts,

I run Runestone Academy as a “labor of love.” For me it is a great way of paying back all those who have helped me be successful over the years. The mission of Runestone is to democratize textbooks for the 21st century. One implication of that mission is that everyone who wants to learn computer science should have access to free, high quality materials.

I’m not trying to turn Runestone into a major corporation or even make a profit. If I can run it as a small business that keeps me from becoming a greeter at Wal-Mart that is even better! I’m perfectly transparent about the finances, please take the time to read about that in our Annual Report It also does a nice job of clarifying our mission.

Districts, you are driving me crazy with your student data privacy agreements! Do you even understand what you are asking for? I understand that you must follow laws that have been thrust upon you by your well meaning state legislators, I understand that student data privacy is a good thing. I understand that you don’t want some corporation making money by selling your students PII for fun and profit. I agree! Read my Privacy Policy and our Terms of Service We are on the same side.

But here is the problem:

  1. As a free and open source project we don’t have lawyers, nor do we have the budget to pay for them as needed. I am definitely not a lawyer, and frankly Its very risky for me to spend my time pretending to be one to read your agreements.

  2. As a free and open source project I work on this because I love to write educational materials that help students learn and I love to code systems like Runestone that help you and your students teach and learn more effectively. This does not include spending my time filling out attachements A, B, C and H of yet another data privacy agreement.

  3. As a free and open source project we don’t have any money to promise to reimburse you for anything if something goes wrong.

  4. Approximately every district in every state has a different agreement!

As a professor of 15 years I am very aware of FERPA and all that it entails. But seriously here is what I am asked to agree to on a regular basis. This is taken directly from contract sent to me by a school district like yours.

The Contractor warrants that the service it will provide to District is fully compliant with the following regulations:

  • Children’s Online Privacy and Protection Act (COPPA)

  • Family Educational Rights and Privacy Act (FERPA)

  • Health Insurance Portability and Accountability Act (HIPAA)

  • Payment Card Industry Data Security Standards (PCI-DSS)

  • Student Data Privacy Pledge

And any changes to the preceding that may occur from time to time.

The vendor hosting data center meets the following standards (proof must be provided if PII is stored at rest in the vendor’s data center):

  • ISO27001:2005

  • SOC2

  • SSAE 18

Sometimes I swear you and your lawyers just Google everything related to privacy and add it to a laundry list for us to figure out. Honestly what does HIPPA have to do with computer science? I asked a lawyer friend about HIPPA compliance and he laughed and said that only a handful of people in the country even know what that means.

As a non-lawyer computer scientist I have learned some heuristics that help me weed through these agreements in record time. All I need to do is search for the words “indemnify” and “reimburse” as in:

  • If requested by the District, Provider shall reimburse District for costs incurred to notify parents/families of a breach not originating from District’s use of the Service.

  • The Provider shall indemnify and hold harmless the District from and against any loss, claim, cost (including attorneys’ fees) or damage of any nature arising from or in connection with the breach by the Provider or any of its officers, directors, employees, agents or representatives of the obligations of the Provider’s or its Authorized Representatives under this provision or under a Confidentiality Agreement, as the case may be.

Really!? If Equifax, major banks, and pipeline companies cannot always prevent hacks! How are we supposed to be any better? We do our best, try to follow good practices and take all reasonable precautions, but you now want me to promise to pay you back for a service that you are using for FREE. If I find a clause like that I will ask you to remove it. I simply cannot enter into a contract that obligates Runestone to pay you back.

Trying to guilt me into signing something will not work. “If you don’t sign this then our students will not be able to use your materials.” Yes I want your students to use the materials, its why I do this, but I don’t make anything from your students, Its not like a for profit corporation where if I don’t sign you don’t pay me. In fact every district that uses Runestone just adds to the costs of running Runestone. More students means more servers means more money we have to pay to keep running the service. I am not going to put the project at risk just to make your lawyers happy.

Every minute I spend trying to deal with your red tape is a minute a don’t get to spend making Runestone better or writing new materials to help your students learn.


First, everything on Runestone is open source and free for you to use however you see fit. This includes every book and the Runestone server that supports it! This allows for several solutions.

  1. You can run your own Runestone Server and serve any of the books you want on your very own server. This totally eliminates the need for you to use Runestone Academy and keeps you 100% in control of your students data. This was actually the model I envisioned for Runestone in the beginning, but I quickly learned that many people do not have the time or the skill to maintain their own server, and so to help those people Runestone Academy was born. No good deed goes unpunished as they say.

  2. You and your students can read and use any and all of the books on Runestone Academy without creating an account. Ok, yes this makes life a lot harder for teachers to gather their students homework and grade it, but it is 100% an option. I am not trying to force anyone to create an account so I can gather PII to sell!

  3. You could agree to send Runestone some money so that I can have your agreement properly reviewed by a lawyer at around $350 / hour. Lets call it $1200. In that case I would be happy to agree to refund that entire $1200 as reimbursement for a data breach. I’m happy to put that in writing and send you an invoice.

  4. You, as a teacher, can register all of your students anonymously with NO PII. See this post for instructions on how to do this. I would be happy if everyone who sent me contract would just do this instead. I have no interest in knowing anything about your students, the only reason I collect names and email addresses is to deal with password resets and to make life easier for instructors when they are grading assignments. This solution makes it a bit less convenient, but does so without any PII.

In the back of my head I have an addition to Runestone that would allow everyone to use option 4, but allow instructors to keep a mapping in their browser (NOT on Runestone) that would let them see who the anonymous students really are. That is just a gleam in my eye at this point, no code has been written, so please don’t ask when it will be ready.

We Are in this Together

I firmly believe that our goals are aligned! I want to provide great free resources to help students learn, and you as school districts with limited budgets want to use great free resources to help your students learn. I am not the enemy! I have signed the student data privacy pledge, I have published a privacy policy. I sincerely care about student privacy!

But, I believe these laws put projects like Runestone at a real disadvantage. We do not have the resources to hire lawyers to review and negotiate all of these contracts. We are not big enough to make you sign our own. So we need to work together and treat each other with respect and maybe engage in a little give an take. It is a win win! If you don’t use Runestone are you going to pay a big publisher $100 per student every semester?

If you have other suggestions for how we can make this work please feel free to reach out to me, I am open to hearing ideas and working on alternatives that do not create more legal hassle for any of us!

Thanks for reading!


Brad Miller

Professor Emeritus, Luther College

Founder, Runestone Interactive

Fri, 28 May 2021 00:00:00 -0500 <![CDATA[Runestone Roadmap 2021]]> Runestone Roadmap 2021

There is a lot going on in the Runestone world right now and I wanted to let you in on the current roadmap for Runestone development. I’m doing it now as summer is on the horizon and I know for many of you summer is the one time of the year that you may be able to help with development! More on that later. Here is what is on the Horizon for Runestone.

Roadmap Overview


Although the web2py framework has served us well since 2011, I have wanted for a long time to rewrite the core book server in Runestone using modern web frameworks and technologies. This will allow us to scale Runestone more elegantly and save us money as well! The Instructor interface, which gets 10% of the traffic that the book server does will stay on web2py until I or someone else finds the time to port it as well. They will continue to work together just fine as they will share a common database. The BookServer project is using a fully async framework called FastAPI as well as SQLAlchemy for the database work. It will be tightly integrated with the Runestone Components so that you can easily spin up a server to serve a Runestone book on your own infrastructure, With access to all your data in a sqlite database. Production on Runestone Academy or another large installation will still use Postgresql. This is the highest priority for me at the moment as I don’t want to do any further student facing development on the old server.

Funded Projects

There are several high priority projects that are being funded by NSF money through colleagues like Barbara Ericson at the University of Michigan, or the SPLICE project. Others may be on the way…

Support for Peer Instruction

The peer instruction project is part of Barbara Ericson’s research at the University of Michigan and has a two year timeline.

1. Allow Runestone to ingest question banks from various projects so that we increase the number and quality of questions available for instructors to assign.

2. Integrate a discussion system into Runestone and the assignment system to allow students to first answer questions and then discuss them with each other synchronously or asynchronously before they answer the question again. This is the heart of the Peer instruction model.

3. Analytics for selecting questions suitable for peer instruction.

  1. An instructor interface to allow an instructor to see and share aggregate information about how students are answering a particular question.

  2. Support for automatically pairing students for peer instruction.

Publishing Anonymous Runestone Data

This is an ongoing project funded by the SPLICE project. If you are interested in analytics or machine learning and how we might improve the learning experience by applying machine learning to Runestone books I have made available several datasets using the DataShop system at Carnegie Mellon University. Contact me for access to the data.

Support for Large Online Courses

This is a project that is adding support for courses on Coursera using Runestone as the underlying textbook and backend. And includes a number of miscellaneous tasks.

  • Supporting the new toggle question. Given a particular coding problem allow a student to answer a series of hint problems (maybe even a parsons problem) to guide students that are struggling to a solution. Conversely many students would rather write code than drag parsons blocks around so in that case allow the student to simply write the code.

  • Improve the performance of the autograder and the student progress dashboard for large classes.

  • Modify the assignment page interface to allow for a group of students to work together on an assignment.

Smaller Projects

Change the Runestone Onboarding (Registration) Process

Separate registration from adding a course. The flow should look more like the following:

  1. Register for Runestone Academy.

  2. Are you an instructor?

    1. Provide a bit more information so that we can verify that you are.

    2. Here is an overview of the books we offer. Select one to create your course.

    3. Guide you through the course creation process and give you some resources to make the most of Runestone.

  3. Are you a student?

    1. If you are an independent learner here are some open courses you can join.

    2. If you are part of a course here is a nice interface to find the course by name and register for it.

Support for Custom Textbooks

This is a feature than lots of people ask for, but I’ve never found a scalable way to implement it… Until one day as I was registering Runestone on “Read the Docs” it occurred to me that the model would work perfectly well for Runestone!

This development effort would add webhooks for git and Runestone to allow an instructor to Fork any of the Runestone textbooks (or create their own) and link it to Runestone.Academy for hosting. When changes are pushed to the forked textbook Runestone will automatically rebuild and publish the book as a draft. The instructor can then choose when to upgrade the draft to published using the instructor interface.

Even for instructors that do not want to get deeply into restructuredText or python programming this would allow a book to be customized using github’s web based editor. With rebuilds run automatically it would allow authoring a book with no setup.

A somewhat related part of this project is to update our library page to make it database driven. Right now we simply walk the directory of books and rely on the presence of a file that we import. This is error prone and does not allow us to distinguish books that should remain private versus books that are available to create classes from, etc. This part of the project would make the library page as well as the course creation page database driven.

Real time UPDATE I just found out this project will be funded through a grant from Elliott Hauser at University of Texas at Austin !! Elliott is the founder of Trinket and a long time collaborator on the Skulpt Project used to power Python in the browser.

Editorial Interface for Editing and Updating Questions

As Runestone has grown and as more instructors gain experience and confidence in using Runestone the size of our question bank has grown. However my ability to manage all of these questions is far less than what is needed. This project would build out an interface for volunteer editors to clean up and remove bad questions as well as make slight editorial updates to typos or unclear instructions.

Tag all Questions within a Taxonomy of Competencies

With the addition of the exam generator to Runestone we have a real need to create a way of tagging questions with the various competencies that they exercise or test for. This could be related to the editorial interface, but another great suggestion is to crowd source this and have students participate. This project would enhance the question interface to allow students to tag/categorize key concepts for each question. Ultimately editors may have to make the final decision, but it would be a rich dataset to see what students think.

Enhance the Web Interface to allow Instructors to Publish Projects or Labs

Although it is already possible to create coding exercises a number of books are more project oriented. It would be great to allow instructors to publish units of work that are larger than a single exercise but smaller than an entire book. More like a project size unit that may consist of text plus a number of exercises.

Convert Runestone output to XML (PreTeXt)

My work with the PreTeXt project has brought several math books to Runestone. PreTeXt is an authoring system that is primarily used for math books but also some others. It has the advantage of allowing you to write once and choose several output formats. Including PDFs, this is something that I get asked about all the time. My usual answer is why would you want a pdf of an interactive book? That just eliminates the interactivity! But the truth is there are lots of situations where people want or need a pdf version in addition to the interactive version. This is work that I’ll be at least starting on at a workshop in late June hosted by the PreText Project.

Improve the Navigation of Runestone Books

  • Make the main table of contents have collapsible headings

  • provide an omnipresent outline in a sidebar to allow for faster navigation to different parts of the book.

Upgrade LTI to Version 1.3

Currently Runestone supports LTI 1.1, the future is 1.3. Anyone who would like to take ownership of Runestone’s LTI integration is more than welcome to do so. Since I do not use an LMS as part of my daily work anymore I have little motivation to work on this. More work is also needed to verify and document how to use LTI with other LMS systems (besides Canvas and Moodle)

Template Questions

We have a prototype of a fill in the blank questions that would allow you to write a question where every student would get the question but it would be slightly different. For example you might want to write a question like What is X + Y? where each student would get a different X and Y chosen from a range of integers. The question would be smart enough to use a computed answer based on the X and Y given to each student. We would also like to expand this concept to multiple choice questions. Not only is this great for exams and keeping students from cheating but it also gives the students more practice as they could see a different version of the same problem each time they refresh the page.

Blockly on Runestone

Thanks to the authors of the amazing Mobile CSP curriculum and book (coming to Runestone for Fall 2021) and the hard work of Ralph Morelli we will have a great addition to the runestone library, and the ability to use blockly in your textbooks. I think this is awesome for beginners and I’ve always wanted to update The How to Think Like a Computer Scientist (HTTLACS) book to illustrate each new programming construct in Blockly first. I think this is a really cool way to give students a visual understanding of loops and conditionals before they see the syntax.

As is usually the case with all things Runestone we start small, and then iterate, so if you are into blockly and would like to help evolve this new component (which goes by quizly in the source) we would love to have your help.

Fixing Bugs and Even Smaller Projects

Many small coding projects are available on Runestone and are identified as issues. These issues are all typically tagged with the “Good First Issue” and or “Help Wanted” tags. I try to identify these projects this way because they are good starter projects that can get you into the Runestone system. See

Video Production Anyone?

The YouTube videos that accompany the HTTLACS book are very popular on YouTube, but I consistently get complaints about their resolution. Unfortunately the original recordings are 8 years old at least, and have been lost through many migrations of computers. They also feature version 0.1 of Runestone so that pages don’t match at all anymore. My new years resolution for 2021 was to work on one of these a week. So far we are at week 17 of 2021 and I have redone exactly 0 videos. If you would like to try your hand at re-recording any of these I would love it! You can just take the transcript of an existing video and work your way through it.

How You Can Help

Runestone is an Open Source project. Everything on Runestone is freely available as part of our Github Organization. Summer may be a good time for you to help hack on something, test something, or write something! I would be happy to have help on any and all projects. Funded or not it does not matter to me. I only point out the funded projects to show how awesome it is that some big partners have put their faith and money behind Runestone to keep it sustainable.

You do not have to be an expert coder to help out. We welcome people of all abilities to our community and would be grateful for your help.

Another way you can provide coding help to Runestone is to consider us as an option for a student capstone project. We have had great success in working with students at Berea college and other colleges that have adopted Runestone for their capstone course. Many of the projects identified above would be great as capstone projects for a group of students. The students really love, especially if they have benefitted from using a Runestone book as part of their education. This is a pathway for them to give back to a project that has helped them.

Another way you can help is to talk to encourage your department to budget some money to help support Runestone. I’ve heard from a number of instructors this spring asking about this, and I’m very grateful. We are happy to send an invoice for whatever amount you decide. Other departments have decided to charge a lab fee of $10.00 for the course, and have designated part or all of the per student lab fee to go to Runestone. We are happy to work with you on billing and payment!

Wed, 28 Apr 2021 00:00:00 -0500 <![CDATA[Annual Report 2020]]> Annual Report 2020

Happy 2021! Quite a few people have asked through various channels the last few weeks about how Runestone is doing. So it made sense to answer that question in the form of an annual report. The mission of Runestone Interactive LLC is to “Democratize textbooks for the 21st century”. There are two important components to that mission:

  1. Democratizing Textbooks — what that means is that ALL students should have equal access — for free — to high quality books. Expanding outward a bit I think the ideals of democracy also encompass our committment to student privacy. If it was possible to run Runestone with zero student personally identifiable information, I would do it.

  2. What are textbooks for the 21st Century? This was really the genesis of Runestone, that textbooks, especially computer science textbooks ought to encourage active learning and therefore BE interactive. I’m really proud that not only are Runestone’s textbooks highly interactive, but many of our features have PhD quality research behind them. I can trace six different contributors to Runestone who have used Runestone or developed features for Runestone as part of their PhD Work.

    I think there is more to come here as we learn how to apply machine learning to the data we have been collecting over the years. Imagine if the textbook could intervene and give the student intelligent hints? Or what if the textbook could “see” the early warning signs of a student about to lose their way in the course? There is much to aspire to in creating textbooks for the 21st century!

In the rest of this report I want to give you an overview of a few of our accomplishments during 2020, particularly in the areas of growth and financial viability. I’ll try to restrain myself and not dive into all the Python and Javascript code I’ve written this year!


2020 was an exciting and good year for Runestone along many fronts.

  1. We experienced huge user growth, partly due to the pandemic, and partly to new books with great support and large reader bases.

  2. We hosted nine new (to Runestone) textbooks.

  3. We hosted five Math textbooks, our first outside of CS, thanks to a partnership with the PreTeXt project.

  4. We made strides toward making Runestone sustainable, experimenting with ad serving and ramping up our efforts in getting support from institutions and individuals.

  5. We hosted over 200 instructors in our first virtual workshop, held in June.

  6. Several colleges have used Runestone as an integral part of their capstone project experience for students. This is an amazing way for students who have learned from the Runestone textbooks to give back — by helping with development.

  7. We put in place legal Privacy Policy, Terms of Service, and a formal information security policy.

  8. The University of Michigan funded a major development effort to create an exam builder. This is still in Beta but you’ll hear more about this in 2021.

Key Metrics

Registered Users

The graph below shows the cumulative number of registered users on Runestone Academy starting in January of 2019. You can see the sharp uptick in the number from about mid august of 2020 to late september of 2020. The number of users is very much tied to the rhythms of the school year as we see big numbers at the typical starts of each term. However as we serve students around the world this is somewhat moderated by the fact that in the southern hemisphere the cycles of the school year are the opposite of the northern. Much of the growth in fall registrations is due to the CSAwesome curriculum and the many high schools that have adopted that for their AP CSA courses.


graph of user growth over the last two years

You can see the cyclic nature more clearly in the following graph that illustrates the number of new courses created each week. Particularly interesting in this graph is the spike of new courses created in March 2020, around the time of widespread stay at home orders in the USA.



Graph of the number of new courses starting each week

Courses for each Book

The majority of the Runestone traffic comes from a few of our books. The CSAwesome book has been a huge hit, taking the number one spot from the How to Think Like a Computer Scientist (thinkcspy) book in 2020. The Foundations of Python Programming book continues to show good adoptions as an alternative to thinkcspy in the Python area. The Webfundamentals book, which has continue to evolve since I wrote it for my own course about five years ago continues to do well. Finally the How to Think Like a Data Scientist book, written with the support of Google, is seeing some traction outside of the original group of 20 schools that were guinea pigs for that book.

One of the challenges is that we have great materials on Runestone Academy but have always relied on word of mouth as our marketing and advertising strategy. I think many of our books are poised for big growth if we can figure out effective ways to spread the word. One of the reasons that CSAwesome is so popular is because of the training and professional development opportunities provided by the CSAwesome team that brings in hundreds or thousands of teachers each summer and introduces them to the materials.


Graph of all courses created in 2020 by the book used


Graph of the number of students using each book in 2020

Page Views

The number of registered users and formal courses tell only about half of the story for Runestone. Roughly 50% of the traffic we experience is from search engine traffic. Daily we see an over 300,000 page views on, or over 2 million page views a week. According to Alexa Internet, that puts in the top 10,000 websites in the United States. And the top 1% of all websites in ithe world! Hard to believe.

As you can see, the weekends are dramatically lower traffic as this graph over the last two weeks confirms what we have all long suspected about our students work habits. It is also why I update all the books and push bug fixes on saturday mornings.


Total pages views each day for the last two weeks (partial day for Jan 27).


In May of 2018 I took an early retirement package from Luther College to work full time on Runestone. This was a big leap of faith, but softened by the package offered by Luther. After making that leap I was very fortunate to have the opportunity to get a contract with Google’s Engineering Education group that allowed me to work on the Runestone platform and develop content for one of their initiatives. That contract (originally for five months) stretched until June of 2020. So revenue was not really a concern until June of 2020. Since then we have taken some steps to meet the goals of making Runestone sustainable, and of allowing me to continue to devote full time to working and supporting all of the students, teachers and authors using this platform.


On the expense side we had a total of $18,000 in expenses. We can break things down into the following categories:

  • Hosting: $5,000

  • Health Insurance: $6,000

  • Financial service charges: $750

  • Legal Fees: $450

  • Other Business Expenses: $5800

Hosting charges have more than doubled for 2020 due to the increased traffic to the books as well as the increased use of languages that require service side support — Java and C++. The legal Fees were largely for reviewing our privacy policy and terms of service, but increasingly school districts are asking for us to sign legal contracts in order for their teachers to use Runestone. Many times those agreements run 20+ pages long and are hard to read for a non-lawyer. Most of them are in my view quite unfair, and expect Runestone to promise to reimburse the school district for any expenses they incur in the case of a data breach. All for a service they are receiving for free!


Given that we really started to focus on Revenue in mid-summer I’m very pleased with our progress on this front. We had a total of $65,000 in revenue. Our major sources of revenue break down as follows:

  • Support from individuals through Paypal, Venmo, Patreon: $10,200

  • Support from Colleges and School Districts: $20,000

  • Grant and Consulting Income: $31,000

  • Advertising Income: $3,500

I’m really pleased with how this is working out. The difference between Revenue and Expenses leaves about $47,000. As I easily put in 40+ hours a week coding, maintainnig, and supporting instructors and students Runestone, that does not work out to a great hourly salary. It is less than half of what I would like it to be, but it is a great start! It is also a labor of love. The support from researchers at Elms college, and the University of Michigan has been wonderful. The $100-$300 level of support from many high schools and college departments obviously adds up and I hope that will continue to grow. It is pretty easy to project that our Advertising income will grow quite a lot in the next year as the number for 2020 only includes the fall quarter. As a reminder, Advertisments are ONLY shown to users who are not logged in, which means that the vast majority of ads are shown to people who arrive on Runestone through some kind of search.

If you are an instructor and can encourage your students to support Runestone or can persuade your department to provide some support that would be awesome! If you are a educational researcher and would like to include Runestone in a grant, lets talk!

Many people have suggested that I simply charge every student $10 or $20. However, as I mentioned in a blog post earlier this year one of the “True North” guiding principles for Runestone is to keep textbooks free for students. It makes it harder for us to stay viable but I believe with your help we can do it.

Challenges and Goals for 2021

The challenges for 2021 are many and varied

  1. I am currently in the midst of rewriting the core Runestone Server. We cannot overcome our scale issues by simply spending more on hosting fees. Runestone is built on a foundation that is over 10 years old. Moving to a modern web framework requires a lot of work, but will pay off.

  2. Growing overhead for Privacy and other legal agreements. I am 100% in favor of student privacy, and as I’ve repeated many many times, I will never sell student data, I will never give away any data that risks exposing any student’s identity. But there has to be a solution to dealing with schools that want me to sign onorous legal agreements for a free service.

  3. Growing the community. If there is one thing I have learned in the last two years it is the importance of building community around the Runestone books. The huge growth for the CSAwesome books is partly due to great content and partly due to the amazing community that Beryl Hoffman and Barbara Ericson have built around that book. If you have some time and energy to devote to creating a community of practice around any other book I would be happy to have your help.

  4. Continue to grow revenue. I can only dream of the day when our revenue brings in enough money to hire a second full time developer or a support person. If you have an interest in Search Engine Optimization, and would like to help us out in an ethical way I would love to hear from you. One additional project that I’m committed to is to remove the links to twitter, Facebook and other social media platforms. Although, in the beginning I thought this was a great way to increase the visibility of Runestone through social media, it seems that now it is mostly a benefit to their advertising platforms. Time fo those links to go.

  5. I’m happy to say that the National Science Foundation has awarded an IUSE grant to Barbara Ericson for Peer+, a system to support both in-person and remote Peer Instruction in Runestone ebooks. In Peer Instruction students vote for an answer to a hard multiple-choice question individually, then discuss it with a peer, and then vote again. The instructor then displays the results of both votes and discusses the answers and misconceptions. We will mine historical data from the current ebooks to identify good questions for Peer Instruction and also add questions from public question banks. We will test the effect of Peer+ on learning, student satisfaction, and student retention.

  6. I would like to develop a formal advisory board for Runestone along with a succession plan in the event that I am eaten by a bear on one of our camping trips, or run over by a car on one of my many bike outings! We already have an informal advisory group consisting of a group of Authors that meet on zoom every couple of weeks. Namely Bryan Jones of Mississippi State, Beryl Hoffman of Elms College, Barbara Ericson of University of Michigan, Jan Pearce of Berea College, and James Juett University of Michigan. Others pop in from time to time but I am very grateful for this group that provides regular feedback and wisdom! In addition if you are an operational person and would have an interest in helping Runestone by being the go to person for keeping the servers up and Running or have an interest in becoming a Docker/Kubernetes expert I would be only too happy to give up the responsibility for care and feeding of our servers.

As we begin 2021 I am feeling very hopeful. Although most days I feel like I’m a juggler juggling one too many balls, I think Runestone Interactive is poised to live up to our big goals! That cannot happen without your help. If you like to code, there are lots of easy starter tasks to take on to get you started. If you like to interact with other instructors, how about adopting a textbook and helping to build community for that book? If you like to write or make videos, how about helping us with our documentation? If you have some influence or some spare change laying around your department, how about supporting in future of Runestone?

We have a great mission — Democratizing textbooks for the 21st century! Help us achieve it.

Wed, 27 Jan 2021 00:00:00 -0600 <![CDATA[Runestone Academy gets Math]]> Runestone Academy gets Math

Two important announcements this week!

  • Math books on Runestone academy!

  • Runestone Academy will host WebWork courses

Read on for the details…

It all began last December at the American Institute for Mathematics Workshop on OER assessments. Its a partnership that makes a lot of sense as our goals and values are so clearly aligned around making great textbooks available to students for free. Whereas the PreTeXt project has put a lot of focus on the write once and publish in many different mediums paradigm, Runestone has put more effort into developing an html only interactive textbook with an integrated LMS offered as a service. PreTeXt books can be published as HTML, EPUB, PDF and other formats including braille! Now PreTeXt books can also be published for Runestone.

I’m pleased to announce that we will be hosting several textbooks written using PreTeXt. beginning today you can use the following books:

Over the last 9 months I’ve been working with Rob Beezer and the team to integrate PreTeXt books into Runestone. We have things like activecode, short answer, and videos working seamlessly with Runestone. This means that Math teachers can use one of these books on Runestone and get all the benefits of grading, and following student progress. One of our key challenges was getting Runestone to work with WebWork problems. WebWork is widely used in the math world and has an enormous catalog of homework problems.

In order to make the integration work well, we decided that Runestone should host its own WebWork server. This allows us the flexibility to run a custom server that supports the needs of Runestone on our schedule. The load on the server is fairly light for Runestone’s purposes so Alex proposed that we use that spare capacity to serve the community. Alex Jordan is a WeBWorK developer partnering with Runestone Academy for this service, and will be managing the WebWork server and providing faculty with some level of support. Alex has been invaluable in getting the integration complete.

Runestone Academy is hosting free WeBWorK courses for Fall 2020. If you would like to use this service (or if you have questions), Check out our Webwork Page

  • Active Prelude to Calculus

  • Active Calculus

  • Active Calculus Multivariable

  • ORCCA (basic and intermediate algebra)

This is a great addition to Runestone. We can learn a lot from the WebWork community as they have done an excellent job of evolving their question authoring system over the years.

Why are we doing this? First, we recognize that in the era of remote teaching during a pandemic, some faculty are still searching for an affordable online homework platform. Second, if this semester’s WeBWorK trial is successful, we plan to offer this as an inexpensive but fee-based add-on service to Runestone Academy. We would charge between $50 and $100 per course (not per student). This is part of a multifaceted plan to make Runestone Academy sustainable. All revenue exceeding expenses would support some open source educational technology, such as Runestone and the WeBWorK Project.

In case you did not know, WeBWorK problems and problem sets are easily portable. Any PreTeXt book with WebWork can export and archive of problems for use ona WebWork server. If you try this service in the Fall and would like to move to some other server once Runestone academy starts charging for the service, you will be able to export everything with no hassle.

If you are new to Runestone, this YouTube Playlist will demonstrate every aspect of Runestone from the instructors side. To get a good idea of the student experience you can take a look at The Overview Book The PreTeXt project also has a number of samples available.

Sun, 23 Aug 2020 00:00:00 -0500 <![CDATA[New Titles]]> New Titles

This week I’m really proud to announce that I’ve been promoted to “Senior Executive Bartender.” Ha, If you got that reference congratulations you are old enough to have been a fan of Cheers! If you don’t know what Cheers is, then we will just move along.

But seriously folks I really am excited to announce two new books available for your courses this Fall. The first is How to Think like a Computer Scientist C++ interactive edition. This is an Allen Downey book in the same basic pattern as the thinkcspy book on Runestone but in C++. Barb Ericson and her students at the University of Michigan (The other U of M) have been working on doing the conversion to an interactive eBook this summer and it looks great, and really exercises all of the work we have done on improving C++ activecode, CodeLens, and unit tests for C++.

A few people have asked me about the Java version in this series, and all I can say is that I hope someone will take up the task! I think that with the work Barb and her students have done here, and the fact that the books are quite similar that it would be easy to adapt the C++ to Java. Any takers?

The second new book Mastering Relational Databases by Libby Shoop at Macalester College has a wonderful back story that fills my heart with joy. This is a great new book that I hope many undergraduate institutions will adopt. It is also the first major work to feature SQL in the browser with unit tests! But thats not the reason I am so happy to see this published? I’ll let Libby give you a preview:

A great deal of the material in part 2 was originally written by John Carlis for an audience that included a wide range of people, from undergraduate and graduate students to experienced computing professionals. His unpublished manuscript was entitled “Mastering Database Querying and Analysis”. Its primary focus was relational algebra precedence charts. In 2015, three years before his death, he had given me permission to edit a copy of his material to make it approachable for undergraduate students. This online book is the result of my attempt at that task, most of which I am completing after his death.

John was my Masters advisor at the University of Minnesota (the *real* U of M), and an amazing mentor to many many students. He was Libby’s PhD adisor, and we crossed over for a few years when we were both John’s students. John was the person who encouraged me to write my first book. He was an amazing teacher, mentor and huge proponent of active learning and practicing. I believe he would be very happy to see this book come to life on the Runestone Platform! I’ve written more about John’s influence on my life’s path on my personal blog

There are more new titles in the works, but I’ll save them for another day. Please help us spread the word and tell your friends and colleagues.

Sun, 26 Jul 2020 00:00:00 -0500 <![CDATA[Runestone Funding]]> Runestone Funding

In 2018 I took advantage of an “early retirement” opportunity from Luther College to pursue working on Runestone full time. That opportunity afforded me a year to figure out a business model for Runestone so that I could continue to give Runestone my full attention. However, before the end of the 2017/2018 school year, I got an even cooler opportunity to work with a group Google working to bring computer science education to underserved institutions. Google would use Runestone as the platform for the Applied Computing series and I would work with them, developing both content and features for Runestone. This contract was originally written for five months, but ultimately extended to the maximum contract length of two years. Earlier this month the contract came to an end.

During this time period Runestone has experienced tremendous growth, pushed further this last spring as everyone had to move online. Since the fall of 2018 over 116,000 students have registered on! That is a lot of students, and if you do the math, we have saved students around the world millions of dollars in textbook costs. Just last week we hosted an online workshop for Runestone instructors with over 240 people registered! If you missed it you can find the videos on our YouTube channel

While the last two years have been exciting, fun, and profitable, consulting for Google really isn’t a business model for Runestone. So, for the last many weeks I’ve been having conversations with some of the major stakeholders and contributors to Runestone including authors, and code hackers, and instructors like most of you. We have big hopes and dreams to grow the platform! Today I write to tell you the outcome of these discussions and how that will effect Runestone moving forward.

My goals for a funding model for Runestone include:

  1. Keep Runestone books FREE for all students. This has always been the North Star for me.

  2. Provide a first class educational experience for all students using Runestone.

  1. Make Runestone sustainable. In other words, cover the server costs, pay myself a small salary, ideally provide for summer internship opportunities for a few students.

  2. Provide a solid foundation to continue the growth and development of the Runestone platform. More authors, more books, better features. For example

    1. Developing more user friendly tools to encourage new authors.

    2. Creating additional opportunities to customize textbooks.

    1. Funding for workshops for new instructors, hackers, and authors

    2. Continuing to develop innovative new features for Runestone books

    3. Develop communities of practice around each textbook where all instructors and authors can contribute ideas, teaching tips, assignments, teaching materials.

We considered many different models, and wow is it difficult to find a model that works for everyone given the wide range of people that use Runestone. We serve everyone from independent learners to public high schools to large universities. But starting soon, here are the three mechanisms we are going to try.

  1. All pages viewed by non-registered users will contain an advertisement. Around 40-50% of the traffic on Runestone is realized as the result of a Google search. These users typically view one or two pages and then move on. But the volume of this alone is quite large. Let me be super clear When your students login and are viewing your course, they WILL NOT see an ad! This is to generate money from the large number of transient users of the Runestone pages. If your student sees an ad that should be a very clear indication that they are not logged in.

  2. Beginning this Fall, we will do a WikiMedia style fundraiser a few weeks into the term. By this I mean we will display banners to registered students that have not chosen to support Runestone. If your class is identified as a public high school course they will be automatically excluded from this as I realize it is not appropriate. Less than 1% of students choose to support Runestone at registration time. However they haven’t experienced a Runestone book yet. Given a few weeks and the value that we provide I believe we can increase that percentage significantly by asking students a few weeks after they have used the books. This ask will be limited in time so we will not nag them forever. Just a week or so.

  3. I will continue to work with researchers at academic institutions to participate in grants. Getting grant money to support the development of significant Runestone features is an awesome way to fund that development. I would welcome the opportunity to partner with any of you that are writing grants and using Runestone in your research or classes.

I think these are some of the least intrusive ways we can generate diverse revenue streams that will help us realize our goals. First and foremost I want to ensure that Runestone is robust and available for years to come. I welcome your feedback, thoughts and further ideas.



Mon, 29 Jun 2020 00:00:00 -0500 <![CDATA[Runestone Workshop June 22-25 2020]]> Runestone Workshop June 22-25 2020

Barb Ericson and I are pleased to announce that our scheduled SIGCSE Workshop will now take place virtually, and you are all invited to participate free of charge. The workshop will take place the week of June 22, Monday through Thursday and will use a blended format of asynchronous videos and assignments along synchronous discussion sessions. The live discussion sessions will happen at 3PM EDT. You are welcome to attend any or all of the sessions depending on your schedule and previous experience with Runestone.

To Sign up for the workshop please visit If don’t have an account please make one and register for the course UsingRunestone20. If you already have an account on Runestone then add yourself to UsingRunestone20. Note this will enroll you in the Foundations of Python Programming course which we will use as a starting point. We will also spend a lot of time with the instructors guide.

Module One - Runestone Student Experience

  • Introduction

    • goals for the workshop

    • history of Runestone

  • Be a student

    • enroll in course

    • check out your first assignment - reading assignment

    • checkout your second assignment - problem set

    • Using the Practice Tool

Module Two - Teacher Experience Creating a course

  • Creating a course (base course versus bespoke course)

  • Student Management

    • Enrolling and dropping students

    • Adding a TA

  • Monitoring Student Progress

  • Instructor only features of the textbooks

  • Setting up practice

Module Three - Creating your own Assignments Writing and Customizing

  • Creating assignments and questions

  • Making assignments

    • Reading Assignments

    • Problem Sets

  • Grading assignments

    • Using the autograder

    • Manually grading

  • Editor Role and flagging bad questions

Module Four - Stories from the Instructor Community

  • Discussion with guest instructors

  • Creating a community of practice

Tue, 26 May 2020 00:00:00 -0500 <![CDATA[Runestone 5.0]]> Runestone 5.0

Today I am extremely pleased to announce Runestone 5.0! This is a combined new release of both the server as well as the components. Not a lot of this release is highly visible but there is a lot going on behind the scenes.

Perhaps the biggest change for everyone is that pages should load faster. We have moved to using the webpack system to bundle all of our javascript files into a single runestone.js file. Pages used to grab more than 100 javascript and css files, but that has been reduced to just a few now. This makes Runestone MUCH more interoperable with things like Markdown or vanilla html. It was also a really good chance to clean up, refactor and modularize a lot of ugly javascript that had accumulated in the last 9 years!

For Authors

  1. jUnit is now support for writing auto grade-able Java questions

  2. catch.hpp is now supported for writing auto grade-able C++ questions

  3. The python tutor support is upgraded to the latest version and supports writing embedded visualizations for Python, C, C++ and Java

  4. Skulpt – our in-browser version of Python supports all Python 3 syntax and lot more of Python3! f-strings are even supported now! This will provide students with a much more consistent experience when moving

  5. .. datafile:: – if you need data for your program or auxiliary cpp or java files for your programs you should use the datafile directive to provide them. This will ensure that these files are available anywhere in the book, including assignments and the grading interface.

  6. There is a video now that walks you through the process of getting runestone installed and ready to go if you want to write your own content.

For Instructors

  1. The Runestone instructor interface has accumulated a lot of great changes in the last 9 months. I attribute many of these changes to wonderful discussions I have had with teachers on the CSAwesome mailing list.

  2. I have set up Google Groups for each book, with the goal of using these groups to develop communities of practice around each book. There is a lot we can all learn from each other and share with each other. These groups will also serve as the gatekeeper for access to instructor-only materials. In particular both FOPP and HTTLADS have very highly developed instructor manuals now.

Summer is a great time to get involved

We have a lot of projects in the pipeline for this summer! Including lots of great new content for you to use this Fall. Here are some links to help you get started.

We can use help writing unit tests for exercises, editing exercises, adding new features, whatever your interest there is sure to be a project for the common good that you can get involved with. You don’t need to be an expert, none of us were when we started, and we are happy to help get you up and running.

Fri, 22 May 2020 00:00:00 -0500 <![CDATA[Runestone Partners with Edfinity to bring homework system to CS1]]> Runestone Partners with Edfinity to bring homework system to CS1

Today I’m pleased to announce Runestone’s partnership with Edfinity, our first commercial partnership! Edfinity provides affordable access to exceptional homework problems and assessments customized for any textbook. While Edfinity has focused on the Math and Physics markets thus far they have chosen to partner with Runestone as they move into the computer science world. Edfinity was created with the NSF support (Awards #1758301 and #1943530) - they are best characterized as a for-profit social venture with equal focus on educational research and educational technology. Since the launch of their Math/Physics homework in Fall 2019, over 125 institutions have adopted Edfinity for one or more introductory courses.

Shivram Venkatasubramaniam, CEO of Edfinity has this to say: “Runestone offers high-quality instructional content on a mature, open platform for introductory computer science that has gained adoption at hundreds of institutions across the US. They have a relentless focus on educators needs with a view to delivering superior learning outcomes, and a vision borne out of their deep experience as educators. We, therefore, view Runestone as a natural complement to the Edfinity homework/assessment platform - together Runestone and Edfinity are very well-positioned to serve the needs of educators and students at scale - not just for Computer Science, but other STEM subjects as well. Runestone’s value proposition as an open, interactive textbook platform applies equally to other STEM disciplines”

In eight years of providing textbooks to the computer science community the Runestone team has learned that even for a free textbook, that is only the beginning of what schools are looking for. Many instructors are looking for a commercial class homework system to compliment the Runestone textbooks. The partnership with Edfinity will fill that void. Instructors, especially those teaching CS for the first time, need pre-packaged assignments and assessments. Over the next few months we will be working with the Edfinity team and the community to develop homework and assessment packs that not only complement the How to Think like a Computer Scientist and Foundations of Python Programming text books, but can be easily paired with any commercial textbook for a fraction of the price. A math educator commented recently - ‘Edfinity is as close to free as you can get for a commercial quality homework system’ - I expect this to be true of the Edfinity/Runestone CS homework offering as well. Longer term we will work together to make the experience of using Runestone with Edfinity as seamless as possible.

As an immediate first step, Edfinity is funding several paid positions for undergrads, faculty and high school students (openings and postdoc - please contact me if you are interested) to help write and test exercises, unit tests, quizzes and exams to accompany the textbooks. I’m really excited for this opportunity to work with the team at Edfinity, it’s clear that they are laser focused on bringing a world class homework and assessment system at an affordable price to instructors that really need it. To see how Edfinity works you can check out some of their math offerings on their website at or drop a line to Sid and Shivram at

Runestone and Edfinity will be actively piloting our new offerings with several of you during the coming months, with full availability scheduled for Fall semester 2020. This is designed to be a community-centric effort and we invite your participation and assistance to any degree both in terms of conveying your needs to us, and in helping to build out an exceptional corpus of questions and unit tests. Please contact me with ideas and suggestions.

Sun, 26 Jan 2020 00:00:00 -0600 <![CDATA[PreTeXt + Runestone: a New Partnership]]> PreTeXt + Runestone: a New Partnership

I’ve just returned from an inspiring weeklong workshop on Interactive Assessments in Open Source Textbooks, hosted at the American Institute of Mathematics offices in San Jose. There were about 25 participants in the workshop, including, authors, instructors, developers, administrators, and people who wear combinations of those hats. Not only was the workshop inspiring, but the structure of the workshop encourages concrete progress, and I’m thrilled to say that we were able to demonstrate several aspects of a working integration between Runestone and PreTexT. Although our teams have been working in parallel for several years, it is clear that our goals are well aligned. The PreTeXt markup system provides over 50 different online resources to the math community including textbooks for Calculus, Linear Algebra, Discrete Math and much more. Both PreTeXt and Runestone have a lot to bring to a combined table and here are some of the ways that the entire community may benefit from an integrated system:

  • PreTeXt does not have a central hosting system like Runestone.Academy, nor does it collect the data for the kinds of instructor analytics that Runestone does.

  • While PreTeXt makes use of WebWork for interactive exercises WebWork does not have all of the different components that runestone has

  • PreTeXt has a different markup language than Runestone - based on XML - it is not complicated to write and less fussy than restructuredText in the use of whitespace in its Syntax

  • PreTeXt can produce documents in many different output formats including online as well as PDF, ePub, and in-progress work on Braille!

Running a PreTeXt book on a Runestone Server with integration to webwork

Serving a PreTeXt book with a Runestone Server integrating a WebWork question

During the week we were able to make PreTeXt output html that was compatible with several different Runestone components, including activecode, multiple choice, and short answer questions, the rest will follow over the coming months! This means that an author could write in PreTeXt and host a book on In fact were were able to demonstrate that we could already host existing PreTexT books, and with a little work we were able to capture student responses from WebWork problems and save them to the Runestone database.

In addition to the integration work, there was a lot of conversation and drawing of pictures and making of lists that revolved around topics that both Runestone and PreTeXt could benefit from. Some of it is great design work, some of it is a vision of how the future could look, But the answers to the questions will provide lots of work for the next few years.

  1. How can we write and present instructor editions of textbooks? What kinds of resources should be included in an instructor edition and how best to expose those materials.

  2. How can we build effective communities around the various textbooks we provide to encourage instructors to help and learn from each other?

  3. What kinds of analytics would be useful for book authors? How can the data collected for authors be visualized effectively to help authors write more effective textbooks?

  4. How can we write questions that can have many equivalent variations, to promote better student practice and learning.

  5. To what extent should instructors be able to customize a textbook? How can we allow customizations without a dizzying and confusing proliferation of minor variations of the text?

  6. How can we collect data to enhance student learning, while still respecting student’s privacy and respecting an increasing number of privacy laws?

  7. What can we do to ensure that creators of open source educational resources get the proper credit towards tenure and promotion for work that has a wide impact? Sure a journal paper that is read by 6 other people is fine, but isn’t a book that educates thousands of students and moves pedagogy forward at least as valuable?

I’m not going to provide answers to those questions here as that work will be ongoing and we could use a lot of help. If you are interested in tackling any of these questions or just participating in the discussion, please let me know!

I also learned about the DOENET project happening across many universities with the project manager located at the University of Minnesota. This would be a longer term integration project that could address many of the privacy issues surrounding open source education materials. And since I’m only seven minutes away from DOENET central by bike, we decided to focus on other integration issues during this workshop, and get together after the conference to start this discussion.

In closing, I want to thank the organizers of this workshop – Jim Fowler, Mitch Keller, Matthew Leingang, and Oscar Levin for inviting me to participate. Thanks to the other participants who made the non-math guy feel extremely welcome! Thanks to Rob Beezer and David Farmer for inventing and writing PreTeXt and being so open to working on integration, even well into the evening after the workshop was done for the day.

I hope someone from the ACM will read this, and look into creating and funding workshops with a similar format. This was 100% a week well spent – Not something you can say after every conference! This workshop was a great beginning and I look forward to seeing what we can do together.

Sun, 15 Dec 2019 00:00:00 -0600 <![CDATA[Privacy and Runestone]]> Privacy and Runestone

In this week’s update I want to tell you about some new account management features I added to Runestone, and give you some idea about how you could set up your class if you don’t want Runestone to capture any Personally Identifiable Information (PII) about your students.

Runestone collects a lot of data. I think it is all justified, and if you are an instructor you can gain some real insight into how your students are doing based on the data we collect and the reports we create for you. None of that requires any PII about your students. Yet we do collect some… what do we collect? and why?

Here is what we we ask for when a student registers:

  1. username - this can be anything and it is easy for this to NOT contain PII

  2. email - This is the only reliable way to do password recovery.

  3. First Name and Last Name - this has always been just for the instructors so that when they are grading assignments or looking at analytic reports, they know who they are looking at.

  4. Password - encrypted and stored securely.

  5. Name of the course. The instructor has full control over this call it whatever you want.

Setting up a class with no PII

We will walk through two strategies for managing a Runestone Course that doesn’t store any PII about a student. The first strategy has the students register themselves following the instructions I’ll tell you about. The second strategy puts you in complete control and allows you to create all of the usernames for your students!

The way to use Runestone without PII is to ensure that your students choose a username that they remember but has nothing to do with their real identity. They can also put in fake first names and last names.

The real trick is how to manage password recovery Thats where this cool email alias trick comes in. Most email systems (including gmail) support the username+alias convention. That is you can add a + and an additional alias to your email address. For example or both of those emails will end up in the runestone mailbox, but I can also see the alias. We can use this feature so that students don’t have to enter their real email address, they enter yours plus their username as the alias. This means that you will end up fielding the password reset emails. If you are an email whiz you can set up some rules to forward these to the right student.

Experience has shown that having students register themselves and get all of those pieces correct is a losing battle. So why not put you in charge of registering them? Get out your favorite spreadsheet app and prepare a file with your students information.

You can upload a CSV file with the following format: One row for each student


In order to protect your students privacy you can create a spreadsheet like the following:



It is important that you check your CSV file and make sure there are NO SPACES before or after the commas!

From the Manage Students tab on the Instructor’s Page, you can now upload this CSV file using the interface.


Once you have uploaded the CSV, you can add some columns to the spreadsheet that contain the real student information to help you remember. You will probably learn quickly who your made up names correspond to, and can probably think of something anonymous that will help you remember who F1 L1 really is. On the first day of class you may want to hand out paper for the students with their username, and walk them through changing the password, or maybe you want to assign each student a unique password when you are making up the spreadsheet. I just would not recommend a scenario where all the students have the same password.

Of course all of this is optional. You can also create a csv file with usernames of your choice and include the real email addresses and real first / last names of your students, or you can have them register themselves as students have been doing on Runestone since 2012.

Password and Account Management

There is no doubt about it, password management is a headache for everyone. Runestone handles hundreds of password reset requests each week. But that does not always work. Some schools (inexplicably) block email from Runestone. Other schools run software that simply mangles the URL’s found in emails that look like they are password reset emails. This is understandable to some degree as Phishing scams are so common, and its important to protect the students. I’ve done everything I can do from my side to validate the email domain. But, other times students simply type in their email incorrectly.

When the email system for password resets do not work, I have now given instructors the ability to change the passwords for the students in their class. Hopefully, this will eliminate the emails asking me to manually reset passwords when the reset emails are now flowing. Runestone has gotten too big for me to deal with these requests, and I don’t have any minions for that.

Instructors cannot change their own passwords through this mechanism. who among us hasn’t walked away from the computer and left ourselves logged into Runestone? As an instructor, you will still have to use the normal email password reset mechanism. If your school blocks emails then I suggest you just get a free gmail address to use for this purpose.

One final note on account management. When you use the Remove Student button, you are not actually deleting the student. What I do is move the student out of your course so they do not clutter your gradebook or Student Progress page. They are moved into the public version of the course you based your own course on. This allows them to keep their username, as well as retrieve any of their work they did. If a student wants to truly delete themselve they do have that capability! From the user menu select edit profile and from there the student can really delete themselves and all of their work.

What other Data Does Runestone Collect

Here is a graph of all of the events that runestone has collected in the last 4 weeks.


As you can see that is a lot of activecode and a lot of pages loaded. Also over 1 million multiple choice questions answered! I would love to see more clickable area questions and more fill in the blanks. Some of our preliminary research shows that those kinds of questions are much more effective at getting students to review the content in order to get a correct answer, rather than simply clicking on different answers until they guess correctly for multiple choice. All of these events are captured and tied to a username. So the more anonymous your username the better as far as I’m concerned.

What I’m really interested in is figuring out how to apply machine learning algorithms to help identify students that may struggle, to help weed out bad questions, to help us write better questions, to help us order the textbook in different ways, to adapt to each student to help them achieve their maximum potential.

Runestone also collects the source code each time your student changes and runs the code in an activecode activity. This is to provide a timeline for the student as well as for yourself.

  • Make sure you explain the timeline to your students. Sometimes there is great fear in them about changing code that is working. Having this timeline present encourages them to experiment, knowing that they can always get back to a good working version.

  • For you the timeline can be an invaluable tool for talking through an assignment with a student. You can see their progress and if they make a wrong turn you can usually see that in the timeline and talk to them about why they decided to a particular direction.

  • The timeline is also available in the scratch activecode which some students like to use as a notebook for the course.

Fri, 25 Oct 2019 00:00:00 -0500 <![CDATA[Grading Updates]]> Grading Updates

Lots of updates to the Grading interface this week, as well as some important bug fixes. They are now live. The main things to look for include:

1. I have plugged a couple of holes in session handling, where students could mistakenly end up in the open fopp book instead of in the correct course.

2. In addition, if the student is not logged in, the progress bar at the bottom of the page will be gray instead of green. Another visual indicator that they are not logged in.

3. The grading interface has been redesigned into a new layout, and streamlined significantly. Important note – these changes are to make manual grading workflow less tedious. The autograder workflow is unchanged.

  1. Now you only have one choice to make. Start with either a Chapter or an Assignment.

  2. Questions for chapters are sorted alphabetically

  3. Its easy to select all question or all students or both.

  4. Chapters, Assignments, Questions, and students are all searchable. So if you have a class of 125 and need to regrade that one student it is easier to find them.

  5. If an assignment has not had its grades released there is a bright red message to that effect that will remind you to press the button.

  6. If you select multiple students you can navigate from score to score with the tab key and when you update the score it autosaves when you move to the next input field. You’ll know it has saved because it will turn light green.

  7. If you like grading things one at a time, the old save and save/next buttons are still there.

Here is quick video tour of the new features:


Please don’t hesitate to leave a comment on other things I could tweak to make the workflow more clear or efficient.

Sat, 12 Oct 2019 00:00:00 -0500 <![CDATA[Help Wanted - Make Autograding Awesome]]> Help Wanted - Make Autograding Awesome

What if I told you that just 10 minutes a day for one week between now and September 30th could improve your life dramatically for the entire semester?


Did I get your attention? How much time would you save if every programming problem in Runestone had unit tests so it could be auto graded? I know I would save me many hours! In addition my research has shown that students work harder, and end up with much better solutions when an exercise has a good set of unit tests. It seems like a win for everyone! When there is that positive affirmation that you are correct, students build confidence and feel better about doing the work. In addition, all the emails I get asking for solutions will go away because if all the unit tests pass people will be happy that they have succeeded.

Most of the programming exercises for Foundations of Python Programming, as well as those in How to Think Like a Computer Scientist lack good unit tests for the autograder. This is actually an easy problem to solve if we crowd source the solution. If all of us using Runestone in our classrooms write a unit test every day we could have 100% coverage.

Adding unit tests to nearly every activecode is an achievable goal, if we work together to write them. All the cool kids are doing crowdsourcing today, so why can’t we? Here is the best thing, I’ll take a set of tests for any activecode in (nearly) any format. Just send me the title of the book, the activecode ID (shown below the activecode) and a set of assertions, and I (or another volunteer) will get that information in the book. With the new architecture, as soon as I do a rebuild all of us will immediately benefit from the new tests. You can send me an email. You can post it in the comment section of this blog. You can create an issue on github Or you can post the message on our Slack Channel. (See the Support tab for the link to join).

The test you send me could be as simple as.

Dear Brad,

For activecode ac_2_5_blah we should test the following conditions

    for starting input of 44 and 15
    variable foo should have the value of 29
    variable bar should have the value of "hello world"
    The output should contain the word awesome
    the source code should not include the word append


<insert your name here>

Thats it, we’ll take it from there. You could also make Pull Request on github if you know how to do that.

You do not need to be an expert to do this. You could even make this an assignment for your more advanced students. I promise I won’t yell at you if you leave some corner case uncovered. Someone else will probably think of it. I, and a couple of other folks will act as the editors to make sure we have good tests that cover a reasonable set of cases for each problem.

What does a unit test look like?

Here is the ideal submission. (Again, I’ll take much less than this)

from unittest.gui import TestCaseGui

class MyTest(TestCaseGui):

    def test_XXX(self):
        assertEqual(variable, correct_value, feedback)
        assertEqual(function_call(), correct_return_value, feedback)


What about problems without functions?

No problem, you can make an assertion that a variable should be equal to a value just as easily as a function. Now, many problems are not stated precisely in the sense that we have not stated clearly. Store the result in “variableX”. That is easily changed as well,and you get the satisfaction of knowing that your variable name will be used by thousands of students.

You can also make assertions about the output as well as the assertions words that appear in the source code.

For example lets say you want to write a unit test to check that the ever popular “hello world” program works. We will use the getOuput() method to grab the output of the program. Its just a string so we can manipulate it and assert that substrings be in the output!

print("Hello World")

from unittest.gui import TestCaseGui

class myTests(TestCaseGui):

    def testOne(self):
        self.assertIn("Hello World", self.getOutput(), )


Note that the ==== tells Runestone to hide the test code following the ==== from the student.

Looking for multiple answers in the output is easy to do with a number of assertions. So, this would be a good strategy for having them try several test cases. Even when using the input function!

Or, lets say you want the students to calculate the total of a list of numbers but you don’t want them to use the sum function, and you want the result to be stored in the variable total. We also have access to the source through the getEditorText method.

numbers = [1, 3, 5, 11, 25]

total = sum(numbers)

from unittest.gui import TestCaseGui

class myTests(TestCaseGui):

    def testOne(self):
        self.assertEqual(total, sum(numbers) )
        self.assertNotIn("sum", self.getEditorText(), "Do not use the sum function")


This would result in:


What if I have already written a unit test for problem xxx?

Even better! Just send me the unique id for the problem you created, and (if you know it) the id for the problem you started with.

I know we are all super busy, but with just a few minutes from each of us we can really make a huge dent in reducing our grading workload. Think about it this way: when you make the assignment, jot down a few ideas about what you are going to look for when the students hand in their work, or even write up the unit tests in an editor. If you get them to me, and we get them into the book before the deadline, all you have to do is make sure your students run them one time with the tests in place and your grading for that problem is done! I think this can pay for itself with a single assignment!

Thanks in advance for your participation!


Thu, 15 Aug 2019 00:00:00 -0500 <![CDATA[User Interface Polish]]> User Interface Polish

In my last update I detailed some improvements to Runestone that had been a long time in the making. Today, I want to share some improvements to the UI that have come directly from new contributors that are also actively using Runestone in the classroom. These are designed to help make your experience using Runestone more pleasant. I’m sure that there is much more we can be doing, but its really great when the changes and requests from from people who are using Runestone every day.


Self Grading Have your students ever complained that they don’t know what they have left to do to complete either a Reading Assignment or a Problem Set? Probably, have you resorted to running the autograder a few days early and releasing grades so that your students can see what they have left to do? Now students can “grade” their own assignments. At least that parts of it that can be automatically graded.


After the student presses the button they will see their scores and progress for every part of the assignment. If they are not getting full credit for a section of the reading, they can click on the link and finish the activities on that page. If they are not getting credit for a programming problem they can continue to work on it until all the unit tests pass. If a problem requires manual grading it will just show a message “Not Yet Graded”.

Multiple Selections: Have you ever wanted to streamline your manual grading by selecting a bunch of students at once and having all of their code visible rather than selecting them one at a time? You can now do that!


Not all of us have a good naming convention in mind for our assignments at the beginning of a semester. So, one simple but popular request is to be able to rename assignments. You can do that now.


What can be autograded? When making an assignment, I know this was a bit frustrating. Now the Tree View of the avaialable exercises tells you what kind of thing each question is, AND if there is a checkmark next to it then you know that it will be autograded. Better yet, the system now does a much better job of setting the correct default grading option for you when you add something to an assignment.


Help Wanted

We know there are lots of coding exercises that could be autograded but are not. This is due to a lack of hours in the day. Almost anyone can help solve this problem. Writing tests is not that hard to do, and I’ll take the tests any way you are willing to give them to me. I’d love it if they were a Pull Request on github, but I’ll take them in the form of emails, or posts on the Google Group, or tweets, or Slack messages or whatever way you want to get them to me. Here is an example of how easy it is to add a unit test to the code in any exercise. Unit Tests for Code . This problem is so solvable if we can get a community effort behind it. If everyone teaching using runestone sent me one unit test a week, think of how much better your lives would be to have more programming exercises that could be automatically graded. – Not to mention how much better your students learn when exercises are automatically graded.

How about offering a little extra credit to your advanced students for writing unit tests to exercises they have already done??

Student Progress

Speaking of student progress a little polish has been applied there too. The “donut charts” are now organized by subsection. And the dropdown menu has Chapter Numbers! This makes it a lot easier to guage what is going on in your class at a glance when you can see the donut charts grouped by sub chapter.


Squashing Bugs

We have also squashed a fair number of bugs this summer. Just two days ago I corrected a really annoying grading problem thats been around for 3 years! I’ve also found ways to eliminate (or ignore) the cruft in our database that sometimes throws off the count of exercises for reading assignments! Little by little we are making things better and more reliable. Please do report any problems that you run into. Again preferably on Github That makes it so much easier for us to track.

Sat, 27 Jul 2019 00:00:00 -0500 <![CDATA[Mid-Summer Update]]> Mid-Summer Update

The July 4th holiday is just three days away, which for many of us in the academy marks the midpoint of the summer. To say that we have made some big strides in Runestone so far this summer is an understatement, and they are all coming to before the holiday. So, put on your seatbelts and get ready for the ride.

New Stuff for Instructors and Students

There are several big changes to Runestone that should make life better for instructors this summer.

  1. Dynamic Pages – wait, what? I thought Runestone pages were already far from static. You’re right, but you know that course building and rebuilding process you had to go through to create a course? Its gone! We now have a single copy of the most up-to-date version of each textbook that we can serve dynamically for you and your own bespoke course. You won’t really notice anything except that creating a course is now instantaneous, and the URL to your book is now going to be /runestone/books/published/yourbook instead of the old /runestone/static URL.

  2. In page progress indicators Reading assignments are awesome, but they do have some potential pitfalls. Such as students doing work in the wrong course, or not understanding what they need to do on each page. So at the bottom of each page we have added a progress bar to show how many of the interactive activities on each page the student has tried.

  1. One scratch activecode per book – One thing I learned last year in working more closely with instructors and having more time to listen to students is that most students were confused by the scratch activecode. In the past we had a different scratch activecode for each page for each student. This was confusing as students use these scratch activecode’s a lot, but were often confused about how to find something they were trying on a page somewhere that they could not recall. Now their entire history of all interactions will be available and scrollable through the history slider. Making history searchable is on the todo list somewhere…

  2. Navigation menu within the chapter – another request that I’ve heard from many of you is to improve the navigation in the book. There are many different opinions on what is “best” in this area, but the one I like the best, and the one that keeps the book pages the most uncluttered, is to have a “This Chapter” menu in the navbar. This lets you quickly move to any page in your current chapter. If you want to change to a different chapter then you can click on the course name and that will bring you to the table of contents for the whole book. So, closer is one click away, and anything outside the current chapter is two clicks away. That makes sense to me. But I would love to hear feedback on this. Nothing is set in stone.


New Stuff for Authors

  1. New activecode language for writing about and working with SQL! Oh yeah, I love SQL! I love writing queries and I love teaching about how to “squeeze information out of a database,” a phrase my masters advisor John Carlis, repeated every day in our databases class. SQL queries are also “autogradeable” with a simle set of asserts. The database engine runs in-browser and is a fully operational version of SQLLite!

  1. New Spreadsheets directive When trying to write a data science book last year, it was clear that both SQL and a Spreadsheet in the browser would really help. Now, thanks to the jExcel Project We also have a very functional spreadshet running in the browser. Many many Excel/Sheets functions are supported, and it is also autogradeable through a series of assert statements. In addition, cells that students should be adding formulas to are colored in light blue to start with, and after grading are colored green or red to indicate whether they got it right. In addition there is text that also shows whether each test passed or not, and what the expected value was.

  1. New Instructor Only Reveals So many people have asked for instructor materials for all of our books. One of the things the last year of working with Google has helped with is in actually getting some materials written. Its at least as much work as writing the book itself! So how to deliver these new materials? This is one of the first great features of the new dynamic pages model of serving. Buttons that are only included on the page if you are an instructor.


When you click on it the answer or any other helpful material is displayed.


Students will not even see the buttons. In some books these instructor materials will be “paid for” features. You will also need to do an extra level of proof that you are an instructor in order to gain access to these.

Inside Baseball

  1. Unit testing means better reliability and more agile development. We have invested a lot of time this summer in getting a good unit testing framework set up for the Runestone server. I’ve spent some portion of each day writing one or more unit tests. For the functionality required to serve books and collect data reliably we now have excellent test coverage. Not as good for some of the instructor supporting pages, but we are working on it, and they get a lot less traffic than the book pages. I’ve also done a lot of load testing on this new architecture and it seems that it should hold up quite well.

  2. New dynamic architecture for more dynamic features in the future. In addition, the new dynamic architecture changes how we check that login/logout is handled. This should help with making sure that students are always in the course they should be in, rather than some random course they found with a Google search.

  3. is gone. Well its not really gone, it has a permanent redirect to None of the old classes on made the move. Registrations have been disabled for a year. This move has been coming for two years, so hopefully it doesn’t surprise anyone.

  4. Both RunestoneComponents and RunestoneServer are running Python3. We had to keep Python2.7 a lot longer than I would have liked due to the web2py framework, but happily web2py now fully supports Python 3.7. I look forward to simplifying and dropping support for 2.7 once all this new stuff is live on academy.


That is a LOT of change! And, I plan to deploy all of these new features over the next week. Even with all of the testing we have done there is bound to be something we have missed that will break something. Please be patient, and please report it on our GitHub page!

  1. If you have an existing course, in progress, on Runestone Academy that course should continue to work just fine. You will not be able to rebuild it as that functionality is gone.

  2. Newly created courses will use the new dynamic pages feature. This means you don’t need to rebuild, you will automatically get updates that fix typos or squash bugs.

  3. I’m introducing these changes during our summer “downtime” although I fully realize that our summer downtime is now busier than our peak traffic the first couple of years! This means that by the time classes start for people in the Northern Hemisphere in late august – the majority of our users – things should be in good shape.

Mon, 01 Jul 2019 00:00:00 -0500 <![CDATA[SQL]]> SQL

Today, I built something I’ve been wanting to add to Runestone for several years. I added support for the SQL language. I’ve been holding off on adding support for it, because I thought I was going to have to support SQL as a server side language rather than in the browser. This means a lot of thinking about security, and making sure that one student can’t wipe out the data in a test database for all students, and setting up accounts, etc. Every time I’ve taught introduction to SQL we get to that point in the semester where I know I have to keep a close eye on the server as students begin to write bad cartesian products and create result sets with millions of rows! This effectively slows down or stops the database engine for the entire class.

However, last week, as I was thinking about it again I stumbled across the SQL.js project This project takes the SQLite C code, and compiles it to Webassembly. Thus making it possible to run an authentic version of SQLite in the browser! SQLite is a good tool for teaching as it is lightweight, comes with Python, and runs small databases directly in memory.

From a teaching perspective this really simplifies things a lot! Now each student has their own copy of a database that I can distribute to them over the Internet. Students can’t mess upanyone else’s results or hang the server. If they really do mess up a query they just need to reload the page. In addition, this also means that I can add autograding of SQL queries. As I begin to work on my Web Development course for next fall, and we begin to revise our intro to data science course, having SQL in the browser is going to be a game changer!

Its available today in the github version of Runestone.

Stay tuned.

Mon, 17 Jun 2019 00:00:00 -0500 <![CDATA[On Runestone for Intro Courses]]> On Runestone for Intro Courses

By Jackie Cohen

I began using Runestone as a computer programming course instructor in 2014. In the ensuing five years, I’ve taught a variety of programming courses, with a variety of levels and audiences, which used a variety of Runestone books and features – more and more, as more have been developed – and I have worked on some Runestone features myself.

This means that I’ve explored Runestone in a variety of ways, and for a variety of reasons. Many Runestone Interactive materials, and features available in the platform, have proven particularly valuable to me in one type of use case: using a Runestone textbook as the source material for an true introductory programming course, where the course is primarily made up of students who have never done any computer programming before.

This is the kind of use case I’ll share some things about here: an example of which material(s) I’ve selected for intro courses, why I think Runestone is a useful choice for intro programming courses, and particular ways that I’ve relied on a couple of Runestone features to help me achieve pedagogical goals in my class.

Of course, “an introductory programming course” can look like many different things, for many different reasons, and have many different audiences and structures, but here I am speaking generically from my experience being an instructor of college-level students who have never programmed before, enrolled in a class of between 50 - 250, as our course sizes vary from semester to semester!

(Note also that while I refer to a bunch of ideas about pedagogy and what students struggle with, below, what I’m talking about here are all anecdotal experiences of my own as an instructor – there is a lot of great research about related things as well if you’re looking for more detail on computer science, engineering, and programming education.)

So - Which Runestone Book(s)?

The introductory courses I have usually taught over the past several years in my department are taught in the Python programming language. I’ve used a couple different versions of course materials over the years, but for the purposes of this explanation, I’ll examine a case where I choose Foundations of Python Programming as the course textbook.

I’ve also used selections at times from other Runestone materials, which is a nice backup for a time that one book has, say, 98% of what I want.

Why Runestone?

A primary highlight of selecting a Runestone-based textbook as my course textbook is the stability it offers my class, on a few different axes, and this is why I’ve chosen to use it repeatedly.

First, and primary to my choice of Runestone for introductory classes, is the fact that using Runestone to learn computer programming material does not involve any installation or dealing with the varieties of operating systems and software, but it does require learning new vocabulary and practices for saving and running code.

I find that installation is a huge barrier for many introductory programming students. It can be difficult, time-consuming, and annoying. The difficulty or annoyance of installation of a programming language interpreter, a text editor, a command prompt software, or anything else, can feel like proof that a student new to these ideas “doesn’t belong here [in this class]”.

Students who have never done any computer programming before, especially students who do not have a lot of experience with software tools or other types of computing, often struggle (as many computer-types do for a long time into our careers, really!) with the difference between a problem that requires struggle and creative problem-solving, and a problem that is not truly creative or “computationally challenging” and is rather caused by an error you can’t control, or by forgetting to click a single button at the start of the installation process. And for large courses without a large staff of a TAs/et cetera, helping students complete installations with a ton of different systems and versions thereof can also be a huge pain, and not a great note on which to start a class – plus it’s difficult to balance this fact with the desire to start writing code and seeing things happen on your screen rigt away. Using a Runestone textbook solves these problems, and allows students to focus both on concepts AND upon the joy of seeing code “work” and even pass tests – on the in the very first week of class.

Secondly, everything is in one place, rather than having a plethora of blog posts, links, a textbook, code files… all from Day 1. While I do integrate non-Runestone resources into many such courses, the Runestone textbook is always “home” for students, who often express (in my experience) that they feel overwhelmed by having a lot of new material in programming as well as a lot of new resources to use.

Using a Runestone textbook also allows me to integrate graded work of various types with un-graded work, helping me to emphasize during class that learning to program usually requires a lot of practice. If such and such thing is required/graded, but there’s also A, B, and C helpful conceptual activities we will discuss in lecture next week… isn’t it useful that they’re just right there, too? A simplification, of course, but something I’ve valued especially when students in class don’t have a lot of experience learning a programming language and are still adapting their own learning and practice expectations for the course. What they need at first is all there on one website.

(In some courses I’ve moved “off” Runestone in a second half of the course, e.g. to using files and command prompts, but the Runestone experience they have is something they can continue to rely on and can use for practice, and remains a way of gaining new information about syntax and concepts since it is still our course textbook.)

What’s in Runestone?

With these ideas in mind, there are a few Runestone features that I’ve relied on heavily during introductory classes like those I describe.

One is, of course, the ActiveCode feature – we can write a program on the first day of class, and begin to understand the process of writing, thinking about, and running code, and how to see and check the results of code that’s been run. The CodeLens tool is a huge help with this, because explanations that I previously needed to include in additional resources or slides are built right into readings, so class time can focus on in-depth exploration that supports understanding even more.

Another is the ShowEval Trace and ShowEval Replace modes – as we begin writing code, I like to ensure I have a focus on understanding complex expressions and debugging, and this feature is a nice interactive way to focus specifically on evaluation of complex expressions without generating them correctly prior to evaluating them.

The availability of the Parsons Problems for a variety of reasons has proven valuable in my introductory courses – one way I like to use these features in class is that after I’ve, for example, assigned these as part of readings before class, I can return to them during a lecture so we can analyze together what ways of solving them are productive (rather than simple trial-and-error, for example), and we can really dig into useful ways of practicing and learning programming so students can experience them with me. I’ve used Parsons Problems both with code, like in the code based example at the bottom of this page, and with non-code words, like in the non-code example at the bottom of this page, and find a mix of the two helpful.

And finally, because I can assign reading assignments as part of the course, and I can choose a book with a structure I want (e.g. the way FOPP has chapters followed by exercises and sets of “chapter assessment” exercises) I can take advantage of the above to include all of those features (and others) in assignments/course work/student lab time/office hours. Students know that they will need to interact with these interactive features during the out of class time they spend reading, and we come back to the online course “home” to review and explore new concepts during class time.

When it comes time to introduce native programming on one’s own computer in the introductory courses that I teach, there’s a place to return to for conceptual things (the online Runestone textbook!), so it allows me in-class time to focus on identifying, and then solving, those problems that do come from ‘simply’ forgetting to click one button or remembering the type of your operating system – since identifying that kind of struggle/problem is also a very useful skill to identify and develop while students begin learning programming.

Mon, 13 May 2019 00:00:00 -0500 <![CDATA[Docker: an Open Source Story]]> Docker: an Open Source Story

I love the open source world! You never know what amazing new project you will find, or who you will meet that is willing to contribute some amazing new skills to your project. This last week provided a great example of the kinds of synergies that can happen here. It started with a very short conversation on another project I’m involved in called Skulpt – it is the Javascript implementation of Python that we use in the textbooks.


Just a couple of days later @vsoch and another contributor to the RunestoneServer who turned out to be already know each other found each other on another line of conversation! A couple days later Runestone got 100 times easier to install! Runestone Server has been containerized!! Why am I so excited??

As a small-ish open source project, with a limited number of contributors, it is really hard to provide a server like the Runestone Server that runs under all possible configurations of Windows, macOS, and Linux. Those three variations alone are a lot to keep track of, but when you add in all of the individual customizations that developers tend to do on their systems, the number of possible permutations is HUGE. Docker helps solve that by allowing US to control the environment in which the server runs. Yay! That is a giant sized win for us.

Even if you don’t know any of the details, you’ve probably heard of Docker. Docker is a tool designed to make it easier to create, deploy, and run applications like the Runestone Server by using containers. Containers allow a developer to package up an application with all of the parts it needs, such as database servers, libraries, Python packages, and other dependencies, and ship it all out as one easy to install package.

So, YOU get a virtual environment that runs on your machine that we can specify, that makes it easy for you to install; and WE get a system that lets us control all of the libraries and dependencies so that you can get a running version of our server with a minimum of fuss! If that is not a win-win scenario I don’t know what is.

Wow, that sounds Awesome, how do I get started?

Well, you will still need a little familiarity with the command line. You will need to be able to run some docker and git commands.

For potential developers and contributors to the project, or for instructors who may need to run their own server due to university privacy policies or whatever the process of getting Runestone Server up and running is pretty straight forward. You can see the details here But lets walk through a conceptual overview of the process here.

  1. Install Docker Yes, its a pretty big thing you have to install, but its been done millions of times and it really just works.

  2. Clone the Runestone Server project from github (git clone

  3. Choose one or more books you want to use in your course, or use in development, and clone them into the books folder of Runestone.

  4. If you want to pre-populate users and or instructors you can make a csv file called instructors.csv and another called students.csv

  5. Build the docker container

  6. Run the docker container

The above steps will build the books you want to use in the course, populate a database with all the necessary tables and data to run Runestone as well as any instructors or students you want set up, and then start the RunestoneServer.

Again, detailed instructions are linked above, but it really is a lot easier than it was before. In addition, If you are a developer, you can simply make changes to the code right where you have cloned it and the Docker container will pick up those changes on your next request.

I think this could be a real breakthrough for getting more developers up and running quickly, and reduce support time for people who want to run their own server at their school. In any case I encourage you to give it a try, and please, please do report any problems you run into with getting a server up and running.

Runestone and Open Source

Much of Runestone is built with the help of other open source projects. I like to think of it as the “family tree” of the project. The root system goes very deep as every one of the projects that I mention is built on top of many other other open source projects. We are truly standing on the shoulders of thousands of giants who make make this open source eco system work.

Here is a list of our first level ancestors. I’m sure I’m leaving something or someone out and I apologize if that is you. Please let me know and I’ll update the post.

  • Skulpt - Javascript Implementation of Python

  • Sphinx A tool for making beautiful documentation that is highly extensible.

  • Python Tutor Awesome tool for visualizing code

  • web2py Our web development framework

  • Docker

  • Python

  • ShowEval

  • How to Think Like a Computer Scientist - The original text before we made our interactive edition.

  • The number of Python packages approximately 75 are too many to mention individually.

  • All of the books published on Runestone are open source projects.

Sun, 05 May 2019 00:00:00 -0500 <![CDATA[Patreon: Become a Runestone Patron]]> Patreon: Become a Runestone Patron

As you have seen from the data in previous post Runestone is growing and helping to introduce many students at many institutions to computer science, data science, web programming, you name it. I think we are helping to make the world a better place.

When I left my full time teaching job at Luther College a year ago, it was with the goal of making Runestone my full time work. That is still my goal. Originally I had given myself a year to figure out a business model. Then Google presented one to me. Come and consult with us, they said. Help us with our project. That has been, and continues to be, a great opportunity for me, but it won’t last forever. And, its not really a business model, And I need to strike a better balance between writing books and developing the Runestone platform!

Make Textbooks Free

I could follow a model of charging every student a modest sum of money to access books on Runestone Academy But did you know that about half of the schools that use Runestone are public high schools? That model does not work for public school students, and I don’t want to leave them behind. Runestone has saved students millions of dollars in textbook costs, and I’d like to keep it that way.

I could create a freemium business model, where the basic books are free, but more advanced features and teaching materials would cost the school money. That may still be a path to follow, but that makes the code more complex to check whether I should expose a feature to someone based on some flag that tells me if you’ve paid or not.

So instead, I’m doubling down on the supporter model by starting a Patreon campaign. Patreon is for people like us, for content creators that need support to keep on creating! You can use a trusted website to make a regular monthly payment that will come to Runestone.

We have so much work that we could do to make Runestone better than it is today. In my ideal world I would like to be able to hire students that have been through one or more Runestone courses. How cool is that? You got your introduction to CS through Runestone and now you can help develop the very platform that helped you get a start. At the same time they would get valuable resume building experience!

So, today I’m asking you to please support Runestone. Become a patron. It doesn’t have to be a lifetime commitment, but if Runestone has helped you this year why not help us in return?? Go to our page on Patreon and become a patron today.

Then, make sure you tell in the comments what you would like to see most in the next generation of Runestone tools and books.


Mon, 29 Apr 2019 00:00:00 -0500 <![CDATA[New Feature Friday]]> New Feature Friday

Yes I’m posting this on Monday, but Friday sounded like a better title. Many instructors have asked for the ability to download student data to import into a spreadsheet. This was a feature we removed a couple years back because downloading the raw stream of click information really wasn’t that useful, and we take data security of student information very seriously.

So, this weekend I pushed a new release of the Runestone server that gives you four new reports that you can either view online or download as a CSV file to import into your favorite spreadsheet program.


Click on the Overview Reports link to access

The four new reports are:

  • Count of Subchapter Activities

  • First Interaction with all Activities

  • Last Interaction with all Activities

  • Number of interactions with each Activity

The count of subchapter activities is a good cross check on reading assignments. If a student wants to know why they are not getting full credit, this report will show you how many distinct activities in each subchapter a student has attempted.


Count of unique components a student has interacted with.

As you can see, this report is organized by Chapter and then Subchapter. And then one column for each student. So you can see that btest has been looking at the Classes and ClassesBasics chapters, but hasn’t done any of the work in the AdvancedFunctions chapter. Meanwhile, bmiller has been busy with Conditionals and Dictionaries. Yes, sorting by chapter number would be nice, and will be coming soon, it requires a couple of other pieces to be in place first.

The Number of interactions with each activity takes you one level deeper and shows the list of known activities in each Subchapter and the number of interactions each student has taken with each activity. Compare this next figure with the last. In the previous figure you see that bmiller had 6 interactions in the graphing_with_altair subchapter. Now you see exactly which six components they were, and how many times bmiller interacted with each of them.


Count of interactions with each component.

The First Interaction and Last Interaction reports are just variations on this one, that show you the first time a student interacted with some component. “How come I didn’t get credit for doing this?” Because this report shows that you didn’t do it until 2 hours after the deadline.

All four of these are downloadable by clicking the “Download CSV” button.

These reports came about because I was writing about making pivot tables with Pandas, and I realized how easy it would be to apply my lesson to real life to make some useful reports for you. I know there may be more reports like this that you would find useful, so please feel free to make requests in the comments section, or make an issue on our Github page

Wandering Students

One problem with grading on Runestone is that I have noticed that students don’t always end up doing their work in the right class. Sometimes students don’t remember where they should be and end up Googling their way into Runestone. But often times this googling lands them in the “open to anyone” version of the textbook the instructor has chosen for the course. This means their activities are recorded under that course and not your own.

To combat this behavior I have added two features that require you to rebuild your course to activate.

  • The Navbar for your course should now contain your course name, and not the name of the textbook. Teach them to pay attention to that. We are going to develop more visual cues to clearly differentiate between the open version of a course and a bespoke course this summer.

  • When students are logged we now check to make sure that if they are in a course for which they have not registered we will alert them, and then take them to the course selector to make sure they are in the proper course.

We love to hear from yo, so let us know what you think of these updates!

Mon, 22 Apr 2019 00:00:00 -0500 <![CDATA[A Lab in Three Easy Steps]]> A Lab in Three Easy Steps

This weeks post is an update of a post that has been continually one of the most popular on my Personal Blog So I thought it would be good to update it a bit and publish it here. These instructions can be used to create a lab or any kind of in-class activity that you want to use with your students. Its a good “Getting Started with Runestone” tutorial.

Getting Started

The major steps in getting started are

  • Installing Python

  • Installing the Runestone Components

  • Building your first lab

Install Python

  1. If you are on a Mac you are already done with this step.

  2. If you are on Windows you will need to go to

If you are an advanced Python user you may want to may want to create a virtualenvironment for this project but it is not a requirement.

If you are on Windows you may want to edit your PATH environment variable following the instructions here. Again, Mac users can ignore this.

Installing the Runestone Components

You are going to need to use the command line for the rest of this tutorial, so start up a terminal (/Applications/Utilities/Terminal) on macOS or run Powershell or cmd.exe on Windows) I will repeat myself here. These commands need to be run from the command line, not from the Python shell.

Run the pip command

$ pip install runestone

Or on Windows if you have not modified your PATH try:

C:\Python34Scriptspip.exe install runestone

From now on I’m only going to give the Mac way of running the commands. If you are on windows you will need to add C:\Python34Scripts to the beginning of the command and add .exe to the end.

You can watch as a lot of text goes scrolling by. But as long as you don’t get any errors you should be good to go. You only need to do these first two steps once. Once you have installed Python and Runestone you will not have to do it again.

Starting your first Runestone Project

Here is a session of me on my computer creating a simple project.

$ mkdir mynewproject
$ cd mynewproject
$ runestone init

This will create a new Runestone project in your current directory.
Do you want to proceed?  [Y/n]: y
Next we need to gather a few pieces of information to create your configuration files
Project name: (one word, no spaces): MyLab
Path to build dir  [./build]:
Path to deploy built site  [../../static]:
Use Runestone Web Services  (true, false) [false]:
Your Name  [bradleymiller]:
Title for this project  [Runestone Default]: My First Lab
Use Simple Python3 Semantics  [false]: true
Default ActiveCode language [python]:
Enable inline Activecode downloads by default (single activecode downloads may be enabled with the :enabledownload: flag) [false]:
Done. Type runestone build to build your project

You should answer no to using the Runestone services as at this point you are just making a static page with no ability to capture and store data about your students. You can write Runestone labs for lots of languages, including C++ and Java but Python, Javascript, and HTML are the only languages supported directly in the browser.

At this point you will have the following files and folders:

  • The _static folder is for things like images or Javascript files.

  • The _sources folder is where you will put your own writing. To start with there are a couple of example files for you.

  • The _templates folder is for styling. There is a default set of templates that match the Runestone interactive look and feel. That is a good thing to start with. Once you become more familiar with the system you may want to customize the templates or even make your own.

  • The file is used by Sphinx, and contains information from some of the questions you answered when you initialized your project.

  • The file is used for building and setting build parameters.

All of these files are important, and you should not delete any of them.

Next run runestone build This command will create a build/mynewproject folder with an index.html file in it. If you want you can now run runestone serve and then go to your browser and open up the following URL http://localhost:8000/index.html Yay! You have a webpage. Feel free to explore a bit to get an idea about some of the components you can use in your lab.

Writing your Own Lab

OK, lets edit _sources/index.rst Initially it looks like this:

This Is A New Project

SECTION 1: Introduction

Congratulations!   If you can see this file you have probably successfully run the ``runestone init`` command.  If you are looking at this as a source file you should now run ``runestone build``  to generate html files.   Once you have run the build command you can run ``runestone serve`` and then view this in your browser at ``http://localhost:8000``

This is just a sample of what you can do.  The index.rst file is the table of contents for your entire project.  You can put all of your writing in the index, or as you will see in the following section you can include additional rst files.  those files may even be in subdirectories that you can reference using a relative path.

The overview section, which follows is an ideal section to look at both online and at the source.  It is pretty easy to see how to write using any of the interactive features just by looking at the examples in ``overview.rst``

SECTION 2: An Overview of the extensions

.. toctree::
   :maxdepth: 2


SECTION 2: Add more stuff here

You can add more stuff here.

If you are not familiar with markup languages, this file should still be quite readable to you, and you can probably easily guess what most things do. Runestone uses a markup language called restructuredText. There is a very nice, short tutorial here. But maybe the most important reminder is that restructuredText is like Python, indentation matters.

To give you an idea of what you see in the example above, the section that starts with .. toctree:: is called a directive and it creates a table of contents for you. the maxdepth part sets the table of contents to show sections and subsections. And the line with overview.rst indicates that it is a file that should be included in the overall web page. More on all of this later. Our first task is simply going to be to wipe everything out, and start over. Using a plain text editor change index.rst to look like this:

My Sample Lab

Part 1: Turtle Graphics

In this section we will do the following:

* Create a turtle
* Make the turtle draw a box

.. activecode:: turtle1

   import turtle

   timmy = turtle.Turtle()
   for i in range(4):

Now it is your turn.  Can you modify the program to make timmy draw an octagon instead of a square?

Now save the file and rerun the runestone build command. Everything should build without a problem and you can now run runestone serve and open up http://localhost:8000 from your browser. Notice that you can change the program and rerun it right from your browser.

It is probably obvious that you can create headings and subheadings. Unordered lists are created using * and the runnable code examples are created by the .. activecode:: directive. The name turtle1 must be unique on the webpage, other than that it is not used for too much at this point. The rest of the activecode directive contains plain old python code, but it must be indented to line up with the a in activecode. All indented lines are included as the body of the activecode directive, regular text processing starts at the first unindented line.

There you have it. You have created a very nice little lesson without a lot of hassle. The Runestone and Sphinx tools take care of all of the formatting for you!


There are lots of resources to help you with writing resources with Runestone.

  • Runestone Overview

  • The Authors Guide

  • Using the runestone doc command. Runestone has some basic documentation built right into the command. runestone doc --list gives you a list of all of the various components. You can guess what they do from the name pretty well.

$ runestone doc --list
Runestone Directives List

To get the detailed Syntax and options for any of the components you can run runestone doc <componentname>

runestone doc fillintheblank

    .. fillintheblank:: some_unique_id_here

        Complete the sentence: |blank| had a |blank| lamb. One plus one is: (note that if there aren't enough blanks for the feedback given, they're added to the end of the problem. So, we don't **need** to specify a blank here.)

        -   :Mary: Is the correct answer.
            :Sue: Is wrong.
            :x: Try again. (Note: the last item of feedback matches anything, regardless of the string it's given.)
        -   :little: That's right.
            :.*: Nope.
        -   :2: Right on! Numbers can be given in decimal, hex (0x10 == 16), octal (0o10 == 8), binary (0b10 == 2), or using scientific notation (1e1 == 10), both here and by the user when answering the question.
            :2 1: Close.... (The second number is a tolerance, so this matches 1 or 3.)
            :x: Nope. (As earlier, this matches anything.)

    config values (

    - fitb_div_class - custom CSS class of the component's outermost div

Giving Students Browser Access to the Lab

At this point the build folder has a static webpage that you can upload to any web server. If you have your own webpage hosted on a school server that you normally use for class you can make your Lab available to the students by simply taking the folder mynewproject inside the build folder and putting that on your website. The folder is self contained and can be hosted on any web server.

If you know the IP Address of your own computer and you simply want to give let students bring up the webpage from your computer you can do that too. For example, lets suppose you know that your IP address is Your students can get everything they need from

In another post, I’ll cover how to host your lab using Github pages.

If you have questions or comments on getting going, please leave them below, or join our community on Slack or our Google Group. You’ll find links to these resources on our Support Page

Sun, 14 Apr 2019 00:00:00 -0500 <![CDATA[Runestone Dashboard]]> Runestone Dashboard

I’ve recently started working on a Runestone Dashboard to monitor our progress in terms of the number of students, and traffic, bandwidth, new courses that are created etc. I thought that it would be interesting to share a few of the graphs with you.

We launched in September of 2017 so we are closing in on the end of our second academic year. After 5 years on our old hosting service it was time for a fresh start. This first graph shows you the number of students that have registered each week since we started.


You can clearly see the beginning of semester registration peaks, and of course its exciting to see the year over year growth. Its interesting to note that fall registrations tend to be larger than spring registrations, but that may be because many students register for a course that continues in the spring so there is no need to re-register.

With the number of new courses in our library growing, I was really interested to see what courses people are using the most. This is not necessarily a fair comparison as thinkcspy has been around the longest and is a very popular book in its own right. But its really fun to see how quickly the Foundations of Python Programming book has taken off, since that was just launched in the fall of 2018. The APCS A Review book continues to be a very valuable resource for many schools. This graph is showing only the courses built in the last 14 weeks so it should be a pretty good indicator of what is happening Spring semester of 2019.


Not surprisingly, the number of students registering by base course has the same shape as the previous graph. But this does count students that are studying the material as part of a formal course, as well as independent learners that just sign up for one of the open textbooks that anyone can learn from.


Finally, many people ask about the daily traffic. There are many ways of thinking about this, from the number of unique students each day to the number of pages served each day to the number of student activities per day. this graph shows just the count of student activities each day for the last 14 months.


Its pretty interesting to note our “summer lull” which definitely shows that we are getting most of our use during the school year. No surprise there. Its also amazing to see how much our traffic has grown from Spring semester 2018 to Spring semester 2019. Finally, you can see the orange colored days are the weekends. Is this definitive proof that our students don’t study as much as we would like on the weekends!?

Fri, 05 Apr 2019 00:00:00 -0500 <![CDATA[Book Spotlight: How to Think like a Data Scientist]]> Book Spotlight: How to Think like a Data Scientist

It is said that the most important characteristic of a data scientist is curiosity. So how do you structure a class that encourages students to be curious and ask questions of the data? When I first taught DS-320 at Luther in 2017 I had to make it up as I went. Luckily I had some great, and very patient students that were willing to go with it. My main goal was to “let the data drive the learning!” My vision for the course was to pick some data sets, do exploratory data analysis on them, generate a bunch of questions as a class, and then figure out what we needed to know in order to answer our questions. As a course, I thought this was an amazing and really fun way to structure the learning. It doesn’t lend itself to a structured day by day syllabus since you can’t necessarily predict everything you are going to learn when you start! But we learned a LOT in that class, and we had a lot of fun doing it.

Finding textbooks for undergraduate data science courses is really hard. There is little agreement on curriculum at the undergrad level and definitely not much with a more liberal arts emphasis. So, I was thrilled when the Applied Computing Series team at Google asked me to take the lead on creating a book for their AC201 course.

This book is an attempt to build some structure around the approach described above, without totally killing the spontaneity of encouraging students to ask good questions of the data. You are never going to find a data set to make everyone happy, but if you pick several data sets hopefully enough of them will interest enough students to keep everyone engaged. In this text (so far) we look at World Happiness data, Movie Reviews, the CIA world factbook, United Nations speeches, Bike Rental data from Washington DC and shopping cart data from Instacart.

The learning objectives of the course are as follows:

  • Articulate the data science processing pipeline

  • Extract data using SQL

  • Gather data from the Internet using web API’s and screen scraping

  • combine data from different sources

  • Clean the data

  • Handle missing data/finding outliers/fixing data

  • Normalizing and rescaling data

  • Visualize the data

  • Translate questions to analysis and analysis to interesting stories

  • Analyze data

    • Single variable regression, logistic regression

    • Market basket analysis

    • Cohort analysis

    • Sentiment analysis, exposure to Bayes Theorem

    • Time series

    • Geographic analysis

    • Simulations, Monte Carlo

  • Understand statistical significance and how to test for it using practical simulation techniques.

You can see how the individual skills learned map onto different data sets and chapters by taking a look at the preface

One of the big challenges of this was how to make the book interactive even while wanting students to install and run their own copy of a Jupyter notebook server. The approach is to have the book lead the students through some analysis while asking them to do work in the notebook and bring answers back into the textbook. For example use the notebook to find the busiest bike rental pickup point, and then paste the id of that station into a multiple choice question in the text.

Maybe at some point we’ll have a way to embed Jupyter notebook cells into a Runestone text, but that will require a LOT more computing power.

In the meantime please take a look. The book, as is, has been classroom tested in four schools this spring, but I think there is a lot more content that could be added, and the existing content still needs work to clean up. So feel free to let us know about any issues on github.

Fri, 29 Mar 2019 00:00:00 -0500 <![CDATA[Runestone History and a Roadmap]]> Runestone History and a Roadmap

Runestone Interactive was created in 2011 during Brad’s sabbatical. I should have been working on a new edition for two paper textbooks, but I had the worst kind of writers block. I just couldn’t stand the idea of a paper textbook for computer science in 2011. Textbooks should let you run the examples! Even better textbooks should encourage you to edit the examples and play around with them. When a google search for python in the browser turned up the skulpt project I knew I was onto something.

After spending a couple of months building a turtle graphics module, I realized that nobody would write a book if they had to do a ton of javascript programming for every example. So I started to look around and found Sphinx and docutils. Although markdown is probably more popular, Sphinx/docutils is so much more extensible. So I set out about writing some extensions to Sphinx, and the rest is history. Now adding an example to the textbook is just as easy as copy/pasting the code into the plain text document!

We first used Runestone in the classroom in 2012 for 60 students at Luther College. From 2012 to now Runestone has grown to serving 25,000 students a day around the world at something like 800 institutions. The real surprise came when I discovered that many of them were high schools. This made me very happy !

Our library now lists 18 books! But there are probably at least another 18 that I don’t know about. The number of translations of Runestone books that I have randomly discovered is amazing. That makes me very happy also.

The tagline “democratizing textbooks for the 21st century”, is really inspired by a class visit with Guy Kawasaki in a class I taught during January Term when I would take 12 students to Silicon Valley to visit with entrepreneurs, at all kinds of companies. It is, in Guy’s terms, a mantra. It means that textbooks should be free! You should not be excluded from learning about CS because you cannot afford $200 for a textbook! If Runestone can play a role in disrupting textbook publishing that would be awesome. I’m hoping that Runestone can serve 2 million students a day in my lifetime! It also means that textbooks should be interactive, intelligent, living documents.

In 2018, I decided to leave my dream job at Luther College to focus all of my energy on a new dream job, Runestone Interactive. I was growing increasingly frustrated that there were not enough hours in the day to teach classes, attend committee meetings, grade homework, prep lectures, and work on Runestone. This turned out to be a great leap of faith, as not long after I made the decision I was contacted by a some team members in Google’s EngEDU organization that wanted to use Runestone as part of their Applied Computing Series of courses. The goals of Runestone and the goals of the AC team could not be more aligned. Runestone is also used as a platform for teaching courses by LaunchCode. I get to work with a bunch of really smart Googlers, and have time to continue to develop Runestone.

A Roadmap for the Future

The sign of a good project is that the todo list never gets shorter. Every time I cross something off the list three new things replace it. There is no doubt that with focus, time and energy that Runestone can be way more awesome than it is today. The details and current development priorities are outlined here .

What I am most interested in is creating a sustainable community around Runestone so that it will continue after I am not caring for it every day. This means a concerted effort on funding, on growing the number of students, and building the number of authors and developers.

All of the above has been happening organically, but we need to accelerate on all fronts. This new home page, is one part of that. YOu will begin to see articles detailing development, as well as posts about how people are using Runestone in the classroom in real life. Please share this site with your friends, and colleagues, introduce influencers to Runestone and help us spread the word.

Sun, 17 Mar 2019 00:00:00 -0500