Runestone Partners with Edfinity to bring homework system to CS1

Today I’m pleased to announce Runestone’s partnership with Edfinity, our first commercial partnership! Edfinity provides affordable access to exceptional homework problems and assessments customized for any textbook. While Edfinity has focused on the Math and Physics markets thus far they have chosen to partner with Runestone as they move into the computer science world. Edfinity was created with the NSF support (Awards #1758301 and #1943530) - they are best characterized as a for-profit social venture with equal focus on educational research and educational technology. Since the launch of their Math/Physics homework in Fall 2019, over 125 institutions have adopted Edfinity for one or more introductory courses.

Shivram Venkatasubramaniam, CEO of Edfinity has this to say: “Runestone offers high-quality instructional content on a mature, open platform for introductory computer science that has gained adoption at hundreds of institutions across the US. They have a relentless focus on educators needs with a view to delivering superior learning outcomes, and a vision borne out of their deep experience as educators. We, therefore, view Runestone as a natural complement to the Edfinity homework/assessment platform - together Runestone and Edfinity are very well-positioned to serve the needs of educators and students at scale - not just for Computer Science, but other STEM subjects as well. Runestone’s value proposition as an open, interactive textbook platform applies equally to other STEM disciplines”

In eight years of providing textbooks to the computer science community the Runestone team has learned that even for a free textbook, that is only the beginning of what schools are looking for. Many instructors are looking for a commercial class homework system to compliment the Runestone textbooks. The partnership with Edfinity will fill that void. Instructors, especially those teaching CS for the first time, need pre-packaged assignments and assessments. Over the next few months we will be working with the Edfinity team and the community to develop homework and assessment packs that not only complement the How to Think like a Computer Scientist and Foundations of Python Programming text books, but can be easily paired with any commercial textbook for a fraction of the price. A math educator commented recently - ‘Edfinity is as close to free as you can get for a commercial quality homework system’ - I expect this to be true of the Edfinity/Runestone CS homework offering as well. Longer term we will work together to make the experience of using Runestone with Edfinity as seamless as possible.

As an immediate first step, Edfinity is funding several paid positions for undergrads, faculty and high school students (openings and postdoc - please contact me if you are interested) to help write and test exercises, unit tests, quizzes and exams to accompany the textbooks. I’m really excited for this opportunity to work with the team at Edfinity, it’s clear that they are laser focused on bringing a world class homework and assessment system at an affordable price to instructors that really need it. To see how Edfinity works you can check out some of their math offerings on their website at edfinity.com/catalog or drop a line to Sid and Shivram at hello@edfinity.com.

Runestone and Edfinity will be actively piloting our new offerings with several of you during the coming months, with full availability scheduled for Fall semester 2020. This is designed to be a community-centric effort and we invite your participation and assistance to any degree both in terms of conveying your needs to us, and in helping to build out an exceptional corpus of questions and unit tests. Please contact me with ideas and suggestions.

PreTeXt + Runestone: a New Partnership

I’ve just returned from an inspiring weeklong workshop on Interactive Assessments in Open Source Textbooks, hosted at the American Institute of Mathematics offices in San Jose. There were about 25 participants in the workshop, including, authors, instructors, developers, administrators, and people who wear combinations of those hats. Not only was the workshop inspiring, but the structure of the workshop encourages concrete progress, and I’m thrilled to say that we were able to demonstrate several aspects of a working integration between Runestone and PreTexT. Although our teams have been working in parallel for several years, it is clear that our goals are well aligned. The PreTeXt markup system provides over 50 different online resources to the math community including textbooks for Calculus, Linear Algebra, Discrete Math and much more. Both PreTeXt and Runestone have a lot to bring to a combined table and here are some of the ways that the entire community may benefit from an integrated system:

  • PreTeXt does not have a central hosting system like Runestone.Academy, nor does it collect the data for the kinds of instructor analytics that Runestone does.

  • While PreTeXt makes use of WebWork for interactive exercises WebWork does not have all of the different components that runestone has

  • PreTeXt has a different markup language than Runestone - based on XML - it is not complicated to write and less fussy than restructuredText in the use of whitespace in its Syntax

  • PreTeXt can produce documents in many different output formats including online as well as PDF, ePub, and in-progress work on Braille!

Running a PreTeXt book on a Runestone Server with integration to webwork

Serving a PreTeXt book with a Runestone Server integrating a WebWork question

During the week we were able to make PreTeXt output html that was compatible with several different Runestone components, including activecode, multiple choice, and short answer questions, the rest will follow over the coming months! This means that an author could write in PreTeXt and host a book on Runestone.academy. In fact were were able to demonstrate that we could already host existing PreTexT books, and with a little work we were able to capture student responses from WebWork problems and save them to the Runestone database.

In addition to the integration work, there was a lot of conversation and drawing of pictures and making of lists that revolved around topics that both Runestone and PreTeXt could benefit from. Some of it is great design work, some of it is a vision of how the future could look, But the answers to the questions will provide lots of work for the next few years.

  1. How can we write and present instructor editions of textbooks? What kinds of resources should be included in an instructor edition and how best to expose those materials.

  2. How can we build effective communities around the various textbooks we provide to encourage instructors to help and learn from each other?

  3. What kinds of analytics would be useful for book authors? How can the data collected for authors be visualized effectively to help authors write more effective textbooks?

  4. How can we write questions that can have many equivalent variations, to promote better student practice and learning.

  5. To what extent should instructors be able to customize a textbook? How can we allow customizations without a dizzying and confusing proliferation of minor variations of the text?

  6. How can we collect data to enhance student learning, while still respecting student’s privacy and respecting an increasing number of privacy laws?

  7. What can we do to ensure that creators of open source educational resources get the proper credit towards tenure and promotion for work that has a wide impact? Sure a journal paper that is read by 6 other people is fine, but isn’t a book that educates thousands of students and moves pedagogy forward at least as valuable?

I’m not going to provide answers to those questions here as that work will be ongoing and we could use a lot of help. If you are interested in tackling any of these questions or just participating in the discussion, please let me know!

I also learned about the DOENET project happening across many universities with the project manager located at the University of Minnesota. This would be a longer term integration project that could address many of the privacy issues surrounding open source education materials. And since I’m only seven minutes away from DOENET central by bike, we decided to focus on other integration issues during this workshop, and get together after the conference to start this discussion.

In closing, I want to thank the organizers of this workshop – Jim Fowler, Mitch Keller, Matthew Leingang, and Oscar Levin for inviting me to participate. Thanks to the other participants who made the non-math guy feel extremely welcome! Thanks to Rob Beezer and David Farmer for inventing and writing PreTeXt and being so open to working on integration, even well into the evening after the workshop was done for the day.

I hope someone from the ACM will read this, and look into creating and funding workshops with a similar format. This was 100% a week well spent – Not something you can say after every conference! This workshop was a great beginning and I look forward to seeing what we can do together.

Privacy and Runestone

In this week’s update I want to tell you about some new account management features I added to Runestone, and give you some idea about how you could set up your class if you don’t want Runestone to capture any Personally Identifiable Information (PII) about your students.

Runestone collects a lot of data. I think it is all justified, and if you are an instructor you can gain some real insight into how your students are doing based on the data we collect and the reports we create for you. None of that requires any PII about your students. Yet we do collect some… what do we collect? and why?

Here is what we we ask for when a student registers:

  1. username - this can be anything and it is easy for this to NOT contain PII

  2. email - This is the only reliable way to do password recovery.

  3. First Name and Last Name - this has always been just for the instructors so that when they are grading assignments or looking at analytic reports, they know who they are looking at.

  4. Password - encrypted and stored securely.

  5. Name of the course. The instructor has full control over this call it whatever you want.

Setting up a class with no PII

We will walk through two strategies for managing a Runestone Course that doesn’t store any PII about a student. The first strategy has the students register themselves following the instructions I’ll tell you about. The second strategy puts you in complete control and allows you to create all of the usernames for your students!

The way to use Runestone without PII is to ensure that your students choose a username that they remember but has nothing to do with their real identity. They can also put in fake first names and last names.

The real trick is how to manage password recovery Thats where this cool email alias trick comes in. Most email systems (including gmail) support the username+alias convention. That is you can add a + and an additional alias to your email address. For example runestone+support@gmail.com or runestone+info@gmail.com both of those emails will end up in the runestone mailbox, but I can also see the alias. We can use this feature so that students don’t have to enter their real email address, they enter yours plus their username as the alias. This means that you will end up fielding the password reset emails. If you are an email whiz you can set up some rules to forward these to the right student.

Experience has shown that having students register themselves and get all of those pieces correct is a losing battle. So why not put you in charge of registering them? Get out your favorite spreadsheet app and prepare a file with your students information.

You can upload a CSV file with the following format: One row for each student

username,email,first_name,last_name,password,course

In order to protect your students privacy you can create a spreadsheet like the following:

hogwarts-1,dumbledore+hogwarts1@gmail.com,F1,L1,owls4all,potions_fall19
hogwarts-2,dumbledore+hogwarts1@gmail.com,F2,L2,owls4all,potions_fall19
hogwarts-3,dumbledore+hogwarts1@gmail.com,F2,L2,owls4all,potions_fall19

Note

It is important that you check your CSV file and make sure there are NO SPACES before or after the commas!

From the Manage Students tab on the Instructor’s Page, you can now upload this CSV file using the interface.

../../../_images/student_management.png

Once you have uploaded the CSV, you can add some columns to the spreadsheet that contain the real student information to help you remember. You will probably learn quickly who your made up names correspond to, and can probably think of something anonymous that will help you remember who F1 L1 really is. On the first day of class you may want to hand out paper for the students with their username, and walk them through changing the password, or maybe you want to assign each student a unique password when you are making up the spreadsheet. I just would not recommend a scenario where all the students have the same password.

Of course all of this is optional. You can also create a csv file with usernames of your choice and include the real email addresses and real first / last names of your students, or you can have them register themselves as students have been doing on Runestone since 2012.

Password and Account Management

There is no doubt about it, password management is a headache for everyone. Runestone handles hundreds of password reset requests each week. But that does not always work. Some schools (inexplicably) block email from Runestone. Other schools run software that simply mangles the URL’s found in emails that look like they are password reset emails. This is understandable to some degree as Phishing scams are so common, and its important to protect the students. I’ve done everything I can do from my side to validate the runestone.academy email domain. But, other times students simply type in their email incorrectly.

When the email system for password resets do not work, I have now given instructors the ability to change the passwords for the students in their class. Hopefully, this will eliminate the emails asking me to manually reset passwords when the reset emails are now flowing. Runestone has gotten too big for me to deal with these requests, and I don’t have any minions for that.

Instructors cannot change their own passwords through this mechanism. who among us hasn’t walked away from the computer and left ourselves logged into Runestone? As an instructor, you will still have to use the normal email password reset mechanism. If your school blocks emails then I suggest you just get a free gmail address to use for this purpose.

One final note on account management. When you use the Remove Student button, you are not actually deleting the student. What I do is move the student out of your course so they do not clutter your gradebook or Student Progress page. They are moved into the public version of the course you based your own course on. This allows them to keep their username, as well as retrieve any of their work they did. If a student wants to truly delete themselve they do have that capability! From the user menu select edit profile and from there the student can really delete themselves and all of their work.

What other Data Does Runestone Collect

Here is a graph of all of the events that runestone has collected in the last 4 weeks.

../../../_images/rsevents.png

As you can see that is a lot of activecode and a lot of pages loaded. Also over 1 million multiple choice questions answered! I would love to see more clickable area questions and more fill in the blanks. Some of our preliminary research shows that those kinds of questions are much more effective at getting students to review the content in order to get a correct answer, rather than simply clicking on different answers until they guess correctly for multiple choice. All of these events are captured and tied to a username. So the more anonymous your username the better as far as I’m concerned.

What I’m really interested in is figuring out how to apply machine learning algorithms to help identify students that may struggle, to help weed out bad questions, to help us write better questions, to help us order the textbook in different ways, to adapt to each student to help them achieve their maximum potential.

Runestone also collects the source code each time your student changes and runs the code in an activecode activity. This is to provide a timeline for the student as well as for yourself.

  • Make sure you explain the timeline to your students. Sometimes there is great fear in them about changing code that is working. Having this timeline present encourages them to experiment, knowing that they can always get back to a good working version.

  • For you the timeline can be an invaluable tool for talking through an assignment with a student. You can see their progress and if they make a wrong turn you can usually see that in the timeline and talk to them about why they decided to a particular direction.

  • The timeline is also available in the scratch activecode which some students like to use as a notebook for the course.

Grading Updates

Lots of updates to the Grading interface this week, as well as some important bug fixes. They are now live. The main things to look for include:

1. I have plugged a couple of holes in session handling, where students could mistakenly end up in the open fopp book instead of in the correct course.

2. In addition, if the student is not logged in, the progress bar at the bottom of the page will be gray instead of green. Another visual indicator that they are not logged in.

3. The grading interface has been redesigned into a new layout, and streamlined significantly. Important note – these changes are to make manual grading workflow less tedious. The autograder workflow is unchanged.

  1. Now you only have one choice to make. Start with either a Chapter or an Assignment.

  2. Questions for chapters are sorted alphabetically

  3. Its easy to select all question or all students or both.

  4. Chapters, Assignments, Questions, and students are all searchable. So if you have a class of 125 and need to regrade that one student it is easier to find them.

  5. If an assignment has not had its grades released there is a bright red message to that effect that will remind you to press the button.

  6. If you select multiple students you can navigate from score to score with the tab key and when you update the score it autosaves when you move to the next input field. You’ll know it has saved because it will turn light green.

  7. If you like grading things one at a time, the old save and save/next buttons are still there.

Here is quick video tour of the new features:

../../../_images/GradingUpdates.png

Please don’t hesitate to leave a comment on other things I could tweak to make the workflow more clear or efficient.

Help Wanted - Make Autograding Awesome

What if I told you that just 10 minutes a day for one week between now and September 30th could improve your life dramatically for the entire semester?

../../../_images/unit_tests.png

Did I get your attention? How much time would you save if every programming problem in Runestone had unit tests so it could be auto graded? I know I would save me many hours! In addition my research has shown that students work harder, and end up with much better solutions when an exercise has a good set of unit tests. It seems like a win for everyone! When there is that positive affirmation that you are correct, students build confidence and feel better about doing the work. In addition, all the emails I get asking for solutions will go away because if all the unit tests pass people will be happy that they have succeeded.

Most of the programming exercises for Foundations of Python Programming, as well as those in How to Think Like a Computer Scientist lack good unit tests for the autograder. This is actually an easy problem to solve if we crowd source the solution. If all of us using Runestone in our classrooms write a unit test every day we could have 100% coverage.

Adding unit tests to nearly every activecode is an achievable goal, if we work together to write them. All the cool kids are doing crowdsourcing today, so why can’t we? Here is the best thing, I’ll take a set of tests for any activecode in (nearly) any format. Just send me the title of the book, the activecode ID (shown below the activecode) and a set of assertions, and I (or another volunteer) will get that information in the book. With the new architecture, as soon as I do a rebuild all of us will immediately benefit from the new tests. You can send me an email. You can post it in the comment section of this blog. You can create an issue on github Or you can post the message on our Slack Channel. (See the Support tab for the link to join).

The test you send me could be as simple as.

Dear Brad,

For activecode ac_2_5_blah we should test the following conditions

    for starting input of 44 and 15
    variable foo should have the value of 29
    variable bar should have the value of "hello world"
    The output should contain the word awesome
    the source code should not include the word append

Sincerely,

<insert your name here>

Thats it, we’ll take it from there. You could also make Pull Request on github if you know how to do that.

You do not need to be an expert to do this. You could even make this an assignment for your more advanced students. I promise I won’t yell at you if you leave some corner case uncovered. Someone else will probably think of it. I, and a couple of other folks will act as the editors to make sure we have good tests that cover a reasonable set of cases for each problem.

What does a unit test look like?

Here is the ideal submission. (Again, I’ll take much less than this)

from unittest.gui import TestCaseGui

class MyTest(TestCaseGui):

    def test_XXX(self):
        assertEqual(variable, correct_value, feedback)
        assertEqual(function_call(), correct_return_value, feedback)
        ...

myTest.main()

What about problems without functions?

No problem, you can make an assertion that a variable should be equal to a value just as easily as a function. Now, many problems are not stated precisely in the sense that we have not stated clearly. Store the result in “variableX”. That is easily changed as well,and you get the satisfaction of knowing that your variable name will be used by thousands of students.

You can also make assertions about the output as well as the assertions words that appear in the source code.

For example lets say you want to write a unit test to check that the ever popular “hello world” program works. We will use the getOuput() method to grab the output of the program. Its just a string so we can manipulate it and assert that substrings be in the output!

print("Hello World")

====
from unittest.gui import TestCaseGui

class myTests(TestCaseGui):

    def testOne(self):
        self.assertIn("Hello World", self.getOutput(), )

myTests().main()

Note that the ==== tells Runestone to hide the test code following the ==== from the student.

Looking for multiple answers in the output is easy to do with a number of assertions. So, this would be a good strategy for having them try several test cases. Even when using the input function!

Or, lets say you want the students to calculate the total of a list of numbers but you don’t want them to use the sum function, and you want the result to be stored in the variable total. We also have access to the source through the getEditorText method.

numbers = [1, 3, 5, 11, 25]

total = sum(numbers)

====
from unittest.gui import TestCaseGui

class myTests(TestCaseGui):

    def testOne(self):
        self.assertEqual(total, sum(numbers) )
        self.assertNotIn("sum", self.getEditorText(), "Do not use the sum function")

myTests().main()

This would result in:

../../../_images/more_units.png

What if I have already written a unit test for problem xxx?

Even better! Just send me the unique id for the problem you created, and (if you know it) the id for the problem you started with.

I know we are all super busy, but with just a few minutes from each of us we can really make a huge dent in reducing our grading workload. Think about it this way: when you make the assignment, jot down a few ideas about what you are going to look for when the students hand in their work, or even write up the unit tests in an editor. If you get them to me, and we get them into the book before the deadline, all you have to do is make sure your students run them one time with the tests in place and your grading for that problem is done! I think this can pay for itself with a single assignment!

Thanks in advance for your participation!

Brad

User Interface Polish

In my last update I detailed some improvements to Runestone that had been a long time in the making. Today, I want to share some improvements to the UI that have come directly from new contributors that are also actively using Runestone in the classroom. These are designed to help make your experience using Runestone more pleasant. I’m sure that there is much more we can be doing, but its really great when the changes and requests from from people who are using Runestone every day.

Grading

Self Grading Have your students ever complained that they don’t know what they have left to do to complete either a Reading Assignment or a Problem Set? Probably, have you resorted to running the autograder a few days early and releasing grades so that your students can see what they have left to do? Now students can “grade” their own assignments. At least that parts of it that can be automatically graded.

../../../_images/self_score.png

After the student presses the button they will see their scores and progress for every part of the assignment. If they are not getting full credit for a section of the reading, they can click on the link and finish the activities on that page. If they are not getting credit for a programming problem they can continue to work on it until all the unit tests pass. If a problem requires manual grading it will just show a message “Not Yet Graded”.

Multiple Selections: Have you ever wanted to streamline your manual grading by selecting a bunch of students at once and having all of their code visible rather than selecting them one at a time? You can now do that!

Assignments

Not all of us have a good naming convention in mind for our assignments at the beginning of a semester. So, one simple but popular request is to be able to rename assignments. You can do that now.

../../../_images/rename.png

What can be autograded? When making an assignment, I know this was a bit frustrating. Now the Tree View of the avaialable exercises tells you what kind of thing each question is, AND if there is a checkmark next to it then you know that it will be autograded. Better yet, the system now does a much better job of setting the correct default grading option for you when you add something to an assignment.

../../../_images/problem_type.png

Help Wanted

We know there are lots of coding exercises that could be autograded but are not. This is due to a lack of hours in the day. Almost anyone can help solve this problem. Writing tests is not that hard to do, and I’ll take the tests any way you are willing to give them to me. I’d love it if they were a Pull Request on github, but I’ll take them in the form of emails, or posts on the Google Group, or tweets, or Slack messages or whatever way you want to get them to me. Here is an example of how easy it is to add a unit test to the code in any exercise. Unit Tests for Code . This problem is so solvable if we can get a community effort behind it. If everyone teaching using runestone sent me one unit test a week, think of how much better your lives would be to have more programming exercises that could be automatically graded. – Not to mention how much better your students learn when exercises are automatically graded.

How about offering a little extra credit to your advanced students for writing unit tests to exercises they have already done??

Student Progress

Speaking of student progress a little polish has been applied there too. The “donut charts” are now organized by subsection. And the dropdown menu has Chapter Numbers! This makes it a lot easier to guage what is going on in your class at a glance when you can see the donut charts grouped by sub chapter.

../../../_images/analytics.png

Squashing Bugs

We have also squashed a fair number of bugs this summer. Just two days ago I corrected a really annoying grading problem thats been around for 3 years! I’ve also found ways to eliminate (or ignore) the cruft in our database that sometimes throws off the count of exercises for reading assignments! Little by little we are making things better and more reliable. Please do report any problems that you run into. Again preferably on Github That makes it so much easier for us to track.

Mid-Summer Update

The July 4th holiday is just three days away, which for many of us in the academy marks the midpoint of the summer. To say that we have made some big strides in Runestone so far this summer is an understatement, and they are all coming to https://runestone.academy before the holiday. So, put on your seatbelts and get ready for the ride.

New Stuff for Instructors and Students

There are several big changes to Runestone that should make life better for instructors this summer.

  1. Dynamic Pages – wait, what? I thought Runestone pages were already far from static. You’re right, but you know that course building and rebuilding process you had to go through to create a course? Its gone! We now have a single copy of the most up-to-date version of each textbook that we can serve dynamically for you and your own bespoke course. You won’t really notice anything except that creating a course is now instantaneous, and the URL to your book is now going to be /runestone/books/published/yourbook instead of the old /runestone/static URL.

  2. In page progress indicators Reading assignments are awesome, but they do have some potential pitfalls. Such as students doing work in the wrong course, or not understanding what they need to do on each page. So at the bottom of each page we have added a progress bar to show how many of the interactive activities on each page the student has tried.

../../../_images/progress.png
  1. One scratch activecode per book – One thing I learned last year in working more closely with instructors and having more time to listen to students is that most students were confused by the scratch activecode. In the past we had a different scratch activecode for each page for each student. This was confusing as students use these scratch activecode’s a lot, but were often confused about how to find something they were trying on a page somewhere that they could not recall. Now their entire history of all interactions will be available and scrollable through the history slider. Making history searchable is on the todo list somewhere…

  2. Navigation menu within the chapter – another request that I’ve heard from many of you is to improve the navigation in the book. There are many different opinions on what is “best” in this area, but the one I like the best, and the one that keeps the book pages the most uncluttered, is to have a “This Chapter” menu in the navbar. This lets you quickly move to any page in your current chapter. If you want to change to a different chapter then you can click on the course name and that will bring you to the table of contents for the whole book. So, closer is one click away, and anything outside the current chapter is two clicks away. That makes sense to me. But I would love to hear feedback on this. Nothing is set in stone.

../../../_images/chapter_nav.png

New Stuff for Authors

  1. New activecode language for writing about and working with SQL! Oh yeah, I love SQL! I love writing queries and I love teaching about how to “squeeze information out of a database,” a phrase my masters advisor John Carlis, repeated every day in our databases class. SQL queries are also “autogradeable” with a simle set of asserts. The database engine runs in-browser and is a fully operational version of SQLLite!

../../../_images/SQL_browser.png
  1. New Spreadsheets directive When trying to write a data science book last year, it was clear that both SQL and a Spreadsheet in the browser would really help. Now, thanks to the jExcel Project We also have a very functional spreadshet running in the browser. Many many Excel/Sheets functions are supported, and it is also autogradeable through a series of assert statements. In addition, cells that students should be adding formulas to are colored in light blue to start with, and after grading are colored green or red to indicate whether they got it right. In addition there is text that also shows whether each test passed or not, and what the expected value was.

../../../_images/sheets.png
  1. New Instructor Only Reveals So many people have asked for instructor materials for all of our books. One of the things the last year of working with Google has helped with is in actually getting some materials written. Its at least as much work as writing the book itself! So how to deliver these new materials? This is one of the first great features of the new dynamic pages model of serving. Buttons that are only included on the page if you are an instructor.

../../../_images/instructor_hidden.png

When you click on it the answer or any other helpful material is displayed.

../../../_images/instructor_reveal.png

Students will not even see the buttons. In some books these instructor materials will be “paid for” features. You will also need to do an extra level of proof that you are an instructor in order to gain access to these.

Inside Baseball

  1. Unit testing means better reliability and more agile development. We have invested a lot of time this summer in getting a good unit testing framework set up for the Runestone server. I’ve spent some portion of each day writing one or more unit tests. For the functionality required to serve books and collect data reliably we now have excellent test coverage. Not as good for some of the instructor supporting pages, but we are working on it, and they get a lot less traffic than the book pages. I’ve also done a lot of load testing on this new architecture and it seems that it should hold up quite well.

  2. New dynamic architecture for more dynamic features in the future. In addition, the new dynamic architecture changes how we check that login/logout is handled. This should help with making sure that students are always in the course they should be in, rather than some random course they found with a Google search.

  3. http://interactivepython.org is gone. Well its not really gone, it has a permanent redirect to https://runestone.academy. None of the old classes on interactivepython.org made the move. Registrations have been disabled for a year. This move has been coming for two years, so hopefully it doesn’t surprise anyone.

  4. Both RunestoneComponents and RunestoneServer are running Python3. We had to keep Python2.7 a lot longer than I would have liked due to the web2py framework, but happily web2py now fully supports Python 3.7. I look forward to simplifying and dropping support for 2.7 once all this new stuff is live on academy.

Caveats

That is a LOT of change! And, I plan to deploy all of these new features over the next week. Even with all of the testing we have done there is bound to be something we have missed that will break something. Please be patient, and please report it on our GitHub page!

  1. If you have an existing course, in progress, on Runestone Academy that course should continue to work just fine. You will not be able to rebuild it as that functionality is gone.

  2. Newly created courses will use the new dynamic pages feature. This means you don’t need to rebuild, you will automatically get updates that fix typos or squash bugs.

  3. I’m introducing these changes during our summer “downtime” although I fully realize that our summer downtime is now busier than our peak traffic the first couple of years! This means that by the time classes start for people in the Northern Hemisphere in late august – the majority of our users – things should be in good shape.

SQL

Today, I built something I’ve been wanting to add to Runestone for several years. I added support for the SQL language. I’ve been holding off on adding support for it, because I thought I was going to have to support SQL as a server side language rather than in the browser. This means a lot of thinking about security, and making sure that one student can’t wipe out the data in a test database for all students, and setting up accounts, etc. Every time I’ve taught introduction to SQL we get to that point in the semester where I know I have to keep a close eye on the server as students begin to write bad cartesian products and create result sets with millions of rows! This effectively slows down or stops the database engine for the entire class.

However, last week, as I was thinking about it again I stumbled across the SQL.js project This project takes the SQLite C code, and compiles it to Webassembly. Thus making it possible to run an authentic version of SQLite in the browser! SQLite is a good tool for teaching as it is lightweight, comes with Python, and runs small databases directly in memory.

From a teaching perspective this really simplifies things a lot! Now each student has their own copy of a database that I can distribute to them over the Internet. Students can’t mess upanyone else’s results or hang the server. If they really do mess up a query they just need to reload the page. In addition, this also means that I can add autograding of SQL queries. As I begin to work on my Web Development course for next fall, and we begin to revise our intro to data science course, having SQL in the browser is going to be a game changer!

Its available today in the github version of Runestone.

Stay tuned.

On Runestone for Intro Courses

By Jackie Cohen

I began using Runestone as a computer programming course instructor in 2014. In the ensuing five years, I’ve taught a variety of programming courses, with a variety of levels and audiences, which used a variety of Runestone books and features – more and more, as more have been developed – and I have worked on some Runestone features myself.

This means that I’ve explored Runestone in a variety of ways, and for a variety of reasons. Many Runestone Interactive materials, and features available in the platform, have proven particularly valuable to me in one type of use case: using a Runestone textbook as the source material for an true introductory programming course, where the course is primarily made up of students who have never done any computer programming before.

This is the kind of use case I’ll share some things about here: an example of which material(s) I’ve selected for intro courses, why I think Runestone is a useful choice for intro programming courses, and particular ways that I’ve relied on a couple of Runestone features to help me achieve pedagogical goals in my class.

Of course, “an introductory programming course” can look like many different things, for many different reasons, and have many different audiences and structures, but here I am speaking generically from my experience being an instructor of college-level students who have never programmed before, enrolled in a class of between 50 - 250, as our course sizes vary from semester to semester!

(Note also that while I refer to a bunch of ideas about pedagogy and what students struggle with, below, what I’m talking about here are all anecdotal experiences of my own as an instructor – there is a lot of great research about related things as well if you’re looking for more detail on computer science, engineering, and programming education.)

So - Which Runestone Book(s)?

The introductory courses I have usually taught over the past several years in my department are taught in the Python programming language. I’ve used a couple different versions of course materials over the years, but for the purposes of this explanation, I’ll examine a case where I choose Foundations of Python Programming as the course textbook.

I’ve also used selections at times from other Runestone materials, which is a nice backup for a time that one book has, say, 98% of what I want.

Why Runestone?

A primary highlight of selecting a Runestone-based textbook as my course textbook is the stability it offers my class, on a few different axes, and this is why I’ve chosen to use it repeatedly.

First, and primary to my choice of Runestone for introductory classes, is the fact that using Runestone to learn computer programming material does not involve any installation or dealing with the varieties of operating systems and software, but it does require learning new vocabulary and practices for saving and running code.

I find that installation is a huge barrier for many introductory programming students. It can be difficult, time-consuming, and annoying. The difficulty or annoyance of installation of a programming language interpreter, a text editor, a command prompt software, or anything else, can feel like proof that a student new to these ideas “doesn’t belong here [in this class]”.

Students who have never done any computer programming before, especially students who do not have a lot of experience with software tools or other types of computing, often struggle (as many computer-types do for a long time into our careers, really!) with the difference between a problem that requires struggle and creative problem-solving, and a problem that is not truly creative or “computationally challenging” and is rather caused by an error you can’t control, or by forgetting to click a single button at the start of the installation process. And for large courses without a large staff of a TAs/et cetera, helping students complete installations with a ton of different systems and versions thereof can also be a huge pain, and not a great note on which to start a class – plus it’s difficult to balance this fact with the desire to start writing code and seeing things happen on your screen rigt away. Using a Runestone textbook solves these problems, and allows students to focus both on concepts AND upon the joy of seeing code “work” and even pass tests – on the in the very first week of class.

Secondly, everything is in one place, rather than having a plethora of blog posts, links, a textbook, code files… all from Day 1. While I do integrate non-Runestone resources into many such courses, the Runestone textbook is always “home” for students, who often express (in my experience) that they feel overwhelmed by having a lot of new material in programming as well as a lot of new resources to use.

Using a Runestone textbook also allows me to integrate graded work of various types with un-graded work, helping me to emphasize during class that learning to program usually requires a lot of practice. If such and such thing is required/graded, but there’s also A, B, and C helpful conceptual activities we will discuss in lecture next week… isn’t it useful that they’re just right there, too? A simplification, of course, but something I’ve valued especially when students in class don’t have a lot of experience learning a programming language and are still adapting their own learning and practice expectations for the course. What they need at first is all there on one website.

(In some courses I’ve moved “off” Runestone in a second half of the course, e.g. to using files and command prompts, but the Runestone experience they have is something they can continue to rely on and can use for practice, and remains a way of gaining new information about syntax and concepts since it is still our course textbook.)

What’s in Runestone?

With these ideas in mind, there are a few Runestone features that I’ve relied on heavily during introductory classes like those I describe.

One is, of course, the ActiveCode feature – we can write a program on the first day of class, and begin to understand the process of writing, thinking about, and running code, and how to see and check the results of code that’s been run. The CodeLens tool is a huge help with this, because explanations that I previously needed to include in additional resources or slides are built right into readings, so class time can focus on in-depth exploration that supports understanding even more.

Another is the ShowEval Trace and ShowEval Replace modes – as we begin writing code, I like to ensure I have a focus on understanding complex expressions and debugging, and this feature is a nice interactive way to focus specifically on evaluation of complex expressions without generating them correctly prior to evaluating them.

The availability of the Parsons Problems for a variety of reasons has proven valuable in my introductory courses – one way I like to use these features in class is that after I’ve, for example, assigned these as part of readings before class, I can return to them during a lecture so we can analyze together what ways of solving them are productive (rather than simple trial-and-error, for example), and we can really dig into useful ways of practicing and learning programming so students can experience them with me. I’ve used Parsons Problems both with code, like in the code based example at the bottom of this page, and with non-code words, like in the non-code example at the bottom of this page, and find a mix of the two helpful.

And finally, because I can assign reading assignments as part of the course, and I can choose a book with a structure I want (e.g. the way FOPP has chapters followed by exercises and sets of “chapter assessment” exercises) I can take advantage of the above to include all of those features (and others) in assignments/course work/student lab time/office hours. Students know that they will need to interact with these interactive features during the out of class time they spend reading, and we come back to the online course “home” to review and explore new concepts during class time.

When it comes time to introduce native programming on one’s own computer in the introductory courses that I teach, there’s a place to return to for conceptual things (the online Runestone textbook!), so it allows me in-class time to focus on identifying, and then solving, those problems that do come from ‘simply’ forgetting to click one button or remembering the type of your operating system – since identifying that kind of struggle/problem is also a very useful skill to identify and develop while students begin learning programming.

Docker: an Open Source Story

I love the open source world! You never know what amazing new project you will find, or who you will meet that is willing to contribute some amazing new skills to your project. This last week provided a great example of the kinds of synergies that can happen here. It started with a very short conversation on another project I’m involved in called Skulpt – it is the Javascript implementation of Python that we use in the textbooks.

../../../_images/happy_meeting.png

Just a couple of days later @vsoch and another contributor to the RunestoneServer who turned out to be already know each other found each other on another line of conversation! A couple days later Runestone got 100 times easier to install! Runestone Server has been containerized!! Why am I so excited??

As a small-ish open source project, with a limited number of contributors, it is really hard to provide a server like the Runestone Server that runs under all possible configurations of Windows, macOS, and Linux. Those three variations alone are a lot to keep track of, but when you add in all of the individual customizations that developers tend to do on their systems, the number of possible permutations is HUGE. Docker helps solve that by allowing US to control the environment in which the server runs. Yay! That is a giant sized win for us.

Even if you don’t know any of the details, you’ve probably heard of Docker. Docker is a tool designed to make it easier to create, deploy, and run applications like the Runestone Server by using containers. Containers allow a developer to package up an application with all of the parts it needs, such as database servers, libraries, Python packages, and other dependencies, and ship it all out as one easy to install package.

So, YOU get a virtual environment that runs on your machine that we can specify, that makes it easy for you to install; and WE get a system that lets us control all of the libraries and dependencies so that you can get a running version of our server with a minimum of fuss! If that is not a win-win scenario I don’t know what is.

Wow, that sounds Awesome, how do I get started?

Well, you will still need a little familiarity with the command line. You will need to be able to run some docker and git commands.

For potential developers and contributors to the project, or for instructors who may need to run their own server due to university privacy policies or whatever the process of getting Runestone Server up and running is pretty straight forward. You can see the details here But lets walk through a conceptual overview of the process here.

  1. Install Docker Yes, its a pretty big thing you have to install, but its been done millions of times and it really just works.

  2. Clone the Runestone Server project from github (git clone https://github.com/RunestoneInteractive/RunestoneServer.git)

  3. Choose one or more books you want to use in your course, or use in development, and clone them into the books folder of Runestone.

  4. If you want to pre-populate users and or instructors you can make a csv file called instructors.csv and another called students.csv

  5. Build the docker container

  6. Run the docker container

The above steps will build the books you want to use in the course, populate a database with all the necessary tables and data to run Runestone as well as any instructors or students you want set up, and then start the RunestoneServer.

Again, detailed instructions are linked above, but it really is a lot easier than it was before. In addition, If you are a developer, you can simply make changes to the code right where you have cloned it and the Docker container will pick up those changes on your next request.

I think this could be a real breakthrough for getting more developers up and running quickly, and reduce support time for people who want to run their own server at their school. In any case I encourage you to give it a try, and please, please do report any problems you run into with getting a server up and running.

Runestone and Open Source

Much of Runestone is built with the help of other open source projects. I like to think of it as the “family tree” of the project. The root system goes very deep as every one of the projects that I mention is built on top of many other other open source projects. We are truly standing on the shoulders of thousands of giants who make make this open source eco system work.

Here is a list of our first level ancestors. I’m sure I’m leaving something or someone out and I apologize if that is you. Please let me know and I’ll update the post.

  • Skulpt - Javascript Implementation of Python

  • Sphinx A tool for making beautiful documentation that is highly extensible.

  • Python Tutor Awesome tool for visualizing code

  • web2py Our web development framework

  • Docker

  • Python

  • ShowEval

  • How to Think Like a Computer Scientist - The original text before we made our interactive edition.

  • The number of Python packages approximately 75 are too many to mention individually.

  • All of the books published on Runestone are open source projects.