Miller Named Distinguished Member of the ACM
Runestone Academy founder and president, Dr. Bradley Miller, has been named a Distinguished Member of the Association for Computing Machinery (ACM) for his contributions to the development of recommender systems for eCommerce and interactive textbooks for computer science and mathematics. The ACM is the world’s largest educational and scientific computing society, and this honor is given to only 1% of its members. Dr. Miller was recognized for his work in the development of the Runestone Interactive platform, which has been used by over 1 million students and teachers around the world to learn computer science. He has also been a leader in the development of open educational resources (OER) and has been a strong advocate for the use of technology in education.
“Each year we look forward to selecting a new class of ACM Distinguished Members from among our worldwide association of 110,000 colleagues,” explained ACM President Yannis Ioannidis. “ACM’s motto is ‘advancing computing as a science and profession.’”
I am really honored to be recognized by the ACM for my work in computer science education and open educational resources. I have been fortunate to work with many talented people over the years, and this award is a reflection of their hard work as well. I am especially grateful to the Runestone team for their dedication and commitment to making computer science education accessible to all. As a Professor Emeritus of Luther College, I’m especially proud of this honor; if you look at the other 55 people named this year they are from Google, Microsoft, and lots of prestigious research universities around the world. I am proud to be a part of this community and to be able to contribute to the advancement of computing as a science and profession. I look forward to continuing to work with my colleagues in the field to make computer science education more accessible and effective for all students.
You can read more about the announcement On the ACM Website.
DDoS Update
The Distributed Denial of Service (DDoS) attack is ongoing. Even after I moved our load balancer to new hardware and a new IP address. This means that the attackers are not just targeting the IP address of the load balancer, but they are targeting “runestone.academy” by name. This is a much more serious attack and it is taking a lot of time to mitigate.
I’ve learned a lot about how to mitigate these attacks and I’m implementing the solutions as quickly as I can without causing a lot of downtime. I’m sorry for the inconvenience this is causing you. I think are to the point where we have minimized it. When a new wave of attacks hits it just takes a minute or so to block the attackers. So, just keep that in mind, take a breath, wait a minute and refresh your page. I’m also sorry that I have to spend so much time on this instead of working on new features for Runestone.
These kinds of sustained attacks do not come for free. There are literally thousands of IP addresses involved over a period of days. This is not just a kid in a basement with a botnet. This is a reasonably well-funded and well-organized attack. I’m not sure what the motive is, but it is clear that they are not going to give up easily. Could it be an evil publisher trying to squash free textbook distribution? Could it be one of our “competitors” trying to take us down? Could it be a foreign government trying to disrupt education in the US? I don’t know. I do know that I’m not going to give up easily either.
The cartoon at the top was generated yesterday by DALL-E. It is a cartoon of me wearing many hats. I thought it was appropriate for the situation. GitHub co-pilot suggested that I should be wearing a tinfoil hat. I’m not sure that would help. It also suggested that I should write that I am working with the NSA, CIA, and Department of Homeland Security to track down the attackers. That is certainly not true. I’m just a guy trying to keep the Runestone servers up and running.
I also want to assure you that, so far, the attack has been limited to, and stopped by the load balancer, our application servers and database are secured behind the load balancer and an additional firewall. So, even if the load balancer were to go down, the application servers would still be up and running. Yes, timeouts of API calls to save data to the database might happen, but you should get a dialog if that happens, so that you can reclick the button and have it save the latest answer again.
Again, please remember to check the Runestone Status Page if you are having trouble accessing the Runestone books. It updates every 5 minutes and is served from a completely different server, so is unaffected by the DDoS attacks.
Distributed Denial of Service 2024
Well, Fall certainly started off with a bang this year. Registrations are ahead of last year, web traffic is also way up, and we are seeing a lot of adoptions of new books in the Runestone ecosystem.
We are also seeing a lot of new attacks. Specifically, we have been subject to a Distributed Denial of Service (DDoS) attack each day. This is where a large number of computers are used to flood a target with traffic, causing the target to become overwhelmed and unable to respond to legitimate requests. For example yesterday from about 7:30AM Central to 1:30PM Central, we had over 5 million bogus HTTP requests sent from around 6,000 different IP addresses. All were identical trying to send some kind of binary request. A little research indicated that they were probably trying to create some kind of buffer overflow in the web server that would allow them to gain access.
The attack stopped on its own yesterday afternoon. But then it started up again this morning, though not with as much intensity.
We have been able to mitigate the attacks by blocking the IP addresses of the attacking machines, but it is a constant game of whack-a-mole. Thankfully, there is an automated system called fail2ban that can help us with this. Until we get it fine-tuned there is a possibility that some legitimate users may be blocked. If you are having trouble accessing the Runestone books, please let us know.
We have also been working with our hosting provider to help us with this. They are supposed to provide some level of protection against DDoS attacks, but they claim that it only applies if the attack is against a large part of their cloud.
I’m not sure why we are being targeted. Maybe it is just mischief, maybe we just got caught up in the middle of a range of IP addresses being attacked, We don’t have any money to pay a ransom and it we haven’t been asked to. No matter the motive, it is a good reminder that the internet is a dangerous place and that we need to be vigilant in protecting our systems.
If you have trouble accessing Runestone and are wondering if it is just you or if the site is down please check The Runestone Status Page
The top bar shows the current status of the Runestone servers. If the bar is green then everything is good. The orange-red bars crossing the green indicate times when the site was not available to the majority of 7 severs around the world that check periodically. the wider those bars, the longer the site was unavailable. The vertical green bars are the “Appdex score” which is a measure of how responsive the site is. The higher the better. This page is served completely independent of our servers so if you can’t reach the Runestone site you should still be able to reach this page. Also, please know that when access to the site is not perfect I get notified by text and email so I can jump in and try to fix things.
Hopefully, things will smooth out now and you can focus on teaching and learning! Thanks for your patience.
Brad
Runestone Origins
I come from a long line of educators, my father, and both of my grandmothers were teachers. I was lucky enough to spend time with each grandmother many summers during my childhood years. Each of them loved to bring me on adventures. My grandmother on my mother’s side was a teacher in rural Minnesota. She took me to see the Petroglyphs and to Pipestone National Monument, and many other places. My grandmother on my father’s side was a teacher in a one room school house in Northern Minnesota. She would take me to school and give me a nickle to say the pledge of allegiance. She would also take me on other adventures, including going to see the Kensington Runestone near Alexandria, MN. I was always fascinated by the story of the Runestone and the mystery of its origins, and of course Vikings! I have always loved to learn and to teach and I think that my grandmothers had a lot to do with that, both of them helped me become an early reader and supplied me with lots of books. I hope that you enjoy learning and teaching from the Runestone books as much as I have enjoyed creating and maintaining Runestone Academy.
A few days ago I got to revisit the Runestone when we visited our friends who live near Alexandria. Here I am on the site where the stone was found, entangled in the roots of a poplar tree.
The Runestone is a 200 pound slab of rock that was found in 1898 near Kensington, MN. The stone is covered with runes that tell the story of a group of Norsemen who came to the area in 1362. The stone is controversial and many people believe it is a hoax. The stone is now on display in the Runestone Museum in Alexandria, MN. I’ve never understood why an illiterate farmer would go to the trouble of carving a 200 pound stone with a story that no one could read. I prefer to think that the stone is real and that it tells the story of a group of Norsemen who came to Minnesota 130 years before Columbus.
We are Hiring
Are you passionate about computer science education? Do you want to “democratize” textbooks for the 21st century?
We have grant money to hire a half time developer/devops/support person. The possibilities are endless! You would get to work with me and other volunteers in the Runestone community to help make Runestone better.
Help out with bug fixes and/or feature requests for Runestone Academy
Answer first line support emails from instructors
Help Monitor Google groups that support our books and proactively address questions from instructors.
Build and push weekly updates to our docker registry
build and push weekly book updates to our servers
Help develop tests for key parts of the system
Should be proficient with at least some of the following: Python, SQL, Javascript, React, Docker. If you know Kubernetes and could help us migrate from docker compose to Kubernetes that would be super cool. You must be able to work efficiently from the command line. Must be able to work independently and be self motivated.
If you are interested fork our repository and submit a pull request with a fix or a feature request. This is your chance to impress me. You can find our repository at https://github.com/RunestoneInteractive/rs Our live site which serves about 90,000 students each semester is at https://runestone.academy
Runestone Academy LTD is a 501(c)(3) non-profit corporation. We are an equal opportunity employer. We are committed to diversity and especially encourage members of underrepresented communities to apply.
Summer Developments
The Runestone developers have been busy this summer. Here are some of the highlights:
Beta of new Assignment Builder
We have been working on a new version of the Assignment Builder that will allow you to create questions without knowing RST or PreTeXt. Its completely rebuilt from the ground up and is much more user friendly, using modern web technologies like React and Redux. We are looking for people to try it out. The current assignment builder will remain available, and contains a link for you to try the new version.
Real Time Score Updates
We now detect when a student has submitted an answer to a question that is part of an assignment. When they submit the answer, we will update the score in the grade book in real time. This is a feature that has been requested by many of our instructors and students. Students will see their current score on the assignment page as well as the assignment selection page. You will also notice that most questions now have an extra little icon to indicate that the question is correct or incorrect. This makes the questions consistent. We plan to also add an icon to indicate that the question is part of an assignment, but thats not ready yet.
Peer Instruction Improvements
Peer instruction now has a great new instructor page for you to view a summary of what happened during peer instruction. You can see how many students answered each question, and how many students got the question correct. You can also see how many students answered each question correctly on the first try. This is a great way to see how well your students are understanding the material. We have also added an option to allow the students to record who they talked with if you don’t want to use the chat feature on the web page but would prefer your students talk to each other face to face.
Chapter Activity Report Addition
Another feature that many people have asked for is a simple summary to show how many of the activities on each page a student got correct. This is now available in the drop down menu of Chapter Activity Report called “Number of activities Correct”
Book Discussion Forums
On the home page for your course, you may see a link to a discussion forum for the book the course is using. These links are only available if you are an instructor, and we encourage you to use them to ask questions, and share ideas and materials with other instructors using the same book. In the coming years we hope to build a community of instructors around each book, and provide more structured ways to share materials, such as quizzes, exams, assignments, slides, and other materials. We know we don’t always provide everything you need, and we hope that by building a community of instructors around each book we can help you find the materials you need. These forums will only work if you participate, so please join in and help us build a community around the books you are using.
Using LTI?
If you are using LTI you will notice a big change that we hope you appreciate. Instead of showing the assignment list to students, they will see a message reminding them that they need to launch the assignment from your LMS. In the past this has been the source of much confusion for students, and we hope this change will make it more clear to them that they need to launch the assignment from your LMS.
Textbook Improvements
Many textbooks have had lots of improvements as well. More of the books have been converted to PreTeXt and have a more modern look and feel, and we keep adding new textbooks all the time. We have added many math books at various levels, We encourage you to tell your Mathematics friends to check us out. We encourage you to tell all your friends to check us out.
Bug Fixes
There have also been many bug fixes and improvements to the system. We are always working to make the system better and more reliable.
More to Come
We have many more features in the works, such as overhauling the grading page, and providing better summary reports about assignments. We are always looking for feedback from our users. If you have a feature you would like to see, or a bug you would like fixed, please let us know. We are always looking for ways to make the system better. Join us in our Discord channel. We would love to hear from you.
Collaborative NSF Grants Awarded to Runestone Academy, Michigan, Duke, Virginia Tech, and Berea and American Institute of Mathematics
Runestone Academy LTD got good news this week. We received our first collaborative NSF Grant! The grant (#2336758) Testing the Effects of Peer Discussion is a collaborative research project with the University of Michigan, Duke University, Virginia Tech, and Berea College! The grant will allow us to study the effects of different modes of discussion during peer instruction sessions on student learning in computer science. As I was writing this post I got a second email from the NSF indicating that a second proposal had also been awarded!! (#2334768) Collaborative Research: PreTeXt-Runestone: Open Textbooks Engaging Undergraduates in STEM which is a Collaboration with Mathematics educators at the University of Michigan and the American Institute of Mathematics.
We are excited to be working with two amazing teams of researchers and educators. We are also excited to be able to provide a platform for this research.
Peer Instruction
The first grant will allow us to improve the Peer+ platform on Runestone Academy. The grant will fund the following activities:
Improve the A/B testing mechanism on Runestone to better compare synchronous text-chat versus verbal discussion in the context of Peer Instruction (PI).
Add hundreds of existing questions for CS2 (in Java and C++) and discrete math to Peer+ on Runestone Academy.
Test the use of Large Language Models (LLMs) to improve the interactivity of the asynchronous text-chat.
Test Peer+ in CS1, CS2, and discrete math courses at the four institutions mentioned earlier.
Improve the instructor interface to reduce the time to find existing PI questions.
Improve the instructor interface for authoring new PI questions.
Conduct an instructor workshop to improve knowledge of PI and best practices.
Add additional instructor support to help writing PI questions for ebooks.
PreTeXt and Runestone for STEM
The second grant will advance our knowledge of the process of developing content for three novel interactive questions for four PROTEUS textbooks (PreTeXt- authored textbooks hosted in HTML format on the learning analytics platform Runestone Academy) and their use in classrooms. The project seeks to answer the following questions: How do groups of textbook users (students, teachers, authors, and researchers) work together to write interactive questions designed to elicit multiple ways of students’ thinking in classrooms? and How do students and instructors use those interactive textbook questions in their classrooms? The interactive questions are (i) Short Answer free response, (ii) Matching Exercises, and (iii) Parsons Problems. These will be embedded in textbooks for calculus and linear algebra (Active Calculus, Boelkins, 2022; APEX Calculus HTML Edition; Hartman et al, 2022; First Course in Linear Algebra, Beezer, 2022; and Understanding Linear Algebra; Austin, 2022), chosen because they vary in their support for active teaching, use of visualizations, and emphasis on proof. This grant also provides support for Runestone Academy to make a new hire in the area of development operations and technical support! Yahoo!!
Summary
The process of getting these grants was no small feat! Writing the grants was the easy part. Then, back in December, we learned that both had been recommended for funding. But that was just the beginning. In order for Runestone Academy to be awarded the money, we had to go through a rigorous review process to demonstrate that we were capable of managing the grant. This forced us to “grow up” a bit and add in more processes and procedures to our small operation. It’s all about checks and balances in the accounting world! To be clear, Runestone has been a part of several NSF grants in the past but as a contractor, not as a collaborating Principal Investigator.
Once we passed the “New Performer” process then we just had to wait. As you may know the NSF budget was held up in congress for a while, and funding is tight. We were not sure if we would get the grant or not. But this week we got the good news! We are excited to be able to start the project and to be able to work with such an amazing team of researchers and educators. We are looking forward to the results of the study and to sharing them with the community. This is a real boost for Runestone Academy and all of our students, teachers, and collaborators! Hopefully this will help demonstrate our legitimacy and help us to secure more funding in the future.
SIGCSE 2024: How do you do it?
A second SIGCSE is in the bag for Runestone as an exhibitor. It was a good year. It was amazing to see so many people who have been using Runestone for years and to hear their stories. The number of people who stopped by the booth to just say thank you was really overwhelming! I hope you all know that your gesture will keep me going for months! It was also great to see so many new faces who were just learning about Runestone. I am always amazed at the number of people who have never heard of Runestone before. I guess that is a good thing, it means there are still lots of people to reach.
It was great to connect with so many colleagues from years past. I was able to catch up with people I have not seen in years. I was also able to meet some new people who I have only known through email, Zoom, or Discord. It was great to put real live faces to names.
It was so interesting to hear the questions that people asked. This year felt different than years past. The top questions were “How do you do it?” “How do you keep Runestone going?” “How do you keep it free?” “Are you really free?”” And, “do you have a book that covers X?” But the question I want to address in this post is how do we keep it going? I understand that in this day and age the assumption is that “if it’s free online then the user is the product.” But that is not how we roll, that is not our mission, that is not our model. We are a non-profit and we are here to serve the community. We are not here to make money off of you or your students. We are here to help you teach and learn computer science and math. Our North Star is to make computer science and mathematics education accessible to everyone.
We had answered that question so many times that we determined that we needed to create a new table banner for next time. We also decided that we could have a tongue in cheek answer for the rest of the conference, so Rob created a tip jar. We had a lot of fun with that! Some people were so amused that they threw in a dollar! We are not going to get rich off of that, but it was fun.
Here is the answer to the question “How do you do it?” We are a non-profit. We are funded by donations, grants, memberships, and ads. We have a small team of volunteer developers who put their effort into Runestone. We have a medium community of authors who help us keep Runestone up to date. We have a large community of students and teachers who help us find bugs and suggest improvements. Yes, professional copyeditors are great, but the masses of students and teachers are even better.
A second part of the solution is to serve ads through the Ethical Ad Network. We don’t like ads in general but the Ethical Ad Network is great because they don’t do any tracking! Also, we don’t show ads to students who are registered for a course.
Finally, we do have a Patreon account, and a Github supporter account, and of course you can donate directly We would love to have your support! We are always looking for new ways to fund Runestone, if you are independently wealthy and want to fund us, we would love to talk to you! If you have a contact at a foundation that might be interested in funding us, we would love to talk to them! If you have a contact at a company that might be interested in funding us, we would love to talk to them! We are always looking for new ways to fund Runestone.
Over the long term we are looking for solutions that will enable us to remain true to our mission of making computer science education accessible to everyone. We know there are many commercial offerings that have been free and then pulled the rug out from under all of the schools that could not pay. That is not who we are. We are committed to keeping the content and as much of the instructor interface free as we can, while planning for the day that Rob and Brad decide to step aside and really retire. It might mean a freemium model, it might mean a subscription model, it might mean a different model. We are open to ideas and suggestions as long as we can remain true to our mission.
In the meantime, we are always looking for new contributors. If you are interested in helping out, please let us know. We are always looking for people to help us with content, with the platform, with the community, with the business side of things. We are always looking for new ideas and new ways to make Runestone better.
Roadmap 2024
A lot of the podcasts I listen to, and quite a few of the blogs I read, encourage you to pick a touchstone word for the year. I’ve never done that before, but I’m going to try it this year. My word for 2024 is “Simplicity”. I’m going to try to use this word to guide my decisions and actions for the year, both in my personal life and in my professional life.
My decision to choose simplicity was reinforced by a recent blog post by Matt Mullenweg, the CEO of Automattic, the company behind WordPress. And a second by Terry Sutton, a front end developer. In Terry’s post he says:
Good open source software is / does / can (roughly in order):
easy to install and setup, for free, both locally and on my own server
powerful, capable
make easy things easy
make hard things possible
opinionated, and so you get:
a lot out of the box, so you don’t have to:
constantly reinvent the wheel to do simple or common things
interoperable, flexible, extensible
Has a core user story in mind (ie: it appeals extremely well to a couple of use cases, while enabling many others)
continuously improved and updated
I think that’s a great list, and I think that the Runestone software ticks a lot of those boxes, but not the first one, and I’ve heard from many of you, not not the third one. It certainly is the fifth one, and I think sixth through tenth follow. The good news is that principles 1 and 3 are closely related to my touchstone word for the year.
Runestone has been around since 2011, and it has grown organically over the years. Myself, students, other contributors added features as I’ve needed them, and as others have asked for them. I’ve tried to keep the code clean, and the architecture flexible, but I think it’s time to take a step back and look at the big picture. I think it’s time to simplify.
Moving to the monorepo was a first step in that direction. I think it’s time to take a second step. I’m going to try to simplify the user experience, particularly for instructors. I’m going to try to make it easier to install and setup, and I’m going to try to make it easier to use. I’m going to try to make it easier to contribute to, and I’m going to try to make it easier to extend.
Here are some some of the things I’m targeting especially for the first half of 2024:
Create a quick programming project interface to serve as a replacement for instructors who used repl.it.
Simplify the experience for creating and grading assignments
Simplify the installation process
Simplify the configuration process
Make the developer experience better for contributors
We are going to have to bite the bullet on LTI 1.3 support soon! People are deprecating 1.1 as we speak.
Please note, these are Goals not guarantees. I had many features I wanted to implement in the Fall of 2023, but most of my time was eaten up answering questions from users, and dealing with growth and scaling issues. I’m hoping that by setting these goals, and by sharing them with you, I’ll be able to stay focused on them.
I’m not going to go into details here, but I’ll post about progress periodically. If you want to get into the details, it is all on Github in the form of six projects (Sort them by name). Under the hood the main goal is to get all of the instructor facing pages ported to the same underlying technology we use for the assignment and book serving pages. This will make it easier to maintain, and easier to extend. It will also make it easier to add new features, and to make the user experience more consistent.
Problem Solving with Algorithms and Data Structures using (Python|C++|Java)
A triumph of open source and open content
In 2005 I was in my second year of teaching computer science at Luther College. I had enough experience to adamantly oppose Java as our introductory language, and I had learned about Python at the 2004 SIGCSE conference. I was excited about Python and had been playing with it for a year. There was an excellent introductory textbook by John Zelle who was at a nearby college. In our departmental discussions we thought that Python would be a good introductory language, but we didn’t want to give students one semester of Python and then a second semester in Java or C++. We wanted to stick with Python. So I said to my colleague David Ranum, “lets write our own data structures book!” And so we did. We contacted Zelle’s publisher who was very interested in our idea. We wrote the book in 2005 and it was published in 2007. It has been used at several universities as the primary textbook for the data structures course taken by second year computer science majors, as well as a supplementary text for students in other disciplines. The book has been translated into Spanish as well as Chinese.
The goals for the book were: * To have a strong focus on problem solving, that introduces the students to fundamental data structures and algorithms. * To introduce algorithm analysis in terms of Big O notation. * To use Python as the programming language. * To illustrate that we are all “standing on the shoulders of giants.” In computer science we are building on the work of those who came before us.
I loved teaching from this book. The students seemed to enjoy the problems we introduced and learned a lot. By the end of the course they were all much more fluent in Python and ready to tackle other languages and more advanced topics. They all had good “gut level” understanding of the major data structures and algorithms. I was very proud of the book and the course.
I’m also proud that in the 17 years since the first edition was published we have only had two more editions. There is no need to gouge students every two years with a trivial update. The book is still relevant and useful today.
Fast forward to 2011, I was on sabbatical and was working on the second edition of this book. But I had a serious case of “writers block” my main problem was that I just hated the idea of a paper textbook for computer science in the 21st century. Books should be interactive, you should be able to run these algorithms right in the book! That was the genesis of Runestone Academy. The next step was to convince our publisher that we should make the book freely available online. I’ll never forget the response of our publisher, Jim Leisy, “Lets go change the world!” With that, we were able to make the book available under a Creative Commons license. The book is still available in print, but the online version is the one that is being actively maintained, and through our strong partnership with the PreTeXt project we can write once in PreTeXt and publish both online and in print.
The Creative Commons license is a great example of the “shoulders of giants” lesson we wanted to teach our students as in 2018, Jan Pearce of Berea College in Berea Kentucky took our book and created a version for her students that used C++ as the primary programming language! She kept the Python as “pseudocode,” and added C++ examples along side. The content was what she was looking for but the programming language was wrong. So she changed it! She also added some new content and examples. She then shared her changes back with us, and we were able to incorporate the book into our collection. The C++ edition of the book has been used by over 3,000 students in over 223 courses around the world.
In the Fall of 2023 J. David Eisenberg of Evergreen Valley College in San Jose, CA, took the book and created a version for his students that used Java as the primary programming language! The content was what he was looking for but the programming language was wrong. So he changed it! This book does not have Python as pseudocode, but has a lot more interactive exercises and more of the examples can be run in the book than in the Python edition. He also added some new content and examples.
I can’t tell you how proud I am of these two projects. Even though I was personally opposed to Java (and C++) as an introductory language, we don’t all need to agree. And I know I’ll never convince the college board they are wrong. These two books are a great example of the power of open source and open content. I hope that you will consider using this book in your own courses, and if you do, please consider sharing your ideas or changes back with us so that we can all benefit from your work. I would suggest that the Java book is very accessible to high school students who have had the AP Computer Science A course. The Python edition is very accessible to students who have had a semester of programming in any language. If you are willing to help with any of the editions, but especially the Python edition I would love to hear from you. I would really like to make the Python edition much more interactive.