New R/exams Version: Improved E-Learning Tools During a Pandemic

Overview

The new version 2.4-0 of the R/exams package has been released on the Comprehensive R Archive Network at https://CRAN.R-project.org/package=exams. This is the first CRAN release after 2.5 years of very active development. Due to the challenges from distance learning during the pandemic a lot of new features were added and enhancements made, especially for the support of various e-learning tools and learning management systems.

Many of changes were prompted by the authors’ own needs (especially in the support of OpenOlat) but also by the many questions on StackOverflow, posts in the R-Forge forum, or tweets and e-mails from users all around the world. In the following the most important features from the update are highlighted. For the full list of changes including technical details and smaller bug fixes, see the NEWS of the package release. More blog posts are planned for introducing some of the new features in more detail, e.g., cloze exercises and evaluation strategies.

Facilitated support of UTF-8 and R/Markdown exercises

Initially, the package relied on Rnw (R/LaTeX) exercises only and optionally supported different encodings for non-ASCII characters, e.g., UTF-8 or latin1 (ISO-8859-1) etc. However, it was cumbersome to correctly set the encoding of input exercises and output file formats.

Given that the majority of users by now seem to rely on Rmd (R/Markdown) exercises and that most output file formats involve using the pandoc document converter (which requires to use UTF-8) for some aspects, the entire exams package and all its function now exclusively use UTF-8 as well. All templates etc. have been modified to support UTF-8 out-of-the-box. This greatly facilitates working with UTF-8 as there is no need anymore to set any encoding argument anywhere.

Support for all other encodings like ISO-8859-* (latin1, latin9, etc.), which had previously been available for Rnw exercises in certain exams2xyz() interfaces, has been disabled. This is a slight reduction of functionality but will be a great benefit for most users of the package.

Finally, to facilitate working with Rmd exercises and embedded graphics or data files, the packages base64enc, knitr, and rmarkdown are now imported in the package (and not just suggested).

Open-ended text questions with text editor or file upload

The processing of string exercises for learning management systems like Moodle, OpenOlat, or other QTI-based systems has been extended. By default, string exercises are intended for closed-format short-text answers that have to be matched exactly by the participants (e.g., asking for the name of an R function or exact value of an output string etc.). Additionally, open-ended text answers can now be enabled by setting the exstringtype meta-information to essay and/or file. The former requests a text editor for entering an answer while the latter requests a file upload dialogue. The essayreg exercise has been modified to leverage this new meta-information. Such exercises then have to be evaluated by the instructors/examiners, i.e., there is no automatic evaluation by the learning management system.

Similarly, the exclozetype meta-information now also accepts essay or file instead of string for elements of a cloze exercise. Currently, the combination of these types with num or schoice elements etc. is only possible for QTI-based systems (i.e., OpenOlat in particular) but not for Moodle (whose cloze format does not support open-ended text answers). For illustration, see the new "essayreg2" and "lm3" exercise templates.

Enhancements for OpenOlat

Several enhancements and extensions have been made in exams2openolat() (or the underlying exams2qti21() function, respectively). The most important new features are:

• Improved processing of the cutvalue (for passing a test/exam/quiz). By default, this is cutvalue = NULL (or equivalently cutvalue = NA) which means that no pass/fail status is computed at all, i.e., only the sum of the points is reported. This is particularly useful if individual online tests during a semester contribute to an overall grade but cannot be passed/failed individually.
• New argument navigation = "nonlinear". This can be switched to "linear" enforcing that questions in the test must be answered sequentially while the default "nonlinear" means that participants can switch back and forth between questions.
• New argument selection = "pool" that controls how exercises and sections are sampled. By default, the function creates one section for each exercise from which one replication will be selected in the exam. If selection = "exam" each section contains all questions and one section will be selected for the exam. The "exam" variant has the advantage that nsamp can be fully used now and that questions that build on each other can be used in the exam.
• New argument shufflesections = FALSE can be set to TRUE in order to randomly shuffle the order of sections/exercises for each participant. For selection = "pool" this corresponds to shuffling the sections that contain the pools of exercises. For selection = "exam" it corresponds to shuffling the exercises within each exam section.
• New argument cloze_schoice_display = "auto" that controls the display of schoice elements in cloze exercises. By default, radio "buttons" are used if the answer list appears in its own paragraph and a "dropdown" menu is used if the answer list appears inline (and has no mathematical markup). Both options can also be enforced explicitly, independently from the answer list appearing in a separate paragraph or inline.
• The default stitle (section title) and ititle (item title) are now NULL so that items are simply numbered consecutively (1, …, n) and section titles are omitted. Similarly, the default sdescription is now empty omitting the section description as it is typically not necessary.

The are some additional new arguments that allow further fine control of a test’s configuration. The most important and flexible is the argument config = TRUE. The logical specification config = TRUE/FALSE uses the default configuration or switches off the extra configuration entirely, respectively. Moreover, a list of options like config = list(cancel = TRUE, scoreprogress = TRUE) can be provided for customizing how OpenOlat renders the QTI 2.1 content, see ?openolat_config for more details.

Finally. a new argument envir has been added that is passed on to xweave(). By setting this to a common environment, e.g., envir = .GlobalEnv (or some new dedicated environment), it is possible to re-use variables generated in one exercise in previous exercises, e.g., for creating a sequence of variables based on the same data set. Note that this needs to be combined with selection = "exam" (see above).

Enhancements for Moodle

The popular exams2moodle() interface has been improved, in particular for more flexible cloze exercises:

• mchoice elements in cloze questions are now properly supported. By default they are shown as MULTIRESPONSE checkboxes and employ Moodle’s default evaluation strategy where each incorrect box eliminates one correct box. A different evaluation strategy can, in principle, be chosen but Moodle might not process all negative points correctly.
• schoice elements in cloze questions are still rendered as MULTICHOICE drop-down menus by default unless they contain math markup. As this is not rendered by Moodle in drop-down menus, a MULTICHOICE_V column of radio buttons is used in this case.
• To allow for customization of both mchoice and schoice elements in cloze questions, the are now both cloze_mchoice_display and cloze_schoice_display arguments. This is not fully backward compatible because in previous versions cloze_mchoice_display was also used to customize schoice elements. Now a warning is issued in this case.
• To fix the maximum width of fill-in-the-blank cells in num and/or string sub-items (e.g., when presented in a table), arguments numwidth and stringwidth have been added to make_question_moodle(). Alternatively, they can also be specified through exextra tags in each exercise. See the fourfold2 exercise for an example and ?make_question_moodle for more details.

Moreover, there is a new experimental function moodle2exams() that can take a Moodle XML quiz file with numeric, multichoice, shortanswer, and essay exercises and convert them to R/exams exercise files, either in Rmd or Rnw format. If the text formatting is more advanced (e.g., containing mathematical notation or tables etc.) the function might not lead to fully satisfactory results but still provide a useful starting point for subsequent manual editing.

New exams2xyz interfaces

Various new learning management systems or quiz engines can now be interfaced using R/exams. The work on exams2testvision() and exams2grasple() was financially supported by the Dutch Ministry of Education, Culture and Science (Project code OL20-06), and the University of Amsterdam.

• exams2ilias() for the open-source ILIAS learning management system. This is essentially a convenience wrapper to exams2qti12(), tweaking a few defaults and employing a somewhat modified XML template. Not all question types are supported, though, mostly string questions with open-ended answers and multiple-choice and single-choice questions. Numeric and cloze questions are not supported, yet.
• exams2testvision() for the Dutch testing platform TestVision. It is essentially a fork of exams2qti21() that incorporates TestVision’s own strict implementation of QTI 2.1. See the online tutorial on how to upload the zip output from exams2testvision() into the system by selecting it in the import menu and then moving the imported material to the appropriate directories.
  Additionally, there is a new function testvision2exams() to convert TestVision’s QTI 2.1 questions to R/exams exercise files, either in Rmd or Rnw format. The supported TestVision question types are ‘invul (numeriek)’, ‘een-uit-meer’, ‘meer-uit-meer’, and ‘open’ which are converted to num, schoice, mchoice, and string, respectively.
• exams2grasple() for Grasple, a Dutch practice platform for mathematics and statistics education. It supports num and schoice questions which are exported to a zip file containing Grasple’s JSON format. Note that currently importing cannot be done by users themselves; it requires a request for manual import by a Grasple team member.
• exams2particify() that can export exercises to a comma-separated values (CSV) format for import in the audience response system Particify, the successor to ARSnova. In particular, single-choice and multiple-choice exercises are fully supported while num and string question are converted to open-ended text questions.
• exams2kahoot() that can export sufficiently simple single-choice and multiple-choice exercises to an Excel sheet via openxlsx::write.xlsx() that can be imported by the game-based learning platform Kahoot! (suggested by Rushad Faridi). Exercises are converted to plain text and questions must not exceed 120 characters, answers must not exceed 75 characters.

Enhancements for written exams (NOPS format)

The exams2nops() function is the R/exams interface for generating exams of single-choice and multiple-choice questions, that can be scanned and evaluated automatically.

Various improvemens have been made in nops_scan(), especially for scanning the boxes pertaining to the student registration ID. Rather than reading a very small area around each box and just shaving off its borders, a larger area is read now and then shaved iteratively. Hence, it is also easily possible to further increase the size of the area which may sometimes lead to improved scanning results.

In the exams2nops() function itself, the handling of reglength < 7 has been improved. Internally, reglength = 7 is still enforced (and thus necessary in the registration CSV file) but the initial IDs are fixed to “0” in the exam sheet and corresponding boxes ticked already.

Finally, language support has been extended by two new languages: Czech (cz, by Jindřich Marek) and Galician (gl, by Marta Sestelo & Nora M. Villanueva).

Enhancements for Blackboard

In the exams2blackboard() interface the new argument mathjax = NULL has been added that optionally embeds the MathJax <script> in each of the exercises. This allows to render mathematical content by the MathJax plugin for all modern browsers. Without MathJax, the browser itself has to do the rendering directly (which is supported by Firefox and Safari but not by Chrome for example). The default is FALSE unless converter = "pandoc-mathjax" is used. But also for the default converters (producing MathML output) mathjax = TRUE can be used. (Suggested and tested by Sean Quallen and Gabriele Cantaluppi.)

Moreover, the following improvements may be useful for Blackboard users: Rendering verbatim code chunks can be fixed from <pre> to <code> tags if fix_pre = TRUE (default) which is necessary in classical Blackboard systems. Points can be specified through expoints.

Further improvements

• When running exercises via knitr::knit() errors in the R code will stop the evaluation now by default. This was always the default behavior for Rnw exercises (i.e., when processed with engine = "sweave") but now is also the default for Rmd exercises and for Rnw exercises via engine = "knitr". In exercises processed via knitr::knit() it is possible to carry on with code evaluation after errors (the default in knitr) by setting the chunk option error = TRUE. Similarly, the default handling of warnings has been set to warning = FALSE so that warnings are reported on the console rather than in the weaved/knitted exercises.
• Added new argument texengine = "pdflatex" to exams2pdf() which is passed on to tinytex::latexmk(..., engine = texengine). Provided that tinytex support is installed, this option can also be set to texengine = "xelatex" or "lualatex" for example.
• Interfaces exams2html(), exams2pdf(), and exams2pandoc() gained an argument exshuffle = NULL which can be modified to overrule the exshuffle setting within an exercise (if any). For example, setting exams2xyz(..., exshuffle = FALSE) retains the full answerlist without any permutation.
• In xexams() a new argument rds = FALSE was added that indicates whether the list returned by the function should also be saved as an RDS data file. If rds = TRUE the file name metainfo.rds is used but alternatively rds can also be specified as a character file name to be used. The new rds argument is leveraged in exams2blackboard(), exams2moodle(), exams2pdf(), exams2html(), exams2qti12(), and exams2qti21(), as well as other interfaces built on top of these.