On this page:
check-deadline
check-max-submissions
update-submission-timestamp!
get-submission-timestamp
set-test-max-score!
score-add-penalty!
add-report-line!
with-output-to-report
add-score-to-report!
write-report
@test
@test/  exn

8 Grading Utilities

 (require handin-server/grading-utils) package: handin

This module provides utilities that can serve as the basis for an automated grading system. In particular, it provides:

The following example illustrates a checker module using this infrastructure:

(module checker handin-server/checker
 
  (require handin-server/grading-utils)
 
  ; Checks that submission is on time and that the user has submissions left
  (pre:
   (check-deadline)
   (check-max-submissions))
 
  ; Ends the report by adding the score and writes it in the user directory
  ; This way, students can see their reports from the web interface.
  (post:
   (add-score-to-report!)
   (write-report))
 
  (check:
   ; Get timestamp of the submission and add it to header and report
   (update-submission-timestamp!)
   (add-header-line! (get-submission-timestamp))
   (add-report-line! (get-submission-timestamp))
 
   ; Acceptance tests: reject the submission if any of these tests fail
   (!test (foo 1) 3)
   ; ...
 
   ; Grading
 
   ; Initialize max score
   (set-test-max-score! 100)
 
   ; Failure discounts 25 points
   (@test "Sample case 1"
          "Error using even? predicate"
          (bar '(1 2 3 4) even?)
          '((2 4)(1 3))
          25)))

procedure

(check-deadline)  void

Checks if the current submission is within the deadline specified for the corresponding assignment. If the submission is past this deadline, a submit error is thrown. If no deadline is specified the check is trivially passed. This function is intended to be used in the pre-checker stage.

Deadlines are configured in the "config.rktd" file (see Server Setup), in a per-assignment basis, through the "deadline" key. This is an optional keyword that holds an association list where each sublist has three elements:

Consider an example configuration:

   (deadline (("Assignment 1" (2014 3 11 23 59 59) 3)))

procedure

(check-max-submissions)  void

If the assignment has a limited number of submissions, it checks whether the user has already used up all of them. If the user has no submissions left, the current one is rejected. This check is intended for use in the pre-checker stage.

By default users have an unlimited number of submissions. Restrictions on the number of submissions must be defined in the "config.rktd" file, through the optional max-submissions key. This key must hold an association where each sublist has two elements: the assignment name, and a non-negative integer specifying the allowed submissions. For example:

  (max-submissions (("Assignment 1 5)))

check-max-submissions only counts the number of existing successful submissions, more specifically, it counts the number of "\"SUCCESS-n\"" directories specific to each user. Therefore, submissions that rejected by the pre-checker or checker do not count against the submission limit.

Sets the current timestamp as a thread-local value that can be later retrieved using get-submission-timestamp. This value can be then used to identify submissions and related files, like reports or grading files. The rationale is that we cannot use the "\"SUCCESS-n\"" names because they will change after new submissions.

Returns an string representing the latest timestamp value stored using update-submission-timestamp!.

procedure

(set-test-max-score! max-score)  void

  max-score : positive?
Sets max-score as the maximum score possible for this submission, and initializes the student’s score to this value. Penalties are only applied to the student’s score, while the maximum score is available for reference only.

procedure

(score-add-penalty! penalty)  void

  penalty : (or/c zero? positive?)
Discounts penalty points from the student’s score.

procedure

(add-report-line! line)  void

  line : string?
Like add-header-line!, but it writes to the report instead of the textual version of the submission.

syntax

(with-output-to-report expr ...)

Evaluates expr ... and redirects output to the report bound to this submission. Similar to with-output-to-string or with-output-to-file.

procedure

(add-score-to-report!)  void

An utility function, defined as:

(define (add-score-to-report!)
  (add-report-line! (format "Final Score: ~a out of ~a"
                            (get-student-score)
                            (get-test-max-score))))

syntax

(write-report)

Writes the report bound to the current submission into the root of the user folder of the corresponding assignment (for example "Assignment 1/tester"). By default, the report is written immediately to this folder. It is possible to configure the delay in the "config.rktd" file, using the report-delay-in-minutes key. The value of this key must be a non-negative number, for example:

   (report-delay-in-minutes 5)

Internally, the macro calls the start-timer procedure in the following way:

(define-syntax (write-report stx)
   ...
   (define report-delay (get-report-delay-in-minutes))
   (start-timer (start-timer-manager) (* 60 report-delay) ...))

The default behavior is obtained because get-report-delay-in-minutes returns 0 if the key is not found in "config.rktd".

syntax

(@test test-desc error-desc expr result penalty)

(@test test-desc error-desc expr result equal? penalty)
Like !test, but is designed to work with a report and a user score. It evaluates (!test expr result) or (!test expr result equal?) and redirects the output to the report. If the test fails, the error message is written to the report and no exception is thrown, and the penalty is added to the score of the user.

syntax

(@test/exn expr penalty)

Like !test/exn but the output is redirected to the report. In case of error, the message is written to the report instead of raising an exception.