Feedback Provided to the Instructor

• Class Summary:
  • Self Confidence: How the students ranked their knowledge of the topic before and after using the problet. This will be listed under PreSurvey and PostSurvey results. Here is an excerpt:

    ---------- Begin PreSurvey Results ----------
    Processed 14 students
    How well do you know Arithmetic Operators (+,-,*,/,%)?
    	Very well - 4
    	Well - 8
    	Average - 2
    	Not well - 0
    	Not at all - 0
    ...
    ---------- End PreSurvey Results ----------
    ---------- Begin PostSurvey Results ----------
    Processed 14 students
    How well do you know Arithmetic Operators (+,-,*,/,%)?
    	Very well - 6
    	Well - 8
    	Average - 0
    	Not well - 0
    	Not at all - 0
    ...
    ---------- End PostSurvey Results ----------
    

    In this example, the number of students who thought they knew arithmetic operators very well went up by 2 from presurvey to post-survey. This shows any change in the self-confidence of the class on the topic of the problet.
  • Improvement in Learning: For each learning objective, the average class performance is listed on the pre-test and the post-test. Here is an excerpt:

    ---------- Begin Pretest Results ----------
    Processed 9 students
    For Arithmetic.*.Coercion:
    Number of students processed: 9
    Number of students who solved problems: 9
    Class Average of Percent Correct Partial Wrong Missed Not Attempted
    0.33 / 0.0 / 0.66 / 0.0 / 0.0 / 
    ...
    ---------- End Pretest Results ----------
    
    ---------- Begin Post-test Results ----------
    Processed 9 students
    For Arithmetic.*.Coercion:
    Number of students processed: 9
    Number of students who solved problems: 9
    Class Average of Percent Correct Partial Wrong Missed Not Attempted
    0.61 / 0.0 / 0.39 / 0.0 / 0.0 / 
    ...
    ---------- End Post-test Results ----------
    

    In this example, on problems involving coercion and multiplication (the learning objectives), students got an average of 33% of the problems correct on the pre-test and 61% correct on the post-test.

    Instructors could use the post-test results to determine the sub-topics that they may want to go over again in class.

  • Feedback about the Problet: What the students thought about using the problet and learning from it. Here is an excerpt:

    ---------- Begin Feedback Results ----------
    Processed 9 students
    It was easy to use this problet.
    	Strongly Agree - 0
    	Agree - 7
    	Neutral - 1
    	Disagree - 1
    	Strongly Disagree - 0
    The generated problems were instructive.
    	Strongly Agree - 0
    	Agree - 6
    	Neutral - 2
    	Disagree - 1
    	Strongly Disagree - 0
    The feedback provided to my answers was clear.
    	Strongly Agree - 0
    	Agree - 7
    	Neutral - 1
    	Disagree - 1
    	Strongly Disagree - 0
    ...
    ---------- End Feedback Results ----------
    

    In this example, students mostly agreed that the problet was easy to use, the problems were instructive and the feedback provided by the problet was clear.
• Student Summary - For each student:
  • Summary of Learning: Summary of the student's performance on the pre-test, practice session and post-test. The following is an excerpt from the summary of the pre-test for John Doe:

     --- Begin summary of the Pretest for John Doe ---
    On problem 1 (Template 300), entered 2 steps, correctly solved 2/2 steps in 28 seconds
    On problem 2 (Template 203), entered 2 steps, correctly solved 1/2 steps in 17 seconds
    On problem 3 (Template 305), entered 4 steps, correctly solved 2/2 steps in 25 seconds
    On problem 4 (Template 424), did not attempt problem
    ...
    0.88 / 0.0 / 0.25 / 0.0 / 0.0 / Arithmetic.%.Correct / 8 / 7 / 0 / 2 / 0 / 0
    1.0 / 0.0 / 0.0 / 0.0 / 0.0 / Arithmetic.%.Precedence / 6 / 6 / 0 / 0 / 0 / 0
    ...
     --- End summary of the Pretest for John Doe ---
    

    Note that the student solved the first problem completely correctly; the second problem partially (only 1 of 2 steps); the third problem partially because the student entered more steps (4) in the answer than there were in the correct answer (2); and did not attempt the fouth problem.
    After the problem summary, the problet lists the summary of the student for each learning objective. In the above example, the student correctly solved 87% of the problems on the correct evaluation of the modulus (%) operator, and 100% of the problems on the precedence of the modulus operator.

    If any data was not used in the analysis, the data is identified in the following format:

    The following data was NOT analyzed for John Doe
    Arithmetic / Registration / Thu, 24 Mar 2005 11:03:46 -0500 (EST)
    Arithmetic / PreSurvey / Thu, 24 Mar 2005 11:04:00 -0500 (EST)
    Arithmetic / Trial Run / Thu, 24 Mar 2005 11:06:55 -0500 (EST)
    

    Data is not analyzed under the following circumstances:
    • The student used the problet more than once - only data from the last attempt is analyzed. The data from the earlier attempts is not analyzed.
    • The student did not attempt all the relevant stages of the problet:
      • To analyze the student's learning, the student must have attempted all three stages: pre-test, practice and post-test
      • To analyze the student's level of confidence, the student must have answered both the pre-survey and the post-survey