Guidelines for Development and Administration of Computerized, Online Tests

 

I.                 Creating an Online Test.

 

A.          Who do I contact about having an  online test programmed for me?

 

Zac March

882-3962

MarchJ@missouri.edu

E215 Vet. Med. Bldg.

 

B.          What sorts of questions and multimedia can I include in my test?

 

1.               Question types supported

• True/False
• Single-Answer Multiple Choice
• Multiple-Answer Multiple Choice
• Matching (where a student draws an arrow from an answer choice to a text description or a picture related to it)
• Fill-in-the-Blank
• Individuals wishing to deploy computerized tests with free text responses should consult with CVM-IT staff regarding currently available options. Currently, fill-in-the blank, discussion or other free-text exam formats can be supported by (a) coordinated email, (b) printable PDF documents, or (3) WWW-based forms. All of these tools can be utilized in either synchronous or asynchronous fashion. Security issues will vary with the specific format utilized.

 

2.               Multimedia types

 

Most image, video, and sound clip formats are supported or can be converted into a format that is supported.

 

3.               Scoring options

 

·       The system can automatically score all objective question formats. Maximum, minimum and partially correct question values can be calculated. These settings can be customized.

·       Fill-in-the-Blank questions can be automatically scored against a preset list of acceptable terms (by text string matching).

 

4.               Punctuation limitations

 

·       Any symbol or formatting may be used in a question stem.

·       Commas, semicolons, apostrophes, and percent signs (%) in the answers should not be used. These symbols disable the item analysis functions of the testing server.   Please try to word your answers so as to avoid using these symbols if possible!

 

C.          What do you need from me in order to program my test?

 

1.          A text document (ideally in the form of a digital file—for example, a Word document) which contains ALL of the following information for each question:

 

·       the text of the question stem

·       a list of the answer choices, with the correct choice(s) indicated

·       the type of question desired (i.e., single-answer or multiple-answer for multiple-choice questions)

·       the point value to be assigned to the question.

 

2.          If you want to use multimedia (pictures, movies, or sounds) in your test, we will need the appropriate multimedia files and an indication with which question(s) each file should be presented.

 

3.               Statistics for the test as a whole (used in calculating student pass/fail feedback):

 

·       the total point value for the test

·       the passing threshold (e.g., 60%)

 

D.          How long will it take to have my test programmed?

 

·       Please allow at least a week for programming before you plan to administer your test. Note that additional time may be required if extensive multimedia processing is needed (for example, to digitize images from kodachromes, digitize or edit movie clips, etc.)

 

II.               Scheduling and Administering an Online Test.

 

A.              Who do I need to see about scheduling my online test?

 

·       CVM-IT staff will assist you in scheduling the exam and reserving the CVM student computer laboratory.

 

 

B.              What do I need to consider as I schedule my test?

 

·       The computer lab can only accomodate 32 students at a time. If you are testing an entire class, they will need to take the test in two shifts.

·       Short, low-point value quizzes  have  been self-administered at the students’ convenience; however, our experience indicates that full-scale examinations should be administered during fixed hours, when support staff can be made available to help students deal with any technical difficulties during the exam, and to employ appropriate examination security.

·       Testing renders the computer lab unavailable to the rest of the college during the test period.   Please be considerate and schedule only the amount of time absolutely required for your test.

·       If you must reschedule your test, please be sure to first coordinate the change with both Stephanie McClaren and Zac March.

 

          C.          Who needs to be present when my test is administered?       

 

·       Faculty members should be available to answer content-related questions during the testing period. At least one CVM-IT staff person will be present in order to deal with any technical problems that arise.  Zac March will coordinate arranging this support as a part of the process of scheduling your test.

 

III.             Reporting and Feedback from Online Tests.

 

A.              What reports are available for me to assess my test results? 

 

·       The Lesson Assignment Summary report provides a listing of all students taking the exam, and their scores.

·       The Line Item Analysis provides a listing of all answers to questions, and the number of students selecting each answer, for use in identifying defective questions.

 

B.              What performance feedback is available for the students taking the test? (Samples of reports attached.)

 

·       As soon as the students submit their tests, their scores will be automatically E-mailed back to them.

·       At the option of the instructor, the test may be made “reviewable;” this means that the student can reopen a blank exam and go through it to review the questions and the answer choices.  This will NOT show the student which answers he or she selected.

·       The student may view a “Review Answers” report, which will show the question number (but NOT the text of the question stem),  each answer the student selected, and whether that answer was correct or incorrect.  (The report will NOT show the student information about any other answers except the ones that he or she selected.)

 

C.             What if there is a problem with a test question?

 

·       Regrading defective test questions, once the computer has scored them, must be done manually and can be quite cumbersome. There is not, at this time, any automated way to report individual students’ answer choices for the test as a whole.

·       Regrading a single question for a particular student is best done by using the “Review Answers” report (see III.b.3.), which documents all answer choices made by the student.

·       Regrading a single question for all students taking the test (for example, to give credit for answers other than those originally coded as correct when the test was programmed) requires that the data be manually extracted from the online testing database by the IT staff.  This can be done, but it is extremely time-consuming and can only be supported on a very limited basis.  If you need this sort of assistance, please coordinate your request with Zac March.

 

IV.           Developing your own computerized tests.                 

 

Faculty members wishing to program their own tests or other learning modules should contact Zac March for information related to software acquisition and training opportunities.