in Business Administration and Economics

The aim of SEAL is to achieve an effective improvement compared to the systems and tools previously used in university teaching and examination organisation.

At the heart of the project is a computerised system that offers options for correcting and evaluating exams, viewing exams, evaluation and role management. For example, it will be possible to flexibly organise examinations and their evaluation schemes and thus adapt them to the individual needs of each course. The evaluation time for around 800 to 1000 multiple-choice exams has already been reduced from three weeks to three days.

SEAL is based on the in-house software development KLE (Klausureinsicht) of the Chair of Microeconomics. KLE has already been used several times to evaluate the exam of the assessment phase course "Grundzüge der VWL" at the Faculty of Business Administration and Economics. The students' answer sheets were entered into the software using scanning software and automatically analysed. This has already reduced the evaluation time of around 800 to 1000 multiple-choice exams from three weeks to three days.

As part of the course "Fundamentals of Economics", 142 students were asked about the user-friendliness of methods for learning assessments. There were two methods to choose from that are suitable for multiple-choice exams: the Scantron sheet and the exam solution sheet (KLB). With the Scantron sheet, the boxes representing the correct answers have to be coloured in with a pencil. Although an answer sheet must also be filled in for the KLB, this procedure has been simplified, as a pencil no longer needs to be used, for example. It is also no longer necessary to completely colour in an area and, unlike the Scantron sheet, corrections are also possible. This KLB is the basis for SEAL.

There are 90 women and 52 men in the sample. Over 90% of the respondents are studying economics or international business studies. They were also asked about their semester of study, the use of electronic and internet-enabled devices and their experience with other forms of examination (Scantron, KLB, free text, e-examination, other).

A standardised questionnaire was chosen to test the user-friendliness of the two forms of examination under investigation. This goes back to Brooke [2]. In addition to the good comparability with other studies, the calculation of the so-called System Usability Score (SUS) should be emphasised in particular. The system usability score asks users about the user-friendliness of an application and presents the result as a percentage usability value (or usability score) of the application. Accordingly, 100 per cent corresponds to a perfect system with no usability problems, a score above 80 per cent indicates a system with excellent usability, scores between 60 and 80 per cent are to be interpreted as good and a usability score below 60 per cent indicates significant problems in the operation of the system.

All questionnaires were completed in full so that 142 SUS could be determined for each type of examination. The average user-friendliness for the Scantron sheet is therefore 52.62%. The standard deviation is around 16.78 percentage points. In contrast, the average user-friendliness for the KLB is around 72%. The standard deviation here is around 15.26 percentage points. The improvement compared to the Scantron sheet is clear to see. The user-friendliness is around 20 percentage points higher than that of the Scantron sheet. This is all the more astonishing when you consider that the required customisation was relatively minor.

SEAL is a computer-based system that enables time-efficient examination organisation, implementation and evaluation. This improves both study and research conditions. The flexibility of SEAL should be emphasised in particular. Each department can create its own examinations, assessment and evaluation schemes. SEAL is suitable for all types of multiple-choice examinations.

SEAL is also intuitive to use and builds on existing Paderborn University architectures. This means that the costs for administration, maintenance and implementation can be kept to a minimum and even reduced compared to the systems used to date.

SEAL is being developed cooperatively by three chairs from two different faculties. From the Faculty of Business Administration and Economics, Prof. Dr Claus-Jochen Haake's Chair of Microeconomics is involved, while Prof. Dr Ilka Mindt from the Institute of English and American Studies and Prof. Dr Doris Tophinke from the Institute of German and Comparative Literature are involved from the Faculty of Arts and Humanities.