Computer-based software systems
A unique software system has been developed to manage the design, translation, test administration and data collection of the survey instruments (the language tests and context questionnaires).
Jostein Ryssevik, from Ideas2Evidence, works closely with Gallup Europe and is Team Leader for this exciting area of SurveyLang’s work, which is creating new approaches to test development.
The platform is fine-tuned to the specific set of requirements of the Survey and is designed to support the delivery of both the paper-based and the computer-based tests.
These are just some of the aspects of this system:
Item authoring tool
The language test items for the ESLC survey are developed by an expert team of over 40 Language Testing item writers located across Europe. Items move through the cycle of authoring, editing, vetting, adding of graphics and audio, pre-testing, Field Trial, Main Study, with each stage involving different tasks, roles and responsibilities.
The Test Item Authoring Tool supports this process. It allows non-technical language testing item writers to create tasks by means of templates for the various task types used. At any stage in the development, a task can be previewed and tested to allow the author to see how it will look and behave when rendered in a test. The authoring tool also supports the capture and input of all the metadata elements associated with a task.
Test Item Databank
The Test Item Databank is the hub of the central system, providing long-term storage, version control and management of the language test-items graphics and sound files. Test items are uploaded to the databank by the item writers to be seen and shared by others in this process. The databank includes a version control mechanism and a secure authentication system, ensuring only authorized personnel can see or change tasks.
Test assembly tool
The assembly of test items into complete test sessions is managed by the language testing team.
In order to allocate students to tests, the test assembly tool will also need to process detailed information about every sampled student; especially the student’s proficiency level (from the routing test), the tested language as well as the student’s national or local language.
The assembly tool will use this information and randomly allocate students across the group of available test sequences matching their respective proficiency level. Finally, the test assembly tool will produce the individualized test material for each student, both for computer-based and paper-based testing.
What makes this an innovative approach is the fact that both computer and paper-based tests are served from the same source and the same system. This reduces manual work and hopefully also manually-induced errors.
Test rendering tool
Security is one of the critical challenges related to computer-based delivery of the Language Test. Protecting the security of the test material and isolating students from external influences such as access to the web or dictionaries are both vital to the integrity of the test results.
SurveyLang has developed a test rendering tool that will boot the test computers from USB memory-sticks or CDs. This will ensure SurveyLang has full control of the local desktop environment, including blocking access to Internet or other resources.