Although the use of OER is growing in the Global South, it is not used as frequently or as broadly as it is in the Global North, perhaps due to a combination of cultural, demographic, socioeconomic and political factors. However, the potential benefits of OER – especially the improvement of everyday life for those on the margins of society – should be understood by scholars, policy makers, and the public so that they can help promote greater social equity and empowerment. The purpose of this presentation is to analyze a survey instrument validation process that I have engaged in so as to conduct a large-scale survey across nine countries and 36 institutions in the Global South concerning the scholarly activities of current and potential users of Open Education Resources (OER).
The development process of the instrument consisted of the following six steps:
- Define the construct: After a comprehensive review of the OER literature, the constructs’ definitions were centred on the factors influencing the current and potential user of OER. The four constructs defined were: individual characteristics, education institution characteristics, OER proficiency, and perceived outcomes of OER.
- Item development and judgment: The item development process associated each construct with several questions derived from the literature and previous OER surveys. We recruited 34 OER experts to assess the instrument’s content and face validity. We scaled the questions within each of the two main constructs using a 5-level Likert scale. The majority (75.5%) of respondents had 6 years or more years of education experience, 53% had distance education experience, and all were at least 31 years old. The selection of qualified experts is an important part of the instrument process to determine the appropriate number of questions to fully explore each construct. This mitigates researcher bias in selecting items to develop. The experts also provided feedback regarding each question. I instructed them to determine to what extent the items represented the constructs using the Likert scale. Only the most collectively agreed items were included in the next version of the instrument.
- Designing and conducting studies to develop a survey: After securing face and content validity, the instrument contained 26/30 questions for student/instructor versions. The final instrument was loaded onto the SurveyMonkey site and made available in English, Spanish and Portuguese. A hard copy option was also made available for several countries. I designed the format of the survey so that participants were able to edit their own survey responses or finish it later in case of internet connection loss. Additional information to support the results is also being collected using interviews and secondary data by Local Coordinators (LC) in the 36 institutions.
- Pilot study: The main idea was to identify errors in the survey’s form and presentations. It allowed me to redesign some problems before it was used. After the pilot test, the 78 respondents provided consistent suggestions to improve grammar and major suggestions regarding the construct “OER proficiency”. The amount of time it took to complete the survey was also noted. The improvements were made and additional pilot testing was done with minor revisions.
- Data collection plan (the current stage at time of writing): The sample from the Global South includes South America, Africa, and South and South-East Asia, with three countries in each region. The selected nine countries are Brazil, Chile, Colombia, Ghana, South Africa, Kenya, India, Indonesia, and Malaysia. Four institutions were chosen from each country for the survey. There is a regional coordinator for each region in charge of three countries with four LCs each. Each LC is in charge of collecting the data from one education Institution. Each LC will collect at least 210 completed surveys. The total number of completed questionnaire will be around 7,560. A random sample will used where every eligible student or instructor should have an equal, non zero chance of being included.
After obtaining ethical clearance at each institution, the LC has to follow the data collection plan with random collection. Initially s/he will send the list of all classes within their campus to the global coordinator who will return to the LC a random ordered list of 30 classes to be surveyed. From that list, the LC will select only 10 classes with 30 or more students and permission to survey the students granted by the instructor. The LC will send the list of all students, from each of the 10 classes selected, to the Global coordinator. The Global coordinator, with support from the statistician, will send to LC a list of 30 ordered random list of students. Only 20 students will be surveyed for each class (online or hard copy). All instructors from the 10 selected class will also be surveyed.
- Finalizing the survey: I plan to use Exploratory Factor Analysis (EFA) and assess the survey’s reliability to validate the instrument after the data collection.
The process of developing and validating an instrument discussed here will hopefully enhance the quality of information derived from the data collected and help to inform policy makers and educators about OER adoption in the Global South.