We just finished our first year piloting an educational intervention which uses tablets to provide access to over one hundred educational games and books for children. At year’s end, my team and I were in charge of conducting a first-level qualitative evaluation to determine the program’s merits for expansion. While the kids were the ones who received tablets, we were particularly interested in the parents’ perceptions of the school’s program.
We work in rural communities in Guatemala, where around half of the parents never speak Spanish in their homes. They instead speak one of Guatemala’s 21 indigenous languages, each stemming from a common proto-Mayan root which existed around 5000 years ago. Since then, each regional variant has changed significantly. And while it has evolved from its ancient beginnings, its vocabulary remains rooted in the communities’ contexts.
So when we were planning a survey asking about tablets, apps, and chargers, we found ourselves in a unique situation. Below I outline each step of the process, with the challenges encountered and some helpful tips.
Creating the survey – I recommend you give your enumerators at least one afternoon to have a group translating session. Have them speak about different words in the tool and how they would translate them, and ask them to settle on one agreed-upon translation. Then have them translate it back to the original language, to ensure that it maintains the original intention of the survey.
This is also incredibly important at the analysis stage. Why? Because if an enumerator differs in the way he/she translates an questionnaire, there is potential for some serious enumerator fixed effects. Avoid the bias and trouble, and make sure the enumerators implement clearly and consistently.
Preparing for implementation – Let’s use the example of “computer.” In the Ixil language, my team would translate it to something like “things used for writing,” while miming a typist. In Q’eqchi’, “metal brain” is the official translation according to the Academia de Lenguas Mayas. Many would choose to say the word in Spanish, computadora, and then provide a description of what that word means, so it can be used throughout the rest of the survey.
Either way, implementation can be made easier with examples. After you decide on the translation, decide on what tools you can use to facilitate understanding. Best case scenario, you bring the actual computer itself – though in many cases the security of the area doesn’t allow for carrying valuables. You can also include photos in your survey so enumerators can use the image for a quick translation.
Implementing the survey – As is good practice with any qualitative tool, the enumerators should try to make respondents feel as comfortable as possible. In the beginning, have the enumerators start talking casually about the technology so that respondents get used to the vocabulary that will be used in the survey.
Then, practice patience. While the language in which the survey is created may allow for a fluid, quick conversation about tech, the language of implementation may require more explanation. Allow plenty of time for each survey in order to ensure quality data.
Interestingly, I found very little online on the topic of tech intervention evaluation across languages. AEA has an active set of blog posts on Indigenous Peoples and Evaluation, but more resources on language and technology in evaluation are needed.
* Obviously the Mayas were kings and queens of technology, and Mayan communities today still use unique technologies helpful for their daily lives. 'Tech' here refers to things like computers, tablets, internet - though I recognize that this is an ethno-centric definition.