Dean Karlan & Jonathan Zinman have quantified something most researchers probably sensed already – that respondents are often very sensitive to social perceptions around “undesirable” activities such as borrowing. In South Africa, they found that roughly 47% of known borrowers at a microcredit bank told surveyors they had not borrowed, with women being more likely to lie than men, and especially more likely to lie to male surveyors than female. (The surveyors had no connection with the bank, and thus no independent knowledge of the correct answer for a given respondent.)
This is the eternal question at the heart of research – what do you do if self-reported data isn’t accurate, but it is all you have? And how does one incentivize truth-telling? An interesting corollary to this experiment would have been to tell a subset of respondents that the surveyors knew they belonged to the bank (without giving the surveyors any other personal information about respondents’ accounts), and measure how the mere mention of the lending institution changed respondents’ rate of truthful reporting. How would this work in situations where there is legitimately no authority with credible information on respondents, like measuring landholding in situations where land tenure is totally informal? Would this be too destructive to surveyor impartiality (and general research ethics) to be an option? The more I delve into these complicated questions of survey design, the more I find them strangely fascinating.