‘Please respond:’ Crafting an effective research survey

What compels a busy health professional to participate in a research survey?

Health professionals have a poor track record of participating in survey research related to medical care and treatments, service use, and patient engagement. In fact, there has been a downward trend in participation over the past decade.

This information provides valuable data for those of us involved in researching and influencing policy, yet we only hear from a small group due to a variety of factors impacting response rates and generalizability of survey results. Nevertheless, some medical journals, such as the JAMA Network, recommend a response rate of 60% or higher for manuscript consideration. What’s a researcher to do?

The Leverage-Salience Theory (LST) of survey participation suggests that participants are positively or negatively influenced by survey characteristics such as topic, incentive, format of survey (online vs. telephone vs. mail, etc.) and time commitment. Topic is the critical attribute that creates leverage over other features such as format, time required, or incentives.

Survey format (web-based, in-person, paper) is another attribute that impacts response rates. Online surveys enable reliable tracking of results, ability to collect large sample sizes in a short time, ease of analyzing and reporting data, and the ability to send reminders with one or two clicks.

But this ease and frequency also causes survey fatigue, as more researchers use this mode to reach participants. Paper-mailed surveys have stronger response rates than electronic versions, which may be attributed to concerns about the ability to secure data and keep information confidential online.

Time commitment is another factor to consider with shorter surveys achieving higher completion rates with a median time of ten minutes as optimal and a maximum of 20 minutes. For the medical community, lack of time was the main reason for non-response.

Incentives play a bigger role than most realize. Several studies found that response rates increased by 30% when even a $2 incentive was offered.  However, some studies found that responses rates for some surveys sent to physicians only saw a bump if the incentives were in the higher range of $30 and up.

Type of incentive matters as much as the value, with options including pre-paid, contingency payment after completion of survey, drawings, gift cards, cash, noncash, or a combination.  Prepaid cash, often used with paper surveys, is shown to have the greatest response, while contingent cash is effective for online surveys, and to a lesser extent, lottery drawings.

One U.S study reported a 12% increase in response rates by physicians offered a $2 incentive compared to a lottery draw to win $250.

What about appealing to doctors’ and researchers’ altruism? Surprisingly, charitable donations were found to have little impact compared to control groups even when the charity was tied to the survey topic and audience. An Australian study actually found that a $10 donation to surgeons’ professional college actually decreased the response rate, as did a Canadian survey to nephrologists offering a $40 donation to a kidney foundation.

One proven method of increasing response rates, regardless of format, is the reminder. The rules about how many to send and length of time between reminders vary, but the general consensus is no more than four reminders and at least one week and up to one month apart. Ideally, sending a reminder in a different format, such as mailed survey follow-up to an online questionnaire, is most effective.

Crafting an effective survey involves consideration of multiple factors that shift depending on the audience resulting in no hard rules in administering surveys. In general, for all audiences, the topic is key to get people’s attention and for a large percentage of potential respondents the format, time involved, and often even compensation have minimal impact on response rates.

Lessons learned: Ensure that the topic is meaningful to your audience, keep it in the 10-minute mark, toss in some incentives – preferably cash – and, send several reminders to potential respondents. Then, cross your fingers and hit SEND.

–By Laura Torgerson, research associate in the Center for Medical Ethics and Health Policy at Baylor College of Medicine