QUANTITATIVE RESEARCH
Last time we talked about Qualitative research (collecting and interpreting non-numerical data, rarely used in behavior and welfare research) and quantitative research (involves numbers) which can be obtained from experiments (which we covered last time), data bases, and surveys. Today we will cover surveys and the various ways surveys can be used to generate data.
SURVEYS & QUESTIONNAIRES
Surveys are frequently used in behavior and welfare research. To paraphrase what Dr. Nina von Keyserlingk (Professor of Animal Welfare at University of British Columbia) mentioned in a recent webinar on dairy cattle welfare “They are cheap!” Research is expensive and funding is often difficult to obtain; this is especially true in areas like behavior, welfare, and the human-animal bond, which may be considered “soft” or “fuzzy” science. And funding is complicated by the universities “take” of overhead (a percentage amount above the awarded grant, for the NIH that is 52%). Universities discourage faculty from submitting grants to groups or agencies that do not provide overhead, and many of the groups that fund behavior, welfare, and human-animal interaction research do not provide overhead or do not provide it at the NIH level. So what are researchers to do? Every graduate student in behavior and welfare (as well as other disciplines) needs to do a research project, and surveys are a relatively inexpensive way to accomplish this. And just because they are cost effective doesn’t mean the science is bad.
In a survey study from the U of Guelph, Jacobs and colleagues examined canine resource guarding (RG) in the presence of other dogs. They hypothesized that avoidance and rapid ingestion would likely occur together and have a negative relationship with aggressive RG. They also hypothesized that aggressive RG would have a positive relationship with positive punishment, negative reinforcement training, and owner manipulation of food. All patterns of RG would have a positive relationship with fear and impulsivity. The behavior categories and questionnaire were developed and validated. Participants were recruited on line using a snow-balling technique (basically participants invite other individuals to participate in the survey). The first part of the survey was a tutorial and eligibility-assessment quiz to inform and assess participants’ ability to correctly identify behavior patterns related to different forms of RG. All terms assessed were defined. Four multi-level logistic regression models were fitted to determine the association between the independent variables and different categories of RG. A statistical model is a mathematical representation (or mathematical model) of observed data and can be thought of a statistical assumption – the assumption allows researchers to calculate the probability of an event. Information was provided for 4857 individual dogs from 3068 individuals, 72% of individuals correctly identified the categories of RG and were included in data analysis. The authors conclude that their research was unique in that it applied a screening tool to increase the validity of participant’s responses and reduce the potential for misclassification bias. Distinct relationships between the patterns of RG in the presence of other dogs were identified. Dogs that express RG aggression were less likely to express RG avoidance or RG rapid ingestion, which were likely to occur together. Taken together, the three studies by Jacobs et al. provide an in depth analysis of RG, and each manuscript stands on its own. My only critique of this is that the journal allowed citations for “submitted” manuscripts. Submitted manuscripts may never be published, however in this case they were.
Now let’s look at another survey study. The stated purpose of the small animal practitioner survey by Kipperman et al. was “to assess the frequency with which ethical dilemmas between client and patient interests occur; identify some of the common circumstances that cause ethical dilemmas; determine the degree of moral stress elicited by these conditions; characterise the views of practitioners regarding euthanasia decisions; determine the methods veterinarians use to resolve ethical dilemmas; determine which interest veterinarians prioritise when client and patient interests conflict; assess measures that might reduce the impact of ethical dilemmas; document the prevalence of ethics training and the influence of such training on managing ethical dilemmas and evaluate differences among veterinarians regarding ethical dilemmas on the basis of gender, practice role (owner vs associate), practice type (referral vs other), years of practice experience, age or number of veterinarians in the practice.” The survey (unavailable unless you contact the owner) was piloted on 12 small animal practitioners and the final survey consisted of 5 sections and 29 questions. Distribution was an email invitation to 4100 members of the California Veterinary Medical Association (CVMA), 4400 members of the Humane Society Veterinary Medical Association (HSVMA), 93 members of the Society for Veterinary Medical Ethics and 1309 small animal specialists of the American College of Veterinary Internal Medicine. Response rate was less than 5%. So already, by reading the purpose(s) and the methods, I see “red flags.” The authors propose a huge number of purposes with no clear question or hypothesis. Distribution of the survey is also problematic, as it does not represent the small animal practitioner in the US (as stated in the title). The two largest groups invited to participate were members of the CVMA and HSVMA. In 2008, the Association of Veterinarians for Animal Rights (AVAR) and The Humane Society of the United States (HSUS) approved a corporate combination agreement that creating the HSVMA. I read this paper because we discussed it in our animal welfare journal club, but for “casual reading” I would not have continued reading after the “red flag alert:” no clear cut questions, survey unavailable on line, skewed sample population, and dismal response rate to the questionnaire (nonresponsive bias).
Vas and colleagues from Miklosi’s group took a different twist, they wanted to develop a questionnaire assessing attention skills, impulsivity, and motor activity in pet dogs using a dog specific questionnaire based on a human parental questionnaire (ADHD RS Parent version). Evidence suggests that the dog could serve as a model to study the evolutionary emergence and regulating mechanisms of human behavior, and to understand the mechanisms of human behavior disorders like attention deficit hyperactivity disorder (ADHD). They assessed reliability and validity of the questionnaire using standard psychological methods for assessing parental perceptions of the behavior of their children. To be useful, the dog-ADHD rating scale owner version of the questionnaire must be reliable and validated. Dogs (n=220) and their owners (168) were recruited on a voluntary basis from participants of the family dog research database. The mean of the two predetermined subscales (inattention and activity–impulsivity) was determined. Cronbach’s alpha (α coefficient) is a common statistical way to measure reliability. Reliability refers to the consistency of a measure: over time (test-retest reliability), across items (internal consistency), and across different researchers (inter-rater reliability). The higher the α coefficient (0 to 1), the more the items have shared covariance and probably measure the same underlying concept (thing). An α coefficient of greater than 0.7 is considered the threshold for statistical significance. In this study the α coefficient was 0.784 for inattention and 0.734 for activity–impulsivity, suggesting that there is consistency with in the predetermined subscales. Validity is the extent to which a measure “covers” the construct (broad concepts or topics) of interest. External consistency was statistically evaluated by group comparisons (age: juveniles/adults/older dogs; gender: females/males; training: untrained/beginner/advanced). The questionnaire and all of the data was included in the published manuscript. The authors conclude “the application of human ADHD questionnaire is a reliable and valid method of assessing attention skills and activity in dogs. To our knowledge this is the first study on dogs using the adaptation of a questionnaire developed for studying children’s behaviour (through the estimation of their parents).“
Surveys can also be combined with other type of research to look at a question from two different angles, or as in the paper by Lit et al from Oberbauer’s group to look at the genetics of behavior. Although originally bred as a herding dog, the Belgian Malinois has recently found a niche in law enforcement, military environments, and bite-sports. Some Malinois owners have described behavioral changes, including unpredictable aggression, eyes glazing over, and seizures, that are a danger to the handler and public health, and may result in behavioral euthanasia. One potential contributor to these behavioral changes could be modification in the production, transport, or metabolism of dopamine. Dopamine is a neurotransmitter involved in locomotor activity, goal-oriented behavior, reward processing, and seizure activity. The dopamine transporter (produced by the dopamine transporter gene) is responsible for removal of dopamine and regulating signal amplitude and duration in the synaptic cleft (space between neurons). There are two alleles (alternative form of the gene) for the dopamine transporter gene. Allele “1” contains one copy of 38-base pair string of nucleotides and allele “2” contains two copies. The authors hypothesized that that the 1/1 genotype was over represented in Malinois and was associated with behavioral changes. Of the 144 Malinois genotyped, 84% were either 1/1 or 1/2 , while 92% of the other breeds genotyped were 2/2. To correlate behavior with genotype, owners (n=131) answered a short behavioral questionnaire about seizures, glazing over, and unprovoked aggression. Of dogs whose owners reported yes to the behavior questions 67.4% were homozygous for the 1/1 genotype, 26.1% of heterozygotes (1/2), and 6.5% of the dogs homozygous for the 2/2 genotype. The authors conclude that in Malinois there is an association, but likely not causation, between the 1/1 and 1/2 genotype and seizures, eyes glazing over, and unprovoked aggression. To identify a specific allele as causative, the phenotype must be clear, however, “behavior genetics are notably complex, with behaviors affected by actions and interactions of many genes; any findings may not be relevant to other breeds because of different genetic backgrounds.” This is an extremely thorough study with a large amount of genetic data and less robust behavioral data.
Writing good surveys and questionnaires is hard. You can’t just write a dozen questions, put them on social media, tabulate the responses and “call it science.” The questions need to be piloted, revised, and tested for validity and reliability. Basically if you answer the questions at two different time points the answers should be roughly the same and you and other members of your group should also have similar responses. What else do you need to consider? Both the number of responses and the response rate. In 2008 the editors of the American Journal of Pharmaceutical Education increased their expectations for survey response rates to 60% and for survey research intended to represent all schools and colleges of pharmacy, a response rate of ≥ 80% is expected. The latter refers to “representativeness” or how well the sample is representative of the population of interest.
SO the next time you read a survey research paper ask yourself: Did they ask an answerable question? How many people who received the survey answered it (response rate)? Did the sample (number of participants) represent the population of interest? Was the sample large enough to draw meaningful conclusions? Before you get to the statistics section you should have a pretty good idea of the quality of the study. Next time we will cover data based research.
CITATIONS
Jacobs JA, Coe JB, Pearl DL, Widowski TM, Niel L. Factors associated with canine resource guarding behaviour in the presence of dogs: A cross-sectional survey of dog owners. Prev Vet Med 161:134–142, 2018. https://doi.org/10.1016/j.prevetmed.2017.02.004
Kipperman B, Morris P, Rollin B. Ethical dilemmas encountered by small animal veterinarians: characterisation, responses, consequences and beliefs regarding euthanasia. Vet Rec 182:548, 2018.
Vas J, Topa ́l J, Pe ́ch E, Miklo ́si A. Measuring attention deficit and activity in dogs: A new application and validation of a human ADHD questionnaire. Appl Anim Behav Sci 103:105–117, 2007.
Lit L, Belanger JM, Boehm D, et al. Characterization of a dopamine transporter polymorphism and behavior in Belgian Malinois. BMC Genetics 2013, 14:45 http://www.biomedcentral.com/1471-2156/14/45
Lana Kaiser MD, DVM, born in Buffalo, NY, received a BA in English from SUNY at Buffalo with plans to be a poet. She is a graduate of Michigan State College of Human Medicine, a Board Certified (Human) Internist, a cattle veterinarian, and a Emeritus Professor in the Departments of Physiology and Medicine, in the College of Human Medicine.
Trained as a biomedical researcher with a research focus on cardiovascular pathophysiology and parasitology, she is also interested in scientifically studying the interaction between humans and animals, and has published in both scientific disciplines. A 1995 graduate of the College of Veterinary Medicine, MSU she resides on a farm in Mason, MI where she raises Maine-Anjou and Red Angus cattle.
She has a mobile beef cattle practice, consults for several national agricultural entities, and has written and lectured about animal welfare, animal health, genetic defects, and human-livestock interactions at the state and national levels. She is involved in issues of animal behavior and welfare at the county, state and national level, was a founding member of MSU’s Human-Animal Bond Initiative and coordinator of MVMAs Annual Animal Welfare Conference and Animal Behavior Conference. She has been active in the MVMA Animal Welfare, Legislative, and Food Animal Practivce Committees, AABP Animal Welfare Committee, and Dean Foods Animal Welfare Advisory Council.
She enjoys clicker training her four dogs, her cow horse, and the occasional cow.
