Keegan & Donato Consulting is often engaged to critique survey work submitted by opposing experts. We have reviewed and rebutted hundreds of consumer research studies and can help you develop an effective rebuttal strategy.
Our firm, which is located in beautiful Rye, New York, just north of Manhattan, has designed, executed, and critiqued hundreds of surveys that have been admitted into evidence in federal and state courts, at arbitration, and to the Trademark Trial and Appeal Board (TTAB).
Principals Mark Keegan and Anthony Donato offer a combined 25 years of expertise in conducting consumer research studies in the context of litigation and have extensive testimony experience.
Rebuttal and Critique
Consumer surveys are complex and require training and expertise to design and execute properly. Although a research study may appear convincing and well-executed to an untrained eye, an experienced survey expert will be able to identify weaknesses that may not be apparent to the casual observer.
When critiquing or rebutting a study, we look closely at how the research was conducted to ensure than an expert’s report is consistent with consumer survey best practices. The factors we examine may include:
- Who conducted the survey? Experts who design, execute or analyze the survey should be appropriately trained and qualified and understand survey methodology.
- What questions were asked and how were they worded? The choice of words and phrases in a question is very important for conveying its meaning and intent and ensuring that all respondents interpret the question the same way. Even small differences can affect the answers respondents give or lead them to respond a certain way.
- In what order were survey questions asked? Survey results can become biased through the ordering and structure of the questions. Questions asked early in a survey can have unintended effects on how respondents answer subsequent questions.
- How was the universe defined? What was the sampling frame used to identify the population? Did the survey results accurately reflect the demographics of people who use the product in question? A survey’s relevance is dependent upon the identification of the target population, and precautions should be taken to decrease the likelihood of sample bias.
- What method did the researcher use to select respondents? Potential respondents should be screened with initial questions to determine their eligibility to participate in the survey. If a sample group doesn’t properly match the target audience or population, it is possible that the results will be incorrect.
- Was the survey data properly analyzed and reported? Was the data analyzed in accordance with accepted statistical principles?
These are just a few of the factors that contribute to a valid and reliable study. It is not uncommon for courts to find consumer surveys poorly designed and lacking in objectivity. This could result in a survey being given little weight or excluded by the court.
For help with your consumer survey rebuttal, call Keegan & Donato at (914) 967-9421. Our education and experience conducting and critiquing consumer survey research have provided us the tools to design studies that avoid the methodological limitations that often hamper competing research studies.