Most attorneys are familiar with focus groups and mock trials. Perhaps you’ve heard about a big data study but don’t know what it is. Perhaps you’ve run one but are still learning how to maximize the impact of the information you gleaned from it. In this post, I will go through what a Big Data study is, how it is different from a focus group or mock trial, and what it is best used for.
Until recently, the only method we had for testing a case was a traditional focus group or mock trial. These tools are still invaluable for testing the case, finding holes, case framing, and guiding you through discovery. But they always had limitations, namely that they could never predict the outcome of trial. Prediction was never their purpose: the purpose was stress testing and finding themes and worst-case scenarios. If any consultant ever told you that they could tell your chances of winning or losing, the value of the case, or a juror profile based off anything less than 6 panels of focus groups (and more like 20 panels for a juror profile) at a bare minimum, they were lying to you. The participant numbers in a focus group or mock trial are simply too small to have any way to determine statistical significance.
Enter, Big Data studies. Big Data fills the gaps that focus groups and mock trials could never fill and these tools all work synergistically together.
So, what is a Big Data study? While some attorneys call them online focus groups, this is somewhat of a misnomer. You won’t see these jurors or talk to them and they won’t talk to each other (this is their main limitation – you can’t get a good feel for WHY people feel a certain way or how they communicate with each other in a Big Data study). A Big Data study is really an online survey. Depending on who runs it (more on AI focus groups/data studies in the next post – be sure to read!), these should be sampled from real human beings and preferably in your venue or similar venues. The studies I run are through Campbell Law, so I will discuss how those are run as I personally believe they are the best quality Big Data studies.
What do jurors see and react to? 300+ jurors read scripts drafted by you (and edited/collaborated on by me if you run them through me) that lay out the plaintiff and defense arguments, including photos and video. Throughout, they are given attention checks to ensure they are paying attention. They answer 50+ juror intake questions, some of which I personally customize based on the case, and then answer leaning/attitudinal questions at the end along with a verdict form.
Importantly, jurors are broken into 3 groups of approximately 100 each. All jurors review the same information except that they are presented with different damages asks at the end. So, one group might be asked to award $10M, the second group asked for $25M, and the third group $40M. You might also add extra groups of 100 jurors to test different variables: what if defense admits liability? What if the video is in or out? What if a party settles? All of that information is analyzed and returned to us in a 60-70 page report.
What is in the report? The report contains a wealth of information, so let’s break down the important parts:
Win Rate Based on Damages Ask: The first thing you will see in the report is your win rate based on what amount jurors were asked to award and any other variables tested. Perhaps surprisingly, your chances of winning or losing can often depend on the number you throw out at jurors. Below, you can see three different damages asks as well as a test with one party removed (NoHouston). Since the high ask tested best overall, we then ran that same High ask with Houston out of the case so we can compare whether having Houston in or out affects the verdict. This chart shows that the best ask is the High ask at an 89% win rate and that if Houston is out of the case, that greatly hurts our chances of winning as the win rate drops by 13%.

Likely Verdict: The next thing you will see in the report is the amount of money you are likely to recover (which accounts for defense leaning jurors and the deliberative process) on your best day, worst day, and average/median. Here is the same case. You can see the average expected recovery is about the same from low to medium, though the medium drops slightly, and then climbs again in the high, but drops if Houston is out. Perhaps shocking is that the best case and worst case scenario is better in the low ask and high ask but worst in the medium ask. This shows how important the damages ask is. Selecting a number in the middle can tank your case. While the best case scenario is higher without Houston, it’s not worth having Houston out of the case as the win rate is lower and average recovery is also much lower:

Themes: The report will give you some information on framing. Jurors are asked to rank the two most important issues in making their decisions. Those are tallied and ranked to show you the most impactful themes, biggest danger points, and what potential issues rank low (and therefore should be eliminated from your main themes). While this is ranking is mostly accurate, there is a limitation in that humans are notoriously horrible self-reporters. Therefore, a juror may list something as the reason they came to their decision when in fact something else entirely was driving it. This is where I like to add in focus groups and mock trials to hear jurors talk. In watching their thought process, you can often determine if something hidden is driving them or if some other issue was more important but perhaps seemed so obvious that they skipped past it. Here are samples from a hospital sexual abuse case showing the top themes for plaintiff jurors and for defense jurors:


How Much Does It Matter?: The report will tell us what jurors think about the case issues (for example, “I think the defendant’s motive was pure but they were just being careless”) as well as how much those attitudes and beliefs matter in terms of your win rate or damages. The charts on these tend to be hard for attorneys to read and decipher, so I won’t share the chart, but in simple terms, it will tell us how much that sentiment mattered for win rate and damages. Someone who agrees the defendant was careless but had pure motives, for example, may be 60% worse for you on liability compared to the average juror in the study and may give 45% less money. Consider a statement “I think the plaintiff was likeable” garners 50% agreement and 50% disagreement. That means half the jurors dislike your plaintiff. But, consider if the data shows that you are still winning over the jurors who dislike your plaintiff. That may mean you don’t have to worry about your plaintiff being unlikeable as it’s not affecting win rate or damages.
Juror Profile: In focus groups and mock trials, we always warned against profiling jurors. It’s not always true that conservatives are bad for you or certain ethnicities and ages are good or bad; but, sometimes it is! The trouble with smaller sample research is that there is no way to know. If we run a four panel mock trial and 4 of the 5 African-Americans are bad for you, does that mean you should be wary of African-American jurors? Perhaps, but perhaps you simply flipped a coin too few times for the average to come out to 50%. If your first 4 flips are tails, it would be a gave mistake to assume a coin flip is 80% tails over time. We need over 200 jurors to find enough data points to compile a juror profile. Sometimes, the strangest things show up, such as people they admire (in one case, those who admired Angelina Jolie were 167% better on damages for us) or TV shows they regularly watch (NFL football watchers was a -44% on liability on another of my cases). But we also see traditional data points show up, such as tort reform attitudes or case-specific backgrounds and beliefs. This can help aid the jury selection process in addition to more holistic questioning. Here is a sample of a scoring sheet I compiled for a client based on their data. In this case, we had very little time for voir dire, so we ran through as many of these questions as we could in the time permitted:

Where Is Data Weak?
Data studies are powerful assets to trial preparation, but they do have some limitations. Personally, I need to hear people talk. It gives me a 3-D view of what the data is showing and allows me to better frame the case. As mentioned, you also have to be aware of self-reporting biases, which is again where mock trials and focus groups come in useful.
Data cannot capture strong personalities of counsel or defendants of whom you do not have video evidence. I had one case that I’m convinced we won due to the horrific personality of the defendant (along with a strong jury selection).
I also worry about really complex cases in data studies as it can be hard to determine when jurors are simply misunderstanding or not entirely following an argument. Mock trials and focus groups would reveal this, but it can hide in the data.
How Do I Use Data?
In very large cases (8-figures and above), I strongly prefer a mix of mock trials and data studies. I believe that captures the full picture and allows you the best all-encompassing view of framing as well as prediction. In 7-figure cases, we may run a smaller focus group and then data. In other cases, we may pick one or the other depending on what is most advantageous to the case.
I have a relationship with Campbell Law in which I can run data for/with you for the same cost as you would pay going direct to the Campbells. This is due to our synergistic relationship in that I can help prepare the data scripts with you, hand off a polished script to them to run, and then help you to understand the report at the end. They do less work, you get a consultant for free, and it gets me involved in case which may need further help such as jury selection or focus groups and mock trials. In fact, it often saves money in the end to have me run the data as it will decrease intake fees for me to get up to speed on the case if you need my assistance otherwise.
Stay tuned for the next post regarding AI testing of cases. If you have not yet subscribed, you may subscribe to this blog to get automatic updates in your email when I post.
If I can help with your case, please contact me at: Jessica@TrialDynamics.net







