I gave it four stars. You gave it two and a half stars. Fair enough. It’s just our opinion, right? Well…sort of. While it is true that match rating is a subjective process, that does not mean we cannot understand it. Too often, subjective behavior (especially as related to art) is either reduced to its simplest form, in which many of the interesting aspects of the phenomenon are lost, or treated as an enigma that cannot be studied empirically. But if we understand match rating as a form of behavior, we can define the parameters of that behavior in order to better understand what people mean when they rate matches. The problem is not the behavior, it is the tools that we use to study it. Simply put, we need better tools to understand the experience of watching and appreciating professional wrestling.
Have you ever read Roland Barthes’ Mythologies? If you are interested in academic studies of professional wrestling, your answer is probably a resounding “yes” (index fingers in the air). It’s beautiful writing and critique, but as a passionate wrestling fan, it is also…missing something. Let’s not forget, the term “fan” is short for “fanatic”. While enthusiastic, Barthes by no means appeared to be a professional wrestling fanatic. He was a scholar interested in the interesting and often paradoxical cultural phenomenon known as professional wrestling. But, at times, his analysis in “The World of Wrestling” seems entirely disconnected from my experience as a fan. Part of that can be attributed to the time (1957) and place (France) of the writing. His experiences are entirely valid but bear little resemblance to my experiences as a modern pro wrestling fan. Would his critique be different if he instead watched NJPW’s Dominion 2018? Perhaps somewhat but probably not in a truly fundamental way. His interpretation is based on a casual interest in professional wrestling rather than that of a devoted fanatic. So, can we compare his interpretation of a match to mine? Probably not with a five-star system that attempts to describe a complex, interconnected series of events involving sport, spectacle and fan engagement with one of 21 fixed categories.
Even if we assign the exact same star-rating to a match, we do not know if our rating was based on the same parameters of judgement. It is important to recognize that interobserver agreement and reliability are not the same thing. Interobserver agreement refers only to an agreement between two observers regarding the occurrence of an event. For instance, I may assign a match three stars while you also assign the match three stars. Does this mean that we had equivalent viewing experiences? It is possible that the experiences are equivalent, but it is also possible that we had vastly different experiences while viewing the match that led to similar match ratings. Perhaps I found the moves performed in the match to be simplistic and sloppy, but I greatly enjoyed the underlying story and dramatic build to a climax. Conversely, you found the execution of moves to be crisp and relevant to the narrative of the match, but you observed little evidence of investment by the live audience. As a result, we both assigned the match three stars, but our three-star ratings have very different meanings. The point is evident: while any two viewers may assign similar ratings to a match, the underlying experience contributing to those ratings may be vastly different. A reliable scale must not only ensure that similar observations lead to similar ratings but that the assigned ratings are equivalent in meaning.
The purpose of this post is not to disparage the five-star system so often (but incorrectly) attributed to Dave Meltzer. It has served as an incredible tool for describing our experience of professional wrestling. But as Meltzer has often acknowledged, it was never intended to be an objective or authoritative indication of match quality. Instead, it is a shorthand tool that allows fans to efficiently communicate the level of enjoyment (s)he experienced from watching a match. But as a tool for the empirical investigation of professional wrestling fandom, it is insufficient. The five-star system fails to get underneath the rating. Consider this classic example: Meltzer gave Hogan vs. Andre from WMIII a dreadful negative four-star rating, but the match is venerated as one of the greatest moments in wrestling history by many WWF/WWE fans. How could they be so far apart? While we can speculate (e.g., Meltzer values in-ring athleticism, whereas WWF fans value spectacle), we cannot truly know the answer given a star rating. The tool quantifies the aggregated experience of watching the match, but it does little to clarify why that experience occurred.
What is the solution to this conundrum? Ultimately, we need a better instrument to empirically investigate the experience of viewing professional wrestling. We need an instrument that better clarifies the why of the professional wrestling experience. With better tools, we can better understand professional wrestling viewing as a behavior and begin to understand the variable underlying that behavior. Not only will such an instrument allow us to understand the factors that contribute to the evaluation of match quality, it will allow us to better understand differences between professional wrestling fans and fans of other sports and entertainment mediums. Such an instrument should clearly identify and define aspects of pro wrestling matches. Such an instrument should provide guidelines for how users should quantify their experience. Such an instrument should include anchor points from which deviation can be understood. Such an instrument should be as objective as possible in order to allow for more meaningful comparisons. Pro wrestling viewing is and always will be a subjective experience, but that does not mean that we cannot move toward a more objective understanding of the phenomenon.