Intrinsic judgment error in men’s championship world surf league: WSL 2021
DOI:
https://doi.org/10.47197/retos.v64.106821Keywords:
Reliability, Validity, Surfing, Judgment, Fair gameAbstract
Introduction: Surfers’ performances are subjectively ranked by 5 judges. Low reliability and validity in judgment may lead to preventable errors and unfair scores.
Objective: The aims of the present study is to describe the judgment error for each wave surfed in world surf league (WSL) championship tour of 2021, establish between-judge reliability, and establish judgment validity comparing each judge score with the final control score of the wave.
Methodology: To describe the judgment error, we analyzed the available WSL data related to the 2021 Men’s Championship Tour (4,095 waves; 20,475 scores).
Results: We found an inverted ‘U’-shaped pattern for the judgment error score vs. control score, explained by a quadratic regression model (R = 0.52; SEE = 0.10). The reliability produced an excellent intraclass correlation coefficient (CI95% = 0.97, 1.00), with a between judge (typical) error of 0.15. Validity analyses indicated a minimal real difference of 0.49 in the sum of two waves between the surfers for having 95% certainty for the heat winner.
Conclusion: We recommend WSL incorporate intrinsic judgment errors to increase fairness and trust in the WSL championship tour.
References
Atkinson, G., & Nevill, A. M. (1998). Statistical methods for assessing measurement error (reliability) in variables relevant to sports medicine. Sports Medicine, 26(4), 217-238. https://doi.org/10.2165/00007256-199826040-00002
Copay, A. G., Subach, B. R., Glassman, S. D., Polly, D. W., Jr., & Schuler, T. C. (2007). Understanding the minimum clinically important difference: a review of concepts and methods. Spine Journal, 7(5), 541-546. https://doi.org/10.1016/j.spinee.2007.01.008
Emerson, J. W., Seltzer, M., & Lin, D. (2009). Assessing Judging Bias: An Example From the 2000 Olympic Games. The American Statistician, 63(2), 124-131. https://doi.org/10.1198/tast.2009.0026
Flessas, K., Mylonas, D., Panagiotaropoulou, G., Tsopani, D., Korda, A., Siettos, C., . . . Smyrnis, N. (2015). Judging the judges' performance in rhythmic gymnastics. Medicine and Science in Sports and Exercise, 47(3), 640-648. https://doi.org/10.1249/mss.0000000000000425
Heiniger, S., & Mercier, H. (2019). Judging the judges: A general framework for evaluating the performance of international sports judges. arXiv, Pre Print(10055), 1-9. https://arxiv.org/abs/1807.10055.
Heiniger, S., & Mercier, H. (2021). Judging the judges: evaluating the accuracy and national bias of international gymnastics judges. Journal of Quantitative Analysis in Sports, 17(4), 289-305. https://doi.org/https://doi.org/10.1515/jqas-2019-0113
Hopkins, W. G. (2000). Measures of reliability in sports medicine and science. Sports Medicine, 30(1), 1-15. https://doi.org/10.2165/00007256-200030010-00001
Koo, T. K., & Li, M. Y. (2016). A Guideline of Selecting and Reporting Intraclass Correlation Coefficients for Reliability Research. Journal of Chiropractic Medicine, 15(2), 155-163. https://doi.org/10.1016/j.jcm.2016.02.012
Leandro, C., Avila-Carvalho, L., Sierra-Palmeiro, E., & Bobo-Arce, M. (2017). Judging in Rhythmic Gymnastics at Different Levels of Performance. Journal of Human Kinetics, 60, 159-165. https://doi.org/10.1515/hukin-2017-0099
Leskošek, B., Čuk, I., Karácsony, I., Pajek, J., & Bučar, M. (2010). Reliability and validity of judging in men's artistic gymnastics at the 2009 university games. Science of Gymnastics Journal, 2(1), 25-34.
Lyngstad, T. H., Härkönen, J., & Rønneberg, L. T. S. (2020). Nationalistic bias in sport performance evaluations: An example from the ski jumping world cup. European Journal for Sport and Society, 17(3), 250-264. https://doi.org/10.1080/16138171.2020.1792628
McGregor, T., & Wills, S. (2016). Natural Assets: Surfing a Wave of Economic Growth. https://EconPapers.repec.org/RePEc:syd:wpaper:2016-06
Muraven, M., & Baumeister, R. F. (2000). Self-regulation and depletion of limited resources: does self-control resemble a muscle? Psychol Bull, 126(2), 247-259. https://doi.org/10.1037/0033-2909.126.2.247
Pajek, M. B., Cuk, I., Pajek, J., Kovač, M., & Leskošek, B. (2013). Is the quality of judging in women artistic gymnastics equivalent at major competitions of different levels? Journal of Human Kinetics, 37, 173-181. https://doi.org/10.2478/hukin-2013-0038
Premelč, J., Vučković, G., James, N., & Leskošek, B. (2019). Reliability of Judging in DanceSport. Front Psychol, 10, 1001. https://doi.org/10.3389/fpsyg.2019.01001
R Core Team. (2021). R: A language and environment for statistical computing. R Foundation for Statistical Computing. https://www.R-project.org/
Revelle, W. (2021). Procedures for personality and psychological research https://CRAN.R-project.org/package=psych
Rice, E. L. (2021). Surfing. J Sports Med Phys Fitness, 61(8), 1098-1103. https://doi.org/10.23736/s0022-4707.21.12847-6
Román, C., Borja, A., Uyarra, M. C., & Pouso, S. (2022). Surfing the waves: Environmental and socio-economic aspects of surf tourism and recreation. Sci Total Environ, 826, 154122. https://doi.org/10.1016/j.scitotenv.2022.154122
Santos González, D. (2024). No vas a dejar el surf. Reflexiones sobre las motivaciones y condicionantes sociales que conducen a su práctica en entornos naturales y artificiales (You won't quit surfing. Reflections on the motivations and social conditions leading to its practice in natural and artificial environments). Retos, 59, 903-911. https://doi.org/10.47197/retos.v59.108656
Ste-Marie, D. M. (1996). International Bias in Gymnastic Judging: Conscious or Unconscious Influences? Perceptual and Motor Skills, 83(3), 963-975. https://doi.org/10.2466/pms.1996.83.3.963
World Surf League. (2022). Rules and regulations. https://www.worldsurfleague.com/pages/rules-and-regulations
Downloads
Published
Issue
Section
License
Copyright (c) 2025 Tony Meireles Santos, Lucas Eduardo Rodrigues Santos, Ítalo Vinicius, Cayque Brietzke, Lucas Camilo Pereira, Paulo Henrique Melo, Thaiene Camila Beltrão Moura, Taddeo De Negri, Hassan Mohamed Elsangedy, Flávio Oliveira Pires
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and ensure the magazine the right to be the first publication of the work as licensed under a Creative Commons Attribution License that allows others to share the work with an acknowledgment of authorship of the work and the initial publication in this magazine.
- Authors can establish separate additional agreements for non-exclusive distribution of the version of the work published in the journal (eg, to an institutional repository or publish it in a book), with an acknowledgment of its initial publication in this journal.
- Is allowed and authors are encouraged to disseminate their work electronically (eg, in institutional repositories or on their own website) prior to and during the submission process, as it can lead to productive exchanges, as well as to a subpoena more Early and more of published work (See The Effect of Open Access) (in English).
This journal provides immediate open access to its content (BOAI, http://legacy.earlham.edu/~peters/fos/boaifaq.htm#openaccess) on the principle that making research freely available to the public supports a greater global exchange of knowledge. The authors may download the papers from the journal website, or will be provided with the PDF version of the article via e-mail.
