‘No Effect’ Conclusions in Studies Reporting Nonsignificant Results Are Potentially Incorrect

Purpose: To examine the spectrum of effect sizes in line with “no effect” claims in clinical studies published in high-impact orthopaedic journals. Methods: Eight orthopaedic journals with the highest impact factors for 2018 were included in this study. The titles and abstracts of all journal articles published in these journals in 2019 were screened for no-effect statements. The effect sizes for effects claimed to be no effect were calculated. The asymmetry of 95% confidence interval (CI) tails of effect estimated in relation to 0 was examined by calculating the ratio between the tail lengths. Results: The analysis included 255 articles reporting coefficients sufficiently. The median Cohen's d value was 0.15 (interquartile range [IQR] 0.06 to 0.25) in the studies comparing means and the median ϕ value in the studies comparing frequency distributions was 0.06 (IQR 0.04 to 0.15). In studies reporting odds ratios or hazard ratios, the median estimate value was 1.16 (IQR 1.09 to 1.34). The median asymmetry ratio for all studies was 1.9 (IQR 1.3 to 4.0). Asymmetry ratio values exceeded 5 in 55 studies (22%). Conclusions: No-effect statements are used for a wide variety of observed effects. Despite the conclusion of no effect, observed effects advocated toward actual effects. In addition, many cases in which the asymmetry of CIs related to the 0-effect estimate showed a high tendency of effect direction. Clinical Relevance: Rather than emphasizing dichotomized interpretation of statistical inference, reporting observed coefficients with elaboration of related uncertainty and compatibility with meaningful effect sizes in the specific context is encouraged.