*Geek Box: ROBINS-E
The Risk of Bias in Non-Randomized Studies of Exposures [ROBINS-E] is a tool for grading risk of bias, one of several tools that may be used in a systematic review or other synthesis of evidence. It is based off seven domains, and within each domain a rating of low, moderate, serious, or critical risk of bias is graded.
However, there are a number of potential issues with the ROBINS-E that make it a rather insensitive tool for assessing bias. The first issue is that the ROBINS-E tool can result in studies being rated as ‘critical’ for bias, even though a study may have critical risk of bias in 7/7 domains, while other studies may have critical risk of bias in only 1/7 domains or 4/7 domains. Only a study with low risk of bias in 7/7 domains will be rated with low risk of bias overall. This is problematic it assumes each domain has equal weighting, when the actual influence of a given domain on the quality assessment of a study differs. It assumes the overall “grade” of each study is equivocal, when they are not.
Another issue is that it is based on comparing observational studies to a randomised controlled trial, but this is an imbalanced comparison because there are numerous facets of an RCT that an observational study simply cannot match. It would be far more efficacious to take the best possible design and execution of an observational study as the benchmark for comparison. This is like taking a rally car and comparing it to a formula-1 car and complaining when the rally car doesn’t have the characteristics of the F-1 car. It also assumes that RCTs, as the comparative design, are impeccable, which is simply not the case: RCTs have plenty of their own limitations.
Overall, it does not appear to be a useful tool for qualitatively assessing bias in observational research.