The development of catheter-based renal sympathetic denervation aroused great hopes for the treatment of hypertension, the principal modifiable cardiovascular risk factor. Initial trials reported reductions of around 30 mmHg in systolic blood pressure in patients with resistant hypertension.1,2 The therapeutic effect appeared to be so strong and obvious that the technique was soon widely adopted in many countries. More than 50 denervation systems were developed3 and national and international registries confirmed significant falls in blood pressure. However, surprisingly, SYMPLICITY HTN-3, the largest randomized trial on renal denervation and the first to be blinded and placebo-controlled, showed no significant effect.4 A lively debate ensued. Theories concerning ethnicity, techniques and concomitant drug therapy were put forward, but there was little discussion of the central question of the design of previous studies and of threats to their validity.

The article by Sousa et al. in the current issue of the Journal5 presents results from a registry of 31 patients with resistant systemic hypertension who underwent catheter-based renal denervation. Patient selection was based on criteria in accordance with the most important clinical trials on this intervention.1,2,4 One-year data focused on patients’ hemodynamic response and renal function parameters. The authors report significant decreases in office and 24-hour ambulatory blood pressure measurements. What is striking about this study is the significant reduction in the urine albumin-to-creatinine ratio – from 25.6 mg/g (interquartile range [IQR] 8.7-382.8 mg/g) to 15.9 mg/g (IQR 4.4-55.0 mg/g), p=0.009 – in the subgroup with significant decreases in blood pressure (responders).5 This finding is particularly interesting because at first sight it appears to be an indication of the effectiveness of the treatment, establishing a causal chain between denervation, falls in blood pressure, and improvement or reversal of target organ damage. In the current climate of skepticism concerning the therapeutic effectiveness of denervation, this result could be seen as a positive sign, suggesting that denervation may have benefits beyond reducing blood pressure. Albuminuria is an independent marker of renal and cardiovascular risk, and is an objective, investigator-independent parameter that is uninfluenced by the placebo effect. This finding is not unprecedented, as there are other reports in the literature of reduced albuminuria following renal denervation.6 The effect appears to be dependent on reducing blood pressure, since in a series of 54 patients in whom blood pressure decreased only slightly following denervation, no significant decrease in albuminuria was seen.7 A similar result was seen in the subgroup on non-responders in the article under discussion.5

However, it is important to point out the main limitation of the study's patient registry. Of the 65 patients enrolled between July 2011 and April 2015,8 only 31 completed the assessment process, even though all patients had had more than 12 months of follow-up by the time the article was submitted. The authors do not specify why over half the patients did not complete the follow-up protocol. This may affect the interpretation of the results. The 31 patients who completed the protocol may have been better at complying with the follow-up schedule and with the drug regimen, in which case the falls in blood pressure and in albuminuria may have occurred independently of the denervation procedure. If this group were particularly motivated to modify their lifestyles and adhere to therapy, and also had access to additional consultations at the hypertension clinic, this by itself could explain the results. In other words, the authors describe a benefit that, due to the characteristics of the study population and the nature of the follow-up, is not necessarily only to be interpreted as a biological response to renal denervation. There is thus an important threat to the study's internal validity, one described in the literature as differential mortality or experimental mortality9 (‘mortality’ in this sense meaning loss of subjects during the study, not necessarily by literal death). Subjects who remain may be more motivated and thus more likely to comply with study requirements.

Other threats to internal validity are commonly found in almost all clinical studies on renal denervation.10 The effect of renal denervation on resistant hypertension is an interesting case study on experimental design and threats to internal validity.10 Most studies are open and do not include a control group. Patient selection is mainly based on blood pressure above a certain limit. Given the natural variability of blood pressure, this method will tend to select individuals who at that point presented a higher value than their long-term average, and naturally, over the course of the trial their observed levels will move closer to the long-term average for each individual – the effect known as regression to the mean,11,12 by which individuals selected on the basis of extreme characteristics will tend to regress to their long-term average irrespective of treatment.

Over the course of the trial, another threat to internal validity may appear, arising from the method of measuring blood pressure. After the patient starts an antihypertensive treatment, the physician will tend in practice to interpret measurements that do not indicate a fall in blood pressure as incorrect and will usually repeat the measurement, ignoring the initial reading.12 Lower readings are interpreted as correct and higher readings as due to the white coat effect or simple measurement error, particularly when the therapy is believed to be effective. If in a registry or open trial not all measurements have to be recorded, this may result in underestimation of blood pressure after beginning of therapy. This effect has been termed asymmetrical data handling12 and is an important threat to the internal validity of trials on hypertension. It can be overcome by implementing rigorous recording protocols, the use of digital blood pressure monitors that record all measurements taken, and/or ambulatory blood pressure monitoring.

Another important threat is when there is an actual fall in blood pressure that is not caused by a biological effect of the treatment under study, but is in fact due to confounding factors. This may result from a strong conviction of the benefit of the therapy, i.e. the placebo effect,13 or the psychological effect of increased motivation, good adherence to medical therapy and adoption of a healthier lifestyle following inclusion in an innovative treatment program that brings access to more consultations and improved surveillance. The mere awareness of being observed improves performance (the Hawthorne effect).14

In a recent meta-analysis, Howard et al.12 elegantly quantified these three threats to internal validity in studies of renal denervation: regression to the mean, asymmetrical data handling, and confounding effects. The effect of regression to the mean was not significant, but this is the only one of the three types of bias that can be overcome by the inclusion of a randomized control group in an open trial. The other two resulted in a combined mean underestimation of blood pressure of around 19 mmHg, and could only have been avoided by blinding and by sham procedures in the control group.12 The authors give the example of a greater fall in blood pressure as measured in the office than in ambulatory monitoring, only in open trials. The same effect has been found in open trials of antihypertensive drugs,15 but was not seen in blinded trials, including those which demonstrated a beneficial effect of the drug under study.15

In conclusion, registries and open clinical trials, even when rigorous and correctly performed, are unable to overcome the most important threats to the internal validity of studies testing new treatments for hypertension. Improvement in markers of target organ damage in patients with blood pressure reductions only increases the likelihood that this reduction is real rather than resulting from regression to the mean or asymmetrical data handling. However, even then, it does not guarantee that a biological effect of the new treatment is responsible, since the observed changes could be due to confounding effects.