Searches were performed on YouTube with a computer connected to the Internet in Portugal, between 7 May and 5 August 2019, without prior login and with cookies deleted. The combination of terms used for the search were, in no order of importance: “ataque + coração” (heart + attack), “ataque + cardaíco”(cardiac + attack), “enfarte – cerebral” and “infarto – cerebral” (infarction - brain) (in order to exclude videos referring to stroke), “enfarte + miocárdio” and “infarto + miocárdio” (myocardial + infarction), “enfarte + agudo + miocárdio” and “infarto + agudo + miocárdio” (acute + myocardial + infarction), , “trombose + coração” (thrombosis + heart) and “trombose + coronária” (thrombosis + coronary). Terms such as “ST-segment elevation” or “ST-elevation” were excluded from the search because they are not frequently searched by the general public. Terms such as “angina”, “angina + chest” or “angina pectoris” were also excluded, given the focus exclusively on AMI.

The search results for each term were sorted according to relevance, without applying any additional search filters. Videos presented after position 100 in the search results for each term were excluded as it was assumed that there was a low probability of the videos after this point being viewed.

From the initial selection of 1000 videos (10 terms x first 100 videos) obtained from the different search terms, the following were excluded: (1) videos considered irrelevant for not addressing the topic under study (n = 316), (2) videos that were not in Portuguese language (n = 106), (3) videos with technical problems or without audio due to low adherence to watching them (n = 24) and (4) duplicate or repeated videos between searches (n = 345, in this case the first result was analyzed and the duplicates excluded). Videos subdivided into multiple parts were counted as a single video (n = 42). Thus, our search returned a total of 242 eligible videos. Supplementary Table 1 lists the 1000 videos evaluated and Figure 1 summarizes the flow chart of video search and selection.

Figure 1.

Flowchart of video selection.

133 videos with multiple exclusion criteria (18 irrelevant videos and non-Portuguese language, 14 irrelevant videos without audio, one video without audio and non-Portuguese language).

(0.2MB).

The evaluation of eligible videos was performed by two observers (I.F. and D.M.). The following parameters were recorded for each video: upload date, number of views, video duration (in minutes), and user opinion (in the form of likes and dislikes). For videos divided into multiple parts, the upload date of the last video was recorded, and the average number of views and like/dislike ratio were counted. It was also documented whether the video provided health information or not, the target audience (patients, general public, or health professionals), and the typology of the video (humorous, musical, animation, advertising a drug, hospital or professional, or regular - where the remaining videos are included). In cases where multiple typologies were identified for the same video, a consensus was reached among three researchers (I.F., D.M., and J.B.A.) about the typology that best represented the video under analysis.

The videos were further characterized as to their source/origin: personal experience, news outlets, entertainment television programs, scientific societies, pharmaceutical companies, colleges or educational centers, health or government institutions, individual trainers without affiliation, or other media (videos not included in the previous categories).

The content of the videos was evaluated with regard to the topic being addressed: pathophysiology, signs and symptoms, diagnosis, treatment, preventive measures (including control of cardiovascular risk factors), and associated complications. The possibility of the same video addressing multiple topics was acknowledged.

The information was analyzed for inaccuracies, noting the type of clinical/scientific inaccuracy. The credibility and quality of the information was also assessed using the Health on the Net Code (HONCode)12 and DISCERN13 scores, both based on the audiovisual information in each video. Health on the Net is an independent organization that provides health information guidelines for websites based on eight principles: authority, complementarity, confidentiality, attribute (origin of the information), justifiability (evidence), transparency of authorship, transparency of sponsorship, editorial honesty, and conflicts of interest. We assessed the adherence of eligible videos to each of the HONCode principles using a table adapted for online videos as previously published,14 and classified the quality of the videos into low (HONCode 0-2), moderate (HONCode 3-5), or high (HONCode 6-8). The DISCERN score is an instrument used to assess the quality of written medical information and was adapted to the assessment of videos as in previously published studies.14 In videos focused on diagnostic considerations alone, only the first eight items of the DISCERN score are applicable, in contrast to videos regarding therapeutic approaches, where all the DISCERN items are relevant (Supplementary Table 2). A score from 1 to 5 is given to each item, where 5 indicates minimal or no gaps; the quality of the videos was graded as low (DISCERN score 1), moderate (DISCERN score 2-4), or high (DISCERN score 5).13

Statistical analysis was performed using SPSS for Windows software, version 22.0 (SPSS Inc., Chicago, IL, USA). Categorical variables are presented as absolute frequencies with percentages. Parametric continuous variables (normally distributed) are presented as mean ± standard deviation, otherwise they are presented as median and interquartile range. We tested the normality of the distribution of continuous variables using the Kolmogorov-Smirnov test and visual assessment of histograms. Comparisons between video groups were made using chi-square test or Fisher's exact test for categorical variables as appropriate; continuous variables were compared using Student's t-test or one-way ANOVA for parametric variables; non-parametric continuous variables were evaluated with Mann-Whitney's U-test or Kruskal-Wallis test as appropriate. Intra- and inter-observer reproducibility was assessed using Cohen's kappa statistics for target audience, source, video typology, topic covered, inaccuracies, and HONCode and DISCERN scores (Supplementary Table 3). A random sample of 25 eligible videos (10%) was used for this purpose. Intra-observer reproducibility was assessed with a time interval greater than one month between analyses. All statistical tests were two-sided and at a 5% significance level.

A total of 242 videos (mean duration 9.2 ± 0.8 minutes, minimum 0.3 and maximum 89.3 minutes), reaching a total of 2226 minutes, were included in the final analysis. The median number of days from upload to analysis of the videos was 823 (384-1693) days, most of which (75.6%) were submitted at least one year ago. The general characteristics of the included videos are summarized in Table 1. Intra- and inter-observer agreement in the evaluation and classification of the videos was good or excellent (Supplementary Table 3). The parameter that showed the lowest inter-observer agreement was the subject of the video (0.63, 0.39–0.87) and the one with the highest agreement was the presence of clinical or scientific inaccuracies (0.84, 0.62–1.00). When disagreements were identified, the final evaluation resulted from a consensus among three researchers (I.F., D.M., and J.B.A.).

Most videos are intended for the general public (n = 199, 77.7%) and originate from trainers without affiliation (n = 95, 39.3%), which also had the highest approval rating by the public with a like/dislike ratio of 44 (19-77). In turn, videos from health/governmental institutions presented the worst like/dislike ratio (like/dislike ratio 17, 3-23).

Entertainment program videos tend to be longer (13.0 minutes, 4.0-21.1). Videos from "other media" (4.2 minutes, 3.1-5.5) are significantly longer than videos from health/government institutions (1.8 minutes, 0.8-3.2, p = 0.002), scientific societies (1.6 minutes, 0.9-6.5, p = 0.003) and news outlets (2.8 minutes, 2.3-4.1, p = 0.010).

Videos originating from personal experience were the most viewed (median 63 305 views, 2 112–683 598) as well as the most commented on (median 69 comments, 2–593). Videos from television programs had significantly more views (median 11 396 views, 1 462-49 073) than videos from news networks (291 views, 15–1 083, p = 0.001), pharmaceutical companies (70 views, 25–128, p = 0.010), and scientific societies (234 views, 58–12 963, p = 0.026). Videos from trainers without institutional ties have a higher number of comments (8 comments, 1–64) than videos from news outlets (0 comments, 0-0, p = 0.002), health/government institutions (0 comments, 0–1, p = 0.023) and scientific societies (0 comments, 0-0, p = 0.028).

The topic of the videos according to the source are summarized in Figure 2. The most covered topic was "signs and symptoms" (62.0%, n = 150), followed by pathophysiology (55.0%, n = 133). "Signs and symptoms" of AMI are the most addressed topic in videos from scientific societies (100.0%, n = 14), personal experience (83.3%, n = 5), news channels (76.9%, n = 10), entertainment programs (73.0%, n = 27), health or government institutions (60.0%, n = 6), and other media (65.9%, n = 29). In turn, pathophysiology is mostly focused on videos from colleges or instructional centers (66.7%, n = 10) and trainers without affiliation (60.0%, n = 57).

Figure 2.

Topics addressed in the videos according to their source.

(0.23MB).

The topic of the videos according to the target audience are summarized in Figure 3. Videos directed at healthcare professionals mostly focus on pathophysiology (60.5%, n = 23), while videos directed at the general population preferably address "signs and symptoms" (67.8%, n = 137).

Figure 3.

Topics addressed in the videos according to target audience.

(0.12MB).

Only 38.8% of the included videos focus on treatment (n = 94) and prevention (n = 94) of AMI, most of them coming from trainers without affiliation (n = 28 and n = 32, respectively).

The analysis of the quality of video information according to the source and topic covered is detailed in Table 2. One third of the videos (n = 79, 32.6%) have clinical/scientific inaccuracies, most of which originated from trainers without affiliation (n = 33, 41.0%). The topic where inaccuracy was most frequent was "signs and symptoms" (n = 46, 58.2%). Videos originating from scientific societies and health or government institutions presented no inaccuracies.

The mean HONCode and DISCERN scores were 3.0±1.1 and 2.4±0.7, respectively. The number of videos conforming to each of the HONCode principles is shown in Table 3. The most respected HONCode principle was the confidentiality principle (n = 223, 95.3%) and the least respected was the justifiability/bibliographic references principle (n = 12, 5.1%).

In the quality assessment using the HONCode, videos from health/government institutions had the best score (HONCode score 4±1), while videos originating from personal experience and news outlets were the worst quality (HONCode score 2±1). Assessment using the DISCERN score showed no statistically significant differences between the different video sources.

The evaluation of the quality of the videos was also performed after excluding the videos from personal experience or news, as they were not considered to be made for pedagogical purposes (n = 19). Of these, only 3 had clinical/scientific inaccuracies. The results of this sub-analysis (Supplementary Table 4) showed no significant differences compared to the overall set of videos.

Only one video (0.4%) had both good quality (HONCode high score and DISCERN high score) and no clinical and scientific inaccuracies. Video quality was also assessed according to video duration, number of views, like/dislike ratio, and number of comments. It was found that videos of longer duration are associated with better HONCode (r=0.191, p=0.003) and DISCERN (r=0.283, p<0.001) scores. On the other hand, clinical or scientific inaccuracies increase in videos with more views (r=0.285, p< 0.001) and higher number of likes (r =0.289, p<0.001), dislikes (r =0.181, p=0.005) and comments (r=0.147, p=0.024). Figure 4 presents the number of likes and views according to the presence of clinical/scientific inaccuracies.

Figure 4.

Distribution of the number of likes (A) and the number of visualizations (B) according to the presence of clinical/scientific inaccuracies.

(0.11MB).

For each search term, the quality of the videos with the best ranking on YouTube was also evaluated, since these are the videos viewed by most users. For this, the first 10 search results were considered, a total of 100 videos, sorted in order of relevance and without the exclusion criteria initially applied in the sample. The prevalence of inaccuracies was 22.0% (n=22) and the mean HONCode and DISCERN scores were 2.9±1.2 and 2.2±0.9, respectively.

The influence of the Internet in the dissemination of health information has grown significantly in recent years. In the United States, it is known that eight out of ten Internet users search for health-related information online, mostly on social media platforms,8 and a third of individuals who search for this type of information do so before being assessed by a doctor.15

There is no doubt that the Internet plays a determining role in the dissemination of medical information to the population and in the way the general public perceives illness. This influence can, however, be a real threat, since exposure to incorrect, unfiltered information can have adverse health consequences and rival the effort to improve health education in recent decades. One of the greatest dangers online is fake news, so often difficult to distinguish from truthful information for lay persons. In an age when it is becoming difficult to distinguish between what is true from what is not on online platforms, there is a critical need to strengthen patient information and protection systems.

Our results are a true reflection of the risk of misinformation: one third of the selected videos were irrelevant and the overall quality of the Portuguese language videos on AMI is low. One third of the videos presented clinical or scientific inaccuracies, especially on the topic of "signs and symptoms", which may contribute to delayed recognition of symptoms by patients, delay in clinical diagnosis and, consequently, worsening of prognosis.15 As an example, one of the evaluated videos with the highest impact (3 052 443 views and 144 000 likes) has a HONCode score of 2 and serious clinical inaccuracies, stating that some foods can replace percutaneous coronary intervention in the treatment of AMI; another video with 1 477 347 views and 12 000 likes has a HONCode score of 1 and claims that the symptoms of AMI appear one month before the event.

It was found that inaccuracies increase with the number of views, likes, dislikes, and comments, which seems to reflect that more sensationalist videos, and therefore those with higher user participation, tend to have a higher number of errors. Other articles have already highlighted this by concluding that health articles with inaccuracies are 28 times more likely to be shared online compared to articles without clinical or scientific errors.15

The danger of fake news on YouTube has been discussed in other countries, not only in relation to AMI16 or other cardiovascular conditions,17 but also in relation to other clinical situations such as respiratory diseases,14,18–20 gastrointestinal diseases,21–23 orthopedic diseases,24,25 vaccination26 or cardiopulmonary resuscitation.27 The conclusions of these studies are in line with what we observed in the Portuguese context and alert us to the urgent need to improve the quality of medical information on this platform.

The general public interacts more with poorer quality videos while better quality videos are the least watched. These problems are enhanced by the fact that online content sharing networks, such as YouTube, allow individual users to upload content without prior validation of quality and veracity of the information.16 Analysis of the "Top 10" YouTube videos revealed that one fifth of these videos contained clinical/scientific inaccuracies and the average HONCode was only moderate. In an age where instantaneity and immediate availability of information is highly valued, it is particularly important that the information presented first be of higher quality than demonstrated in this subset of videos.

In order to decrease the risk of health misinformation on YouTube, several measures can be taken. Since the best quality videos are, as in other studies,8,15,16 videos from scientific societies and health institutions, it is important to increase the number of videos from these sources on online platforms.8 The establishment of partnerships between scientific societies and influential YouTubers may also provide visibility and contribute to the large scale dissemination of quality medical information, as previously done by the Portuguese Cardiology Society.28 Likewise, the partnership between scientific societies and governmental websites , with a high number of daily views, and which have teams specialized in the creation of digital content and communication with citizens, enables medical information of reliable origin to be combined with the transmission of appealing and effective information.

The solution also involves improving the validation systems for medical information shared on YouTube, including implementing peer review systems and flagging videos with clinical or scientific inaccuracies or, for example, videos that are not compliant with the HONCode principles. Revising YouTube's search algorithms for health information could also help ensure that the results presented first come from validated information sources whose accuracy has been confirmed. It is now also possible to use artificial intelligence to create automatic algorithms to exclude YouTube videos that do not comply with the platform's terms of use,29 which could be recoded to exclude videos with clinical/scientific inaccuracies from search results. Also in this regard, the World Health Organization has proposed to the U.S. Internet Corporation for Assigned Names and Numbers that a new Internet domain be created for use exclusively by validated sources of medical information, which would be carefully regulated and monitored to ensure the quality of the information shared.30

The video evaluation process is subjective, but intra- and inter-observer reproducibility was good or excellent. The video search was restricted to a short time period and usually there is turnover in the top 100 results. However, the median time elapsed since upload of 823 (384-1 693) days leads one to believe that the turnover is not so fast, so these results seem to be representative of a longer time period than the one evaluated. It is not possible to obtain demographic information on the viewers of each video, nor is it possible to determine how many Portuguese people access YouTube or specifically which terms are most searched for on this topic. Although the terms selected are not exhaustive (for example, abbreviations such as "AMI" were not searched), we believe that in general they are representative of searches made by the general public. The impact of YouTube on behavior modification in the population was not assessed , but this was not the goal of the study and may be considered in future works.