Open Access
Translator Disclaimer
1 January 2020 Visual Analytics of Surveillance Data on Foodborne Vibriosis, United States, 1973-2010
Jennifer N. Sims, Raphael D. Isokpehi, Gabrielle A. Cooper, Michael P. Bass, Shyretha D. Brown, Alison L. St. John, Paul A. Gulig, Hari H.P. Cohly
Author Affiliations +
Abstract

Foodborne illnesses caused by microbial and chemical contaminants in food are a substantial health burden worldwide. In 2007, human vibriosis (non-cholera Vibrio infections) became a notifiable disease in the United States. In addition, Vibrio species are among the 31 major known pathogens transmitted through food in the United States. Diverse surveillance systems for foodborne pathogens also track outbreaks, illnesses, hospitalization and deaths due to non-cholera vibrios. Considering the recognition of vibriosis as a notifiable disease in the United States and the availability of diverse surveillance systems, there is a need for the development of easily deployed visualization and analysis approaches that can combine diverse data sources in an interactive manner. Current efforts to address this need are still limited. Visual analytics is an iterative process conducted via visual interfaces that involves collecting information, data preprocessing, knowledge representation, interaction, and decision making. We have utilized public domain outbreak and surveillance data sources covering 1973 to 2010, as well as visual analytics software to demonstrate integrated and interactive visualizations of data on foodborne outbreaks and surveillance of Vibrio species. Through the data visualization, we were able to identify unique patterns and/or novel relationships within and across datasets regarding (i) causative agent; (ii) foodborne outbreaks and illness per state; (iii) location of infection; (iv) vehicle (food) of infection; (v) anatomical site of isolation of Vibrio species; (vi) patients and complications of vibriosis; (vii) incidence of laboratory-confirmed vibriosis and V. parahaemolyticus outbreaks. The additional use of emerging visual analytics approaches for interaction with data on vibriosis, including non-foodborne related disease, can guide disease control and prevention as well as ongoing outbreak investigations.

Introduction

Foodborne illnesses caused by microbial and chemical contaminants in food are a substantial health burden worldwide.123456 The gram-negative Vibrio bacterial species, which are found worldwide in ecologically diverse marine and other aquatic organisms contribute to this burden especially through consumption of contaminated seafood.1 The genus Vibrio belongs to the family Vibrionaceae and the class Gammaproteobacteria.7 There are at least 78 known Vibrio species.8 Most of them do not cause disease in humans. However, a small subset of species, notably Vibrio cholerae, Vibrio parahaemolyticus, and Vibrio vulnificus, cause the majority of human disease.9 Vibrio parahaemolyticus and V. vulnificus are ubiquitous in salt waters. Specifically, V. vulnificus is mostly found in estuarine and V. parahaemolyticus is found both in marine and estuarine environments while V cholerae non-01 is ubiquitous in fresh and salt waters. Vibrio infections (vibriosis) can occur through the consumption of contaminated foods and water, or they can be associated with the exposure of skin to aquatic environments and marine animals.10

Annual surveillance data for contaminated food include numbers of outbreaks, illnesses, hospitalization and deaths.111213 In the United States, the Centers for Disease Control and Prevention (CDC) estimates that annually foodborne diseases are responsible for approximately 1,000 reported disease outbreaks, 48 million (about 1 in 6 persons) episodes of illnesses, 128,000 hospitalizations, and 3,000 deaths.11 Since 1973, the CDC has maintained a collaborative surveillance program for collection and periodic reporting of data concerning the occurrence and causes of foodborne-disease outbreaks (FBDOs).14 These multi-year and multi-state data on past foodborne outbreaks caused by bacteria (including Vibrio species), viruses/ prions, protozoa, as well as chemicals, are available from the OutbreakNet Team website of the CDC.15 Other food pathogen-related surveillance mechanisms that include Vibrio species conducted by the CDC include (i) the Foodborne Diseases Active Surveillance Network (FoodNet) initiated in 199616,17 and (ii) Cholera and Other Vibrio Illness Surveillance System (COVIS) initiated in 1988.18 Furthermore, surveillance data on Vibrio species have also been published in the Mortality and Morbidity Weekly Report.14,19 In 2007, vibriosis became a reportable disease in the United States.20 Compared to 2006-2008, the 2010 incidence of laboratory-confirmed Vibrio infection (vibriosis) was significantly higher (39% increase; CI = 12%-72%).11 Further, in 2009 the rates of human Vibrio infection were substantially higher compared to 1996-1998. The total cases reported for vibriosis (noncholera Vibrio species infections) in 2010, 2009, 2008, and 2007 were 848, 789, 588, 549 respectively.15 Between 23 March and 13 April 2011, Florida recorded an outbreak of ten cases related to V. cholerae O75 after oyster consumption.21

Considering the significant increase in vibriosis in the United States, the recent observation of highly unusual oyster-borne toxigenic V. cholerae infections as well as the availability of diverse surveillance systems, there is a need for the development of easily deployed visual integration and analysis approaches that combine diverse data sources in an interactive manner. Current efforts to address this need are still limited or not freely accessible. We propose the use of emerging visual analytics approaches for interaction with data on vibriosis including non-foodborne cases (for example occupational exposure) to guide disease control and prevention. Visual analytics is an iterative process conducted via visual interfaces that involves collecting information, data preprocessing, knowledge representation, interaction, and decision making.22,23 Visual analytics approaches have been applied to understanding spatiotemporal hotspots and syndromic surveillance.24,25 Furthermore, recently a case for visual analytics for concentration of arsenic (a toxicant) in foods was made.26

The purpose of the reported research investigation was to demonstrate that visual analytics approaches could help to identify new relationships and unique patterns of foodborne vibriosis in the United States. Therefore, we identified relevant epidemiological surveillance datasets on foodborne vibriosis. We then used visual analytics tools to demonstrate data integration and interactive data visualization. To the best of our knowledge, this is the first report of the application of visual analytics to over 30 years of foodborne surveillance data on vibriosis in the United States.

Methods

The following four datasets on foodborne vibriosis were assembled: (i) Annual Listing of Foodborne Disease Outbreaks, United States (OutbreakNet); (ii) the Cholera and Other Vibrio Illness Surveillance System (COVIS); (ii) the Foodborne Diseases Active Surveillance Network (FoodNet); and (iv) Epidemiological characteristics of Vibrio parahaemolyticus Infections in the United States, 1973-1998. Subsequently selected visual analytics tools were used to generate views of the datasets. Specific description of method for each dataset is described below.

Annual Listing of Foodborne Disease Outbreaks, United States (OutbreakNet)

The OutbreakNet dataset contains voluntarily reported foodborne disease outbreaks and illnesses by causative agent and food commodities in the United States. A foodborne disease outbreak occurs when two or more cases of a similar illness result from eating the same food.3 The PDF files for 1990 through 1997 were downloaded from the CDC OutbreakNet website.27 The data on Vibrio species were then obtained from each file. Data for 1998 through 2006 were downloaded from the CDC Foodborne Outbreak Online Database (FOOD).28

To facilitate data clustering and visual representations, the names of the etiologic agents reported were processed into 8 categories: (i) Vibrio cholerae; (ii) Vibrio cholerae O1; (iii) Vibrio other; (iv) Vibrio parahaemolyticus; (v) Vibrio parahaemolyticus; other; (vi) Vibrio parahaemolyticus; Vibrio other; (vii) Vibrio unknown and (viii) Vibrio vulnificus. Furthermore, based on a recent report29 on producerelated foodborne illness, the categories developed from the names of the outbreak location categories were (i) Community Event; (ii) Community Event, Other; (iii) Food Service, (iv) Food Service; Community Event; (v) Home, Food Service; (vi) Food Service; Other, (vii) Home, (viii) Home/Community Event, (ix) Other and (x) Unknown.

The Cholera and other Vibrio Illness Surveillance System (COVIS)

The Cholera and Other Vibrio Illness Surveillance System (COVIS) contains multi-year data (1997-2007) on human Vibrio illness in the United States.18 The PDF files for 2002 to 2006 were downloaded from the CDC National Case Surveillance website.30 The frequency of isolates of Vibrio species (excluding toxigenic V. cholerae) isolated from blood, stool, wound and other sites as well as the associated patients and complications (hospitalized or death) were obtained from the files. The data on illnesses were typically divided into patients from Gulf of Mexico Coast states (Alabama, Florida, Louisiana, Mississippi and Texas) and non-Gulf Coast of Mexico states. In the analysis presented here, we did not distinguish between these categories of states.

The Foodborne Diseases Active Surveillance Network (FoodNet)

The FoodNet31 is a collaborative, active, population-based surveillance system from 10 states based on laboratory-confirmed infections caused by enteric pathogens commonly transmitted by food including Vibrio species.16,17 The states cover more than 46 million people (15% of the US population) and include Connecticut, Georgia, Maryland, Minnesota, New Mexico, Oregon, Tennessee, and selected counties in California, Colorado, and New York. The incidence (Cases per 100,000) of vibriosis for each of the 10 states from 2004 to 2009 was obtained from 14 published data sets on FoodNet.3233343536373839404142434445 To compare age group incidence of foodborne vibriosis, the 2010 preliminary data on age group incidence for laboratory-confirmed selected bacteria and parasite in the 10 FoodNet sites were extracted from the 2010 FoodNet report.11

Epidemiological characteristics of Vibrio parahaemolyticus infections in the United States, 1973-1998

Daniels et al46 compiled data on the epidemiology of sporadic infections and foodborne outbreaks caused by V. parahaemolyticus from data reported to the CDC, from state and local health departments. The data from Daniels et al46 were converted into an electronic format suitable for visual analytics (Table 3 of publication). The unique aspect of this dataset is that in some reports it was possible to calculate the attack rate of the infection based on number of persons exposed and number of persons ill.

Results

Annual Listing of Foodborne Disease Outbreaks, United States (OutbreakNet)

The OutbreakNET dataset analyzed consisted of 101 outbreaks and 1,672 cases of foodborne illness linked to Vibrio species from 1990 to 2006 (Fig. 1). Since consumption of raw or undercooked oysters is a risk factor for V. parahaemolyticus infection, we sought to determine oyster-borne outbreaks with V. parahaemolyticus as the causative agent. Figure 2 is a visual representation of the integration of data on causative agent, state, month, and year for outbreaks that occurred in a food service location and which were linked solely to consumption of raw oysters. The visualization helped us to among other findings (i) identify a single outbreak that led to at least 100 cases of illness (Table 1); (ii) determine that Georgia was the only state that reported V. vulnificus (2 cases); (iii) determined that in 1998, Guam Territory (GT) reported a total of 47 cases of V. parahaemolyticus infection associated with consumption of chicken.

Table 1

Highlights for foodborne outbreaks attributed to Vibrio species, United States (1990-2006).

10.4137_EHI.S7806-table1.tif

Figure 1

Foodborne vibriosis in the United States by etiologic agent (etiology field in OutbreakNET), state and year (1990 to 2006). A total of 19 states or territories reported foodborne vibriosis outbreaks. In addition, there were three multi-state (ML) outbreaks caused by V. parahaemolyticus in 1997, 1998 and 2006. The portion of the view with the data is annotated. Number over bars indicates number of ill persons. The data sources are (i)  http://www.cdc.gov/outbreaknet/surveillance_data.html. (ii)  http://www.cdc.gov/foodborneoutbreaks/. Visual analytics resource to interact with data by specifying etiologic agent, location of outbreak and month is available at  http://public.tableausoftware.com/views/ USA_FOODVIBRIO_1990TO2006/USA_Foodborne1990to2006.

10.4137_EHI.S7806-fig1.tif

Figure 2

Visualization of oyster-borne food service outbreaks of V. parahaemolyticus infections integrated by state, month and year. The height of bars indicates number of outbreaks. Outbreaks in Florida are annotated to illustrate data clustering. Number over bars indicates number of ill persons. Dataset on outbreaks caused by raw oysters appended to the bottom of the visualization. Visual analytics resource to interact with data is available at  http://public.tableausoftware.com/views/USA_FOODVIBRIO_1990TO2006/V_para_Oyster_borne_FoodService.

10.4137_EHI.S7806-fig2.tif

The Cholera and other Vibrio Illness Surveillance System (COVIS)

The views generated for the COVIS dataset (2002 to 2006) helped us to identify trends in the anatomic site isolations of Vibrio species. In the dataset analyzed, isolates were reported according to 13 categories including ten Vibrio species. Other categories were “Multiple Sites”, “Species Unknown” and “Other”. Vibrio vulnificus was most frequently isolated from blood and wound isolation sites peaking in 2003 and 2005 respectively (Fig. 3). Furthermore, V. vulnificus had a frequency of isolation approximately six times higher than any other Vibrio species analyzed for the blood isolation site. For isolates from stool specimens, it was noteworthy that V. parahaemolyticus was higher than any of the Vibrio species analyzed, peaking in 2006. The three Vibrio species that showed noteworthy frequencies for wound isolation site were V. alginolyticus, V. parahaemolyticus, and V. vulnificus, with the highest peak recorded in 2005 for all three species.

In terms of reported complications from vibriosis, V. parahaemolyticus and V. vulnificus had the highest frequencies relative to the other Vibrio species studied in all patients and complications (hospitalized and deaths) (Fig. 3). Vibrio vulnificus had the highest frequency for patient hospitalization and deaths compared with the other Vibrio species. Infections due to V. vulnificus resulted in approximately 30 deaths in 2002 and hospitalization of almost 100 individuals in 2005. Furthermore, V. parahaemolyticus infection led to the highest numbers of patients ranging from approximately 200 patients in 2002 to approximately 400 patients in 2006. The comparison of the shapes of the line graphs in the boxes of Figure 3 could reveal novel insights on reporting features. For example, the shape of the line graph for patients with V. parahaemolyticus infection (see Column 3, Row 1 in Patients and Complications chart) is similar to stool site of V. parahaemolyticus isolation (see Column 3, Row 2 in Site of Isolation chart). In both reporting features, cases were approximately the same in 2002 and 2003, followed by an increase in 2004, a decrease in 2005, and then an increase in 2006. Data used to generate Figure 3 can be downloaded from website addresses provided along with the figure legend.

Figure 3

Vibriosis in United States: Patterns of patients, complications and site of isolation of Vibrio species, 2002 to 2009. The comparison of the shapes of the line graphs in the boxes could reveal novel insights on reporting features. For example, the shape of the line graph for patients with Vibrio parahaemolyticus infection (see Column 3, Row 1 in Patients and Complications chart) is similar to stool site of V. parahaemolyticus isolation (see Column 3, Row 2 in Site of Isolation chart). Row 1 (Patients) is the bottom row while Column 1 (V. alginolyticus) is the left-most column. Visual analytics resources to interact with data is available at  http://public.tableausoftware.com/views/USA_vibrio_isolates_2003to2006/Patient_complication;  http://public.tableausoftware.com/views/USA_vibrio_isolates_2003to2006/Site_Isolation.

10.4137_EHI.S7806-fig3.tif

The Foodborne Diseases Active Surveillance Network (FoodNet)

The incidence of vibriosis available from FoodNet for 2004 to 2009 is presented in Table 2 and visualized in Figure 4. The weight of the line is proportional to the incidence (per 100,000 cases). A comparison of the incidence per state for the six years revealed that Connecticut had the highest incidence of 0.77 per 100,000 cases in 2009. Furthermore, the visual representation revealed that, in addition to Connecticut, Oregon in 2009 reported incidence values higher than previous years. The highest incidence for the period was 1.15 per 100,000 cases, which occurred in California in 2006.

Figure 4

Visualization of incidence of vibriosis from 10 States in the United States, 2004 to 2009. The size of the line indicates the incidence. Visual analytics resource to interact with data is available at  http://public.tableausoftware.com/views/FoodNetIncidence2004to2009_1/FoodNet_Vibrio_Incidence_2004to2009.

10.4137_EHI.S7806-fig4.tif

Preliminary data on the 2010 incidence of laboratory-confirmed bacterial and parasitic infection cases, by age group and pathogen, were published by Food-Net in 2011. A view of Table 2 of the FoodNet article is presented in Figure 5. This view allowed us to produce a non-redundant list of incidence values and then identify pathogens that have identical incidence values to Vibrio species. The incidence of infection reported for Vibrio species from FoodNet sites in 2010 ranged from 0 (age group < 5) to 0.8 (age group ≥ 60). The highest incidence was in the ≥60 years age group. Further, foodborne illnesses caused by Vibrio and Yersinia had identical incidence (0.2) for the age group 10 to 19 years.

Table 2

Incidence of foodborne vibriosis in 10 states in the United States, 2004-2009.

10.4137_EHI.S7806-table2.tif

Figure 5

Incidence (cases per 100,000 population) in United States of laboratory-confirmed bacterial and parasitic infection cases, by age group and pathogen, 2010. The source of the data is the Foodborne Disease Active Surveillance Network, United States, 2010 ( http://www.cdc.gov/mmwr/preview/mmwrhtml/mm6022a5.htm). Visual analytics resource to interact with data available at  http://public.tableausoftware.com/views/foodnet2010/incidence.

10.4137_EHI.S7806-fig5.tif

Epidemiological characteristics of Vibrio parahaemolyticus infections in United States, 1973-1998

A total of 40 outbreaks of V. parahaemolyticus infections in the United States were compiled for the period 1973-1998 (Tables 3 and 4). Data were available for 14 years of the possible 26 years. We used Parallel Sets software for categorical data visualization47 to compare the epidemiological characteristics of V. parahaemolyticus infection outbreaks with the food vehicle reported as “Shrimp” or “Raw oyster” (Fig. 6). The horizontal bars in the visualization show the absolute frequency of how often each category occurred. The top horizontal bar shows the months when the outbreaks occurred. The visualization revealed that July had the highest frequency of outbreaks. Between the dimension bars are ribbons that connect categories and split up. This illustrates how combinations of categories are distributed and how a particular subset, for example, Month of July (Jul), can be further subdivided into vehicle (in this example, shrimp and raw oyster). It also illustrates that the vehicle (food) for the pathogen in WA (Washington) and OR (Oregon) was raw oysters while in GU (Guam) it was shrimp. The third horizontal bar represents the Year of outbreak and shows that two raw oyster-borne outbreaks in WA took place in 1990 and one outbreak took place in 1987. In the case of shrimp, the commonalities observed is that in October for both 1977 and 1980 outbreak with shrimp as vehicle occurred in Guam and Arizona.

Table 3

Epidemiological characteristics of Vibrio parahaemolyticus infection outbreaks in the United States (1973-1987).*

10.4137_EHI.S7806-table3.tif

Table 4

Epidemiological characteristics of Vibrio parahaemolyticus infection outbreaks in the United States (1990-1998).

10.4137_EHI.S7806-table4.tif

Figure 6

Interaction with data on epidemiologic characteristics of Vibrio parahaemolyticus infection outbreaks caused by shrimp and raw oysters in United States (1973-1998). This Parallel Sets visualization is for the rows with vehicle of pathogen labeled as “Shrimp” and “Raw oysters” (Table 4). A commonality observed from a visualization of the data was that in the month of July, outbreaks occurred in Guam (GU) in 1980, Washington in 1987 and 1990; and Oregon 1997. Additionally, one outbreak with “Shrimp” as vehicle occurred in GU in 1977 and in Arizona in 1980. See text for interpretation of data visualization.

10.4137_EHI.S7806-fig6.tif

Websites for interactive visual analytics of foodborne vibriosis surveillance data

Table 5 presents the dataset, sources of dataset, and links to web-based visualization. These views are designed to enable users interact with the dataset as well as to download raw or processed data for re-use. The surveillance data visualized were OutbreakNET, COVIS, and FoodNet.

Discussion

There are increasing efforts to develop visual analytics software that facilitates discussion between user and disease surveillance data for environmental and public health policy formulation, resource allocation, and decision making.484950 In this report, we have used a combination of data extraction and data visualization software to illustrate the integration and interactive visualization of surveillance data on foodborne vibriosis in the United States. The three datasets (OutbreakNet, COVIS and FoodNet) were individually processed in the Tableau Software51,52 and the results of analysis were published on websites (Table 5). The websites permit download of datasets and further exploration of the data based on filters available on the website. In addition, Parallel Sets,47 a visual analytics software for categorical data, was used to analyze epidemiological characteristics of V. parahaemolyticus infection outbreaks.

Through the data visualization it was possible to identify unique patterns and/or novel relationships within and across datasets regarding (i) causative agent, (ii) foodborne outbreaks and illness per state, (iii) location of infection, (iv) vehicle (food) of infection, anatomical site of isolation of Vibrio species, (v) patients and complications of vibriosis, (vi) incidence of laboratory-confirmed vibriosis, and V. parahaemolyticus outbreaks.

In the dataset analyzed for 1990 to 2006, Vibrio parahaemolyticus was the major cause of outbreaks (including multistate outbreaks) of foodborne vibriosis (Fig. 1, Table 1). Vibrio parahaemolyticus is recognized globally as a major cause of foodborne gastroenteritis associated with seafood consumption with a higher incidence in Japan and East Asian countries.53 Three major types of clinical illnesses cause by V. parahaemolyticus are gastroenteritis, wound infections, and septicemia. Recent classifications of V. parahaemolyticus have been made based on the presence of particular genes, some of which correlate with pathogenicity. For general species delineation, the thermolabile hemolysin (tlh) gene is used. Vibrio parahaemolyticus strains are considered “pathogenic” if the thermostable direct hemolysin (tdh) and/or TDH-related hemolysin (trh) genes are present. Pathogenicity of V parahaemolyticus is associated with beta-hemolysis, adherence factors, various enzymes, and the products of the tdh, trh, and ure genes.54

Table 5

Websites for interactive visual analytics of foodborne vibriosis surveillance data.

10.4137_EHI.S7806-table5.tif

Gastroenteritis caused by V. parahaemolyticus is almost exclusively associated with seafood consumed raw or inadequately cooked or contaminated after cooking. We have used the visualization in Figure 1 to identify outbreaks resulting in at least 100 cases (Table 1) to prioritize retrieval of additional information including journal articles and reports from states on foodborne outbreaks. We discuss those publications to further identify localized information that may have promoted the transmission of the Vibrio species.

The July to August 1997 multistate outbreak of culture-confirmed V. parahaemolyticus had 209 cases and was associated with eating raw oysters harvested from California, Oregon, and Washington in the United States and from British Columbia (BC) in Canada.55 The largest V. parahaemolyticus oyster-borne outbreak (caused by single clone of O3:K6) was in 1998 with over 400 cases associated with oysters harvested from Galveston Bay, TX.56 The exact number of cases in Texas was 296 (Table 4) with 120 other cases reported from 12 other states (California, Florida, Georgia, Oklahoma, Tennessee, Colorado, Virginia, Alabama, Kentucky, Massachusetts, New Jersey, and Missouri).56 This number of cases was not classified as a multistate outbreak because OutbreakNET defines a multistate outbreak as one in which exposures occurred in more than one state, while an outbreak affecting residents of more than one state due to exposures in a single state is considered to be a single-state outbreak.

In 2003, an outbreak of V. parahaemolyticus in Florida resulted in 115 infected persons (Fig. 1). We obtained additional information on the outbreak through the 2003 Food and Waterborne Illness Surveillance and Investigation Annual Report, Florida.57 The outbreak was due consumption of Seafood Newburg prepared by a caterer in Pinellas County.57,58 The 2006 V. parahaemolyticus infection outbreaks in New York resulted in 108 cases (Fig. 1) with three outbreaks reported in New York City.59 Further, there were 3 confirmed and 2 suspected V. parahaemolyticus related outbreaks in 2006, compared with only 1 confirmed outbreak and 1 suspected outbreak in the previous five years. This trend is captured in Figure 1 (see NY row for V. parahaemolyticus). The outbreak in New York was linked to the multistate outbreak in Oregon, Washington, and British Columbia.606162 Between 2004 and 2009, the FoodNet determined that the incidence of foodborne vibriosis in a California population of 3.21 million was highest in 2006 at 1.15 per 100,000 cases (Table 2 and Fig. 4). Additionally in 2006, OutbreakNET reported two V. parahaemolyticus infection outbreaks of 10 and 27 cases in California (Fig. 1).

Subtyping of selected V. parahaemolyticus isolates in the 2006 outbreaks identified serotype O4:K12 as the causative serotype. In 1997, isolates from serotype O4:K12 were one of the serotypes that caused outbreaks in the Pacific Northwest.55 The draft genome sequence of a V. parahaemolyticus serotype O4:K12 strain has recently been announced.63 The availability of genomic sequences of diverse Vibrio species ushers a new era for genome-based surveillance of foodborne vibriosis.

Vibrio vulnificus infection can cause septicemia in a person with an open wound who has been in close contact with waters infected with V. vulnificus.64 Further, there are two key features of V. vulnificus pathogenesis: (1) extreme destruction of host tissues and (2) the rapid proliferation of the bacteria in the host. The most important virulence factor for V. vulnificus is its capsular polysaccharide (CPS), which assists the species in avoiding phagocytosis by host defense cells and complement. Vibrio vulnificus carries one of the highest mortality rates of any bacterial pathogen and is the leading cause of reported death in the United States related to seafood consumption.65 The hospitalization rate and case-fatality rate for V. vulnificus infection have been reported as 0.910 and 0.390, respectively.66 These high hospitalization and mortality rates are illustrated in the integrated view of the patterns of patients, complications, and sites of isolation of Vibrio species (Fig. 3). Blood and wound are the predominant isolation sources of V. vulnificus, consistent with known pathogenic characteristics of the bacteria.65,66 Most cases of V. vulnificus infection are found in males whose immune system is suppressed or who have underlying diseases with concomitant elevated serum iron levels, primarily liver cirrhosis secondary to alcoholism.67 The age group with highest incidence of Vibrio infections according to the 2010 FoodNet data was 60 years or over (captured in Fig. 5). The virulence factors of V. vulnificus are still poorly characterized. However, the increasing use of genomic and genetic analysis in conjunction with detailed animal models is shedding new light into the pathogenesis of V. vulnificus disease.68

Using Tables 4 and 5, we have illustrated integrated visualization of categorical data (year, month, state or territory and food vehicle) associated with V. parahaemolyticus outbreaks from 1973 to 1998. Public and environmental health data are often collected in categories requiring statistical methods to model and find associations between categorical variables.69,70 Interactive exploratory analysis via visual interfaces to identify partitions in a dataset can prevent missing information on noteworthy associations between variables.71 A total of 17 pathogen vehicle types were present in the dataset (Fig. 6) including “Raw Oysters” and “Shrimp”. In both oysters and shrimps, V. parahaemolyticus is part of their natural bacterial flora.72,73 Consumption of raw oysters is a recognized risk factor for foodborne vibriosis caused by V. parahaemolyticus and V. vulnificus.747576 Visual analytic methods such as geovisual analytics storytelling77,78 that integrates multidimensional data including spatial and temporal data could help enhance the education of producers and consumers of seafood on foodborne vibriosis. In particular, public health advisories as well as annual reports on vibriosis outbreaks when presented in the context of spatial and temporal events can be supplement to advisories and reports disseminated via electronic and print versions.

Foodborne vibriosis surveillance data for United States from 1973 to 2010 were analyzed in this research. Vibrio species are among the 31 major known pathogens transmitted through food in the United States.13 There are also concerns in Europe and other parts of the world of the increasing numbers of illness by non-cholera Vibrio that may be linked to rising temperature of the oceans.7980818283 The increasing incidence of non-cholera vibrios linked to global climate change may lead to worldwide surveillance system to document data on temporal and spatial incidence of non-cholera Vibrio infections.82 Further, new estimates of the burden of foodborne pathogens based on better data sources and methods are now available through the CDC.12,13,84 These recently published datasets as well as future global surveillance data present opportunities for the application of visual analytics methods to gain actionable insights from foodborne disease surveillance data.

Conclusion

In this research article, we have demonstrated the use of visual analytics techniques to facilitate user discourse with datasets on foodborne vibriosis in the United States. The additional use of emerging visual analytics approaches for interaction with data on vibriosis, including non-foodborne (ie, occupational exposure), can guide disease control and prevention as well as during outbreaks by identifying commonalities.

Disclosures

Author(s) have provided signed confirmations to the publisher of their compliance with all applicable legal and ethical obligations in respect to declaration of conflicts of interest, funding, authorship and contributorship, and compliance with ethical requirements in respect to treatment of human and animal test subjects. If this article contains identifiable human subject(s) author(s) were required to supply signed patient consent prior to publication. Author(s) have confirmed that the published article is unique and not under consideration nor published by any other publication and that they have consent to reproduce any copyrighted material. The peer reviewers declared no conflicts of interest.

Acknowledgements

We acknowledge research and student training grants from the following funding agencies: National Biodefense Analysis and Countermeasures Center, Frederick, Maryland; US Department of Homeland Security Science and Technology Directorate (2007-ST-104-000007; 2009-ST-062-000014; 2009-ST-104-000021); National Science Foundation (EPS-0903787; DBI-0958179; DBI-1062057); and National Institutes of Health (NIH-NCRRG12RR13459;NIH-NIGMST36GM095335; NIH-NIMHD 1P20MD002725-01; NIH-NCRR-P20RR016476; NIH-NCRR-P20RR016460). The opinions expressed by authors contributing to this journal do not necessarily reflect the opinions of the journal, the funding agencies or the institutions with which the authors are affiliated.

References

1.

Iwamoto M., Ayers T., Mahon B.E., Swerdlow D.L.Epidemiology of seafood-associated infections in the United States. Clinical Microbiology Reviews. 2010; 23(2): 399–411. Google Scholar

2.

Kuchenmuller T., Hird S., Stein C., Kramarz P., Nanda A., Havelaar A.H.Estimating the global burden of foodborne diseases—a collaborative effort. Eurosurveillance. 2009; 14: 18. Google Scholar

3.

Mead P.S., Slutsker L., Dietz V.. Food-related illness and death in the United States. Emerging Infectious Diseases. 1999; 5(5): 607–25. Google Scholar

4.

Newell D.G., Koopmans M., Verhoef L.. Food-borne diseases—the challenges of 20 years ago still persist while new ones continue to emerge. International Journal of Food Microbiology. 2010; 139 Suppl 1: S3-15. Google Scholar

5.

Schier J.G., Rogers H.S., Patel M.M., Rubin C.A., Belson M.G.Strategies for recognizing acute chemical-associated foodborne illness. Military Medicine. 2006; 171(12): 1174–80. Google Scholar

6.

Venuto M., Halbrook B., Hinners M., Lange A., Mickelson S.Analyses of the eFORS (Electronic Foodborne Outbreak Reporting System) surveillance data (2000-2004) in school settings. Journal of Environmental Health. 2010; 72(7): 8–13. Google Scholar

7.

Farmer J.J.I.The family of Vibrionaceae. In: Barlows A., Truper H.G., Dworkin M., Harder W., Schleifer K.H., editor. The prokaryotes. A handbook on the biology of bacteria: ecophysiology, isolation, identification, and applications, 2nd ed. Vol 2nd. Berlin, Germany; 1992. Google Scholar

8.

Tindall B.J., Kampfer P., Euzeby J.P., Oren A.Valid publication of names of prokaryotes according to the rules of nomenclature: past history and current practice. International Journal of Systematic and Evolutionary Microbiology. 2006; 56(Pt 11): 2715–20. Google Scholar

9.

Lilburn T.G., Gu J., Cai H., Wang Y.Comparative genomics of the family Vibrionaceae reveals the wide distribution of genes encoding virulence-associated proteins. BMC Genomics. 2010; 11: 369. Google Scholar

10.

Tantillo G.M., Fontanarosa M., Di P.A., Musti M.Updated perspectives on emerging vibrios associated with human infections. Letters in Applied Microbiology. 2004; 39(2): 117–26. Google Scholar

11.

Centers for Disease Control and Prevention. Vital signs: incidence and trends of infection with pathogens transmitted commonly through food— foodborne diseases active surveillance network, 10 US Sites, 1996-2010. MMWR. Morbidity and Mortality Weekly Report. 2011; 60(22): 749–55. Google Scholar

12.

Scallan E., Griffin P.M., Angulo F.J., Tauxe R.V., Hoekstra R.M.Foodborne illness acquired in the United States—unspecified agents. Emerging Infectious Diseases. 2011; 17(1): 16–22. Google Scholar

13.

Scallan E., Hoekstra R.M., Angulo F.J.. Foodborne illness acquired in the United States—major pathogens. Emerging Infectious Diseases. 2011; 17(1): 7–15. Google Scholar

14.

Bean N.H., Goulding J.S., Lao C., Angulo F.J.Surveillance of foodborne disease outbreaks, United States, 1988-1992. MMWR Morbidity and Mortality Weekly Report. 1996; 45(SS-5): 1–55. Google Scholar

15.

Centers for Disease Control and Prevention. Center for Disease Control and Prevention Outbreak Response Team. 2011;  http://www.cdc.gov/outbreaknet/. Accessed 2011/07/21, 2011. Google Scholar

16.

Jones T.F., Scallan E., Angulo F.J.FoodNet: overview of a decade of achievement. Foodborne Pathogens and Disease. 2007; 4(1): 60–6. Google Scholar

17.

Scallan E.Activities, achievements, and lessons learned during the first 10 years of the Foodborne Diseases Active Surveillance Network: 1996-2005. Clinical Infectious Diseases. 2007; 44(5): 718–25. Google Scholar

18.

Dechet A.M., Yu P.A., Koram N., Painter J.Nonfoodborne Vibrio infections: an important cause of morbidity and mortality in the United States, 1997-2006. Clinical Infectious Diseases. 2008; 46(7): 970–6. Google Scholar

19.

Centers for Disease Control and Prevention. Surveillance for foodborne disease outbreaks—United States, 2006. MMWR Morbidity and Mortality Weekly Report. 2009; 58(22): 609–15. Google Scholar

20.

Council of State and Territorial Epidemiologists. National Reporting for non-cholera Vibrio Infections (Vibriosis). 2006;  http://www.cste.org/PS/2006pdfs/PSFINAL2006/06-ID-05FINAL.pdf. Accessed 7/21/2011, 2011. Google Scholar

21.

Onifade T.J., Hutchinson R., Van Zile K., Bodager D., Baker R., Blackmore C.Toxin producing Vibrio cholerae O75 outbreak, United States, March-April 2011. Eurosurveillance. 2011; 16(20): 19870. Google Scholar

22.

Keim D.A., Mansmann F., Schneidewind J., Ziegler H.Challenges in visual data analysis, Proceedings of Information Visualization (IV 2006). IEEE Computer Graphics and Applications. 2006; 26(1): 10–3. Google Scholar

23.

Thomas J.J., Cook K.A.Illuminating the path: the research and development agenda for visual analytics. IEEE Computer Graphics and Applications. 2006; 26(1): 10–3. Google Scholar

24.

Maciejewski R., Hafen R., Rudolph S.. Generating synthetic syndromic-surveillance data for evaluating visual-analytics techniques. IEEE Computer Graphics and Applications. 2009; 29(3): 18–28. Google Scholar

25.

Maciejewski R., Rudolph S., Hafen R.. A visual analytics approach to understanding spatiotemporal hotspots. IEEE Trans Vis Comput Graph. 2010; 16(2): 205–20. Google Scholar

26.

Johnson M.O., Cohly H.H., Isokpehi R.D., Awofolu O.R.The case for visual analytics of arsenic concentrations in foods. International Journal of Environmental Research and Public Health. 2010; 7(5): 1970–83. Google Scholar

27.

Center for Disease Control and Prevention. Outbreak Surveillance Data. 2011. Accessed 7/21/2011, 2011. Google Scholar

28.

Centers for Disease Control and Prevention. Food Outbreak Online Database. 2010;  http://wwwn.cdc.gov/foodborneoutbreaks/. Accessed 7/21/2011, 2011. Google Scholar

29.

Durman M.L. Analysis of Produce Related Foodborne Illness Outbreaks. 2010;  http://www.foodandfarming.info/docs/386Produce_Analysis_2010_ Final.pdf. Google Scholar

30.

Centers for Disease Control and Prevention. National Case Surveilliance: Cholera and Other Vibrio Illness Surveillance System (COVIS) 2011;  http://www.cdc.gov/nationalsurveillance/cholera_Vibrio_surveillance.htmlGoogle Scholar

31.

Centers for Disease Control and Prevention. FoodNet. 2011;  http://www.cdc.gov/foodnet/. Accessed 7/21/2011, 2011. Google Scholar

32.

Centers for Disease Control and Prevention. Foodborne Diseases Active Surveillance Network, 1996. MMWR Morbidity and Mortality Weekly Report. 1997; 46(12): 258–61. Google Scholar

33.

Centers for Disease Control and Prevention. Incidence of foodborne illnesses—FoodNet, 1997. MMWR Morbidity and Mortality Weekly Report. 1998; 47(37): 782–6. Google Scholar

34.

Centers for Disease Control and Prevention. Incidence of foodborne illnesses: preliminary data from the Foodborne Diseases Active Surveillance Network (FoodNet)—United States, 1998. MMWR Morbidity and Mortality Weekly Report. 1999; 48(9): 189–94. Google Scholar

35.

Centers for Disease Control and Prevention. Preliminary FoodNet data on the incidence of foodborne illnesses—selected sites, United States, 1999. MMWR Morbidity and Mortality Weekly Report. 2000; 49(10): 201–5. Google Scholar

36.

Centers for Disease Control and Prevention. Preliminary FoodNet data on the incidence of foodborne illnesses—selected sites, United States, 2000. MMWR Morbidity and Mortality Weekly Report. 2001; 50(13): 241–6. Google Scholar

37.

Centers for Disease Control and Prevention. Preliminary FoodNet data on the incidence of foodborne illnesses—selected sites, United States, 2001. MMWR Morbidity and Mortality Weekly Report. 2002; 51(15): 325–9. Google Scholar

38.

Centers for Disease Control and Prevention. Preliminary FoodNet data on the incidence of foodborne illnesses—selected sites, United States, 2002. MMWR Morbidity and Mortality Weekly Report. 2003; 52(15): 340–3. Google Scholar

39.

Centers for Disease Control and Prevention. Preliminary FoodNet data on the incidence of infection with pathogens transmitted commonly through food—selected sites, United States, 2003. MMWR Morbidity and Mortality Weekly Report. 2004; 53(16): 338–43. Google Scholar

40.

Centers for Disease Control and Prevention. Preliminary FoodNet data on the incidence of infection with pathogens transmitted commonly through food—10 sites, United States, 2004. MMWR Morbidity and Mortality Weekly Report. 2005; 54(14): 352–6. Google Scholar

41.

Centers for Disease Control and Prevention. Preliminary FoodNet data on the incidence of infection with pathogens transmitted commonly through food—10 States, United States, 2005. MMWR Morbidity and Mortality Weekly Report. 2006; 55(14): 392–5. Google Scholar

42.

Centers for Disease Control and Prevention. Preliminary FoodNet data on the incidence of infection with pathogens transmitted commonly through food—10 states, 2006. MMWR Morbidity and Mortality Weekly Report. 2007; 56(14): 336–9. Google Scholar

43.

Centers for Disease Control and Prevention. Preliminary FoodNet data on the incidence of infection with pathogens transmitted commonly through food—10 states, 2007. MMWR Morbidity and Mortality Weekly Report. 2008; 57(14): 366–70. Google Scholar

44.

Centers for Disease Control and Prevention. Preliminary FoodNet Data on the incidence of infection with pathogens transmitted commonly through food—10 States, 2008. MMWR Morbidity and Mortality Weekly Report. 2009; 58(13): 333–7. Google Scholar

45.

Centers for Disease Control and Prevention. Preliminary FoodNet data on the incidence of infection with pathogens transmitted commonly through food—10 states, 2009. MMWR Morbidity and Mortality Weekly Report. 2010; 59(14): 418–22. Google Scholar

46.

Daniels N.A., MacKinnon L., Bishop R.. Vibrio parahaemolyticus infections in the United States, 1973-1998. Journal of Infectious Diseases. 2000; 181(5): 1661–6. Google Scholar

47.

Kosara R., Bendix F., Hauser H.Parallel sets: interactive exploration and visual analysis of categorical data. IEEE Transactions on Visualization and Computer Graphics. 2006; 12(4): 558–68. Google Scholar

48.

Kamel Boulos M.N., Viangteeravat T., Anyanwu M.N., Ra Nagisetty V., Kuscu E.Web GIS in practice IX: a demonstration of geospatial visual analytics using Microsoft Live Labs Pivot technology and WHO mortality data. International Journal of Health Geographies. 2011; 10: 19. Google Scholar

49.

Livnat Y., Gesteland P., Benuzillo J.. Epinome—a novel workbench for epidemic investigation and analysis of search strategies in public health practice. American Medical Informatics Association Annual Symposium Proceedings. 2010; 2010: 647–51. Google Scholar

50.

Hafen R.P., Anderson D.E., Cleveland W.S.. Syndromic surveillance: STL for modeling, visualizing, and monitoring disease counts. BMC Medical Informatics and Decision Making. 2009; 9: 21. Google Scholar

51.

Chabot C.Demystifying visual analytics. IEEE Computer Graphics and Applications. 2009; 29(2): 84–7. Google Scholar

52.

Mackinlay J., Hanrahan P., Stolte C.Show me: automatic presentation for visual analysis. IEEE Transactions on Visualization and Computer Graphics. 2007; 13(6): 1137–44. Google Scholar

53.

Infectious Disease Surveillance Center. Vibrio parahaemolyticus, Japan, 1996-1998. Infectious Agents Surveillance Report. 1999; 20(7): 233. Google Scholar

54.

Drake S.L., DePaola A., Jaykus L.An Overview of Vibrio vulnificus and Vibrio parahaemolyticus. Comprehensive Reviews in Science and Food Safety. 2007; 6: 120–44. Google Scholar

55.

Centers for Disease Control Prevention. Outbreak of Vibrio parahaemolyticus infections associated with eating raw oysters—Pacific Northwest, 1997. MMWR Morbidity and Mortality Weekly Report. 1998; 47(22): 457–62. Google Scholar

56.

DePaola A., Kaysner C.A., Bowers J., Cook D.W.Environmental investigations of Vibrio parahaemolyticus in oysters after outbreaks in Washington, Texas, and New York (1997 and 1998). Applied and Environmental Microbiology. 2000; 66(11): 4649–54. Google Scholar

57.

Florida Department of Health. Food and Waterborne Illness Surveillance and Investigation Annual Report, Florida, 2003. 2003;  http://www.doh.state.fl.us/environment/medicine/foodsurveillance/pdfs/annual2003.pdf. Accessed 7/21/2011, 2011. Google Scholar

58.

Farrell J. Seafood seems to be culprit in illness. 2003;  http://www.sptimes.com/2003/12/23/Northpinellas/Seafood_seems_to_be_c.shtml. Accessed 7/21/2011, 2011. Google Scholar

59.

New York State Department of Health. Foodborne Disease Outbreaks In New York State 2006. 2006. Accessed 7/21/2011, 2011. Google Scholar

60.

Cummings K., Karon A., Vugia D.J. Epidemiologic Summary of Non-Cholera Vibriosis in California, 2001-2008. 2009;  http://www.cdph.ca.gov/data/statistics/Documents/vibriosis-episummary.pdf. Accessed 7/21/2011, 2011. Google Scholar

61.

Foodborne Illiness Outbreak Database. California Restaurant, Picnic, or Private Home Raw Oysters 2006. 2006;  http://www.outbreakdatabase.com/details/california-restaurant-picnic-or-private-home-raw-oysters-2006/. Accessed 7/21/2011, 2011. Google Scholar

62.

Balter S. 2006Alert # 21: Vibrio parahaemolyticus in New York. 2006;  http://www.nyc.gov/html/doh/downloads/pdf/cd/06md21.pdf. Accessed 7/21/2011, 2011. Google Scholar

63.

Gonzalez-Escalona N., Strain E.A., De Jesus A.J., Jones J.L., Depaola A.Genome Sequence of the Clinical O4: K12 Serotype Vibrio parahaemolyticus Strain 10329. Journal of Bacteriology. 2011; 193(13): 3405–6. Google Scholar

64.

Gulig P.A., Bourdage K.L., Starks A.M.Molecular Pathogenesis of Vibrio vulnificus J Microbiol. 2005; 43 Spec No: 118-31. Google Scholar

65.

Horseman M.A., Surani S.A comprehensive review of Vibrio vulnificus: an important cause of severe sepsis and skin and soft-tissue infection. International Journal of Infectious Diseases. 2011; 15(3): e157–166. Google Scholar

66.

Oliver J.D.Wound infections caused by Vibrio vulnificus and other marine bacteria. Epidemiology and Infection. 2005; 133(3): 383–91. Google Scholar

67.

Oliver J.D., Karper J. Vibrio vulnificus in Oceans and Health: Pathogen in the Marine Environment, 2nd edn. In: Belken S.S., Colwell R.R., editors. 2005: 253–76. Google Scholar

68.

Gulig P.A., de Crecy-Lagard V., Wright A.C., Walts B., Telonis-Scott M., McIntyre L.M.SOLiD sequencing of four Vibrio vulnificus genomes enables comparative genomic analysis and identification of candidate clade-specific virulence genes. BMC Genomics. 2010; 11: 512. Google Scholar

69.

Preisser J.S., Koch G.G.Categorical data analysis in public health. Annual Review of Public Health. 1997; 18: 51–82. Google Scholar

70.

Agresti A.Modelling ordered categorical data: recent advances and future challenges. Statistics in Medicine. 1999; 18(17-18): 2191–207. Google Scholar

71.

Seo J., Gordish-Dressman H.Exploratory data analysis with categorical variables: an improved rank-by-feature framework and a case study. International Journal of Human-Computer Interaction. 2007; 23(3): 287–314. Google Scholar

72.

Prapaiwong N., Wallace R.K., Arias C.R.Bacterial loads and microbial composition in high pressure treated oysters during storage. International Journal of Food Microbiology. 2009; 131(2-3): 145–50. Google Scholar

73.

Oxley A.P., Shipton W., Owens L., McKay D.Bacterial flora from the gut of the wild and cultured banana prawn, Penaeus merguiensis. Journal of Applied Microbiology. 2002; 93(2): 214–23. Google Scholar

74.

Vieira R.H., de Sousa O.V., Costa R.A.. Raw oysters can be a risk for infections. Brazilian Journal of Infectious Diseases. 2010; 14(1): 66–70. Google Scholar

75.

Su Y.C., Liu C.Vibrio parahaemolyticus: a concern of seafood safety. Food Microbiology. 2007; 24(6): 549–58. Google Scholar

76.

Washington State Department of Health.Eating raw oysters source of recent cases of vibriosis. 2011;  http://www.doh.wa.gov/Publicat/2011_news/11-118.htm. Accessed 8/18/2011, 2011. Google Scholar

77.

Jern M., Brezzi M., Lundblad P.Geovisual Analytics Tools for Communicating Emergency and Early Warning. In: Konecny M., Bandrova T.L., Zlatanova S., editors. Geographic Information and Cartography for Risk and Crisis Management: Springer, Berlin Heidelberg; 2010: 379–94. Google Scholar

78.

Chen H., Yan P., Zeng D.Data Visualization, Information Dissemination, and Alerting. Infectious Disease Informatics. Springer US; 2010; 21: 73–87. Google Scholar

79.

Gonzalez-Escalona N., Cachicas V., Acevedo C.. Vibrio parahaemolyticus diarrhea, Chile, 1998 and 2004. Emerging Infectious Diseases. 2005; 11(1): 129–31. Google Scholar

80.

McLaughlin J.B., DePaola A., Bopp C.A.. Outbreak of Vibrio parahaemolyticus gastroenteritis associated with Alaskan oysters. The New England Journal of Medicine. 2005; 353(14): 1463–70. Google Scholar

81.

Paz S., Bisharat N., Paz E., Kidar O., Cohen D.Climate change and the emergence of Vibrio vulnificus disease in Israel. Environmental Research. 2007; 103(3): 390–6. Google Scholar

82.

Baker-Austin C., Stockley L., Rangdale R., Martinez-Urtaza J.Environmental occurrence and clinical impact of Vibrio vulnificus and Vibrio parahaemolyticus: a European perspective. Environmental Microbiology Reports. 2010; 2(1): 7–18. Google Scholar

83.

Oberbeckmann S., Wichels A., Wiltshire K.H., Gerdts G.Occurrence of Vibrio parahaemolyticus and Vibrio alginolyticus in the German Bight over a seasonal cycle. Antonie Van Leeuwenhoek. 2011; 100(2): 291–307. Google Scholar

84.

Centers for Disease Control and Prevention.Estimates of Foodborne Illness in the United States. 2011;  http://www.cdc.gov/foodborneburdenGoogle Scholar
© 2011 SAGE Publications. This article is distributed under the terms of the Creative Commons Attribution-NonCommercial 3.0 License (http://www.creativecommons.org/licenses/by-nc/3.0/) which permits non-commercial use, reproduction and distribution of the work without further permission provided the original work is attributed as specified on the SAGE and Open Access page (https://us.sagepub.com/en-us/nam/open-access-at-sage).
Jennifer N. Sims, Raphael D. Isokpehi, Gabrielle A. Cooper, Michael P. Bass, Shyretha D. Brown, Alison L. St. John, Paul A. Gulig, and Hari H.P. Cohly "Visual Analytics of Surveillance Data on Foodborne Vibriosis, United States, 1973-2010," Environmental Health Insights 5(1), (1 January 2020). https://doi.org/10.1177/EHI.S7806
Published: 1 January 2020
JOURNAL ARTICLE
PAGES


Share
SHARE
KEYWORDS
bioinformatics
data visualization
foodborne diseases
human-computer interaction
surveillance
Vibrio species
visual analytics
RIGHTS & PERMISSIONS
Get copyright permission
Back to Top