National and international surveillance

Table I.

First author	Year of publication	Country	System	Indicators	Setting	Objectives	Main outcomes
Haley	1985	USA	NNIS SENIC project 1970-1976	BSI, pneumonia, UTI, & SSI	Random sample 338 acute care hospitals > beds	Effectiveness of surveillance and infection control programs	32% HAI reduction
Jarvis	1991	USA	NNIS 1986-1990	BSI, UTI & VAP	179 adult & pediatric ICU from 79 hospitals	Determine best denominator in ICU indicators	Device days and ICU type to stratify ICU rates
Mertens	1994	Belgium and Netherlands	1991-1992	SSI	35 Dutch and 62 Belgian hospitals	Comparing national rates	Need harmonization between national networks
Gaynes	1996	USA	NISS1986-1994	BSI, pneumonia, GI. Microbiological etiology of HAIs	99 HRN	Descriptive epidemiology of HAI in HRN	BSI most frequent HAI, CNS most frequent pathogen
Ronveaux	1996	Belgian	Belgian network 1992-1993	SSI	51 hospitals	Descriptive epidemiology & % antibiotic prophylaxis	Improtance of post discharge surveillance in SSI & antibiotic prophylaxis as process indicator
Richards	1999	USA	NNIS1992-1997	BSI, pneumonia & UTI	61 pediatric ICUs	Descriptive epidemiology of pediatric ICU	1st – BSI, CNS2nd – Pneu, pseudomonas3rd – UTI, E.coli
CDC	2000	USA	NNIS1990-1999	BSI, VAP & UTI in ICU	405 adults & 73 pediatric ICUs	Descriptive epidemiology	Reduction of BSI, VAP & UTI rates
Richards	2000	USA	NNIS1992-1998	VAP, BSI & UTI	205 combined medical-surgical ICU	Descriptive epidemiology	1st – VAP, S aureus2nd – UTI, E. coli3rd – BSI, CNS
Geubbels	2000	Netherlands	PREZIES1996-1997	SSI	38 hospitals	Descriptive epidemiology	SSI rates for 16 procedures
Gaynes	2001	USA	NNIS1992-1998	SSI	225 hospitals	Use of risk index, influence of laparoscope in risk	NNIS surgical risk index usefulness.Incorporation of laparoscope to risk index in four procedures
NNIS	2003	USA	NNIS1992-2003ICARE/AUR	ICU device associated indicators, SSI,antimicrobial resistant pathogen in ICUs,antimicrobials usage in DDD	ICU, HRN, surgical procedures	Descriptive epidemiology	Lower SSI when laparoscope; resistance rates higher in ICUs
NNIS	2004	USA	NNIS1992-2004ICARE	SSI, antimicrobial resistant pathogen in ICUs, antimicrobials in DDD	ICU, HRN, surgical procedures	Descriptive epidemiology	60% of MRSA40% of Klebsiella resistant to 3rd generation cephalosporins
Gastmeier	2006	Germany	KISS1997-2003	CR-BSI, VAP in ICUsSSI	150 ICU133 surgical departments	Effectiveness of nationwide surveillance system	Significant reduction in CR-BSI, VAP & SSI
Brandt	2006	Germany	KISS1997-2004	SSI	133 surgical departments	Effectiveness of SSI surveillance	Reduction in SSI rates
Schwab	2007	Germany	NEO-KISS2000-2005	BSI & pneumonia	24 NICU	Effectiveness of surveillance in NICU	BSI reduction
Mannien	2007	Germany	PREZIES1999-2004	SSI	40 hospitals	Validation study of the surveillance system	Positive predictive value 0.97Negative predictive value 0.99
Klevens	2008	USA	NHSN2006	BSI	32 outpatients dialysis centers	Descriptive epidemiology of BSI in dialysis	BSI rates in relation with fistulas, grafts, permanent & temporary catheters
Geffers	2008	Germany	NEO-KISS2000-2005	BSI, pneumonia, necrotizing enterocolitis	52 NICU	Descriptive epidemiology of HAI in < 1,500g infants	BSI, pneumonia and enterocolitis rates
Wilson	2008	England	SSISS2004-2005	Hip replacement SSI	125 hospitals	Way to identify poor performing hospitals	Funnel plots of rates improves accuracy
Szilagyi	2009	Hungary	NNSR2004-2006	SSI	41 hospitals	Descriptive epidemiology	Comparative data of NNSR versus NNIS
Gastmeier	2009	Germany	KISS2007	C. difficile associated diarrhea	34 hospitals	Descriptive epidemiology	46.5 CDAD/10,000 admissions73% nosocomial
Astagneau	2009	France	ISO-RAISIN1999-2006	SSI	838 hospitals	Effectiveness of surveillance in SSI	SSI rates reduction
Edwards	2009	USA	NHSN2006-2008	BSI, CAUTI, VAP in ICUs, NICUsSSI	1,545 hospitals	Descriptive epidemiology:Effects of mandatory reporting in 11states & enrollment od < 100 beds hospitals	BSI reductionNICU rates reductionNo differences between states with and without mandatory reporting
Gravel	2009	Canada	Canadian Nosocomial Surveillance System2004-2005	HAI C. difficile infection	29 hospitals	Descriptive epidemiology	No changes in incidence, great variability among hospitals
Zuschneid	2010	Germany	ICUs in Germany	CR-BSI, VAP & CAUTI in ICUs	50 ICU	Validation study on representativeness of KISS ICU sample	Agreement between study group and KISS group
Simor	2010	Canada	Canadian Nosocomial Surveillance System1195-2007	MRSA incidence	48 hospitals	Descriptive epidemiology	Increase of nosocomial and community MRSA
Huotari	2010	Finland	Finnish Hospital Infection Program(SIRO)1999-2004	Hip and knee replacement SSI	12 hospitals	Validation study of the surveillance system	Rates higher than previously published
Berg	2011	Norway	NOIS2005-2009	CABG-SSI	6 hospitals	Descriptive epidemiology of CABG-SSI in a mandatory reporting system	No reduction
Viseur	2011	Belgium	Belgian network2008-2010	Mandatory report of CDAD	150 hospitals	Descriptive epidemiology of CDAD in a mandatory system	Decrease of incidenceDecrease of % of CD ribotype 027
Mu	2011	USA	NHSH2006-2008	SSI	847 hospitals	New risk adjustment	New risk models increases predictive performance

Controversies in detail.

On the techniques of surveillance

Since the SENIC project published it results, other cohort studies have demonstrated the effectiveness and also the efficiency of HAI surveillance systems. All these studies were performed in hospitals and with inpatient populations. Since that time the great majority of hospitals have significantly reduced their hospital stays and many inpatient procedures have become ambulatory. These facts have a radical importance in the surveillance of some indicators such as surgical site infection. As the length of stays decreases the way to ascertain the cases is more difficult because most HAI will appear after hospital discharge. Unfortunately we lack a standard for post-discharge surveillance methodology because the way to perform this duty is very hospital-dependent. This change produces a tendency to under-diagnose surgical site infections and can hinder the comparison not only with historical rates, but also with other hospitals and also in the case of aggregate data can preclude comparison between different systems.

In 2013 NHSN changed 1 year surveillance in prosthetic surgeries to 90 days-surveillance. This change, which implies a lower sensitivity to the detect related infections has been accepted in most surveillance systems. The reasons that support this change are: the decrease of surveillance workload that allows to increase the ICP time to implement preventive measures, and also because the greater immediacy for feedback to surgeons. The simultaneous improvement in post-discharge surveillance methods has shown that the lower sensitivity is acceptable and the measure can be efficient.

The situation has become even more complicated in the era of outpatient major surgery in which patients come to hospital only for the intervention and the rest of their follow-up is done in other outpatient healthcare units. This circumstance has not only influence the methodology to ascertain the cases but also the way to apply the diagnostic criteria of HAI. The healthcare facilities where continuity of care is better ensured and which share their electronic medical records along the entire care process are more likely to conduct surveillance in a manner more similar to what is still considered the surveillance standard.

Moreover, in addition to the migration, which occurred in the last two decades of the acute care of patients from inpatient wards to ambulatory care units, another phenomenon is happening at the same time. Many patients who some years ago were considered “acute patients” and admitted to acute care facilities are being sent to short-term rehabilitation centers, nursing homes, home-care or long-term facilities. This fact prevents the comparison of acute care centers with their own historical data. The different organizational structures, the heterogeneity of patients and often the limited resources in these “non acute care” facilities have had to change the rules on HAI surveillance for these patients. Recently SHEA and APIC, published guidelines for conducting infection control and surveillance of HAI in these facilities but results on efficiency and effectiveness are still unknown.

On the methodology of surveillance systems

It is well known that to harmonize the methodology of surveillance is the cornerstone of any surveillance system to permit comparisons over time and between centers. The overall strength of the techniques for HAI surveillance is well demonstrated but there are some gaps which remain controversial.

In reference to case definition, despite efforts over the last 25 years to standardize the diagnostic criteria of HAIs, difficulties remain to implement some of them. Thus, the diagnostic criteria of ventilation-associated pneumonia, of catheter-associated urinary infection and some of the criteria that we use to define surgical site infections have a margin for subjectivity, which means that the sensitivity and specificity can be influenced by the observer. New definitions such as NHSN periprosthetic joint infection (based in the Musculoskeletal Infection Society definition) try to have a lower margin of subjectivity when applied. These facts may hinder the comparison of rates over time and between hospitals. Because of these technical difficulties inherent to these indicators and regardless of the importance of the problems which they measure, in some institutions they are being abandoned in favor of other indicators that are easier to apply, for which risk adjustment is easier or not needed and data validation is easier.

In reference to case ascertainment there are also controversial aspects. With day surgery, hospital stays becoming shorter and when health care is done predominantly on an outpatient basis, the search method of the cases has become much more difficult to homogenize, largely because this process is mostly hospital-dependent. Wide variations, depending on whether the providers are public or private, on the geographic location (urban or rural), and also on the organization of health systems in the different countries, have been identified.

The dissemination of results is without any doubt an essential part of surveillance, but there are also aspects which are not known. Is it best to deliver rates of surgical wound infection to the surgeon in private or is it better to give them to all the people involved in that particular healthcare process. The first approach is the classic one and it has proved to be effective. As a counterpart it is well known that several professionals can be involved in surgical site infections and their prevention. If they receive this information, they all have in hand the possibility of incorporating improvements into clinical practice, which ultimately can help to improve the outcome.

The ideal frequency for giving feedback on the rates to professionals is unknown. It would seem that the more frequent the communication of results, the easier it is to introduce corrective measures if a diversion is detected. This approach can be valid for catheter-associated bacteremia in the ICU. However, with surgical site infection the numbers are low and even lower when standardized and reporting results too often can be counter-productive.

In the last two decades there has been a tendency to abandon hospital-wide surveillance and to switch to targeted surveillance. This change has happened because of workload and costs but the efficiency of both systems to prevent HAIs has not been compared in controlled trials. Targeted surveillance is composed of various elements. The most common are the intensive care indicators (catheter-related bacteremia, catheter-related urinary infection and ventilator-associated pneumonia corrected by device days), surveillance of surgical site infections (which often includes several procedures) and surveillance of multi-resistant bacteria infections and Clostridium difficile associated diarrhea.

Some systems have added other indicators such as hospital-wide BSI or process indicators such as hand hygiene compliance, the percentage of central lines correctly inserted or the percentage of correct surgical antibiotic prophylaxis. Hospitals select a mixture of these indicators to design their surveillance programs. Frequently, the selection of indicators can be influenced by the requirements of accreditation agencies or by the mandatory surveillance systems. Currently there is a lack of agreement on which are the best recommended HAI to monitor in each institution. This circumstance does not permit it to be known if some surveillance systems are more efficient than others.

Another aspect of the methodology of surveillance that may be controversial is the fact that currently much of the information needed for monitoring is obtained from electronic medical records. This fact undoubtedly reduces the workload of the ICP and helps with the accuracy of the data but it may influence the effectiveness of the surveillance because it reduces the Hawthorne effect that the visibility of the ICPs working in the wards has.

On the organization of surveillance systems

Regarding the organization of surveillance systems there is a general feeling that a robust national or international protocol to obtain comparable and unbiased data is needed. Perhaps the two most important controversies are whether monitoring should be voluntary or mandatory and whether the results should be kept confidential or made public. Most of the knowledge that we have on the effectiveness and efficiency of surveillance systems to prevent nosocomial infection come from voluntary and confidential systems.

On mandatory versus voluntary surveillance systems:

In the voluntary systems the main advantages are:

• All the outcome indicators were defined for voluntary reporting systems

• The efficiency and effectiveness in the prevention of HAIs have been demonstrated

• The professionals involved in surveillance activities are highly motivated, and interested in the quality improvement. This can be beneficial for the quality of data collection, to accept the need for training and also to better understand the need for performing laborious validation studies to ensure the data consistency.

• Countries such as USA, England or Germany with experience in voluntary schemes have seen that the degree of participation has increased over the years.

On the contrary, their major disadvantages are:

• Not all hospitals participate and aggregate rates may not be representative of the territorial rates.

• Hospitals join and abandon the system or some of its indicators every year. This makes it very difficult to establish benchmarks and also to make interannual comparisons to evaluate trends.

With mandatory surveillance systems, the main advantages are:

• It is an exercise in transparency and accountability for the institutions also when there is no obligation to disclose the rates.

• The fact that all the centers must report their data means that aggregate rates of the system are most likely representative of reality in the territory.

• The fact that all hospitals are required to participate in some indicators, makes inter-annual comparisons and assessment of trends easier.

Moreover, the most important disadvantages are:

• Some indicators are not sufficiently robust to allow comparisons between hospitals and even less so when comparisons have other implications apart from the main objective of improving HAI prevention. The methodology of some of these outcome indicators of surgical site infection is being evaluated to suit the needs of mandatory surveillance.

• There is a tendency to replace outcome indicators with process indicators.

• Hospitals are forced to meet certain surveillance goals that are general for the system and this can lead to a worse prioritization of some HAI prevention efforts in some institutions.

• Regardless of whether or not results are made public, the fact that they are known at least by the health authorities who are responsible for the system can represent a stimulus not to report. Systems can obtain benefits from not reporting.

On public versus confidential reporting systems:

The last area of controversy regarding the national surveillance systems is whether the results should be kept confidential or whether instead it is better to make them public. Today, it is unknown whether to publish the results is more or less efficient than to maintain them confidential.

The potential advantages of making the results public are:

• It is an exercise of transparency and accountability that can allow citizens to make better informed decisions. This is only valid as long as the organization of the health system allows them to choose freely.

• It can permit to insurance companies to hire those who achieve better results and therefore the institutions with better performance will benefit.

• In countries like England and USA, since the results have been made public, an increased interest in HAI and its prevention has been observed.

The main disadvantages are:

• A benefit of public reporting on effectiveness of care remains unproven.

• In the US the lack of agreement between the different states has meant that the results published by each state are different and comparison between them is difficult to do.

• HAI indicators are of very difficult to interpret and leagues based on results may cause undue economic harm to some institutions.

• Poor performance on the particular indicators published can determine economic losses and in turn discourage the reporting of HAI or prioritize HAI prevention efforts erroneously.

What national and international guidelines exist related to surveillance systems?

Although JAHCO established the need for surveillance in 1964, the first standardized approach to surveillance was available in 1969. The first specific guidelines on the methodology on nosocomial infection surveillance were published in 1970 by the CDC and they remained unchanged until 1986. Since that time, few specific guidelines on surveillance have been developed but several evidence-based guidelines on HAI prevention have been published and updated, and in most of them, the indication and the methodology of the surveillance for that specific HAI is clearly specified.

In 1991 new guidelines specific for surveillance were published by the CDC. In these document specific instructions with the methodology for hospital-wide, intensive care unit, high-risk nursery and surgical wound infection surveillance are described. The responsibility to select which components each institution should use is not defined and they recommend that hospitals should determine the composition of their own surveillance program.

One of the most important contributions of the document is the precise definition of the denominators for each HAI. The switch from number of admissions in the denominator to patient-days was the first attempt to adjust by exposure to risk. In the case of surgical site infection surveillance, two optional methods were described: the detailed and the limited option. The document already indicates the problems with hospital-wide surveillance and the need to combine different elements of surveillance and not to use a single method. These guidelines already cited problems that remain unresolved even today, such as post discharge methodology and adjustments to risk to permit the comparability of the rates among different hospitals.

A second edition of these guidelines was published in 1998, and advanced in detail on the surveillance methodology techniques. They refer to the need to have a written plan, the necessity for maintaining the intensity of surveillance stable, for maintaining the consistency of the complete surveillance process over the time and the need for periodic evaluation and validation of the data to allow comparisons over time and with other hospitals. The guidelines are subdivided into seven recommended practices: assessing the population, selecting the outcome or process indicators, use of standardized definitions of HAI, characteristics of the data collection process and of the way of performing the rates analysis, the risk adjustment and also the way of feeding back the data obtained. In all cases, practical examples are also described.

In 1999 the Hospital Infection Control Practices Advisory Committee (HICPAC) published the Guideline for Prevention of Surgical Site Infection. It is a comprehensive guideline that includes all the knowledge to that date on the prevention of surgical site infection. An extensive part of the document is dedicated specifically to surveillance of surgical site infection. It describes the usefulness and limitations of the different attempts at risk adjustment and recalls that stratification for risk is indispensable to make comparisons but does not always guarantee the usefulness of these comparisons.

Authors describe the two main methods to identify the patients: direct observation and indirect detection. Even when direct observation is still the most accurate method and considered the gold standard, the indirect method has good sensitivity and specificity when compared with the direct method. The indirect method basically consists of obtaining information through the infection control practitioner from different sources (review of laboratory reports, patient records and discussion with care providers). These guidelines also emphasize the need to perform post-discharge surveillance and the absence of consensus about the best way to do it.

The Society for Healthcare Epidemiology of America (SHEA) the Infectious Diseases Society of America (IDSA) and the Association for Professionals in Infection Control and Epidemiology (APIC) and partner organizations (AHA, JAHCO) jointly published science-based and practical recommendations for acute care hospitals for the prevention of common HAIs. These guidelines that entitled Compendium of Strategies to Prevent Healthcare-Associated Infections in Acute Care Hospital are based on previous HICPAC guidelines but they are a further step to give practical evidence-based recommendations to all the stakeholders participating in HAI prevention. In each HAI not only is the need for surveillance described, but also the strength of the recommendation is classified and the quality of evidence is also informed for each different HAI. In 2016 WHO has published new recommendations on surgical site infection prevention.

In 2005 the HICPAC published the first guidelines on public reporting of the HAI. In them, they recommend using the established surveillance methods when designing and implementing a mandatory HAI reporting system. They recall the need to choose appropriate process and outcome measures. These have to be robust indicators, easy to validate and they have to permit comparisons to be made among hospitals and over time but their interpretation has to be unequivocal. The main process measures that they recommend for mandatory systems are: the percentage of correct central line insertion practices, percentage of appropriate surgical antibiotic prophylaxis, and influenza vaccination compliance. The two outcome measures that they consider best suited to report in a mandatory system are central-line associated bacteremia and risk-adjusted surgical site infection. Indicators such as catheter-related urinary infection are refused because of the lower associated morbidity and ventilation-associated pneumonia because of the difficulty to detect it accurately.

In 2008 the CDC published a new edition of its NHSN definitions of healthcare-associated infections and the criteria to apply the definitions of the infections and in 2013 some other modifications were incorporated in its manual. These definitions have been adopted by most of national and international surveillance systems.

The SHEA and HICPAC have also published a consensus guideline on the metrics to use to monitor the incidence and prevalence of multidrug-resistant organisms. They explain the most appropriate system to measure each MDRO in different situations and they recall the important unresolved difficulties to use these indicators in the mandatory reporting systems. In fact they recommend not using them for public mandatory reporting until we have better validation studies. In summary they consider that most of these methods are mainly useful to measure trends and also to measure the effect of interventions within a hospital but they are not yet suitable to allow inter-hospital comparisons. Indicators such as MRSA or VRE bacteremia incidence are the only ones sufficiently robust to permit comparison among hospitals and have demonstrated success in measuring prevention of MDRO in the United Kingdom.

What other consensus group statements exist and what do key leaders advise?

Most of the documents cited in the previous guidelines paragraph are consensus documents. There are other consensuses that have been historically important in infection control.

In 1991 the CDC published a consensus document on the limitations of the different HAI rates to make inter-hospital comparisons. Most of the systems that we use today to adjust by risk were defined in this consensus statement. They identified the inability to make inter-hospital comparisons with hospital-wide rates. The adjustment of critical area indicators by device-days and stratification by ICU type was introduced and in the case of High Risk Nursery a further stratification by weight was established. In the Surgical Component, the NNIS risk index was consolidated against the adjustment index. The use of the 75th percentile cut point for the duration of the surgery demonstrated a better discriminatory effect than the two hour cut point. The CDC, SHEA, APIC and SIS Consensus on Surgical Wound Infections Surveillance marked a before and after for Surveillance of SSI and its basic principles remain, despite the fact that updates have been published.

In 1998 a Consensus Panel Report on the Requirements for Infrastructure and Essential Activities of Infection Control and Epidemiology in Hospitals was published by SHEA and APIC and received the endorsement of JAHCO, AHA, HIP-CDC, PIDS, IDSA and NFID. In this document a well-designed surveillance program is considered essential to perform all the rest of infection control activities. They summarize the most important surveillance activities in three recommendations:

• Surveillance of HAI must be performed (category I) and the process should have: standardized case definitions, definition of the population at risk, appropriate methods of measurement, identification of data sources, definition of numerators and denominators, dissemination of the results and selection of specific events to be monitored.

• Surveillance data must be appropriately analyzed and used to monitor and improve the outcome of the infection control process (category I).

• Clinical performance and assessment indicators used for external comparative measurement should meet SHEA and APIC criteria (category II).

In 2005, under the sponsorship of WHO, a group of experts from the European Union coming from countries with well-developed national surveillance systems and from other countries without, established the current needs for national and international HAI surveillance systems and they asked for international case definitions, guidelines to correctly perform surveillance and to analyze the data and to translate the results into practice, reference laboratories for antimicrobial susceptibility testing and typing and international specific training on these duties. They emphasized that a consensus of the minimum national standards of infection prevention must be achieved and in this sense WHO recommendations on Surgical Site Infection prevention have been published in 2016, but they limit to update the preventive measures but not to surveillance methodology aspects.

A group of representatives from the different European countries, many of them involved in their country’s national surveillance systems held a consensus conference to analyze the main problems of infection control for the next decade. On surveillance they declared their concern about the possible undesirable effects of mandatory reporting of healthcare-associated infections and also emphasized the importance of the validity and comparability of data, which is essential to harmonize surveillance among countries.

References

Buehler, JW, Rothman, KJ, Greenland, S. “Surveillance”. 1998. pp. 435-457.

Haley, RW, Culver, DH, White, JW, Morgan, WM, Emori, TG, Munn, VP, Hooton, TM. “The efficacy of infection surveillance and control programs in preventing nosocomial infections in US hospitals”. Am J Epidemiol. vol. 121. 1985. pp. 182-205.

Haley, RW, Morgan, WM, Culver, DH, White, JW, Emori, TG, Mosser, J, Hughes, JM. “Update from the SENIC project. Hospital infection control: recent progress and opportunities under prospective payment”. Am J Infect Control. vol. 13. 1985 Jun. pp. 97-108.

Wenzel, RP. “The Lowbury Lecture. The economics of nosocomial infections”. J Hosp Infect. vol. 31. 1995 Oct. pp. 79-87.

“National Nosocomial Infections Surveillance System. Nosocomial infection rates for interhospital comparison: limitations and possible solutions”. Infect Control Hosp Epidemiol. vol. 12. 1991. pp. 609-21.

McKibben, Horan, TC, Tokars, JL. “Healthcare Infection Control Practices Advisory Committee. Guidance on public reporting of healthcare-associated infections: recommendations of the Healthcare Infection Control Practices Advisory Committee”. Infect Control Hosp Epidemiol. vol. 26. 2005. pp. 580-587.

Pittet, D, Allegranzi, B, Sax, H, Bertinato, L, Concia, E, Cookson, B, Fabry, J, Richet, H, Philip, P, Spencer, RC, Ganter, BW, Lazzari, S. “Considerations for a WHO European strategy on health-care-associated infection, surveillance, and control”. Lancet Infect Dis. vol. 5. 2005 Apr. pp. 242-50.

Lee, TB, Baker, OG, Lee, JT, Scheckler, WE, Steele, L, Laxton, CE. “Recommended practices for surveillance. Association for Professionals in Infection Control and Epidemiology, Inc. Surveillance Initiative working Group”. Am J Infect Control. vol. 26. 1998 Jun. pp. 277-88.

Lee, TB, Montgomery, OG, Marx, J, Olmsted, RN, Scheckler, WE. “Association for Professionals in Infection Control and Epidemiology. Recommended practices for surveillance: Association for Professionals in Infection Control and Epidemiology(APIC), Inc”. Am J Infect Control. vol. 35. 2007 Sep. pp. 427-40.

Yokoe, DS, Mermel, LA, Anderson, DJ, Arias, KM, Burstin, H, Calfee, DP, Coffin, SE, Dubberke, ER, Fraser, V, Gerding, DN, Griffin, FA, Gross, P, Kaye, KS, Klompas, M, Lo, E, Marschall, J, Nicolle, L, Pegues, DA, Perl, TM, Podgorny, K, Saint, S, Salgado, CD, Weinstein, RA, Wise, R, Classen, D. “A compendium of strategies to prevent healthcare-associated infections in acute care hospitals”. Infect Control Hosp Epidemiol.. vol. 29. 2008 Oct. pp. S12-21.

Anderson, DJ, Kaye, KS, Classen, D, Arias, KM, Podgorny, K, Burstin, H, Calfee, DP, Coffin, SE, Dubberke, ER, Fraser, V, Gerding, DN, Griffin, FA, Gross, P, Klompas, M, Lo, E, Marschall, J, Mermel, LA, Nicolle, L, Pegues, DA, Perl, TM, Saint, S, Salgado, CD, Weinstein, RA, Wise, R, Yokoe, DS. “Strategies to prevent surgical site infections in acute care hospitals”. Infect Control Hosp Epidemiol.. vol. 29. 2008 Oct. pp. S51-61.

Coffin, SE, Klompas, M, Classen, D, Arias, KM, Podgorny, K, Anderson, DJ, Burstin, H, Calfee, DP, Dubberke, ER, Fraser, V, Gerding, DN, Griffin, FA, Gross, P, Kaye, KS, Lo, E, Marschall, J, Mermel, LA, Nicolle, L, Pegues, DA, Perl, TM, Saint, S, Salgado, CD, Weinstein, RA, Wise, R, Yokoe, DS. “Strategies to prevent ventilator-associated pneumonia in acute care hospitals”. Infect Control Hosp Epidemiol.. vol. 29. 2008 Oct. pp. S31-40.

O’Grady, NP, Alexander, M, Dellinger, EP, Gerberding, JL, Heard, SO, Maki, DG, Masur, H, McCormick, RD, Mermel, LA, Pearson, ML, Raad, II, Randolph, A, Weinstein, RA. “Guidelines for the prevention of intravascular catheter related infections. Centers for Disease Control and Prevention”. MMWR Recomm Rep. vol. 51. 2002 Aug 9. pp. 1-29.

“National Clostridium difficile Standards Group: Report to the Department of Health”. J Hosp Infect. vol. 56. 2004 Feb. pp. 1-38.

Pratt, RJ, Pellowe, CM, Wilson, JA, Loveday, HP, Harper, PJ, Jones, SR, McDougall, C, Wilcox, MH. “epic2: National evidence-based guidelines for preventing healthcare-associated infections in NHS hospitals in England”. J Hosp Infect.. vol. 65. 2007 Feb. pp. S1-64.

Mangram, AJ, Horan, TC, Pearson, ML, Silver, LC, Jarvis, WR. “Guideline for Prevention of Surgical Site Infection, 1999. Centers for Disease Control and Prevention (CDC) Hospital Infection Control Practices Advisory Committee”. Am J Infect Control. vol. 27. 1999 Apr. pp. 97-132.

“Council recommendation of 9 June 2009 on patient safety, including the prevention and control of healthcare associated infection”. Official Journal of the European Union. 2009. pp. C151/1-C151/6.

Scheckler, WE, Brimhall, D, Buck, AS, Farr, BM, Friedman, C, Garibaldi, RA, Gross, PA, Harris, JA, Hierholzer, WJ, Martone, WJ, McDonald, LL, Solomon, SL. “Requirements for infrastructure and essential activities of infection control and epidemiology in hospitals: a consensus panel report. Society for Healthcare Epidemiology of America”. Infect Control Hosp Epidemiol. vol. 19. 1998 Feb. pp. 114-24.

Haustein, T, Gastmeier, P, Holmes, A, Lucet, JC, Shannon, RP, Pittet, D, Harbarth, S. “Use of benchmarking and public reporting for infection control in four high-income countries”. Lancet Infect Dis. vol. 11. 2011 Jun. pp. 471-81.

Weinstein, RA, Siegel, JD, Brennan, PJ. “Infection-control report cards–securing patient safety”. N Engl J Med. vol. 353. 2005 Jul 21. pp. 225-7.

Klompas, M, Platt, R. “Ventilator-associated pneumonia–the wrong quality measure for benchmarking”. Ann Intern Med. vol. 147. 2007 Dec 4. pp. 803-5.

Edmond, MB, Bearman, GM. “Mandatory public reporting in the USA: an example to follow”. J Hosp Infect. vol. 65. 2007 Jun. pp. 182-8.

Rosenthal, MB. “Nonpayment for performance? Medicare’s new reimbursement rule”. N Engl J Med. vol. 357. 2007 Oct 18. pp. 1573-5.

Gaynes, RP, Culver, DH, Horan, TC, Edwards, JR, Richards, C, Tolson, JS. “Surgical site infection (SSI) rates in the United States, 1992-1998: the National Nosocomial Infections Surveillance System basic SSI risk index”. Clin Infect Dis. vol. 33. 2001 Sep 1. pp. S69-77.

Jarvis, WR, Edwards, JR, Culver, DH, Hughes, JM, Horan, T, Emori, TG, Banerjee, S, Tolson, J, Henderson, T, Gaynes, RP. “Nosocomial infection rates in adult and pediatric intensive care units in the United States. National Nosocomial Infections Surveillance System”. Am J Med. vol. 91. 1991 Sep 16. pp. 185S-191S.

Mertens, R, Van den Berg, JM, Veerman-Brenzikofer, ML, Kurz, X, Jans, B, Klazinga, N. “International comparison of results of infection surveillance: The Netherlands versus Belgium”. Infect Control Hosp Epidemiol. vol. 15. 1994 Sep. pp. 574-8.

Gaynes, RP, Edwards, JR, Jarvis, WR, Culver, DH, Tolson, JS, Martone, WJ. “Nosocomial infections among neonates in high-risk nurseries in the United States. National Nosocomial Infections Surveillance System”. Pediatrics. vol. 98. 1996 Sep. pp. 357-61.

Ronveaux, O, Mertens, R, Dupont, Y. “Surgical wound infection surveillance: results from the Belgian hospital network”. Acta Chir Belg. vol. 96. 1996 Feb. pp. 3-10.

Richards, MJ, Edwards, JR, Culver, DH, Gaynes, RP. “Nosocomial infections in pediatric intensive care units in the United States. National Nosocomial Infections Surveillance System”. Pediatrics. vol. 103. 1999 Apr. pp. e39

“Centers for Disease Control and Prevention (CDC). Monitoring hospital-acquired infections to promote patient safety–United States, 1990-1999”. MMWR Morb Mortal Wkly Rep. vol. 49. 2000 Mar 3. pp. 149-53.

Richards, MJ, Edwards, JR, Culver, DH, Gaynes, RP. “Nosocomial infections in combined medical-surgical intensive care units in the United States”. Infect Control Hosp Epidemiol. vol. 21. 2000 Aug. pp. 510-5.

Geubbels, EL, Mintjes-de Groot, AJ, van den Berg, JM, de Boer, AS. “An operating surveillance system of surgical site infections in The Netherlands: results of the PREZIES national surveillance network. Preventie van Ziekenhuisinfecties door Surveillance”. Infect Control Hosp Epidemiol. vol. 21. 2000 May. pp. 311-8.

“NNIS System. National Nosocomial Infections Surveillance (NNIS) System Report, data summary from January 1992 through June 2003, issued August 2003”. Am J Infect Control. vol. 31. 2003 Dec. pp. 481-98.

“National Nosocomial Infections Surveillance System. National Nosocomial Infections Surveillance (NNIS) System Report, data summary from January 1992 through June 2004, issued October 2004”. Am J Infect Control. vol. 32. 2004 Dec. pp. 470-85.

Gastmeier, P, Geffers, C, Brandt, C, Zuschneid, I, Sohr, D, Schwab, F, Behnke, M, Daschner, F, Rüden, H. “Effectiveness of a nationwide nosocomial infection surveillance system for reducing nosocomial infections”. J Hosp Infect. vol. 64. 2006 Sep. pp. 16-22.

Brandt, C, Sohr, D, Behnke, M, Daschner, F, Rüden, H, Gastmeier, P. “Reduction of surgical site infection rates associated with active surveillance”. Infect Control Hosp Epidemiol. vol. 27. 2006 Dec. pp. 1347-51.

Schwab, F, Geffers, C, Bärwolff, S, Rüden, H, Gastmeier, P. “Reducing neonatal nosocomial bloodstream infections through participation in a national surveillance system”. J Hosp Infect. vol. 65. 2007 Apr. pp. 319-25.

Manniën, J, van der Zeeuw, AE, Wille, JC, van den Hof, S. “Validation of surgical site infection surveillance in the Netherlands”. Infect Control Hosp Epidemiol. vol. 28. 2007 Jan. pp. 36-41.

Klevens, RM, Edwards, JR, Andrus, ML, Peterson, KD, Dudeck, MA, Horan, TC. “NHSN Participants in Outpatient Dialysis Surveillance. Dialysis Surveillance Report: National Healthcare Safety Network (NHSN)-data summary for 2006”. Semin Dial. vol. 21. 2008Jan-Feb. pp. 24-8.

Geffers, C, Baerwolff, S, Schwab, F, Gastmeier, P. “Incidence of healthcare-associated infections in high-risk neonates: results from the German surveillance system for very-low-birthweight infants”. J Hosp Infect. vol. 68. 2008 Mar. pp. 214-21.

Wilson, J, Charlett, A, Leong, G, McDougall, C, Duckworth, G. “Rates of surgical site infection after hip replacement as a hospital performance indicator: analysis of data from the English mandatory surveillance system”. Infect Control Hosp Epidemiol. vol. 29. 2008 Mar. pp. 219-26.

Szilágyi, E, Böröcz, K, Gastmeier, P, Kurcz, A, Horváth-Puhó, E. “The national nosocomial surveillance network in Hungary: results of two years of surgical site infection surveillance”. J Hosp Infect. vol. 71. 2009 Jan. pp. 74-80.

Gastmeier, P, Weitzel-Kage, D, Behnke, M, Eckmanns, T. “Surveillance of Clostridium difficile-associated diarrhoea with the German nosocomial infection surveillance system KISS (CDAD-KISS)”. Int J Antimicrob Agents.. vol. 33. 2009 Mar. pp. S19-23.

Astagneau, P, L’Hériteau, F, Daniel, F, Parneix, P, Venier, AG, Malavaud, S, Jarno, P, Lejeune, B, Savey, A, Metzger, MH, Bernet, C, Fabry, J, Rabaud, C, Tronel, H, Thiolet, JM, Coignard, B. “ISO-RAISIN Steering Group. Reducing surgical site infection incidence through a network: results from the French ISO-RAISIN surveillance system”. J Hosp Infect. vol. 72. 2009 Jun. pp. 127-34.

Edwards, JR, Peterson, KD, Mu, Y, Banerjee, S, Allen-Bridson, K, Morrell, G, Dudeck, MA, Pollock, DA, Horan, TC. “National Healthcare Safety Network (NHSN) report: data summary for 2006 through 2008, issued December 2009”. Am J Infect Control. vol. 37. 2009 Dec. pp. 783-805.

Gravel, D, Miller, M, Simor, A, Taylor, G, Gardam, M, McGeer, A, Hutchinson, J, Moore, D, Kelly, S, Boyd, D, Mulvey, M. “Canadian Nosocomial Infection Surveillance Program. Health care-associated Clostridium difficile infection in adults admitted to acute care hospitals in Canada: a Canadian Nosocomial Infection Surveillance Program Study”. Clin Infect Dis. vol. 48. 2009 Mar 1. pp. 568-76.

Zuschneid, I, Rücker, G, Schoop, R, Beyersmann, J, Schumacher, M, Geffers, C, Rüden, H, Gastmeier, P. “Representativeness of the surveillance data in the intensive care unit component of the German nosocomial infections surveillance system”. Infect Control Hosp Epidemiol. vol. 31. 2010 Sep. pp. 934-8.

Simor, AE, Gilbert, NL, Gravel, D, Mulvey, MR, Bryce, E, Loeb, M, Matlow, A, McGeer, A, Louie, L, Campbell, J. “Canadian Nosocomial Infection Surveillance Program Methicillin- resistant Staphylococcus aureus colonization or infection in Canada: National Surveillance and Changing Epidemiology, 1995-2007”. Infect Control Hosp Epidemiol. vol. 31. 2010 Apr. pp. 348-56.

Huotari, K, Lyytikäinen, O, Ollgren, J, Virtanen, MJ, Seitsalo, S, Palonen, R, Rantanen, P. “Hospital Infection Surveillance Team. Disease burden of prosthetic joint infections after hip and knee joint replacement in Finland during 1999-2004: capture-recapture estimation”. J Hosp Infect. vol. 75. 2010 Jul. pp. 205-8.

Berg, TC, Kjø rstad, KE, Akselsen, PE, Seim, BE, Lø wer, HL, Stenvik, MN, Sorknes, NK, Eriksen, HM. “National surveillance of surgical site infections after coronary artery bypass grafting in Norway: incidence and risk factors”. Eur J Cardiothorac Surg. vol. 40. 2011 Dec. pp. 1291-7.

Viseur, N, Lambert, M, Delmee, M, Van Broeck, J, Catry, B. “Nosocomial and non-nosocomial Clostridium difficile infections in hospitalised patients in Belgium: compulsory surveillance data from 2008 to 2010”. Euro Surveill. vol. 16. 2011 Oct 27. pp. 20000

Mu, Y, Edwards, JR, Horan, TC, Berrios-Torres, SI, Fridkin, SK. “Improving risk-adjusted measures of surgical site infection for the national healthcare safety network”. Infect Control Hosp Epidemiol. vol. 32. 2011 Oct. pp. 970-86.

Sands, K, Vineyard, G, Platt, R. “Surgical site infections occurring after hospital discharge”. J Infect Dis. vol. 173. 1996 Apr. pp. 963-70.

Nicolle, LE. “Infection control in long-term care facilities”. Clin Infect Dis. vol. 31. 2000 Sep. pp. 752-6.

Smith, PW, Bennett, G, Bradley, S, Drinka, P, Lautenbach, E, Marx, J, Mody, L, Nicolle, L, Stevenson, K. “SHEA; APIC. SHEA/APIC guideline: infection prevention and control in the long-term care facility, July 2008”. Infect Control Hosp Epidemiol. vol. 29. 2008 Sep. pp. 785-814.

Edmond, MB, Wenzel, RP. “National and International Surveillance Systems for Nosocomial Infections”. 2003. pp. 109-119.

Pittet, D, Allegranzi, B, Sax, H, Bertinato, L, Concia, E, Cookson, B, Fabry, J, Richet, H, Philip, P, Spencer, RC, Ganter, BW, Lazzari, S. “Considerations for a WHO European strategy on health-care-associated infection, surveillance, and control”. Lancet Infect Dis. vol. 5. 2005 Apr. pp. 242-50.

Gaynes, RP, Culver, DH, Emori, TG, Horan, TC, Banerjee, SN, Edwards, JR, Jarvis, WR, Tolson, JS, Henderson, TS, Hughes, JM. “The National Nosocomial Infections Surveillance System: plans for the 1990s and beyond”. Am J Med. vol. 91. 1991 Sep16. pp. 116S-120S.

Astagneau, P, L’Hériteau, F. “Surveillance of surgical-site infections: impact on quality of care and reporting dilemmas”. Curr Opin Infect Dis. vol. 23. 2010 Aug. pp. 306-10.

Tu, JV, Donovan, LR, Lee, DS, Wang, JT, Austin, PC, Alter, DA, Ko, DT. “Effectiveness of public report cards for improving the quality of cardiac care: the EFFECT study: a randomized trial”. JAMA. vol. 302. 2009 Dec 2. pp. 2330-7.

Fung, CH, Lim, YW, Mattke, S, Damberg, C, Shekelle, PG. “Systematic review: the evidence that publishing patient care performance data improves quality of care”. Ann Intern Med. vol. 148. 2008 Jan 15. pp. 111-23.

Eickhoff, TC, Brachman, PW, Bennett, JV, Brown, JF. “Surveillance of nosocomial infections in community hospitals. I. Surveillance methods, effectiveness, and initial results”. J Infect Dis. vol. 120. 1969 Sep. pp. 305-17.

“Centers of Disease Control. Outline of Surveillance and control of nosocomial infections”. Atlanta: Centers of Diseases Control. 1970.

Emori, TG, Culver, DH, Horan, TC, Jarvis, WR, White, JW, Olson, DR, Banerjee, S, Edwards, JR, Martone, WJ, Gaynes, RP. “National nosocomial infections surveillance system (NNIS): description of surveillance methods”. Am J Infect Control. vol. 19. 1991 Feb. pp. 19-35.

Lo, E, Nicolle, L, Classen, D, Arias, KM, Podgorny, K, Anderson, DJ, Burstin, H, Calfee, DP, Coffin, SE, Dubberke, ER, Fraser, V, Gerding, DN, Griffin, FA, Gross, P, Kaye, KS, Klompas, M, Marschall, J, Mermel, LA, Pegues, DA, Perl, TM, Saint, S, Salgado, CD, Weinstein, RA, Wise, R, Yokoe, DS. “Strategies to prevent catheter-associated urinary tract infections in acute care hospitals”. Infect Control Hosp Epidemiol.. vol. 29. 2008 Oct. pp. S41-50.

Horan, TC, Andrus, M, Dudeck, MA. “CDC/NHSN surveillance definition of health care-associated infection and criteria for specific types of infections in the acute care setting”. Am J Infect Control. vol. 36. 2008 Jun. pp. 309-32.

Cohen, AL, Calfee, D, Fridkin, SK, Huang, SS, Jernigan, JA, Lautenbach, E, Oriola, S, Ramsey, KM, Salgado, CD, Weinstein, RA. “Society for Healthcare Epidemiology of America and the Healthcare Infection Control Practices Advisory Committee. Recommendations for metrics for multidrug-resistant organisms in healthcare settings: SHEA/HICPAC Position paper”. Infect Control Hosp Epidemiol. vol. 29. 2008 Oct. pp. 901-13.

“Nosocomial infection rates for interhospital comparison: limitations and possible solutions. A Report from the National Nosocomial Infections Surveillance (NNIS) System”. Infect Control Hosp Epidemiol. vol. 12. 1991 Oct. pp. 609-21.

“Consensus paper on the surveillance of surgical wound infections. The Society for Hospital Epidemiology of America; The Association for Practitioners in Infection Control; The Centers for Disease Control; The Surgical Infection Society”. Infect Control Hosp Epidemiol. vol. 13. 1992 Oct. pp. 599-605.

Dettenkofer, M, Ammon, A, Astagneau, P, Dancer, SJ, Gastmeier, P, Harbarth, S, Humphreys, H, Kern, WV, Lyytikäinen, O, Sax, H, Voss, A, Widmer, AF. “Infection control–a European research perspective for the next decade”. J Hosp Infect. vol. 77. 2011 Jan. pp. 7-10.

Roth, VR, Mitchell, R, Vachon, J, Alexandre, S, Amaratunga, K, Smith, S. “Periprosthetic Infection following Primary Hip and Knee Arthroplasty: The Impact of Limiting the Postoperative Surveillance Period”. Infection Control & Hospital Epidemiology. 2016. pp. 1-7.

Fan, Y, Gao, F, Wu, Y, Zhang, J, Zhu, M, Xiong, L. “Does ventilator-associated event surveillance detect ventilator-associated pneumonia in intensive care units? A systematic review and meta-analysis”. Crit Care [Internet]. vol. 20. 2016 Dec 24. pp. 338

Neelakanta, A, Sharma, S, Kesani, VP, Salim, M, Pervaiz, A, Aftab, N, Mann, T, Tashtoush, N, Karino, S, Dhar, S, Kaye, KS. “Impact of changes in the NHSN catheter-associated urinary tract infection (CAUTI) surveillance criteria on the frequency and epidemiology of CAUTI in intensive care units (ICUs)”. Infect Control Hosp Epidemiol. vol. 36. 2015 Mar. pp. 346-9.

Chinn, R, Lempp, JM, Huang, SS, Murthy, R, Torriani, FJ, Daley, J, Dekker, E, Goss-Bottorff, B, Kaler, W, Meyer, K, Myers, F, Nichols, A, Kathleen, Quan, Birnbaum, D. “Standardized Infection Ratio for Surgical Site Infection after Colon Surgery: Discord in Models Measuring Healthcare Quality”. Infect Control Hosp Epidemiol. vol. 37. 2016 Nov. pp. 1378-1382.

Torres, D, González, ML, Loera, A, Aguilera, M, Relyea, G, Aristizabal, P, Caniza, MA. “The Centers for Disease Control and Prevention definition of mucosal barrier injury-associated bloodstream infection improves accurate detection of preventable bacteremia rates at a pediatric cancer center in a low- to middle-income country”. Am J Infect Control. vol. 44. 2016 Apr 1. pp. 432-7.

Cirulis, MM, Hamele, MT, Stockmann, CR, Bennett, TD, Bratton, SL. “Comparison of the New Adult Ventilator-Associated Event Criteria to the Centers for Disease Control and Prevention Pediatric Ventilator-Associated Pneumonia Definition (PNU2) in a Population of Pediatric Traumatic Brain Injury Patients”. Pediatr Crit Care Med. vol. 17. 2016 Feb. pp. 157-64.

Løwer, HL, Eriksen, HM, Aavitsland, P, Skjeldestad, FE. “The quality of denominator data in surgical site infection surveillance versus administrative data in Norway 2005-2010”. BMC Infect Dis. vol. 15. 2015 Nov 30. pp. 549

Magill, SS, Dumyati, G, Ray, SM, Fridkin, SK. “Evaluating Epidemiology and Improving Surveillance of Infections Associated with Health Care, United States”. Emerg Infect Dis. vol. 21. 2015 Sep. pp. 1537-42.

Snyders, RE, Goris, AJ, Gase, KA, Leone, CL, Doherty, JA, Woeltje, KF. “Increasing the Reliability of Fully Automated Surveillance for Central Line-Associated Bloodstream Infections”. Infect Control Hosp Epidemiol. vol. 36. 2015 Dec. pp. 1396-400.

van Mourik, MS, van Duijn, PJ, Moons, KG, Bonten, MJ, Lee, GM. “Accuracy of administrative data for surveillance of healthcare-associated infections: a systematic review”. BMJ Open. vol. 5. 2015 Aug 27. pp. e008424

Farrell, L, Gilman, M, Teszner, E, Coffin, SE, Sammons, JS. “Present or absent on admission: results of changes in National Healthcare Safety Network surveillance definitions”. Am J Infect Control. vol. 43. 2015 Oct 1. pp. 1128-30.

Koek, MB, Wille, JC, Isken, MR, Voss, A, van Benthem, BH. “Post-discharge surveillance (PDS) for surgical site infections: a good method is more important than a long duration”. Euro Surveill. vol. 20. 2015 Feb 26. pp. 21042

Allegranzi, B, Zayed, B, Bischoff, P, Kubilay, NZ, de Jonge, S, de Vries, F, Gomes, SM, Gans, S, Wallert, ED, Wu, X, Abbas, M, Boermeester, MA, Dellinger, EP, Egger, M, Gastmeier, P, Guirao, X, Ren, J, Pittet, D, Solomkin, JS. “WHO Guidelines Development Group. New WHO recommendations on intraoperative and postoperative measures for surgical site infection prevention: an evidence-based global perspective”. Lancet Infect Dis. vol. 16. 2016 Dec. pp. e288-e303.

Freeman, R, Ironmonger, D, Puleston, R, Hopkins, KL, Welfare, W, Hope, R, Staves, P, Shemko, M, Hopkins, S, Cleary, P, Patel, B, Muller-Pebody, B, Li, X, Alvarez-Buylla, A, Hawkey, PM, Johnson, AP, Woodford, N, Oliver, I. “Enhanced surveillance of carbapenemase-producing Gram-negative bacteria to support national and international prevention and control efforts”. Clin Microbiol Infect. vol. 22. 2016 Oct. pp. 896-897.

Dettenkofer, M, Humphreys, H, Saenz, H, Carlet, J, Hanberger, H, Ruef, C, Widmer, A, Wolkewitz, M, Cookson, B. “Key priorities in the prevention and control of healthcare-associated infection: a survey of European and other international infection prevention experts”. Infection. vol. 44. 2016 Dec. pp. 719-724.

Copyright © 2017, 2013 Decision Support in Medicine, LLC. All rights reserved.

No sponsor or advertiser has participated in, approved or paid for the content provided by Decision Support in Medicine LLC. The Licensed Content is the property of and copyrighted by DSM.