INTRODUCTION

The Behavioral Risk Factor Surveillance System (BRFSS) is a state-based system of health surveys. The objective of the BRFSS is to collect uniform, state-specific data on health risk behaviors, clinical preventive health practices, and health care access that are associated with the leading causes of morbidity and mortality in the United States. Data are collected from a representative sample in each state, and the sampling is designed to provide national estimates when all state data are combined (19). For most states, the BRFSS is the only source for this type of information. Currently, data are collected monthly in all 50 states, the District of Columbia, Puerto Rico, the Virgin Islands, and Guam. More than 430,000 adult interviews were completed in 2007, making the BRFSS the largest telephone health survey in the world. The BRFSS is not only a unique source of risk behavior data for states, but also useful to measure progress toward the U.S. Healthy People 2010 objectives.

The BRFSS field operations are managed by state health departments, with assistance provided by the Centers for Disease Control and Prevention (CDC). These health departments participate in developing the survey instrument and conduct the interviews either in house or through the use of contractors. The data are transmitted to the CDC's Behavioral Surveillance Branch for editing, processing, weighting, and analysis. An edited and weighted data file is provided to each participating health department for each year of data collection, and summary reports of state-specific data are prepared by CDC. Health departments use the data for a variety of purposes, including identifying demographic variations in health-related behaviors, targeting services, addressing emergent and critical health issues, proposing legislation for health initiatives, measuring progress toward state and national health objectives, and designing program and policy evaluations.

PAST

In behavioral surveillance, the BRFSS has a relatively long history. The BRFSS started in 1981 as a point-in-time survey. Beginning in 1984, 15 states participated in an ongoing surveillance system with their interviews spread out across the whole year (50). Data were collected on the six individual-level risk factors associated with the leading causes of premature mortality among adults: cigarette smoking, alcohol use, physical inactivity, diet, hypertension, and safety belt use. From its inception, the BRFSS was designed to allow states to add to their individual survey questions of their own choosing. Beginning in 1988, optional, standardized sets of questions on specific topics (optional modules) were made available to states. By 1993, the BRFSS had become a nationwide system, and the total sample size exceeded 100,000. The system was then redesigned, with certain questions asked every year (fixed core) and others asked every other year (rotating core). As part of the 1993 redesign, up to five emerging core questions for newly arising topics were included each year for all states.

By collecting behavioral data at the state level, the BRFSS has proven to be a powerful tool for building health-promotion activities. As the demand for data has increased, the number of requests to add questions to the survey has increased. Currently, almost every Division in the National Center for Chronic Disease Prevention and Health Promotion (NCCDPHP) and other CDC Centers, Institutes, and Offices (CIOs) has questions on the BRFSS. Interest in the BRFSS has also grown outside of CDC. Other federal agencies, such as the Health Resources and Services Administration (HRSA), the Administration on Aging (AoA), the Substance Abuse and Mental Health Services Administration (SAMHSA), and the Veterans Administration (VA), have added questions to the survey. Other countries, eager to develop similar surveillance systems, notably China, Australia, Canada, Russia, and most recently Mexico and Brazil, have also requested technical assistance.

For most states, the BRFSS is the only source of population-based health behavior data related to chronic disease. As the system has expanded, so have the number of optional modules and state-added questions. In 2000, 19 optional modules were supported from which states could select to add to their surveys. These are in addition to the approximately 80 core questions that are asked by all States.

With the growth of the system, some notable successes have been realized. All states use BRFSS data to establish and track state health objectives, plan health programs, or implement a broad array of disease-prevention activities. Nearly two-thirds of states use BRFSS data to support health-related legislative efforts. For example, in Delaware, data were used to support legislation in publications, public hearings, and legislative sessions. A bill to create the Healthy Lifestyle and Tobacco-Related Disease Prevention Fund successfully passed. Two successful legislative initiatives were supported by data on the prevalence of smoking and mammography screening in Illinois—an act requiring no-smoking areas in public buildings and one requiring the inclusion of mammography screening in all health insurance coverage. In Nevada, BRFSS data documenting the state's high rates of chronic and binge drinking were used to support legislation to place a per-gallon tax on distilled alcohol at the wholesale level. Only because the data were state specific could these efforts succeed.

In addition, investigators have performed numerous analyses of BRFSS data and have published several articles and reports, both by CDC and by individual states. Every topic area has been examined and a number of methodological investigations have been undertaken. Most of the articles and reports have received wide dissemination in the literature and other media. State-specific reports have been used by several states for program planning and policy development. The availability of state-specific data gave the BRFSS the power to identify key health problems affecting the nation through the ability to use visual presentations. Indeed, the BRFSS obesity and diabetes maps have been widely credited for identifying the epidemics of obesity and diabetes in the United States (39, 40) (Figure 1). Thousands of articles have cited BRFSS data and/or methods focusing on different topics such as aging (41), depression (22, 52), cardiovascular diseases (17, 54), disability (46, 47), alcohol consumption (38, 53), immunization (1), language barriers (49), firearms (48), and air pollution (4).

To facilitate state use of the data, information and products developed by CDC are disseminated through the BRFSS Web site. All BRFSS data, reports, methodology, and questionnaires are available at http://www.cdc.gov/brfss.

PRESENT

The challenge for the BRFSS is to manage effectively an increasingly complex surveillance system that serves the needs of multiple programs while adapting to changes in communications technology such as the increased use of cellular telephones and call-screening devices, societal behaviors (concerns about privacy and declining participation in surveys), and population diversity (the growing number of languages spoken in the United States along with greater cultural and ethnic diversity) (21). To address these challenges, the BRFSS maintains an ongoing program of improvement and adaptation that involves the following:

These efforts are critical for improving the quality of BRFSS data, reaching populations previously not included in the survey, and expanding the utility of the surveillance data. Pilot studies are conducted in collaboration with the states, and the information garnered from these studies is widely disseminated in reports, conference presentations, and peer-reviewed publications. By addressing current challenges and keeping an eye on future issues, these studies help prepare the surveillance system for design and implementation changes when needed. In this way, the BRFSS team ensures that public health surveillance efforts meet the highest scientific standards, use the most effective and cost-efficient approaches, and produce valid and reliable data and results.

BRFSS Expert Panel Meetings

Guidance on system improvements comes from a variety of sources, including state partners, other CDC Centers and Programs, and other outside experts in the fields of survey research, statistics, and epidemiology. In 2002, the BRFSS held its first biannual BRFSS Expert Panel Meeting, inviting ∼20 survey statisticians, methodologists, and operational experts to a 2-day meeting to discuss the challenges facing the field of survey research and implications for the BRFSS (42). Repeated in 2004 and 2006, these meetings were held with the goal of developing options and prioritizing recommendations for maintaining data quality in the face of societal and technological changes.

At the most recent meeting in November 2006, the panel made a number of specific recommendations, including using advance letters by all states, conducting pilot studies with cellular telephone users, and identifying the appropriate mix of sampling frames and survey modes to maintain the validity of BRFSS estimates. These and other recommendations made by the panels are critical for improving the BRFSS, ensuring the quality and validity of the data, and reducing the potential for bias in BRFSS estimates.

In part on the basis of recommendations from the Expert Panel Meetings, the BRFSS has undertaken a number of innovative and informative pilot studies and analyses, including the following.

Advance Letter and Answering Machine Message Experiments

The BRFSS examined whether a prenotification of an impending call from a survey research would help to improve participation rates. Split-sample experiments were conducted to examine the impact on BRFSS participation rates of sending advance letters and leaving scripted messages on the answering machines of potential sample members. The studies were conducted in 11 states in April and May 2003. The findings indicate that whereas advance letters significantly increased response rates, decreased initial refusal rates, and increase refusal conversion rates, leaving messages on telephone answering machines was not an effective strategy (30, 32). In fact, letters improved response rates, on average, 6 percentage points. The letters were also cost efficient in that the cost of obtaining a fixed number of completed surveys using advance letters was lower than the cost without letters. As a result, advance letters are recommended for use with the BRFSS in all states.

Assessing the Impact of the Do Not Call Registry

More than 100 million telephone numbers have been listed on the National Do Not Call Registry since it began in 2003. To assess the potential impact of the registry on participation rates in the BRFSS, case outcomes were examined from nearly 4.5 million telephone numbers called between January 1, 2002, and June 30, 2005. Using trend analyses and time series modeling, the findings indicated that once pre-Registry trends in response rates and other potential covariates were accounted for, the do-not-call rules appeared to have had no significant impact on state-level response rates in either a positive or a negative direction (35).

Use of Real-Time Telephone Survey Interpreters

Real-time interpretation during a survey can expand the number of languages in which surveys are offered. A detailed assessment of the quality of this approach was conducted as part of the BRFSS in California using behavior coding of interviews conducted with respondents who otherwise would have been finalized as “language barrier nonrespondents.” Interviews were recorded and behavior coded, quantifying for each question (a) the accuracy of the question interpretation, (b) the accuracy of the interpreted response, (c) the degree of difficulty administering the question, (d) the number of times the question was repeated, and (e) the number of times the interpreter and respondent engaged in conversation that was not relayed to the interviewer. The approach produced favorable results, with less than a 4% error rate for interpretation of the questions and a 1% error rate in interpretation of survey responses (36).

Use of Web and Mail Questionnaires

Participation in most random-digit-dialed (RDD) telephone surveys is declining, prompting researchers to consider using alternative survey modes to increase participation. In October and November 2003, a set of experiments was conducted using Internet and mail versions of the BRFSS questionnaire administered to potential respondents drawn from the standard BRFSS telephone sampling frame and reverse-matched to identify valid mailing addresses. Telephone survey follow-up was conducted with Internet and mail survey nonrespondents. Results were compared with those from the ongoing computer-assisted telephone interviewing (CATI) BRFSS. The findings suggest that self-administered modes when used in conjunction with telephone follow-up can improve levels of participation, but may also increase differences between respondents and nonrespondents on measures of interest (31, 33). As a result, overall nonresponse bias may not have been reduced despite increases in response rates.

Use of Address-Based Sampling

Advances in electronic record keeping have allowed researchers to develop and sample from a frame of addresses, which provides coverage rivaling that obtained through RDD sampling methods. The BRFSS examined the possibility of using address-based frames for surveillance by examining the quality of the frame of addresses. The quality and potential of such frames were promising (28). As a result, a pilot study conducted in 2005 compared use of traditional RDD telephone survey methodology with an approach using a mail version of the questionnaire completed by a random sample of households drawn from an address-based frame. The findings indicate that the mail survey approach can achieve higher response rates in low-response-rate states (<40%) than can RDD (particularly when two mailings are sent). Additionally, the address frame with mail survey design provides access to households with only cellular telephones and offers cost savings over the telephone approach (23, 27). The address frame seems to provide a potential alternative for RDD (24).

Improving the Current BRFSS Weighting Methodology

Postsurvey adjustments are becoming an increasingly important means of maintaining the representativeness of survey data. Using statistical raking techniques, the approach to weighting BRFSS data has been reevaluated. The new approach adjusts the data in terms of respondents’ sex and age, and also race (in a more consistent manner), education, marital status, and telephone coverage—variables all found to be significantly related to key health and risk outcomes on the BRFSS (5). In addition, the new weights will include adjustment for nontelephone coverage using the interruption in telephone services methodology (16). The new weighting methodology was implemented in 2007 with a plan to release the new and old methodology in 2010. This will allow investigators to examine the impact of the new weights on estimates and to monitor trends without the effect of changing methodology, especially for 2010 objectives.

Mixed-Mode Survey Approaches

The BRFSS conducted a pilot to examine the feasibility of using mixed-mode surveys involving mail surveys with telephone follow-up. The study compared data from a sample of telephone numbers drawn using RDD methods with a sample of addresses drawn using U.S. Postal Service records. The study was conducted in California, Florida, Massachusetts, Minnesota, South Carolina, and Texas. The pilot showed that the BRFSS could use a multimode approach for data collection to reduce bias (6, 25).

Surveying Cellular Telephone Users

To meet the challenge posed by the growing number of cellular-telephone-only households (Figures 2 and 3), the BRFSS conducted interviews with individuals sampled from known cellular telephone exchanges. The studies were designed to determine the feasibility of conducting BRFSS interviews by cellular telephone, their cost, and the impact on survey estimates of including such interviews. Results clearly showed that the BRFSS can reach the cell phone populations, especially the young age groups that were most likely to be excluded by RDD. In addition, the health behavior of the cell phone population was different from that of the RDD (26). Hence, the BRFSS concluded that cell phones should become a routine mode of data collection, although cell phones pose a major challenge in terms of location and number portability (37). The BRFSS is currently conducting interviews of at least 250 individuals in 21 states with cell phone only as part of the 2008 ongoing BRFSS.

INNOVATIONS

During the past five years, the BRFSS implemented several innovative procedures into the system. These innovations included the following.

Data Quality and Monitoring System

A data submission system via a secured Web link was introduced in 2003. This system allowed for rapid data transmission, editing, and reports generation. The system has a built in procedure to check incoming data in terms of adherence to collection protocols and deviation from expected prevalence of key demographics. For example, the data submitted for Georgia on a monthly basis is compared by age, sex, and race with census data indicating deviation from “true values” and prompting examination of procedures and operation. This is a routine mechanism that monitors the data on a monthly basis with feedback reports to data collectors to fix problems. As a result, BRFSS data quality has improved tremendously (13). In fact, BRFSS data have always been considered valid and reliable when compared with other national household surveys (43, 45). Indeed, this consideration is due to the historically rigid standards applied by the BRFSS when adding questions to the system (2, 3, 9, 51).

An added benefit of this innovative system is the ability of the BRFSS to add questions and provide reports in real time. For example, the BRFSS was used by the CDC to monitor the uptake of the influenza vaccination during the 2004–2005 shortage. Questions were added to the BRFSS within two weeks, and reports were generated on a weekly basis for same-week data collection. Hence, the BRFSS allowed the CDC to examine adherence to its policy (provision of reserve vaccines to individuals at high risk) and to change the policy in mid season as data showed the availability of vaccines in December (10, 20, 34). Moreover, the BRFSS can now respond to emergencies and natural or man-made disasters by providing much needed data to plan for adequate responses (15, 18).

Selected Metropolitan/Micropolitan Area Risk Trends (SMART)

As the usefulness of the BRFSS has increased, demand has also increased for more local-level data, that is, data at the district, county, or city levels. Although the BRFSS was designed to produce state-level estimates, growth in the sample size has facilitated production of smaller area estimates. The need for prevalence estimates at the local level has led to the creation of the Selected Metropolitan/Micropolitan Area Risk Trends (SMART) project to analyze the data of selected metropolitan and micropolitan statistical areas (MMSAs) with 500 or more respondents. The first SMART data provided estimates for 100 metropolitan areas in 2000. Preliminary results showed that the prevalence of certain behaviors varied across cities, not unlike the differences found across states (14, 44). Variation in prevalence was also observed when cities were compared with their surrounding metropolitan areas and with the rest of the state. This new use of BRFSS data fills a critical public health need for local-area surveillance data to support targeted program implementation and evaluation, and these data should help cities to plan and direct their prevention efforts better. The 2007 SMART BRFSS provided data on 198 metropolitan areas and 298 counties.

Child Health Assessment and Monitoring Program (CHAMP)

To assist state and local health officials in obtaining uniform data on the health conditions and behavioral risks of children and adolescents, the BRFSS has launched the Child Health Assessment and Monitoring Program (CHAMP). Although national estimates of health conditions and risk behaviors among U.S. children and adolescents had been periodically obtained through national surveys, these data are not typically available on a state-specific basis. This deficiency is viewed as critical for state health agencies, which have the primary role of targeting resources to reduce behavioral risks and their consequent illnesses. National data may not be appropriate for any given state; however, state and local agency participation is critical to achieve national childhood health goals. CHAMP facilitates the use of standardized questions and data collection methodologies for those aged 17 and younger in the general public. The information is collected via an adult proxy, who responds to questions about the child or adolescent. Use of a random child selection process and population-based weighting allows researchers to develop valid and reliable health and risk estimates for this population.

Although many states have previously used the BRFSS as a tool for collecting health information on children, these efforts tended to be state specific. For example, in 1998, Idaho, Kansas, Minnesota, and Rhode Island asked questions regarding child health. Since then, many other states added their own child health questions, including questions about asthma, immunization, safety belt use, bike helmet use, lead poisoning, chicken pox, dental sealants, and health insurance coverage. In November 2004, CHAMP was used for the first time to collect information across all states about a randomly selected child within the household as part of the influenza monitoring activity.

Chronic Disease and the Environment

In 2003, BRFSS data were used in conjunction with supplementary information sources. Although the BRFSS does not directly measure environmental quality (e.g., air pollution, water pollution, community-wide pesticide spraying), environmental information available from other sources such as the EPA, the United States Geological Survey, and state and local monitoring networks were used in conjunction with BRFSS data to compare measures of environmental quality and chronic disease. Pollutants in the environment have been linked to chronic diseases such as cancer and asthma and to cardiovascular health problems. The 2001–2005 Air Quality System (AQS) database measurements of ozone, sulfur dioxide, carbon monoxide, and particulate matter were used to calculate annual summary statistics for counties and metropolitan areas in the BRFSS. Most recently, the BRFSS released county variables using other data sources. The new release included more than 20 variables such as the number of alcohol outlets and fast-food restaurants in the county. The release of these new county variables with BRFSS data will allow researchers to better examine the associations among risk factors, the environment, and diseases.

Follow-Up Surveys

The demand for more questions and topics to be added on BRFSS surveys led to the creation of follow-up surveys. The idea is to identify respondents with a certain condition of interest and ask for permission to call them back to administer another survey. The BRFSS currently conducts follow-up surveys for asthma in 35 states. The BRFSS Asthma Follow-up Survey collects information on medications, family history, and environmental factors for adults and children who have been diagnosed with asthma at some point in their lives.

BRFSS Maps

The BRFSS introduced on its Web site a function to provide maps for all its risk factors. This utility allows for selection of variables and years to be mapped. The program runs on demand and allows for the addition of SMART data. Users could select their cut points for the maps and save their work for future use.

Web-Enabled Analysis Tool (WEAT)

The newest innovation is a program that allows users to run cross-tabulation and logistic analysis on the Web with no prior knowledge of statistical software for complex survey design. The idea is to increase the data usability and utility. This procedure allows the user to generate frequencies, cross-tabulations, and stratified analyses of BRFSS data. In addition, the user can generate logistic regression models to calculate the effects of one or more predictors, or independent variables, on a dependent variable that has two levels: at risk and not at risk.

FUTURE

The BRFSS is a valuable system for public health. Hence, maintaining the BRFSS and ensuring its high quality is a priority for the CDC and state health departments. However, the system is facing several challenges, which require secured funding. Recent data from the National Health Interview Survey (NHIS) indicate that 15.8% of households had only wireless telephones, whereas 13.1% received all or almost all calls on wireless telephones despite having a landline telephone in the home (8). Therefore, the BRFSS will have to use a mixed-mode or a dual-frame approach for data collection instead of RDD only. Indeed, a mixed-mode approach involving cellular telephones, land-line telephones, and mail survey is key for BRFSS survival. As household access to high-speed Internet services increases, Web surveys should also be incorporated into such mixed-mode survey designs. The mail and RDD frames could be used in complementary fashion, as part of a dual-frame approach: The RDD frame could be used primarily in areas where mail coverage is poor, and the mail frame could be used primarily to provide access to households without landline telephones.

This transition presents four major challenges. First, the use of different systems for data collection may introduce a mode effect. However, the BRFSS has intensively examined the potential of mode effect on its estimates to adjust for this effect in its final weights (29). Second, investigators need to develop a new methodology for combining data from different modes to produce local, state, and national estimates. The BFSS is currently working on a new weighting methodology to incorporate cellular telephones in its data. Third, because the BRFSS is a state-based system, ensuring that all state health departments are ready to make such a transition is a challenge. The BRFSS has worked diligently with all its states partners to address all the challenges from technical, operational, and contractual standpoints. Finally, the real challenge is how to maintain an efficient system that all states can afford and support. Indeed, moving to a mixed-mode approach may increase costs and affect the flexibility and timeliness of the system. For example, mail surveys may take longer to process compared with RDD surveys. Therefore, the new system has to maximize the use of RDD (both landlines and cellular telephones) and supplement it with the other modes to keep the impact of changes at a minimum.

The BRFSS is well equipped to make such a transformation in its methodology. The pilot studies conducted by the BRFSS were designed to gain the experience needed to switch to the mixed-mode approach. Indeed, the pilots showed that mail survey appear to be reasonably complete, and the estimates derived were largely equivalent to those produced with the use of current telephone survey methods. In addition, the pilots showed that the BRFSS could successfully contact and collect data from cellular telephones and Web users. The pilot experience has been widely disseminated to state health departments and other partners to ensure a smooth transition from the current system to the mixed-mode approach with minimal impact on data quality and trends of risk behaviors.

The BRFSS has been very successful in addressing operational challenges that arise during data collection. For example, the RDD design allows the interviewer to list the adults in the house and select one at random to avoid selection bias. However, in mail surveys, it is up to the respondents to randomly select an adult or child from the household. Therefore, the BRFSS has experimented successfully using the next birthday or previous birthday approach (7). Interviewing through a cellular telephone, however, has its challenges. The cost is higher compared with RDD. Moreover, the safety of the respondent (driving when receiving the call), cognitive engagement, and privacy (whether the respondent is in a place where he/she can discuss private matters) have to be considered. In addition, the current cost associated with receiving a call on the cell phone has to be addressed. The BRFSS has conducted several studies to ensure its capability to address the cellular telephones challenge (26). A reimbursement for the air time was instituted to encourage participation. The BRFSS is currently offering an electronic reimbursement on its cell phone operation in the 21 states. The electronic gift (e-gift) ensures the privacy of the respondent because there is no need for a name or address to mail the gift. Another challenge with cellular telephones is the geographic location. This has been addressed by asking the respondent about his/her county of residence and asking the appropriate state or county questionnaire. Because the BRFSS is conducted in every state, the system will allow for the exchange of interviews among states. Hence, no respondent will be excluded on the basis of his or her state of residence.

The BRFSS should collect physical measurements as part of its ongoing operation. Such data are valuable and will enable the BRFSS to adjust its data for self-reporting bias and facilitate validation of key BRFSS questions. Several studies have examined the limitations of self-reported BRFSS data and their impact on estimates such as obesity and blood pressure using national surveys (11, 12). The BRFSS could easily collect some basic measurements from a subsample of respondents. These variables should include height, weight, blood pressure, cholesterol, and glucose as a start and expand the list in the future. These new data will expand the utility of the BRFSS and create a tremendous opportunity for state health departments. For example, the data will allow states to determine the prevalence of diagnosed blood pressure (from regular BRFSS), the prevalence of undiagnosed blood pressure (from the measurements), and the prevalence of controlled and uncontrolled blood pressure. The BRFSS has the best system to undergo such an endeavor because states could use their own public health clinics for data collection. The CDC could provide technical assistance to ensure standardization.

As the BRFSS transitions to new methodologies for data collection, it is important to ensure that the impact is minimal on the ability of data users to monitor trends in risk behaviors and progress toward health objectives. To address this concern, the BRFSS is releasing data files on its Web site using the old and new methodologies of data collection until 2010. For example, the BRFSS 2007 data has two sampling weights: The first uses the same methodology of 2006, and the second has the new weighting methodology (which includes in the weighting formula the addition of marital status, education, and interruption in telephone services). Researchers will be able to monitor trends by using the old weights in both years and can see the impact of the methodology changes on their variables of interest. Indeed, this ensures a smooth transition into a bright future for the BRFSS.

In conclusion, the BRFSS has been a powerful tool for public health officials at the local, state, and national levels. The BRFSS is leading the research and development of alternative modes of data collection to meet the changes in telecommunication. The continued success and improvements are imperative to ensure the survival and utility of the system. As the cost of data collection and the need to examine different ways to reach respondents increase, so will the cost of the system increase. Indeed, maintaining adequate funding and support for the BRFSS is key for its survival.

disclosure statement

The author is not aware of any biases that might be perceived as affecting the objectivity of this review.