Center Identification Number: 527-06
Project Title: Commuter Choice Program Case Study Development and Analysis
Sara J. Hendricks, Research Associate
Philip L. Winters, TDM Program Director
Center for Urban Transportation Research
University of South Florida
External Project Contact:
I. Project Objective
The objective of this project is to improve decision-making and guide investments in commuter choice programs based on a case study analysis that will aim to determine what characteristics make commuter choice programs successful.
II. Project Abstract
Commuter choice providers, transportation demand management (TDM) professionals – even part-time employee transportation coordinators – are asked not only to explain but to justify and give examples of the benefits of commuter choice programs. More often than not, such justifications include general statements such as, “These programs reduce vehicle trips, improve employee recruitment and retention, enhance job satisfaction, improve air quality, and reduce employee parking demand.” However, corporate decision makers and government officials often look to successes in comparable situations and to documentation in order to gauge the merits of implementing and investing in commuter choice programs. In the 2001 Association for Commuter Transportation (ACT) End-of-Year Survey, 59% of respondents requested that ACT “provide information on TDM-oriented statistics and case studies” as a means of improving the value of membership (ranking first among 10 possible choices).
It is observed that, although anecdotal information about existing commuter choice programs is generally available, the information usually presents these programs as snapshots in time and without a thorough examination of the context of the program. This anecdotal information does not go into enough detail to make verifiable determinations of cause and effect. As a result, employers and funders consider many of these existing “case studies” to be unconvincing sources of information regarding program effectiveness and benefits to businesses. In addition, other study methods, such as CUTR’s development of a work site trip reduction model using a neural network, built with thousands of “before” and “after” plans from employers, quantify the change in average vehicle ridership (AVR) of various combinations of commuter choice program elements. While this method uses employee data aggregated to the employer level to capture the many complex social and institutional influences that determine whether a commuter choice program is successful, it does not separate out and make known what these influences are such that commuter choice program managers can use this knowledge to tailor program elements to maximize effectiveness. For example, are there other explanations for dramatic swings in program effectiveness due to the loss of a long-term employee transportation coordinator or change in management?
This study aims to augment these quantitative assessments with the social and institutional contexts within which the programs operate. The value of a case study approach is the use of qualitative data that permit dealing with values and politics that may be an important part of the success (or lack of success) of commuter choice programs. Such factors may include issues relating to management, the corporate environment, local economic conditions, commuter attitudes toward transit, and company attitudes toward providing “work/life” benefits. The case study approach sets out to understand what the observed programs mean to the participants. Value increases as qualitative data is combined with quantitative data on the corporation and its employees, such as commute characteristics, proximity to transit, size, and whether the company is multi-site or single-site.
This project will collect and organize case study information that can be easily accessed and used for comparison and planning purposes. It will examine commuter choice programs that are considered to be among the most successful, for the purpose of evaluating overall program effectiveness that results from their unique combinations of services, incentives, and operating environment.
This project will use applied social research methods, specifically case study analysis, as detailed in Case Study Evaluations, November 1990, by the U.S. General Accounting Office, Program Evaluation and Methodology Division, and other authoritative sources. Using the Case Study Evaluations definition, “A case study is a method for learning about a complex instance, based on a comprehensive understanding of that instance, obtained by extensive description and analysis of that instance taken as a whole and in its context.” For this research project, the complex instance to be studied is the commuter choice program for a carefully selected number of work site locations.
The case studies will be collected by CUTR in consultation with the ACT's Transportation Demand Management Institute, FDOT staff, USF Computer Science and Engineering, and National Center for Transit Research (NCTR) staff and students.
III. Task Descriptions
Task 1: Establish Expert Panel and Conduct Literature Review
Existing case study material will be compiled based upon a literature review, from previous and ongoing studies including “Quantifying the Business Benefits of TDM” and the data sets obtained for the project “Work Site Trip Reduction Model and Manual.” Many regional and local commuter choice programs conduct annual evaluations from which information can be gleaned. CUTR already has data on several thousand employer records (though anonymous). The information now being obtained from Ventura County California, the South Coast Air Quality Management District, Seattle, Tucson and other locations can enable us to pinpoint potential case studies of successful commuter choice programs because we will know the change in average vehicle ridership (thus leading us to employers that report significant changes in vehicle trips). ACT has identified potential/partial listing of 200 case studies.
The purpose of this task will be to develop a list of existing commuter choice programs that are considered to be successful. This list will constitute a pool from which up to 12 programs will be selected for in-depth case study analysis. Case study selection will be finalized during Task 2, Case Study Design.
“Successful” will be defined for study purposes. Characteristics of successful commuter choice programs will be identified, including but not limited to the following:
Such characteristics will be studied to identify emerging patterns. Some of these characteristics may be selected for use as criteria for choosing the appropriate programs for further study.
An expert panel will be assembled, and teleconferences and net meetings will be held to allow the panel to provide input to the development of the commuter choice case study evaluations. Representatives who initially will be invited to participate on the panel will include the ACT Executive Director as well as ACT’s TDM Institute Chair, the Chair of the TRB Committee on TDM, a representative from FHWA, and a representative from FDOT. ACT will be consulted and requested to provide recommendations for persons to serve on the expert panel and to advise on the development of parameters for case study selection and analysis.
The role of the expert panel will be advisory and additive to the study approach devised by NCTR staff. The panel will provide input as to purpose and parameters for case study selection and analysis. The expert panel will be asked to provide input on the following questions and others to ensure that important considerations have not been overlooked:
After the initial identification of potential user groups of the case studies, additional user group representatives will be identified and invited to serve on the expert panel.
The end product of this task will be a memorandum summarizing what commuter choice case studies have been done in the past, what questions the case studies have attempted to answer, and any information regarding identifying factors that affect the success of commuter choice programs. The memorandum will also include a list of commuter choice programs that are considered to be successful with the development of opinions or hypotheses about the relationships between program attributes/conditions and degree of success. Such hypotheses may replace the initially proposed evaluation question or augment it as a theme within the proposed evaluation question.
To be completed 3 months from notice to proceed (NTP).
Task 2: Develop Case Study Design
There are several types of case study approaches designed to answer different evaluatory questions. These include illustrative, exploratory, critical instance, program implementation, program effects and cumulative. Assuming that the evaluation question of this study does not change significantly after receiving input from the expert panel, it initially appears that the program effects case study approach may be best suited for this study. This will be verified during the case study design process.
Program effects case studies can determine the effects of programs, the reasons for success or failure and examine causality. They are usually multi-site and can be stand alone or multi-method (integrating case study findings with other sources of information) and can be conducted before, during or after other methods. Causality is established through development of internally consistent explanation of what led to what and the conscientious use of information from within the site and from contrasting sites to rule out alternative explanations. Usually generalization is wanted but, for a highly diverse program, it may or may not be possible to answer the questions adequately and still have a manageable number of sites. Generalizability establishes that program causes and effects can be duplicated elsewhere.
Based upon the proposed evaluation question, site selection for a program effects case study can be based upon best case, worst case, representative cases, typical cases, or clustered cases. For purposes of this study and the proposed evaluation question, “What makes successful commuter challenge programs successful?”, site selection will be based upon the best or most successful commuter choice programs.
One challenge in the design of the case study approach is that trip reduction programs are highly diverse. Each trip reduction program may have one or more goals, service components, and combinations of incentives and disincentives, in addition to other determining characteristics, such as regulatory environment, availability of transit, land development patterns, number of employees and location relative to other employers, degree of traffic congestion, and the nature of the work conducted by the company. Generalizability can be achieved by choosing an appropriate number and type of sites. Highly diverse programs generally require selection of more sites. The criteria used in the selection of the case study sites must result in a group of case studies that is representative of successful trip reduction programs as a whole. Based upon the results from Task 1, it is possible that the commuter choice programs will be segmented in some way, such as by location, size, CBD/suburban, etc., as a means for ensuring proper representation among the chosen case studies.
If the number of case study sites that are necessary to achieve generalizability is more than 12, then the evaluation question to be answered by the study will be more narrowly focused, in which the pool of potential case studies is narrowed and representativeness can be achieved by a fewer number of sites. An example of a narrower evaluation question might be: What makes successful commuter choice programs that are located in a suburban environment successful? Or we might look only at voluntary programs to narrow the pool of potential sites. Careful specification of the purpose of the study matched with appropriate site selection will enable the study to efficiently choose the minimum number of case study sites while allowing for generalizability.
The expert panel will be consulted to provide input and feedback on the process of case study site selection as well as the resulting number and sites selected.
After completion of Task 1 and the initial information is assembled, it may be apparent that we should either conduct a survey before or after the case study data collection and analysis. This may be the case if it is determined that it is not possible to answer the evaluative question and achieve generalizability and still keep the needed case study sites to a manageable number.
Option 1: One solution would be to conduct the case studies first in a set of sites chosen to be representative of a broader population, then verify findings from the case study through targeted examination of administrative data, annual reports or a survey.
Option 2: Alternatively, examination of documents or the conduct of a survey first would identify findings of particular interest, after which case studies would be conducted in sites selected to maximize the ability to get the specific understanding required.
The end product of this task will be:
To be completed four months from notice to proceed (NTP).
Task 3: Devise Case Study Data Collection Plan
Data collection for a case study approach aims for measurement validity (or construct validity), which means that the information reflects what it claims to reflect and not something else. Measurement validity can be achieved through the use of multiple sources of evidence and the use of the chain-of-evidence technique.
Effort will be made to identify as many different sources of information as possible for each case study to corroborate findings. These include trip reduction annual reports and other documents. Interviews will be arranged with multiple contacts at all levels to gain broader perspective and reduce bias. Such contacts may include commuter choice program regulators, TMA representatives, work site top managers, mid-level managers, ETCs, employee participants and nonparticipants. Other data collection options may include the conduct of a survey before or after case study analysis.
A chain of evidence is the sequence from observations to conclusions. In a strong chain of evidence, an independent second evaluator can follow the documentation of evidence and come to the same conclusions as the first evaluator. A data collection plan will be devised that will incorporate methods to build such an audit trail through the use of procedures such as indexing and referencing.
The data collection plan will also detail what information will be collected from the case study sites initially, guided by the development of emerging themes, while providing data collectors the flexibility to follow leads as they identify them. Where possible, questions will be tailored to each case study site.
The expert panel will be asked to provide input on potential themes that should be initially covered for exploration in the data collection plan. Such themes would aid in the detailed description of successful commuter choice programs, such as the roles of management support, the economic environment, the availability of transit, the capabilities of the ETC and the available commuter choice program budget.
If more than one researcher will be collecting data, then orientation/training of data collectors will be undertaken to ensure consistency and comparability of information from one case study to another. Prompt review and comparison of workpapers among multiple data collectors will also safeguard against bias.
The final product of this task will be a data collection plan that will:
To be completed four months from notice to proceed (NTP).
Task 4: Conduct Case Study Data Collection and Analysis
A unique feature of case studies is that data collection and analysis are concurrent. As data come in, they are analyzed and the emerging results are used to shape the next set of observations. Case study analysis will be iterative, using the “observe, think, test and revise” (OTTR) cycle. The use of multiple sources of data will also allow for triangulation, which is the comparison of multiple, independent sources of evidence before deciding there is a finding. Once findings are determined, alternative interpretations of the findings are developed, then tested through the search for confirming or disconfirming evidence.
Causality in case studies is established through the development of internally consistent explanations of what led to what and the conscientious use of information from within the site and from contrasting sites to rule out alternative explanations. Reproducibility of findings is established through analysis of multiple sites and data over time.
Because this study will most likely be analyzing multisite case study data sets, methods will be selected and applied to integrate the case study information among case studies in addition to outside sources of information. Such methods may include the development of a matrix of categories, graphic data displays, tabulating event frequencies, cross tabulations, or time series analyses. It will be determined whether data collectors will be assigned to collect all information about a particular site and/or whether data collectors will be assigned a particular theme to investigate across all sites.
Depending upon what the NCTR staff and expert panel initially formulate regarding expectations of what conditions make commuter choice programs successful, a model of data analysis will be selected. If there are many strong expectations, then a technique of pattern matching can be applied. Pattern matching matches findings to previously formulated hypotheses or assumptions. If there are no strong expectations, then a technique of explanation building will be used. Explanation building uses the data compiled from case studies to structure the hypotheses or assumptions and explore emergent themes.
The final product of this task is the completion of the working papers containing the case study information documentation and analysis.
To be completed 12 months from notice to proceed (NTP).
Task 5: Prepare Final Report and Information Materials
A final report will be prepared describing the study approach and detailing case study findings. As part of final report preparation, report drafts will be submitted to the expert panel as well as to people from whom data were collected and their critiques will be included in an appendix.
A very detailed narrative mode in describing the findings of case studies is inherent in the purpose of the case study approach and the nature of the inquiry. However, it is suspected that the end user of the results of this study will want information in some more easily used format. In addition to the final report, a pamphlet or other appropriate format (web page, CDROM) will be developed for use by TDM professionals.
The final product of this task will be 30 copies of the final report and 300 copies of collateral material provided to ACT for distribution.
To be completed 14 months from notice to proceed (NTP).
IV. Project Schedule, Milestones
V. Project Budget
Notes: The project team will include faculty, students, and secretarial
and other support staff who will work directly on the project and
whose costs are reflected in the direct costs of the project as listed
VI. Student Involvement
Graduate and undergraduate students will assist in the development of a data collection plan, the assembling and analysis of case study information, and preparation of the final report.
VII. Relationship to Other Research Projects
The NCTR Work Site Trip Reduction Program (in progress) will collect anonymous datasets from a variety of cities with trip reduction requirements. CUTR_AVR neural network software developed by the research team and the USF Computer Science and Engineering in 1997 predicts change in average vehicle ridership based on employer before and after plans obtained from L.A. Phoenix and Tucson. We propose working with USF's Computer Science and Engineering faculty again to help use numerical and symbolic data (allowed by neural networks) that may be obtained from case studies to develop a model that looks at these programs from an employer's perspective as well as a transportation and air quality perspective, EPA's COMMUTER model estimates impacts using model coefficients and look-up tables. The model has not been validated against actual experiences. This model is used to estimate emissions and transportation impacts.
VIII. Technology Transfer Activities
NCTR will assist ACT to communicate the results of the case study analysis of successful commuter choice programs as a resource for TDM professionals and others involved in commuter choice activities. This aspect of the project will be funded by ACT through other federal sources.
IX. Potential Benefits of the Project
The results of this program could substantially influence the adoption of TDM and transit programs or expand participation in programs such as telecommuting and employer-provided commute tax benefits. These transit and vanpool co-pay programs by employers could have a dramatic impact on transit ridership as well as other alternatives to driving alone. According to APTA's “Fare Elasticity and Its Application to Forecasting Transit Demand” report - a 10 percent decrease in bus fares would result in a 2.3 percent increase in ridership in the peak period. Given that the maximum amount an employer can give is $100 per month to transit and vanpooling employees, they could be receiving very low cost or even free transit without public subsidies. As a result, ridership potential could be significantly higher. ACT also has identified this project as one of its top priorities. The end result will be increased ridership and improved performance of these programs as well additional public and private investment in TDM strategies that are the most effective.
Commuter choice, commute trip reduction, TDM, case studies
National Center for Transit Research · at the Center For Urban Transportation Research · University of South Florida · 4202 E. Fowler Ave., CUT100 · Tampa, FL 33620-5375 · (813) 974-3120 · (813) 974-5168 · www.nctr.usf.edu · Comments: firstname.lastname@example.org