Center Identification Number: 77911
Project Title: Development of a Program Assessment Instrument for the Certified Transit Technician Program – Phase One
Janet L. Davis, Senior Research Associate
Ed Bart, Program Manager Transit Maintenance Analysis and Resource Center
Center for Urban Transportation Research
External Project Contact:
Start and End Dates
Start Date: April 2009 Expected End Date: April 2010
I. Project Objective/Problem Statement
Since 1990, the Center for Urban Transportation Research (CUTR) at the University of South Florida (USF) has managed the Transit Maintenance Analysis and Resource Center (TMAARC), which seeks to foster a skilled workforce to maintain the public transportation vehicle fleet in Florida. Historically, the TMAARC Program Manager has been responsible for developing and implementing a curriculum for transit technician certification and apprenticeship programs. A new program, the Certified Transit Technician (CTT), is currently in the final stages of the planning, design, and approval process. CTT intends to prepare new students for employment in the transit industry, train existing employees toward certification and prepare current transit technicians for National Institute of Automotive Service Excellence (ASE) testing and certification.
In cooperation with Hillsborough Community College, the CTT curriculum is a planned sequence of three programs of instruction consisting of a total of 15 modules (referred to as occupational completion points) of study. The three programs are delineated as Transit Technician levels I, II, and III, and each tier contains five occupational completion points (OCP). Transit Technician I and II are prerequisites for Transit Technician III, and ultimately, transit technician certification. All candidates entering the program will begin instruction with the first module of tier one and are assumed to possess minimal technical knowledge. Candidates are expected to develop skills, progress through the course of study, and successfully fulfill all certification requirements. The courses may be taken in any sequence within a respective tier, but tiers must be taken sequentially, starting with tier 1, followed by tier 2, and ending with tier 3. Prior to advancing to the next tier, students must complete the five corresponding OCPs for each technician level and are required to demonstrate proficiency in a tier.
In order to demonstrate the value of the technical instruction program, evaluate the curriculum, and utilize the results for ongoing improvement of the instructional course, it is important to assess the overall effectiveness of the new maintenance training curriculum. The results will assist transit groups in fostering and maintaining a highly skilled maintenance workforce; students will receive vital feedback on performance, which will allow the educational program to be tailored to individual areas of need; and, over the long term, advanced skills developed as a result of this project are likely to produce a positive impact on overall transit maintenance performance measures throughout Florida’s transit agencies.
The success of this research effort depends on a productive collaboration between two CUTR internal program areas. Specifically, the TMAARC group will apply its extensive knowledge and expertise in the area of transit bus maintenance training to work with the Transportation Program Evaluation and Economic Analysis (TPEEA) group. The CUTR Program Manager for TMAARC has responsibility for the development and implementation of CTT, a new transit technician certification program. For prior research efforts, TPEEA has overseen the development of several performance monitoring systems for individual transportation agencies, as well as for the Florida Transportation Commission (FTC) and other transportation industry coalitions. The Program Manager for TMAARC requested the assistance of TPEEA in developing a program assessment instrument as a component of CTT. TPEEA will manage daily oversight of this project and work with the TMAARC Program Manager to generate the intended results. Researchers will follow the first five program participants through the Certified Transit Technician Program. The purpose of this project is to develop the CTT program assessment instrument after sufficient research, analysis and dialogue with transit maintenance managers, Hillsborough Community College education specialists, and human resource managers.
The proposed Principal Investigator for this project will be Janet L. Davis. Ms Davis has performed extensive organizational development assistance to Miami-Dade Transit and Florida’s Turnpike Enterprise. Prior to joining CUTR she was the chief operating officer of a large transportation agency that included a maintenance workforce of over 400.
The project team will also include Ed Bart as Co-Principal Investigator. Mr. Bart is the current TMAARC Program Manager.
This project is expected to take place in two phases over a period of two years. Execution of Phase 1 by the Research Center in no way indicates the Department’s approval of Phase 2. Work not included in this scope of services is not to be performed and will not be subject to compensation by the Department. The proposed scope of services for phase one of this project, which represents the first year of the CTT program, consists of the following supporting tasks:
Task 1: Review of Program Materials and Research Assessment Tools
The project team will review existing programmatic and course materials to ensure familiarization with all aspects of the Certified Transit Technician program and curriculum. TPEEA will then examine and research existing assessment methods that are being employed in the transit area, educational arena, and in related industries. This task will include a literature review, conversations with transit managers, HCC education specialists, and interviews with at least three Florida agencies. The agencies to be interviewed will include a large, mid-sized and small organization to gain some diversity in consultation.
The results of this task will be documented in a letter report to the TMAARC Program Manager.
Task 2: Develop Participant Survey
A survey tool will be developed to explore and identify scheduled participant’s attitudes and perceived abilities prior to entry into the program. Pre-participation survey results will be compared with actual performance of the first program (5 OCPs) in order to assist in the program assessment process.
Task 3: Develop Recommended Assessment Approach
Based on the findings of Task 1, TPEEA will develop and formalize a recommended approach to assessing the CTT program. The approach will address:
HCC evaluation criteria
Movement of participants through the program
Level of interest in program participation
Requests for additional instruction
Participant survey results
Relevance of course content to technician job duties
Recommendations in the form of a technical memorandum will be presented to the TMAARC Program Manager for review and discussion.
Task 4: Formalize Assessment Process
After making any adjustments as a result of comments received during Task 3, TPEEA will work closely with the TMAARC Program Manager to implement the developed approach. This implementation phase will include selection of an existing set of parameters for inclusion in the assessment instrument in addition to integration of these parameters into a CTT tracking database.
Task 5: Year One Review
At the conclusion of the phase one project period, researchers will prepare a written summary report, which will chronicle the development of the CTT program assessment instrument as developed over the course of Tasks 1 through 4. This comprehensive document will include a review of program materials and research assessment tools, a description of the recommended assessment approach, the CTT tracking database, and the formal assessment process. The summary report will also include all survey instruments and other assessment materials developed during the course of the research effort
Progress Reports The University of South Florida will submit quarterly progress reports to the Research Center. The first report will cover the activity that occurred in the 90 days following the issuance of the Task Work Order.
Reports should be submitted within 30 days of the end of the reporting period. Reports are due even if little or no progress has occurred (in which case, the report should explain delays and/or lack of progress). Progress reports should be sent in MS Word to Sandra Bell, firstname.lastname@example.org.
Progress reports must include the following information:
1. Contract Number, Task Work Order Number, and Title
2. Work performed during the period being reported
3. Work to be performed in the following period
modifications (i.e., to funding, schedule, or scope). This section is for
reporting/informational purposes, not for officially requesting an amendment.
5. A Progress Schedule (figures A, B, and C) updated to reflect activities for the period being reported.
Failure to submit progress reports in a timely manner may result in termination of the work order.
Draft Final Reports At the conclusion of phase one of the project period, CUTR will prepare a written summary report, which will chronicle the development of the CTT program assessment instrument as developed over the course of tasks 1 through 4. This comprehensive document will include a review of program materials and research assessment tools, a description of the recommended assessment approach, and the formal assessment process. The summary report will also include all survey instruments and other assessment materials developed during the course of the research effort.
The draft final report will be submitted to Sandra Bell, email@example.com. It should be edited for technical accuracy, grammar, clarity, organization, and format prior to submission to the Department for technical approval. The Research Center expects contractors to be able to provide well-written, high-quality reports that address the objectives defined by the scope of service. Draft final reports must be prepared in accordance with the Guidelines for Preparing Draft Final and Final Reports (http://www.dot.state.fl.us/research%2Dcenter/Program_Information/Guidelines%20for%20Preparing%20a%20Final%20Report%2012-07.pdf). This document provides information on all report requirements, including format requirements, the technical report documentation form, disclaimer language, and so forth.
Final Reports Once the draft final report has been approved, the university shall prepare the final report. The university will deliver a minimum eight (8) copies of the final report in MS Word on CD no later than the end date of the task work order, to:
The Florida Department of Transportation
Research Center, MS 30
605 Suwannee Street
Tallahassee, FL 32399-0450
Each copy will be provided on a CD or DVD (i.e., for a total of eight disks). If the project manager requires additional copies, such provision must be indicated in the scope.
The project manager will review the final report to insure that all issues identified for correction in the draft final report have been addressed.
I. IV. Project Kick-Off Meeting
A kickoff meeting will be held via teleconference and shall be scheduled to occur before any work begins. At a minimum, the project manager and the principal investigator will attend. The Research Center staff must be advised of the meeting and given the option to attend. Other parties may be invited, as appropriate. The subject of the meeting will be to review and discuss the project’s tasks, schedule, milestones, deliverables, reporting requirements, and deployment plan.
IV. Project Schedule
V. Project Budget
The project budget includes printing costs in the amount of $100 to cover the costs associated with producing deliverables outlined in Section VI.Salaries and Fringe 55,206.47
Fixed Price Sub Total 55,206.47
Indirect Cost (fixed price subtotal x 10%) 5,520.65
Total Fixed Price Amount 60,727.12
Total Lump Sum Amount (Salaries and Benefits) 60,727.12
Expenses (Cost Reimbursable) 206.00
Cost Reimbursable (Subtotal) 956.00
Indirect Costs (cost reimbursable subtotal x 10%) 95.60
Total Project Cost 61,778.72
VI. Use of Graduate Student(s) and Other Research Assistants
At present, there are no student roles envisioned for this research project.
No equipment is envisioned to be purchased under this project.
In lieu of traveling to three transit agencies to conduct in-person interviews, CUTR associates will conduct interviews with transit agency personnel via teleconference and/or web conference. In order to successfully complete this research project, CUTR associates will be required to travel in order to complete in-person contact with the first group of program participants (five transit technicians), who will be monitored as they progress through the training program. Input, necessary data, and information provided by these technicians along with observation of the technicians’ participation in actual training will form the groundwork for development of the program assessment tool. (Daytona Beach is the currently designated training site; however, the location may be changed to Pinellas County.) Specific travel destinations within Florida will be determined during the early stages of the project period. All travel shall be in accordance with Section 112.061, Florida Statutes. FDOT employees may not travel on research contracts. All travel must be approved in writing in advance by the Department’s Project Manager. A copy of the approval shall be forwarded to the Research Center for the official file.
National Center for Transit Research · at the Center For Urban Transportation Research · University of South Florida · 4202 E. Fowler Ave., CUT100 · Tampa, FL 33620-5375 · (813) 974-3120 · (813) 974-5168 · www.nctr.usf.edu · Comments: firstname.lastname@example.org