دانلود رایگان مقاله ارزیابی از آموزش تدارکاتی عبور

Abstract

We evaluated a training course called “Orientation to Transit Procurement”, designed and conducted by the National Transit Institute. This course is designed to provide Federal Transit Administration (FTA) grantees an overview of regulations and best practices related to the procurement process. Our objective in conducting the evaluation was to understand how transit agency staff made changes in procurement practices in response to the course training. The evaluation was mixed mode: an Internet survey followed by in-depth interviews with a small group of respondents. Survey respondents were also provided with an open-ended question providing us with additional context for our evaluation. Results show that the training is substantially successful at meeting the goal of improving procurement practices at transit agencies; indeed, most respondents report making changes at their agencies as the proximate result of the training. This was at odds with our exploration of knowledge of procurement topics, as most respondents gave inaccurate answers on multiple-choice “knowledge questions”. This may have been due to question structure or, more likely, the nature of online surveys. Suitable training on the procurement of information technology was also a main concern. The lack of training in this area is indicative of the broader challenge facing public transit agencies in how to incorporate new forms of technology into their existing practices and bureaucratic structures.

4. Conclusions

Our evaluation sought to examine four key aspects of NTI’s “Orientation to Transit Procurement” course. These were overall satisfaction with the course, knowledge retention, changes in agency practices, and participant suggestions for improvements. Transit professionals who take the course overwhelmingly report satisfaction with the course and with the instructors. The course provides a useful introduction to the work they do and they use the skills they learned when they return to their offices. Knowledge retention (or learning) results were, on the face of it, disappointing.However,we are confidentthisfinding is anartifactof the program evaluation methodology; perhaps our choices of questions were too difficult to answer in a survey context, or respondents treated the questionnaire, as suggested above, as a “closed-book exam”. Despite this, respondents seemed familiar with the basic concepts of the course as they were making necessary changes at their agencies. We recommend that when conducting a program evaluation of a training course, evaluators include instructions for any knowledge questions authorizing, if not encouraging, respondents to treat the questions as if they were arising in a real-world circumstance, and encourage the respondent to take the time to consult with the training materials or other sources. To some extent, these results support Alliger and Janak’s (1989) contention that Kirkpatrick’s levels are not causal. While we did find high customer satisfaction, this did not necessarily translate to evidence of learning. However, the behavioral changes we found seem to directly contradict the assertion of no causal link between course satisfaction and course-earned value. In other words, while our knowledge questions suggested ineffective learning, almost half of the survey respondents reported that they made changes to their agency’s procurement processes after attending the course. Indeed, we found ample detail of these changes. Some interviewees stated that they corrected processes that were in violation of FTA guidelines, while others streamlined their processes to improve efficiency. When no post-course changes were made it seems most likely that was because the agency already had sufficient and compliant processes in place. FTA seeks to improve compliance with their regulations, and the procurement training clearly contributes to this goal. Moreover, we found marginal-to-no evidence that unsupportive work environments hinder implementation of the knowledge brought back to the agencies at which our respondents worked.