دانلود رایگان مقاله انگلیسی متدولوژی تصمیم گیری چند معیاره منعطف برای حمایت از مدیریت استراتژیک برنامه های علوم، فناوری و نوآوری - الزویر 2018

Abstract

Research funding programs are a policy instrument utilized by governments to influence the innovation process. They are usually elaborated, launched and managed by research funding agencies. In order to select the most adequate research projects, agencies often rely on the peer review process. This paper introduces a methodology to support funding decisions based on the peer review process. The methodology involves the use of a multicriteria decision model to support the assessment, evaluation, prioritization and selection of applications, under a multi-step decision-making process, which fits into a strategic management cycle within the agency. The Multiattribute Value Theory, being considered under a Value Focused Thinking approach, provides a basis for the construction of the multicriteria decision model. The good practices in peer review and also a logical framework for program management are considered by the methodology. A pilot study, presented in the paper, involved a retrospective implementation of a peer review process in the context of a program launched by the Ministry for Science, Technology, Innovations and Communications and the National Council of Technological and Scientific Development, in Brazil. The methodology allowed a clear distinction of roles. The agency staff in the role of decision-makers was responsible for making value judgments on behalf of the agency. The experts, in the role of committee members and ad hoc reviewers, contributed with their expertise by providing objective assessments. Such assessments served as a basis for evaluating the applications, characterizing the possible portfolios, and can be considered as data in future program evaluation studies.

5. Discussions and Conclusions

The CNPq staff involved in the pilot test could implement the methodology within the SMP cycle. They agreed that DEMUCTI encouraged the consideration of rational criteria, enabled addressing the program objectives in a more explicit way and also enabled the agency to orient the portfolio selection in the directions indicated by the strategic planning of CNPq. As demonstrated in results of Step 4 (refer to Section 4.3), the majority of reviewers were favorable to the new format of the Assessment Questionnaire. However, given that the main objective of the test was to confirm the applicability of DEMUCTI, the pilot had the following main limitations:

• The applicants were not provided with the application template, which brought difficulties for the assessment of some proposals, as discussed in Section 4.3.

• The Assessment Questionnaire did not undergo a complete pretesting that allows, among other things, to identify the most adequate wording, format and order of items and instructions (Colton & Covert, 2007)

• Since there were no partner institutions involved in the funding decision, the pilot test did not allow to check the feasibility of incorporating the objectives and preferences of external actors into the models considered by the methodology.

• In Step 5, only the a posteriori approach of the funding decision was considered. Hence, the pilot test did not allow to verify the performance of a decision model adjusted before the application submission, as in the a priori decision approach.

• As the performance evaluation of Step 7 was not implemented, the pilot study did not allow to confirm if the information gathered by executing DEMUCTI does benefit the subsequent decisions related to the ongoing program.