System of evaluation
If a proposal requests some hundreds thousands or million Euros, it will be evaluated at least by three independent experts from three Member States. The bigger proposals, requesting sometimes more than 10 million Euros, will be evaluated by five experts at least. The selection process of the experts is also regulated. The experts sign a confidentiality agreement, which means they cannot be involved in the proposals they received, they cannot discuss the proposals with anybody (except during the so called "consensus meeting") and they are not permitted to identify the other experts. During the evaluation they must be impartial and consequent and cannot use computer with Internet connection or mobile phone. Following the Calls, hundreds of proposals are evaluated in one or two weeks, on a very tight schedule.
A description of the evaluation procedure can be downloaded together with the other documents of the Call. The experts evaluate a concept, the progress beyond the state-of-the-art, the quality of the objectives, the effectiveness of the work packages. Then the professional structure of the proposed project will be analysed; the project management, the competencies of the partners, the distribution of resources. Finally they evaluate the proposals according to the potential impact focusing on the European objectives. All evaluations are expressed in scores ranging from zero to five. Zero means that the proposal is not in line with the Call. 1= only partially covers the objectives. 2= fair, but with significant failures. 3= good, but improvements would be necessary. 4= very good with some minor improvements, 5= excellent.
Very important: the experts cannot evaluate anything that may be likely but is not written explicit in the proposal!
Usually 100 proposals are evaluated by 30-40 experts. After the experts read the proposals, they assign the appropriate score based on strict criteria. Then the three (or five) experts have a meeting where they have to reach consensus on the score and the total. Every expert must justify their written opinion. It often happens that the experts fall in disagreement. In such a case, first they try to convince each other and agree on the score. If not, the next step is voting (with minority opinions noted) or the proposal will be further evaluated by two or three more experts. The meeting is moderated by an official from the EC and recorded by a rapporteur but they don't give an opinion. Their responsibility is to keep confidentiality, impartiality, consistence and to observe correct drafting.
Then the proposals submitted to a particular Call will be ranked in a so-called "panel" meeting. This time all the experts discuss together which proposals suits best to the European objectives and create a final ranking among the proposals. The proposals below threshold will not be discussed. The threshold is usually 3 by criteria. Meanwhile the total threshold is usually higher than 9 (based on the three criteria). But being above threshold is not enough. The negotiations will be started with the applicants whose high ranking is within the announced frame of the budget.
In case of the ICT only about 10% of the proposals gain support from the EU R&D&I Calls! This means that it makes sense to start this work only with a very strong commitment and awareness. But the potential advantage is much more than some Euros. An innovative business needs extended professional contacts, best practices, knowledge of the latest technologies, trends and research possibilities.
This is why I've created this web site.
"The ability to learn faster than your competitors may be the only sustainable competitive advantage."
Arie de Geus