REACH Incubator lifts the veil on its evaluation process
At REACH Incubator transparency and trust is taken very seriously, and this is why we would like to add a disclaimer that all of the information below is public. We would like our applicants and incubated startups to have the utmost confidence that by becoming part of the REACH community, they are going to be evaluated based on clear and predetermined criteria.
We spoke to Iñaki Armendáriz Olcoz, R&D and Innovation Consultant, part of the Zabala Innovation team, responsible for the Open Call management and evaluation of proposals at REACH Incubator.
Open Call selection
What is the selection process after the applications have been submitted to an Open Call?
Only proposals submitted and finalized through the F6S platform are evaluated (check out F6S Privacy Policy).
A full list of applicants is drafted containing their basic information. Then a three-step evaluation process starts:
1. Eligibility filter: An automatic filtering to discard non-eligible proposals follows the short list → Criteria given in the next question.
Proposals marked as non-eligible get a rejection letter including the reasons for being declared as non-eligible. No further feedback on this part of the process is given.
2. Tractions indicators’ control: there is a set of traction indicators, used as warnings, to automatically discard proposals which do not have the structure or enough resources to commit with the programme. For this purpose, the proposals are categorized in two groups → early stage SMEs with 3 or less financial years closed, and established SMEs with more than 3 financial years closed.
Proposals not passing the automatic sorting receive a rejection letter indicating that the proposal does not reach the internal traction indicators needed for accessing the incubator.
3. Remote evaluation: After the tractions indicator’s control, the eligible proposals are subject to an evaluation process under the following criteria: (1) technical approach, (2) business and (3) team. An external Big Data expert and a business expert reviews each proposal, scoring each on the mentioned criterion. In addition, Digital Innovation Hubs (DIHs) and Data Providers, respectively, under Theme-Driven and Ready-Made tracks assess the challenge fit of the proposals with the DVCs defined for the call.
Each criterion (except for the challenge fit criterion of Track 1 and Track 2 challenges which is a yes/no flag) has a score from 0 to 5. A minimum score of 3 for each criterion and an overall score of 12 for the 3 criteria is needed as a minimum threshold. Only proposals reaching all the criteria are eligible for accessing the EXPLORE phase.
Iñaki Armendáriz Olcoz
R&D and Innovation Consultant, Zabala Innovation
Which are the necessary criteria that all startups must cover in order to get accepted into the programme?
Section 4 of the Guidelines of Applicants gives full details about all the eligibility criteria that a company applying to REACH incubator must meet. As a summary:
What steps are being taken to maximize transparency when it comes to candidate selection?
A complete Guidelines for Applicants document is prepared and openly shared through the REACH website which provides a full set of information regarding the open call for proposals. It includes specifics sections where an overview of the REACH programme is given, the whole process of evaluation is described and the eligibility criteria are explained.
Furthermore, while the open call for startups remains open, two online info webinars take place to support applicants during that application period. REACH also offers a dedicated support channel available for proposers at opencall@reach-incubator.eu.
Apart from the criteria marked as private (traction indicators), the rest of information is made public so anyone who aims to apply to the programme has open access to project documentation.
Once the evaluation process is finished and the applicants classifying to EXPLORE are confirmed, the REACH team remains available to solve any request, inquiry or clarification regarding the evaluation results. If, even so, the applicant considers that a mistake has been made or that the evaluators have acted unfairly or have failed to comply with the rules of this REACH Open Call, and that their interests have been prejudiced as a result, there is an appeal procedure available. The REACH Team will investigate the complaints with a view to arriving at a decision to issue a formal notice or to close the case within no more than twenty days from the date of reception of the complaint, provided that all required information has been submitted by the complainant.
Pitch Day evaluation
How does a pitching day look for the startups and the evaluators?
I would say that giving the startups the possibility to meet face to face, not only with their DPs and evaluators, but also with the rest of REACH participants and even members of the consortium, is always a positive and beneficial aspect. Most of them are startups in a very early stage of growth, and therefore need to expand their contact networks, get to know new investors that may see potential interest in supporting their solutions.
From the evaluators perspective, and the REACH project in general, personal meetings always favor a better defense and interaction with the applicants than those done remotely. It is the moment when candidates can give their best, convince the jury and be chosen to move on to the next phase.
What are the startups scored on? What can automatically disqualify a startup?
On the pitching day, participants are given a slot of 5 min (3 for the presentation and 2 for Q&A) to defend their solutions in front of the jury. Three experts in business and startup development will evaluate the experiments proposed based on their pitching skills and the operational capacity of the team, which contains the following subcriteria:
1. Pitching skills
PRESENTATION SKILLS AND STYLE: The presentation is clear and simple. Is it a catchy presentation? They know how to explain their business idea. Presentation timing has been properly calculated, the employed language body and attitude is adequate. Supporting slides for the presentation are not text/colour abusive. Are the slides without mistakes and easy to understand?
PROGRAM FIT: Do they fit into the program? Can we still coach them? Do they truly address the challenge? Do they present a feasible and high potential solution?
BUSINESS MODEL: How do they expect to make money, what is their revenue stream? Do they have a realistic forecast? Is it scalable?
2. Operational capacity of the team
TEAM EXPERIENCE BUILDING BUSINESS/ START-UPS: even if previous experiences in building a business have not been successful; the team does count with previous experience and has learnt from them.
TEAM MEMBERS BALANCE (BUSINESS AND TECHNICAL PROFILES): The team is well balanced and includes both technical and business profiles. The team must demonstrate technical (data sharing management) and Data Science competence during the demo and explanation of the solution (technical and data science acquaintance of the team) as well as business and commercial skills and experience.
COMPANY AWARDS AND RECOGNITION: The company counts with a recognized trajectory. The meaning of the presented awards/recognition for the company is clear.
Regarding the pitching, the only case a startup does not qualify for the next phase is not reaching the minimum threshold (12 out of 15 points) after the scoring of the evaluators. We strongly encourage all the participants to be present physically at the event, but in some specific cases, due to travel restrictions or visa issues, we allow them to defend their proposals remotely, not being this a ground for disqualification.
Are there any challenges that evaluators and organisers may face when it comes to the evaluation process?
I think the rules are pretty clear and well explained both for the participants and for the evaluators, so there is not too much room for misunderstandings.
The main challenges that we may face are related to the high amount of proposals we have to process and the short time we have to do it. Even if the eligibility and evaluation processes are done as much automatically as possible, there is always the possibility to have human errors when managing information and data between different files, and also the fact that the opinion of an evaluator, which are external experts hired exclusively for the evaluations, may not be the same as the applicant’s. But at the end, when one applies to the call, one has to assume that assessments are not done by machines, and therefore accept the results as they come.
Are startups generally happy with the selection? Have you ever received complaints and if yes – how have you addressed them? Has the complaint been beneficial to the improvement of transparency?
It is true that after each of the two evaluations that announced the list of participants for the EXPLORE phase of both open calls, we have received some complaints regarding the assessments done by the evaluators. Some of them claimed that the results are not fair, others that the comments provided by the evaluators do not match with the solutions proposed, and we also have complaints regarding the fairness and transparency of the evaluation process.
In any case, we try firstly to find an answer that can respond to the concerns of the complainant, always complying with the rules defined in the Guidelines. If that does not work, in some specific cases we have requested the evaluators to revise their evaluations, not with the aim of re-scoring a proposal, but to reshape the comments.
Another resource that REACH offers if the applicant considers that a mistake has been made or that the evaluators have acted unfairly or have failed to comply with the rules, and that their interests have been prejudiced as a result, is a formal appeal procedure. A complaint should be drawn up in English and submitted by email to opencall@reach-incubator.eu including contact details, or the subject of the complaint, or information and evidence regarding the alleged breach. Complaints should be made within five (calendar) days since the evaluation results are presented to the applicants. As a general rule, the REACH Team will investigate the complaints with a view to arriving at a decision to issue a formal notice or to close the case within no more than twenty days from the date of reception of the complaint, provided that all required information has been submitted by the complainant.
Follow REACH on social media:
