Image
Overview
This service performs the crucial step that follows the execution of physical experimentation, i.e., evaluation of the performance of the system under test. This activity requires the application of suitable performance metrics to specific test data collected during the tests and then interpreting the results in view of the features of the system and the experimental setting. Depending on the specific use case, performance metrics can involve pure computer processing, human expertise by agronomists, or a combination of both.
Application of performance metrics may include the development of custom software to extract the necessary information from experimental data and/or to apply suitable processing to the information. If the performance metrics require the data to be subjected to some form of pre-processing, Service S00115 may be leveraged to prepare the data.
Beside the object of this service (i.e., evaluation of results), agrifoodTEF can, on request, support the customer along any other aspect and phase of the physical experimentation pipeline. For instance, for the design of the environment and protocol for the test, the customer can leverage services S00106 and S00107. Preparation of the test environment can be done either by the customer or by agrifoodTEF via service S00110; in both cases, assistance in interconnecting the system under test with agrifoodTEF’s test infrastructure is available via service S00111. Finally, data collection during the test can, if needed, be provided by agrifoodTEF via service S00113.
More about the service
Where complex systems based on AI and/or robotics are involved, performance evaluation is not a straightforward activity. Once suitable performance metrics have been defined, it is important to correctly apply them to experimental data, which often requires that these be processed in a suitable way. Finally, the output of the metrics requires interpretation to extract useful insight on the actual performance of the system, the impact of any environmental factors, and the avenues for improvement that emerge from the analysis. All this is not necessarily part of the expertise of a company focused on product development.This service provides the customer with the expertise of personnel specifically trained in the experimental evaluation of AI- and robotics-based systems (and of advanced agricultural machinery as well).
This service provides the customer with the expertise of personnel specifically trained in the experimental evaluation of AI- and robotics-based systems (and of advanced agricultural machinery as well).
This service provides the customer with the expertise of personnel specifically trained in the experimental evaluation of AI- and robotics-based systems (and of advanced agricultural machinery as well).
The first phase of execution of this service involves discussing with the customer the features of the data to be analysed and the performance metrics to be employed. Additional elements to be discussed concern the features of the system(s) under test that are relevant to interpret the data.
This initial phase may take 1-2 weeks and proceeds via meetings, either in person or remote.The second phase of the service is the one where actual data processing, including application of performance metrics, takes place: as a preliminary step, it requires that the customer give access to the data to the service providers, under NDA if needed.
The duration of the second phase depends on the complexity of the data processing required and on the computational resources it requires but generally does not exceed 4 weeks.The third and final phase of the service is the interpretation of the results of the data processing, which involves expert personnel and usually requires one week.
The final outcome of the service is a report comprising the results of the performance analysis, including the identification of the weakest points of the system’s action and the aspects that, if improved, would significantly improve overall performance.
This initial phase may take 1-2 weeks and proceeds via meetings, either in person or remote.The second phase of the service is the one where actual data processing, including application of performance metrics, takes place: as a preliminary step, it requires that the customer give access to the data to the service providers, under NDA if needed.
The duration of the second phase depends on the complexity of the data processing required and on the computational resources it requires but generally does not exceed 4 weeks.The third and final phase of the service is the interpretation of the results of the data processing, which involves expert personnel and usually requires one week.
The final outcome of the service is a report comprising the results of the performance analysis, including the identification of the weakest points of the system’s action and the aspects that, if improved, would significantly improve overall performance.
This service description is intentionally generic. Every instance of this service is, in fact, customised to adapt it to the needs and requirements of the specific customer. The following is an example of a service instance.Example service: The performance of an autonomous weeding system is evaluated with respect to the classification accuracy, precision, and recall scores for the bean crop and Matricaria chamomilla weed (e.g., as defined via Service S00108). Metrics are computed on predictions generated from a test image collection (e.g., gathered via Service S00113 and later annotated either directly by the customer or by agrifoodTEF via Service S00290) and plotted/tabulated in a detailed report, which also includes qualitative images showing the correct and incorrect predictions. The report also identifies weak spots in the performance of the systems and provides suggestions about what improvement would be most effective in improving real-world performance.
Location
Italy
Remote
Type of Sector
Arable farming
Greenhouse
Horticulture
Tree Crops
Viticulture
Type of service
Data analysis
Performance evaluation
Accepted type of products
Data
