Catalogue of Services

Design of test environments for physical testing

Any physical test activity involves three main components: environment (where the tests take place), protocol (defining what tests are executed and how) and evaluation metrics (used to assess the results of the tests). This service concerns the first element, i.e., the design of a physical testing environment for use cases such as (for instance) weeding, plant phenotyping, and precision spraying solutions. The protocol and the evaluation metrics can – if required – be designed via services S00107 and S00108. Depending on your requirements and reference system/solution to be tested, our team will design an ad hoc setup equipped with all the required features for testing. Environmental features include, for example, the crop and weed species to be prepared and their growth stage. - the plant layout and intra-row configuration, - seasonal weather and climate-related conditions (e.g., lighting conditions, wind, rain), - the type of soil, moisture level, and terrain conditions (e.g., uneven terrain, presence of any slopes, and so forth), - the technical infrastructure supporting the tests (e.g., electrical layout, network infrastructure, environmental sensors, data acquisition systems…). In order to consider all aspects of the environment, this service involves a team comprising both engineers and agronomists.

Design of testing protocols for digital testing

Any test activity involves three main components, i.e., environment (where the tests take place), protocol (defining what activities are executed and how), and evaluation metrics (used to assess the results of the tests). This service concerns the second element, i.e., the design of the testing procedure for digital systems such as (for instance) AI models or computer vision software. The digital environment and the evaluation metrics can be designed—if required—via services S00176 and S00178. In the context of testing customers’ solutions within digital environments, this service is targeted at designing a suitable protocol for digital testing based on the use cases specified by the customer. The components of the testing protocol can include: - Selecting the datasets to be used for testing - Selecting reference AI models to be used for testing (if needed) - Choosing data formats and metadata standards. - Defining data pre-processing and preparation steps - Defining values and ranges of test parameters- Defining the different phases of the protocol- Outlining the operations to be executed in each phase- Thoroughly describing the protocol specifics to ensure reproducibility Datasets to be used for testing can be provided by AgrifoodTEF and/or by the customer; if nothing suitable is available, other AgrifoodTEF services can be leveraged to collect and/or generate tailored data. The technical team executing this service comprises expert engineers but can also involve agronomists when this is necessary to ensure the relevance of the tests for the use case, e.g., to determine the distribution of test repetitions across the variation ranges.

Design of testing protocols for physical testing

Any physical testing activity comprises three main components:

the environment (where the tests are conducted),
the protocol (which defines the tests to be executed and their methodology),
and the evaluation metrics (used to assess the test results).

This service focuses on the protocol, while the environment and metrics can be designed as needed through services S00106 and S00108.

In the context of testing customer solutions within physical facilities, this service aims to create a suitable testing protocol based on the use cases specified by the customer. The components of the testing protocol defined in this phase may include:

Defining the different phases of the protocol and their duration
Outlining the operations to be executed in each phase
Quantifying the effort required for each phase, including the number and qualifications of personnel involved
Distributing testing operations over time, considering seasonal and daily variations, as well as weather conditions
Establishing acceptable and desired variation ranges for each configurable element in the testing environment
Determining the number of test repetitions required and their distribution across the variation ranges

To ensure a comprehensive approach to protocol design, this service involves a collaborative team of both engineers and agronomists.

Desk assessment activities for digital systems or data

Through this service, we offer customers a complete feasibility study of their digital systems and/or data, evaluating their suitability for the test use cases identified by the customer. The study involves both technical and agronomic aspects. The goal of this service is to provide the customer with a comparison between the features of the system or data submitted for review and the requirements of the use cases chosen for (possible) subsequent testing. Examples of systems and data requiring digital testing include (but are not limited to) software modules for data processing (e.g., machine vision modules), AI models, software architectures (e.g., control systems of robots), simulations, datasets collected in the field, synthetic datasets, designs of software systems, and so on. This assessment is crucial to provide customers with key information on elements of their system that need to be further developed to become “fit for testing.” It can also identify changes to apply to either the system or AgrifoodTEF’s testing facilities to meet the requirements for experimentation.

Desk assessment

Desk assessment activities for physical systems

Through this assessment, we offer customers a complete feasibility study of the use cases and systems that customers would like to test, comprising both technical and agronomic aspects. Examples of systems requiring physical testing include (but are not limited to) precision sprayers, weeding machines, robotised fruit picking systems, and other implements; examples of subsystems include (but are not limited to) perception systems (e.g., based on vision), autonomous driving modules, localisation systems, and mapping and navigation subsystems. This assessment is crucial to provide customers with key information on elements of the system that need to be further developed to become “fit for testing”. It can also identify changes to apply to either the system or AgrifoodTEF’s testing facilities to meet the test requirements.

Desk assessment

Evaluation of Automatic Machines and Agricultural Robots

We provide in-depth physical testing services for autonomous and semi-autonomous devices designed for agricultural applications such as irrigation, pest management, and weed control, with a focus on arable crops, specifically cereals, forage, flax, and potatoes. Our service evaluates multiple aspects of hardware performance under real-world farming conditions, including accuracy in task execution, reliability and consistency of results, and potential economic impact through cost savings and efficiency gains. Additionally, we assess the machine’s behaviour in real-world settings as well as its adaptability to various pedoclimatic conditions and finally its agronomic impact to ensure alignment with sustainable and productive farming practices.

Collection of test data

Evaluation of results of digital testing

This service concerns the processing of data collected during digital testing of a system, i.e., experimentation done in a computational environment. Digital testing enables easy collection of experimental data; however, such data then need to be suitably processed to extract information about system performance. The service applies a predefined set of performance metrics (defined either by the customer or via service S00178) to the collected data. Data collection during test execution is expected to have been done by the customer (on request, this activity can be supported by AgrifoodTEF via service S00183). The service includes all the activities needed to apply the performance metrics, including the development of any software component needed for data processing or data preparation. The output of the service is a report providing the quantitative results of the application of the performance metrics to the experimental data. The report also includes an evaluation of these results aimed at highlighting possible issues and pointing out lines for improvement.

Evaluation of results of physical testing

This service performs the crucial step that follows the execution of physical experimentation, i.e., evaluation of the performance of the system under test. This activity requires the application of suitable performance metrics to specific test data collected during the tests and then interpreting the results in view of the features of the system and the experimental setting. Depending on the specific use case, performance metrics can involve pure computer processing, human expertise by agronomists, or a combination of both. Application of performance metrics may include the development of custom software to extract the necessary information from experimental data and/or to apply suitable processing to the information. If the performance metrics require the data to be subjected to some form of pre-processing, Service S00115 may be leveraged to prepare the data. Besides the object of this service (i.e., evaluation of results), AgrifoodTEF can, on request, support the customer along any other aspect and phase of the physical experimentation pipeline. For instance, for the design of the environment and protocol for the test, the customer can leverage services S00106 and S00107. Preparation of the test environment can be done either by the customer or by AgrifoodTEF via service S00110; in both cases, assistance in interconnecting the system under test with AgrifoodTEF’s test infrastructure is available via service S00111. Finally, data collection during the test can, if needed, be provided by AgrifoodTEF via service S00113.

Data analysis

Evaluation of sensor performance for agricultural applications

We provide a service focused on evaluating the performance of a wide range of sensors used in agricultural applications. This includes chemical sensors (for monitoring nutrients and soil pH), optical sensors (for capturing crop health and canopy structure), thermal sensors (for detecting water stress and soil temperature), acoustic sensors (for assessing soil texture or monitoring pests), and electromagnetic sensors (for measuring soil moisture and conductivity). Our evaluation process is conducted under real-world conditions, ensuring that sensors are tested in diverse farming environments and agro-climatic scenarios. The service assesses critical parameters such as accuracy, reliability, durability, and adaptability to different crops, soil types, and operational conditions. These sensors will be integrated into solutions based on AI and robotics to enhance efficiency and decision-making.

Collection of test data

Laboratoire National de Meterologie et d'Essais (LNE)

Evaluation of technical and regulatory aspects for robotic devices or AI systems

No matter the stage in your product's life cycle, LNE supports your project with any required study or regulatory assistance. This service provides you with the know-how of its AI evaluation teams through a range of expertise to answer your needs of understanding the normative landscape and related testing possibilities. Our specialised technical and regulatory service delivers a meticulous evaluation of your robotic device or AI-based software, highlighting critical areas of concern and optimising compliance with European regulation (e.g., AI Act, Machine Regulation, Cyber Resilience Act, ...) and industry standards (ISO 18497, IEC 61508, ...). LNE offers tailored support throughout your product’s life cycle, utilising our expertise to address industrial challenges. Related to your AI or robotic solution, we will investigate and report the standards identified and their requirements to help you better understand and implement them. This service can optionally propose a blank audit to assess any discrepancies and implement corrective actions before certification. Visit the Website for more information.

Desk assessment

ELSA assessment

LCA assessment

Evaluation of the energy efficiency of the embedded hardware associated with the AI functionality

Laboratoire National de Meterologie et d'Essais (LNE)

By having your AI or robotic system tested by LNE, you ensure it meets the highest standards of safety and performance, boosting your product’s reliability and trustworthiness in the market. Partnering with LNE for rigorous testing of your robotic and AI technologies gives you a competitive edge and trusted third-party assessment that enhances your product credibility and opens doors to new market opportunities both locally and globally. This service provides testing of devices with AI-integrated control systems, focusing on energy efficiency and consumption during the product life cycle. By simulating a wide range of scenarios, we provide detailed insights into the energy performance of these AI systems with regard to their expected operational environment. These tests can be conducted under different conditions, either in a completely simulated way, through a hybrid test bench where the actual physical device is assessed within a simulated environment, or using physical infrastructures based on the device's operational environment to test it in a 'real' setting. Tests are conducted under controlled laboratory conditions, ensuring optimal reproducibility and repeatability, providing reliable insights into the system's efficiency and safety-critical functionalities. Typical assessments may cover the energy efficiency of SLAM, various types of end effectors, and other relevant components such as sensors, motors or AI software. This rigorous testing process helps identify potential areas for improvement, enabling companies to enhance the efficiency and sustainability of their products, ultimately leading to cost savings and reduced environmental impact.