Skip to main content

Trustable and Dependable Intelligent Robots for Smart Societies 2024

Track: Benchmarking and Competitions

Date and Time: 13 Mar 2024 12:00-13:20, room Lavatoio

Organisers: Fabio Bonsignorio, Matthew Studley, Diego Torricelli, Pedro Lima

Main questions to be answered – Q1: Reproducibility, benchmarking, standards

Main questions to be answered – Q2: Objective measures of robustness, reliability and agility

Main questions to be answered – Q3: TRL roadmap for new products

Workshop Description: The reproducibility of research results and the objective and operational comparison of different methods is still not mainstream in Robotics, AI and Automation Research. However, in particular thanks to the EU Commission significant progress has been done, This will be a gathering of the TG on Benchmarking and Competions. The SotA will be presented and the road ahead discussed. Info about currently completed and starting projects and results willl be shared. The meeting is open to all. We will present discuss and finalise the ‘Rimini Declaration’, that you find here: Mentimeter data from last year’s workshop: Data from last year’s polls:

Intended Outcome: Workshop on recent developments in technology or applications, Workshop expected to create a roadmap or white paper, Workshop discussion topics of common interest, success stories, use cases, etc, Workshops engaging non-roboticists to understand needs/goals (policy, commercial, technical), ‘The RIMINI DECLARATION’ A 5 pages document summarising the actions to be undertaken to achieve mainstream reproducible and measurable robotics research

Approach: Trustable and Dependable Intelligent Robots for Smart Societies Subtitle: Research Competitions, Benchmarking, Societal and Ethical Issues This is workshop is a follow up to the first very successful edition we had in 2023 in Odense during ERF 2023. We propose the following: • Before the workshop we summarize in a document – the ‘Rimini Declaration on Reproducibility of Research and Benchmarking in AI and Robotics’- and in a database the insights we gathered from 2023 edition of the workshop • We then gather opinions about this document from the industrial and research community, and from the communities involved in funding, legal standards, etc (as far as this is possible and reasonable) • We will use Mentimeter or a similar tool to promote synthesis of knowledge from these opinions, and build an emerging consensus • We will finalize the document after the workshop • The workshop will consist of a short presentation from different angles of the ‘Rimini Declaration’ and of the results of the previous year meeting . There will then be a round table discussion of the community consensus to finalise the document . • After the workshop, the final ‘Rimini Declaration’ will be published. Before the workshop we will spread by mailing lists, social networks etc. a number of crucial thought provoking and outright provocative questions and challenges to the community with the objective of raising awareness and stimulating contributions on the topic. Thought provoking video from recent robotics competitions will be shared before and during the event. Hybrid Event Agenda – 80 mn: – intro + summary of the Declaration: 20 mn : Discussion and work on the document There will be a follow up Insight session with a number of teaser talks and poster presentations with a research orientation-

Contributors: We have a long list: Matteo Matteucci, Enrica Zereik, Davide Scaramuzza, Agnes Delaborde, Isabel Ferreira, Tony Downs (NIST),Ricardo Sanz, Antidio Viguria Jiménez, Alin Albu-Schäffer , Francesco Ferro