Exhibitors 2023



Embodied AI Agent with a real robotic platform

Embodied AI Agent with a real robotic platform

The talk will focus on a real implementation of Embodied AI agent. We will start with an overview of the Machine Learning models covered within Reply R&D, therefore DinoV2 for Object Detection (https://dinov2.metademolab.com/), PALM (https://palm-e.github.io/ ) as a starting point for VLMs (Visual Language Models) and be able to generalize a large number of tasks that require multimodal input (both with images and text). We will then move on to a focus on a robotic agent such as SPOT by Boston Dynamics, therefore its architecture, the potential of this agent and the sensors present in stock. From here we will have the basis to move on to an implementation of Embodied AI Agents controlled completely with voice in natural language. We will show an orchestrator who, by receiving voice commands in natural language as input, will be able to control a robotic agent such as SPOT by Boston Dynamics and use the Machine Learning models necessary to complete the individual tasks within the episode initiated by the user. We will then show current developments in the way related to the use of Visual Language Models, such as RT-2(https://robotics-transformer2.github.io/) for robotic agents and LINGO-1(https://wayve.ai/ ) for autonomous driving.


Embodied AI Agent with a real robotic platform

Maccagni Giacomo, Federico Minutoli

Maccagni Giacomo: Computer Science Engineer at Machine Learning Reply, Master Degree in Computer Science and Engineering at
Politecnico di Milano, Passionated about time series forecasting and robotics. Actually working on Embodied AI Agents with Visual Language Models.


Federico Minutoli: Computer Science Engineer at Machine Learning Reply, Master Degree in robotics and AI at UniGe and more than five years of experience in machine learning and related fields, with a focus in computer vision and NLP, among the others. Lately, he specialized in large language models enabling the visual-language-action (VLA) paradigm and the concerns that may arise in society (the so-called AI safety)


  A5 (pav. 5A.05)
Back
 
Data updated on 2024-05-13 - 11.24.21 am