Personal tools
You are here: Home Results Open-source NIFTi software
About NIFTi
NIFTi is about human-robot cooperation. About teams of robots and humans doing tasks together, interacting together to try and reach a shared goal. NIFTi looks at how the robot could bear the human in mind. Literally. When determining what to do or say next in human-robot interaction; when, and how. NIFTi puts the human factor into cognitive robots, and human-robot team interaction in particular.   Each year, NIFTi evaluates its systems together with several USAR organizations. Rescue personnel teams up with NIFTi robots to carry out realistic missions, in real-life training areas. 
Impressum

This site uses Google Analytics to record statistics on site visits - see Legal information.

 

Open-source NIFTi software

During the term of the NIFTi project a considerable quantity of new modules have been developed related to information processing and robot human interaction in USAR scenarios. A selection of the reusable modules is published as open source. All code should be usable in matching projects covering similar fields. This document contains a collection of all published modules arranged in four main topics: User Interface, Robot Navigation, Robot Perception, and System Infrastructure:

  • The User Interface topic includes all modules that concern the aspects of the NIFTi system that are related to the end user, e.g. GUI, speech recognition, high level control, and user requirements.
  • The Robot Navigation topic addresses modules that enable USAR robots to navigate in difficult terrain and basic capabilities for their localization.
  • The Robot Perception topic demonstrates modules for evaluation and interpretation of various robot sensor data, e.g. object detection, mapping, and image processing.
  • The System Infrastructure topic includes a dataset of indoor and outdoor experiments as well as tools and drivers for the hardware components used in NIFTi.

User Interface


Operator Control Unit


TRex

  • description
    • A geographical information system for visualizing robot locations, areas of interest, etc.
    • RosConnectionApp.exe, Bidirectional_trex_connection.exe (compiled code) (this is server software).
    • Trex Client (compiled code) This is client software.
  • contact person
    • (TNO)
  • licensing requirements
    • Not applicable
  • Link to software
    • Not applicable
  • Link to background information/papers

CLES

  • description
    • This is a prototype for cognitive workload estimation.
  • contact person
    • (TNO)
  • licensing requirements
    • Not applicable
  • Link to software
    • Not applicable
  • Link to background information/papers
    • Colin, T. R., Mioch, T., Smets, N. J., & Neerincx, M. A. (2012, September). Estimating an operator's cognitive state in real time: A user modeling approach. In RO-MAN, 2012 IEEE (pp. 627-633). IEEE.

SCET

  • description
    • A user requirement management tool
  • contact person
    • (TNO)
  • licensing requirements
    • Not applicable
  • Link to software
    • Not applicable
  • Link to background information/papers

Abductive Reasoning for Continual Dialogue Understanding


MRCP4J

  • description
    • A client library for connecting to speech recognisers using the MRCP protocol. An extended fork of an (apparently abandoned?) open-source project.
  • contact person
    • Miroslav Janíček (DFKI)
  • licensing requirements
    • GNU Lesser General Public License v3.0
  • Link to software
  • Link to background information/papers

The Talking Robots Toolkit

  • description
    • Tarot: The Talking Robots Toolkit provides functionality for building robots capable of communicating with human users using natural spoken dialogue. Tarot is based on a core API which provides the primary functionality for building up processes for comprehending and producing spoken dialogue.
  • contact person
    • Miroslav Janíček (DFKI)
  • licensing requirements
    • GNU Lesser General Public License v3.0 or later
  • Link to software
  • Link to background information/papers
    • None yet

Robot Navigation


Trajectory control

  • description
    • Augmented Reality based path planning integrating a trajectory tracking control for the UGV.
  • contact person
    • Mario Gianni (Roma)
  • licensing requirements
    • BSD
  • Link to software
  • Link to background information/papers
    • M. Gianni, G. Gonnelli, A. Sinha, M. Menna and F. Pirri. An Augmented Reality approach for trajectory planning and control of tracked vehicles in rescue environments. In Proceedings of the 11th IEEE International Symposium on Safety, Security and Rescue Robotics. Linkoping, Sweden, 2013.
    • http://www.dis.uniroma1.it/~alcor/site/index.php/publications.html

Mesh-based path planning and high level control

  • description
    • Mesh-based path planning and high level control
  • contact person
    • Bruno Cafaro (ROMA), Federico Ferri (Roma), Mario Gianni (ROMA), Matteo Menna (ROMA)
  • licensing requirements
    • BSD
  • Link to software
  • Link to background information/papers
    • none

TF to Path Converter

  • description
    • The TF to Path Converter listens to the ROS TFs to accumulate the robot's position over time and publish it as a path
  • contact person
    • Benoit Larochelle (DFKI)
  • licensing requirements
    • BSD
  • Link to software
  • Link to background information/papers
    • None

LB3 Visual odometry

  • description
    • Robot pose estimation from images. Point based approach - features, keypoints, landmarks, bundle adjustment.
  • contact person
    • Tomas Svoboda (CTU)
  • licensing requirements
    • BSD
  • Link to software
    • Under active development, not yet in a release form
  • Link to background information/papers

LB3 Visual compass


INSO - Kalman filter for IMU and odometry fusion


Referential tracking system for MATLAB


Robot Perception


Car detector

  • description
  • contact person
    • Karel Zimmermann (CTU)
  • licensing requirements
    • BSD
  • Link to software
    • not yet in a release form
  • Link to background information/papers
    • K. Zimmermann, D. Hurych, T. Svoboda. Non-Rigid Object Detection with Local Interleaved Sequential Alignment (LISA). In IEEE Transactions on Pattern Analysis and Machine Intelligence, pdf

Multi-modal sensor fusion and victim detection

  • description
    • Asus Xtion + Thermo synchronous ROS driver (capturing of color depth and thermal images)
    • laser_render (Semi-dense depth reconstruction from rotating laser scans for a given viewpoint using OpenGL)
    • Victim detector using multimodal input
  • contact person
    • Alexander Shekhovtsov (CTU)
  • licensing requirements
    • BSD
    • see p4p/COPYRIGHT for calibration
  • Link to software
  • Link to background information/papers

Object Detection


icp mapping


Point cloud coloring


Fusion of Thermo-Cam and 3DLS


Virtual PTZ camera


Image processing node


System Infrastructure


DATASET for multi-modal data fusion

  • description
    • UGV sensor data captured during 4km of indoor and outdoor experiments. Stored in ROS Fuerte bag files.
  • contact person
  • licensing requirements
    • BSD
  • Link to software
  • Link to background information/papers
    • to be published

In-Field Pictures Server

  • description
    • The In-Field Pictures Server is a central hub to exchange pictures between different actors connected to a ROS network.
  • contact person
    • Benoit Larochelle (DFKI)
  • licensing requirements
    • BSD
  • Link to software
  • Link to background information/papers
    • B. Larochelle et al., Experiences with USAR Mobile Interfaces: the Need for Persistent Geo-Localized Information, in Proceedings of the International Conference on Intelligent Robots and Systems, IEEE/RSJ, 2013. Accepted.

Ladybug 3 grabbing


xsens driver

  • description
    • ROS Driver for XSens MT/MTi/MTi-G devices.
  • contact person
    • Francis Colas (ETH)
  • licensing requirements
    • BSD
  • Link to software
  • Link to background information/papers

A C/C++ library that provides an interface with the platform actuators

  • description
    • Library providing an API to control the UGV actuators, e.g. platform speed and direction, individual flippers angle, laser scanner rotation speed. Battery level is also available.
  • contact person
    • Patrick Balmer (Bluebotics)
  • licensing requirements
    • GPL
    • GPL licence for EMS Thomas Wuensche
  • Link to software
  • Link to background information/papers

NIFTi-Arm


NIFTi-UAV

Document Actions