🗺 Location: Indian Institute of Science (IISc), Bangalore, India
📅 Duration: March 2022 - August 2024
💰 Funded by: Ministry of Electronics and Information Technology (MeitY), Government of India and Center for Networked Intelligence (CNI), a Cisco CSR initiative.
👩‍🏫 Guided by: Prof. Chandramani Singh and Prof. TV Prabhakar

Note: This work is under progress and the page will be updated. The network used in this work is detailed in the project “Time Sensitive Networking Switch”.

Objective Link to heading

1. Design and implementation of a Tactile Cyber-Physical System for real-time interaction between physical and virtual worlds for applications requiring ultra-reliable low latency communication (uRLLC).
2. To demonstrate a remote surgery scenario where a user with a haptic device(s) can in real time perform physical tasks using a robotic arm at a remote location.

Modules: Link to heading

• Using haptic devices to send kinematic data and receive haptic feedback
• Real-time control of a teleoperator robotic arm
• Intelligence algorithms for performance improvement
• Edge intelligent switch ports on programmable switches using P4

Results Link to heading

Intercity haptic bilateral teleoperation Link to heading

As shown in the video at the top of this page, the robotic arm at IISc Bangalore equipped with a marker pen was controlled by a human operator (yours truly) using a haptic device located about 400km away in IIT Madras Research Park, Chennai trace over an outline of a spiral. A video feed of the operating area from Bengaluru is provided to the human operator at Chennai. Haptic feedback was implemented to indicate operating boundary of the robot and a sense of floor touch.

P4-programmable edge intelligent ethernet switches for TCPS Link to heading

1. Tremor suppression Link to heading

The edge intelligent switch port connected to the haptic device runs the tremor suppression algorithm that:

• Filters out the hand tremors of the human operator that can cause unwanted movement of the teleoperator.
• Uses a threshold-based algorithm to compare the coordinates of the haptic device and discard the packets that are associated with tremors.
• Reduces the network load (up to 99.9%), and movement length of the teleoperator by suppressing tremors as shown.

2. Automatic pose correction Link to heading

A 5x3 array of force-sensitive resistors (FSRs) was built and attached to 3D printed custom fingertips designed using Autodesk Fusion 360. This was fitted on a Robotiq 2F-85 two-fingered gripper. It is used to detect the contact region and force between the gripper and the tool. An Arduino Mega communicates the sensor data to the switch via a host computer.

The edge intelligent switch port connected to the UR3 robotic arm runs the automatic pose correction algorithm that:

• Monitors the grip of the two-fingered gripper on a tool using the force sensor array.
• Calculates and sends the necessary translation and rotation steps to the robot to achieve a firm and accurate grip.
• Uses a geometry-based algorithm and a precomputed table of pose corrections for different sensor combinations as shown in the figure below.
• Reduces the control loop latency (100 μs) and network load by performing the pose correction locally on the switch port as shown.

  

  

Work Done Link to heading

Interaction with Virtual Objects Link to heading

• You can feel the surface of the objects using the Geomagic Touch haptic device and move them around by holding the button.
• The haptic device freely moves when not touching the objects. When the object is touched, a force feedback is felt in normal to the surface of contact.
• At the end of the video, force is demonstrated by placing the pointer on top of the cube and then taking the force away by stopping the program.

Force feedback from a simulated robot Link to heading

• A simulated robot was built using Vrep/Coppelia Sim. You can control the position of end effector using Geomagic Touch
• When force feedback is enabled, it stops you from moving into virtual objects like the cube wall
• Time series plot of x-coordinate of the simulated robot and real Geomagic Touch shows that the robot cannot move beyond the cube wall.

Teleoperation of UR3 robotic arm Link to heading

UR3 from Universal Robots is a high-precision capable collaborative robot. It comes with a patented programming interface – PolyScope. But it has no real-time control capability. A Python program using rospy library was developed for real-time control of the robot using its external control feature.
Some issues of vibration and jerky motion were fixed by developing a few algorithms.

Force feedback - operating boundaries and floor touch Link to heading

Applying force feedback to indicate breach of operating boundary and floor touch. This lead to easier and safer teleoperation of the robot for the writing task.

Intercity haptic bilateral teleoperation Link to heading

The low latency network infrastructure developed in the TSN project was leveraged to achieve the teleoperation of the UR3 robotic arm by a human operator using the Geomagic Touch haptic device across two cities. In the video at the top of the page, I am controlling the robotic arm at IISc, Bangalore from IIT Madras Research Park, Chennai.

Ongoing and future work: Link to heading

• Improving the performance and parallel processing using ROS2
• Intercontinental haptic bilateral teleoperation
• Application that utilizes haptic gloves to feel grasp sensation while interacting with virtual and physical objects.

Related publications Link to heading

• EdgeP4: In-Network Edge Intelligence for a Tactile Cyber-Physical System Testbed Across Cities, IEEE INFOCOM CNERT 2024. Link
• Towards a TSN-DetNet Intercity Testbed for Tactile Cyber-Physical Systems, IEEE INFOCOM CNERT 2024. Link