Social distancing surveillance: how robots will keep you in line

Thought LeadersProfessor Dinesh Manocha
​Adarsh Jagan SathyamoorthyUniversity of MarylandCould a COVID-19 robot help in social distancing surveillance? We spoke to Professor Dinesh Manocha and Adarsh Jagan Sathyamoorthy about how their new robot may be able to do just that.

Please could you introduce yourself and tell us what inspired your latest research into COVID-19 and social distancing?

Social distancing has been encouraged to help reduce the spread of COVID-19. Why is it important that people adhere to social distancing rules?

Image Credit: fatmawati achmad zaenuri/Shutterstock.com

There have been technology-based methods employed to help detect social-distancing breeches including WiFi and Bluetooth. What are some of the problems associated with these methods and how does your new robot overcome them?

WiFi and Bluetooth-based methods are accurate in detecting social distancing breaches and our approach complements them.  The WiFI and Bluetooth-based methods need appropriate sensing technologies and cannot be easily deployed in all kinds of environments (e.g., public places or isolated locations). These methods also need additional infrastructure to be in place for detection.

Our method uses the visual feed from a depth camera onboard a mobile robot and existing CCTV infrastructure (if available) to detect social distancing breaches. In addition, the robot can autonomously navigate and interact with people and encourage them to maintain social distancing. The robot also has a thermal camera, whose thermal image feed can possibly be transmitted to security/healthcare professionals.

If an individual with a much higher temperature than average (symptoms of fever) is detected in a certain location, the healthcare/security professionals in the building can discreetly follow protocols for COVID testing and contact tracing. We do not record people’s faces or identities. Therefore, an individual’s identity or symptoms (or any personal data) are not stored or associated.

Can you describe how you designed and made your new social distancing robot? How does it work?

Our robot is designed to track pedestrians or humans standing at public locations by evaluating the proximities between them.  We use a depth camera for this application. If a certain set or group of people are closer than 6 feet from each other for a certain period, our robot system automatically classifies them as a group.

If multiple groups are detected, the robot prioritizes groups depending on whether a group is static or mobile, the number of members in a group, social interaction between the members, etc., and autonomously navigates to the pedestrians in a top-priority group to encourage them to move apart. For autonomous navigation, our robot uses a Lidar sensor and our machine learning-based navigation algorithm.

Our robot uses a screen to display messages to the groups, and a thermal camera, whose feed can be monitored by healthcare professionals for symptom monitoring.

Image Credit: Sathyamoorthy et al., 2021, PLOS ONE, CC-BY 4.0

Your COVID-19 robot is also able to sort people breaching rules into different groups depending on whether they are mobile or stationary. How is it able to do this?

Using the images from the depth camera and existing state-of-the-art pedestrian tracking algorithms, we can estimate the positions and velocities of pedestrians around the robot.

Our technique to classify groups is based on observing the proximities between humans and pedestrians over a certain period. These proximities can be efficiently calculated from the estimated positions and velocities. The robot is used to provide warnings about social distance constraints to the relevant humans.

How could your robot help to keep people safe and potentially limit the spread of SARS-CoV-2?

Using a mobile robot that can make use of existing CCTV infrastructure in a building is used to warn people that do not follow the social distance constraints.  Furthermore, it encourages and promotes humans to maintain social distances.  Other methods to monitor social distancing are different, in terms of explicit interaction with a robot.

As mentioned before, our robot also aids in contact tracing since it has an onboard thermal camera. All these features would help curb the spread of the disease. We believe that our robot could be another tool for healthcare/security professionals to simplify their workloads.

What further research needs to be carried out before your robot could be used to help control social distancing within crowds?

We believe the robot needs the capability to be socially intelligent and automatically interact with large crowds. The social distancing guidelines are relaxed for the groups consisting of family members, colleges, or humans from the same household, etc.

Our current system lacks the capability to identify such social groups in all scenarios. We are trying to devise algorithms that will help the robot to identify such groups and make it more socially intelligent. We also need to develop better techniques for human-robot interaction.

Deep learning within science has become more widespread over the last few years. How important is deep learning to scienti"c advancements and do you see deep learning becoming more prominent as technology continues to advance?

Deep learning is a popular machine learning technique that has had a significant impact on many applications including computer vision, speech understanding, natural language processing, robotics, etc. They are frequently used in autonomous driving and driver assistant systems, smartphones, IoT devices, etc.  However, deep learning systems need good, labeled datasets.

Moreover, they use high computational power for training and we need to develop power-efficient methods that can be used with small and cheap devices or mobile robots.  

Image Credit: deepadesigns/Shutterstock.com

What does the future look like for autonomous robots within science?

The development of many autonomous robots like self-driving cars, surveillance drones, personal robots is greatly accelerated by recent developments in sensor technologies, machine learning, computer vision, motion planning, etc.

Many companies are developing personal robots that can be used for home monitoring and as social companions.  It is one of the most promising technologies with wide applications in science and daily activities.

What are the next steps for you and your research?

Our upcoming research would focus on improving the human-robot interaction and improving the social intelligence capabilities of such robots.

We are also developing next-generation robot navigation technologies that can enable us to operate in complex indoor and outdoor scenes. Ultimately, these robots have to be safe and friendly, so that they can be widely accepted by society.  They can be used as delivery robots or social robots

Where can readers find more information?

More information can be found here: https://journals.plos.org/plosone/article? id=10.1371/journal.pone.0259713 (https://journals.plos.org/plosone/article? id=10.1371/journal.pone.0259713)

https://gamma.umd.edu/researchdirections/crowdmultiagent/cm

Written by

Emily Henderson

During her time at AZoNetwork, Emily has interviewed over 200 leading experts in all areas of science and healthcare including the World Health Organization and the United Nations. She loves being at the forefront of exciting new research and sharing science stories with thought leaders all over the world.