Smart Video Surveillance of Human Interactions in Crowded and Strongly Occluded Scenarios Using Dense Trajectories from Video and Wearable Cameras

Abstract

The detection and recognition of conversational body gestures while in group discussions and conferences, and in crowded situations is the foremost concept we are addressing. Many related detections and recognition systems have failed to work during dense situations, which has multiple challenges such like cross-contamination among witnesses, heavy occlusions and heavily dynamic backgrounds that are influenced by the condition of the witnesses, light situations or obscurations and shadows. This makes the detection further complicated to analyze with the existing techniques of computer vision. We are approaching this problem by fusing multiple modal resources that is, by using video and body-worn cameras where the data will be continuously recorded. By using video modality, we analyze RGB-D for individual hand and arms trajectory tracking and gesture trajectory recognition concerned with the depth video and synchronized colour. This gesture detecting method uses motion, targeted witness, and depth-based particle filters that will emphasize the efficiency and accuracy largely, in the context of the witness performing gestures toward the camera device within a crowded environment and in front of moving loud and disarrayed environments. We are using the wearable camera to emphasize the natural association between communication, expressions and gesticulations during conversations, which is more efficient than the video. Our proposed approach fuses both the determinations from classifiers from the CCTV video source and body wearable camera modalities, we can increase the area under ROC performance largely. Also, the huge occlusion under the video source is remunerated by the data from the wearables. Therefore, we applied our approach to detect and recognize the speaking state, improving the solid connection found in the discussion between hand gesticulations and conversations.