Action/Event Detection

Human action recognition in videos has been a substantial research topic in computer vision and it is a fundamental tool for video concept detection, animation and synthesis, and so on. Event detection is a challenging problem that has not been yet sufficiently studied. Based on its difficulty, event detection can be roughly categorized into simple event detection, predefined MED and Ad Hoc MED. Much effort has been dedicated to the detection of sports events, news events, unusual surveillance events or those with repetitive patterns. Our research work includes two aspects: we explore knowledge adapatation for multimedia action/event detection (a framework shown in Fig. 1) and construct new models (e.g. deep representation learning) for event detection in surveillance videos ( shown in Fig. 2). 

Fig. 1 The illustration of our framework. We first map the homogeneous features of the auxiliary and target videos, i.e., Modality A into another space by a nonlinear mapping. The video concept classifier and the video event detector obtained from the homogeneous features presumably have common components which contain irrelevance and noise. We propose to remove such negative information by optimizing the concept classifier and the event detector jointly. Meanwhile, another event detector of MED videos is trained based on Modality B. Then we integrate the two event detectors for optimization, after which the decision values from both are fused for the final prediction.

Fig. 2 The proposed framework for detecting anomalous activities is based on two main building blocks. First, SDAE are used to learn appearance and motion representations of visual data, as well as a joint representation capturing the correlation between appearance and motion features. In the second phase, to detect anomalous events, we propose to train three separate one-class SVMs based on the three different types of learned feature representations. Once the one-class SVM models are learned, given a test sample corresponding to an image patch, three anomaly scores are computed and combined. The combination of the one-class SVM scores is obtained with a novel late fusion scheme. 

Selected Papers:

  • Knowledge Adaptation with Partially Shared Features for Event Detection Using Few Exemplars
    Zhigang Ma, Yi Yang, Nicu Sebe, Alexander G. Hauptmann
    IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 36(9):1789-1802, 2014. [PDF][BibTex]

  • Learning Deep Representations of Appearance and Motion for Anomalous Event Detection

    Dan Xu, Elisa Ricci, Yan Yan, Jingkuan Song, Nicu Sebe

    British Machine Vision Conference (BMVC), 2015. (Oral[PDF][BibTex]

  • A Prototype Learning Framework using EMD: Application to Complex Scenes Analysis
    Elisa Ricci, Gloria Zen, Nicu Sebe and Stefano Messelodi
    IEEE Transactions on Pattern Analysis and Machine Intelligence (PAMI), 35(3):513-526, 2013. [PDF][BibTex]