Welcome to our demo system

 

Introduction

 

The Eudemon system tends to demostrate our ideas on content-based video frame copy detection:
  Image feature extraction: We extracted 256-dimensional grey histogram from each image and use the histogram as a compact representation of the light and shade information in each image.
  Distance metric for histogram comparison: We use the Earth Mover's Distance (EMD) which has been proven to be effective for content-based image retrieval as the underlying distance for histogram comparison.
  Challenge : Although EMD is a powerful metric for histogram-based similarity search, it suffers from a cubic time complexity.Therefore, it remains difficult to use EMD in applications with stringent requirement for efficiency.
  New properties of Eudemon system:
      EUDEMON achieves high throughput even when a large number of queries are registered in the system.
      EUDEMON contains self-optimization component to automatically enhance the effectiveness of the filters based on the recent content of the video stream.
      EUDEMON provides a user-friendly visualization interface, named EMD Flow Chart, to help the users to better understand the alarm with the perspective of the EMD.

System Implementation

 
  EUDEMON framework is implemented using C++ with the support of the OpenCV library in version 1.0.
 DOWNLOAD DEMO VIDEO
    [AVI]     5.8M
    [SWF]  28.4M

System Architecture

 

The Eudemon system processes the sampling frames of online videos based on the Filter-and-Refine framework. The overall idea is to use efficient and effective lower or upper bound filters of EMD to rule out unpromising frames in advance. The following Fig. gives an overview of the system. The users need to provide the key frames in their personal videos and register them as the queries in the Eudemon Query Registrar. Another input to the system is the video to be monitored. When the system is started, Eudemon samples frames from the video at a fix rate. When the sampled frame arrives at the Eudemon Query Processor, its grey histogram is constructed. When a histogram is ready, the Eudemon Query Processor pushes it through the Eudemon Filter Chain to verify its similarity to each registered query. The Eudemon Filter Chian has equipped the B+ tree filter[1], R-EMD filter[2], N-IM filter[3], UBp filter[1] before the final EMD refinement. Two more optimization submodules are also embedded into the Eudemon Query Processor, the Eudemon Adapter and the Eudemon Optimizer. Eudemon Adapter utilizes the correlation among the neighboring frames, and adaptively replaces the least efficient B+ tree filter by a new filter that is generated based on the recent arriving frame. The adaptive update will guarantee that the B+ tree filter remains effective. Moveover, Eudemon further reduces the exact EMD computations by incorporating the idea of multi-query optimization which is implemented in the Eudemon Optimizer module. Eudemon also provides a Visualization Interface, named EMD Flow Chart, which illustrates how one histogram is transformed into the other by moving 'earth' among the histogram bins. This help users to gain some understanding on the matching results.

Fig 1.    System Architecture

System Interfaces

 

The user of our EUDEMON system will experience three interfaces, see the following Figures 2-4. At the Basic Interface, the user submits his/her querying images (in the left most panel) and the video to be monitored (in the right most panel). The similarity threshold and the number of B+ tree filters can also be set in this interface. Press any of the query image will pop out its corresponding greyscale histogram. The system transits to the Processing Interface, after the user finishes loading all queries and video into the system. The processing interface displays the qualified frames in the left top panel with the query image listed as the first element of each column. And on the left bottom panel the user can observe the real-time filter rates of different filters equipped in the filter chain. One interesting design of EUDEMON is the way we demonstrate the system performance. Instead of listing the system throughput in the interface, we play the processed frames one by one in the player shown in the right top corner. When the performance is good, the player plays the sampled frames in a smooth way. This will give the user a different but impressive experience to monitor our system performance. After finishing all sampled frames, the system automatically turns to the Result Display Interface. It displays the statistic trend of the system throughput. When clicking on any query image and one of its result, the EMD flow chart is laid out in the right panel. You can also save the query results by pressing the 'Save Rlts' buttom in the toolbar.

Fig.2    Basic Interface

Fig. 3    Processing Interface

Fig. 4    Result Display Interface

System Testing

 

In order to test the effectiveness of EUDEMON system in monitoring copy frames, we makes the following testings:
Test of system roburstness
In practice, a user-concerned frame might be changed artificially in a pirated video. To test the robustness of our system to such circumstance, we do a few modifications on each query frame. Specifically speaking, for each query frame (e.g., shown in the left top corner of Figure 5), we decolor it, alter its lightness, change its colors temperature, vary its saturation and make it distorted separately, see Figure 5 for illustration. Each query image is shown as the first element of each column. The testing results shown in Figure 6 shows that our system is quite robust and it successfully returns similar frames for each modified query image without false positive. The robustness of our system stems from the utilization of the EMD which consider both the aligned bins and neighbor bins.

  


Fig. 5    Query Setting

Fig. 6    Query Results

Test of different datasets
We test the searching effectiveness of our system using different datasets, including an MTV by Lady Gaga (shown in Figure 7), a vedio click about scenery (shown in Figure 8), a vedio click about animals (shown in Figure 9). Each query image is listed as the first image of each column. The testing results show that the Eudemon system sucessfully finds the similar frames to the query image.

  

Fig. 7    Query Results for Lady Gaga Dataset

Fig. 3    Query Results for Scenery Dataset

Fig. 4    Query Results for Animal Dataset

Q&A

 

What is the sample rate for frame capture?
   In our recent setting, it is 5 frames per second.

What is the similarity threshold used in Eudemon for EMD-based image retrieval?
   In our recent setting, it is 5. Intuitively speaking, a large threhold will increase the system recall and decrease the retrieval precision.

How many kinds of image features the Eudemon can support?
   Although recently Eudemon uses the global grey histogram of an image to represent its content, many other image features, such as shape and texture can also be used in Eudemon as long as they are represented by histograms.

What is the maximum throughput Eudemon can achieve?
   Based on our experiment using a movie data set, when the registered query number is as large as 80, the Eudemon system still guarantees an average throughput at 35 frames per second.

Why the System is called Eudemon?
   That is because the meaning of 'Eudemon' is the defender of the good. We do hope our related techniques can be used as the eudemon for the indivial copyrights.:)

References

 

[1] J. Xu, Z. Zhang, A. K. H. Tung, and G. Yu. Efficient and effective similarity search over probabilistic data based on earth mover’s distance. PVLDB, 3(1):758–769, 2010.
[2] M. Wichterich, I. Assent, P. Kranen, and T. Seidl. Efficient emd-based similarity search in multimedia databases via flexible dimensionality reduction. In SIGMOD Conference, pages 199–212, 2008.
[3] I. Assent, A. Wenning, and T. Seidl. Approximation techniques for indexing the earth mover’s distance in multimedia databases. In ICDE, page 11, 2006.

Contact

 

If you encounter any problems with the algorithms or the implementations of EUDEMON, please feel free to contact us (xujia AT ise.neu.edu.cn).