AVScreen: a Real-Time video augmentation method

Francisco J. Hernandez-Lopez and Mariano Rivera

We present an image and video editing tool that allows us to modify a region of any image or video by another image or video. This application is useful for advertisements, commercials, music videos, movies, etc. We name to our application the Augmentation Virtual Screen (AVScreen). AVScreen has different steps when applied to images than when it is applied to videos. The main difference for augmenting videos with respect to fixed images is that the occlusions need be managed. Moving objects in the foreground (FG) may occlude the augmented region in background (BG). So that we use a procedure for FG/BG video segmentation, that is implemented in NVIDIA video cards to fulfill the real-time requirement.


General Outline of AVScreen
Figure 1: General outline of AVScreen. Video 1 (row 1). Video 2 (row 2). Including video 2 in video 1 (row 3).



AVScreen robust to Ilumination Changes (IC):
Original Video AVScreen without IC control AVScreen with IC control
SEC_ORIGINAL_IC_00136
SEC_SEGMENT_IC_withoutTTO_00136
SEC_SEGMENT_IC_TTO

Download video: AVScreenRobustTo_IlluminationChanges.avi

AVScreen robust to Cast Shadows (CtSd):
Original Video AVScreen without CtSd control AVScreen with CtSd control
SEC_ORIGINAL_CtSd_00064
SEC_SEGMENT_CtSd_withoutCtSdControl_00064
SEC_SEGMENT_CtSd_CtSdControl_00064

Download video: AVScreenRobustTo_CastShadows.avi


AVScreen robust to Camouflage Situations (CmSt):
Original Video AVScreen without CmSt control AVScreen with CmSt control
 SEC_ORIGINAL_CmSt_00127
 SEC_SEGMENT_CmSt_withoutCmStControl_00127
 SEC_SEGMENT_CmSt_CmStControl_00127

Download video: AVScreenRobustTo_CamouflageSituations.avi




Get Adobe Flash player
You need Flash Player
for viewing the videos

Results with HD videos:


Original Video                                                                                                                                   Augmented Video
Wall_CIMAT_00070
Download video: Wall_CIMAT.avi


Original Video                                                                                                                                   Augmented Video
Voley_GTO_00116
Download video: Voley_GTO.avi





Augmentation of Unstable Camera Videos and Video Shots in Real-Time

It is an extension of AVScreen. Given an unstable camera video, we stabilize the video and then we apply AVScreen for augmenting the video. On the other hand, given a video shot, we generate a mosaic (panorama) and then we apply AVScreen for augmenting the video shot.

Results in unstable camera videos:

Original Unstable Camera Video                                                  Augmentation of Unstable Camera Video
Shaking_Original&ComposedMosaic_00398
Download video: Shaking_Original&ComposedMosaic.avi


                         Original Unstable Camera Video                              Augmentation of Unstable Camera Video                          
Shaking_Original&Composed_00716
Download video: Shaking_Original&Composed.avi
 


                         Original Unstable Camera Video                                                                                                                                             Augmentation of Unstable Camera Video
Shaking697_Original&Mosaic_00102
Download video: Shaking697_Original&ComposedMosaic.avi


          Original Unstable Camera Video                                                                                                                               Augmentation of Unstable Camera Video
Shaking697_Original&Composed_00150
Download video: Shaking697_Original&Composed.avi


Results in video shots:

Original Video Shot Augmented Video Shot
Stefan_Original_00030
Download video: Stefan_Original.avi
Stefan_ComposedMosaic00030
Download video: Stefan_ComposedMosaic.avi

Stefan_Original&Composed_00110
Download video: Stefan_Original&Composed.avi

skateboarding_00059
Download video: skateboarding.avi
skateboarding_Composed_00059
Download video: skateboarding_ComposedMosaic.avi

skateboarding_Original&Composed_00050
Download video: skateboarding_Original&Composed.avi

DiddyGoesBullJumping_Original_00100
Download video: DiddyGoesBullJumping.avi
DiddyGoesBullJumping_ComposedMosaic_00100
Download video: DiddyGoesBullJumping_ComposedMosaic.avi

DiddyGoesBullJumping_Original&Composed_00081
Download video: DiddyGoesBullJumping_Original&Composed.avi

                                                                                                                                                                                                                    

REFERENCES:

[1] F. J. Hernandez-Lopez, M. Rivera, Binary Segmentation of Video Sequences in Real Time, in: Ninth Mexican International Conference on Arti cial Intelligence, IEEE, 163168, 2010.
[2] F. J. Hernandez-Lopez, M. Rivera, Session and Poster, “VScreen: a Real-Time Augmented Video Method”, GPU TECHNOLOGY CONFERENCE, San Jose CA, USA 2012.
[3] F. J. Hernandez-Lopez, M. Rivera, Session, "Shaking and Shot Video Augmentation in Real-Time", GPU TECHNOLOGY CONFERENCE, San Jose CA, USA 2013.
[4] Hernandez-Lopez, F.J., Rivera, M.: AVScreen: a real-time video augmentation method. Journal of Real-Time Image Processing. pp. 1-13. (2013).




This page was last modified on September 26, 2013 by Francisco J. Hernandez-Lopez.