GetDups  - Get unique duplicate frames

Plugin for Avisynth 2.5

Copyright (C)2004 Alexander G. Balakhnin aka Fizick. http://bag.hotmail.ru

Copyright (C)2005 Turyst04, mailto:Turyst04@nextmail.ru  - synchronous modes and audio info using

Version 0.91 - September 20, 2005


    GetDups - is a special plugin for selecting unique duplicate frames from clip, it return frames which have copies only, by one from the series (group).

 It was developed for processing of clips, produced at capturing of amateur 8 mm (Super 8) films, if we use method with removing of obturator  from projector apparatus.
The plugin support two clip capturing modes: nonsynchronous - at low projection velocity (as a rule below 8 fps) or synchronous - with stabilized projection velocity 16,67 fps. Such source clip contains groups of frames duplicates (very similar frames), separated by intermediate transitional smeared (not-similar) frames. The plugin select one frame per such group.


     Function parameters:


first parameter - input clip
threshold - threshold of difference between fields of neighbor frames (real percent, from 0.0 to 100.0, default= 7.0)
fade - maximum of luma correction (default = 10)
chroma - use chroma information too (default = false )
tff - fields order - top fields first (=true) or bottom (=false) (default = as Avisynth assumes)
mode - processing mode (0, 1, 2 or 4) depending on input capturing mode:

0 – nonsynchronous (old mode, only one up to the version 0.5 inclusive);

1 – synchronous mode (16,67 fps), get on output 2 neighbor fields (from 3 input);

           2– synchronous mode (16,67 fps), get on output 1 field only (from 3 input);

           4 – nonsynchronous mode (like mode=0) with temporal smoothing of difference values;

            (default = 0).

show - mode of processing and info show

     (0 - processing, 1 - processing and info, 2 - test and info,     default = 0)

log - save statistic information about frame difference depending on output format (0, 1, 2, 3 or 4) or loudness params (for audio=1 in 3 columns: min, max and power) for each frame:

    0 - without Log (only one up to the version 0.7 inclusive),

    1 - Log-file contains difference between all neighboring fields - Top with Bottom, then Bottom with Top of next frame,

         1 column,

    2 - Log-file contains difference between  all fields - Top with Top, Bottom with Bottom, of neighboring frames,

         2 columns - 1-st for Top, 2-nd for Bottom (if mode=4 Log-file contains temporal smoothing difference values),

    3 - Log-file contains raw values (without smoothing) of difference between all fields - Top with Top, Bottom with Bottom, of neighboring frames,

         2 columns - 1-st for Top, 2-nd for Bottom,

    4 - Log-file contains both raw and smoothed values of difference between all fields - Top with Top, Bottom with Bottom, of neighboring frames,

         4 columns - 1-st for Top, 2-nd for Bottom (raw), then again Top and Bottom (smoothed),

    default = 0.

logfname - save Log in specified file name and path (default = "c:\\gd_stat.log")

audio - audio info using mode:

    0 - not used audio info (only one up to the version 0.8 inclusive),

    1 - use audio info (detect "good frame" without video info analyze)

    2 - use audio info  (like audio=1, but force protecting from misoperation in case of intermittent mechanism 2-splashed sound)

            (default = 0).

audio_thr - threshold of audio absolute value: from 0 to 32768 (default=5000);  best  value mast lay between lowest pick and highest bottom values

         in all of clip audio (use show mode for analyze).

audio_shift - shift from default detected "good frame" - positive or negative value (reasonable values for projection velocity 4-8 fps is from -2 to 2),

         if default value audio_shift=0 make bad results, try tune this param.

debug - output info for DebugView utility (default=false)  

 

 

   Features and limitations 

  

  1. Works only in YV12 and YUY2 color format.

  2. Directly works with progressive clips.

  3. Tested with Avisynth  2.55.

  4. No arbitrary access to frames. Plugin correctly works only with strictly sequential processing from first frame to end of clip.

  5. Uses method and code of pixels block comparing from Dup plugin  (by Donald Graft/Klaus Post).

  6. Since version 0.2, plugin uses not inter-frame differences, but inter-fields (for correspondent fields of neighbors frames).

It outputs a frame that have a second field chronologically between these duplicate fields.

Correct fields order definition (tff, bff) is very important.

  7. Since version 0.6, synchronous mode is added (by Turyst04) for work with input clip, captured with stabilized projected velocity 16,67 fps, and suitable mode parameter. Input clip  must be processed as separated by fields. This mode allows to increase the capturing velocity (and to minimize the size of input clip file) without any loss of output quality.

  8. Since version 0.9, audio info using for alternative finding method mode is added (by Turyst04, partially used code from AudGraph plugin by Richard Ling) for work with input clip, captured with clear pawl intermittent mechanism (greifer) sound; suitable params added: audio, audio_thr, audio_shift.
 

    Simple sample scripts:

 1) for nonsynchronous mode:

Avisource("input.avi")
loadplugin("GetDups.dll")
AssumeTFF()
GetDups()
AssumeFps(16)
#Trim(0,999)

 

 2) for synchronous mode:

Avisource("input.avi")
loadplugin("GetDups.dll")

fc=Framecount()

AssumeTFF()

SeparateFields()
GetDups(mode=1)

Weave(AssumeFieldBased)
AssumeFps(16)
Trim(0,fc*2/3)


 3) for audio info using mode:

Avisource("input.avi")

loadplugin("GetDups.dll")
AssumeTFF()
GetDups(audio=1)
AssumeFps(16)
#Trim(0,999)

    Usage

Plugin is for one of method for 8 mm films capturing  by camcorder

(not for simple direct shooting from screen, not for per-frame one-shot capturing).

1) modify projector apparatus by removing of obturator to prevent brightness variations;

2) set low projection frequency about  4-6 fps (nonsynchronous mode) or stabilizing velocity 16,67 fps (synchronous mode);

3) project to small screen, shoot and capture video clip to file;

4) review clip by plugin GetDups in test mode (show=2, maybe with Log), check duplicates existence and detection.

If results are bad, then analyze Lof-file, tune threshold, check tff, try mode=4, try use audio=1 or re-capture film with lower fps .

5) make normal processing, write result to output file. Set output clip fps to 16 or 18 by  AssumeFps.

6) use only  selected unique frames at beginning of clip as final video,  remove rest frames by trim.

 

     Additional information

The some (currently small, please post more) discussion is at forum

http://forum.doom9.org/showthread.php?s=&threadid=73577

Great discussion about capturing of 8 mm (Super 8) films and GetDups plugin at Russian forum: http://forum.ixbt.com/0029/016088-8.html#234

How to make stabilizing velocity 16,67 fps projector and other aspects of synchronous mode -  look this supplement article (by Turyst04, in Russian only).

    License

This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2 of the License, or
(at your option) any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program; if not, write to the Free Software
Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
 

Documentation is distributed under CreativeCommons BY-SA 3.0 license.

Please consider to make some donation to be registered user.



    Version changes:
 

Version 0.1, November 2, 2004 - first public. Thanks to Eugen65 for plugin idea and discussions.
Version 0.2, November 3, 2004 - Fields analysis method, mean luma correction,  Show parameter now integer,

        ignoring of drops in captured frames.
Version 0.3, November 7, 2004 - Different frames now defined by one filed (was by both field), final frames now is copy of last duplicate.
Version 0.4, November 8, 2004 - Duplicate is now selected by most similar field if both difference are below threshold.

Version 0.5, November 8, 2004 - Compared block size decreased to 8 instead of 32.

Version 0.6, March 13, 2005 (Turyst04), version for debug only, isn`t published -  added synchronous mode and suitable mode parameter  for work with input clip, capturing with stabilizing velocity 16,67 fps.
Version 0.7, March 23, 2005 (Turyst04) - changed output fields finding method in mode=1 and mode =2:

   threshold not used (in contrast to mode=0), simply looking for local min and max fields differential values (fields with minimal differences  is used for output).

Version 0.8, April 10, 2005. Changed by Turyst04 : added output Log (for external analysis) and mode=4 for temporal smoothing method (smooth difference values only, not picture) combined with old method (with threshold, like in mode=0).

Changed by Fizick:   corrected  mode=0 algorithm: now plugin does NOT detect the frame as a (false) dup, if  the small difference is found for chronological first field only, and  the same field difference was small at previous step too.

Version 0.5.2, April 6, 2005 - was not public, released July 6, 2005 mostly for historical purposes :-). This special version is based on v.0.5 and contains my (Fizick) contribition to v.0.8 (must produce the same output as v.0.8 in mode=0) - Corrected mode=0 algorithm: now plugin does NOT detect the frame as a (false) dup, if the small difference is found for chronological first field only, and the same field difference was small at previous step too.

Version 0.9, June 21, 2005 (Turyst04) - added audio info using for alternative finding method audio=1:

  added params:  audio, audio_thr  and audio_shift.

Version 0.91, September 20, 2005 (Turyst04) - fixed some bugs, added method audio=2

Download GetDups version 0.52

Download GetDups version 0.91

 

 

Return to main page