PROBLEM: Increasing a video's gamma or histogram profile can brighten a seemingly black nighttime surveillance video. However, with only 256 shades of YUV illumination, the results can fail to be forensically useful.

SOLUTION:
Amplify subtle color changes to detect movement (weapon, people in the shadows, etc...), amplify signage (logos, license plates, etc...), and other forensic needs. Besides the obvious use with low quality surveillance video, the exaggerated color shifts allow the detection of seemingly imperceptible temperature changes. For example, machinery heating up (and thus red incremented by one value in RGB space). Here it amplifies the otherwise nearly invisible shadows of two birds (b/w and loops several times), even though this was a very low quality and noisy original. It did the same thing for suspects hiding in the bushes for one of my active cases.

BACKGROUND:
Forensic video analysts often encounter videos with washed out details. A few years back I invented an AVIsynth script that exaggerated subtle changes in pixels as they deviated from the average value of that same pixel across adjacent frames. The filter was painfully slow and suffered from out-of-memory issues. New tools provided a slight speed increase but the memory and speed issues were too constraining, until the script was recently optimized into a plugin.

Because RGB color space retains maximum details, adjacent pixels may have imperceptible color differences. If either pixel is part of a moving object, amplifying the difference will make the variance easily viewable. By restricting the affected color range, the user can target a specific need. The revised script allows the user to define the level of amplification, the temporal range for averaging, and the color range affected.

APPLICATION:
Filters that remove poison or salt-pepper noise may be more effective, and less likely to remove valid details, after RgbAmplifier filter processing. Once amplified, VELA testing can be applied to highlight the previously faint details that are now dominant events (e.g. a knife in a persons hand during a night time fight).

VIDEO PREPARATION:
Videos in RGB color space require a separate byte for each of the red (R), green (G) and blue (B) color values for every pixel. Videos in YUV (4.2.0) color space retain a luminosity (Y) byte value for every pixel, but retain only one red (U) and blue (V) color value for each 2 by 2 group of pixels. Since each byte is an integer value ranging 0 to 255, YUV color space requires only one third of the bytes compared to RGB with minimal perceptual loss in quality. This reduced file size supports faster video transmissions (TV, internet, CCTV, etc..), but is unsuitable for RGB amplification.

Because RGB color space can retain more data, conversion from YUV to RGB will result in skipped RGB values, and a slight loss in clarity due to the rounding errors from conversion. Processes that increase resolution using temporal Gaussian motion compensated de-interlacing or sub-pixel fusion can be used to recover missing details and clarity. The resulting video can then be resized back to its original resolution to retain some of the additional RGB values.

HOW IT WORKS:
Given a clip, this script examines every pixel of every frame and independently multiplies the difference of its specific R, G and B values from the average R, G and B values of that same pixel location spanning a defined radius of adjacent frames. If the new values are above or below the allowable RGB values (0 and 255), they are capped at those limits. The revised RGB values replace the original values, and the amplified clip is returned. If the Multiplier is set to a value of zero, the plugin acts as a temporal frame averager (works great with license plates).

The filter allows the user to define the color range to be affected. Any color outside that range will not be touched. If no color limits are specified, the resulting scene may look over saturated including any poison noise.

This tool will:
 Convert seemingly imperceptible color changes into obvious color changes
 Highlight seemingly imperceptible movements of high contrast pixels
 Sharpen the details of moving objects in low activity videos (e.g. surveillance videos)
 Cause distortion or ghosting/transparency effect to moving objects (an unavoidable side effect)

This plugin requires RGB24 or RGB32 color space, and is intended for forensic analysis of videos created from fixed position cameras. Usage on non-stationary cameras may produce unpredictable or content destructive results. The faster the object is traveling and/or the greater the filter's radius setting, the greater the risk of negatively impacting the visual accuracy of the moving object.

=====
USAGE:
RgbAmplifier(clip clip,int "Radius"=2,float "Multiplier"=2.0,int "Rmin"=0,int "GMin"=0,int "Bmin"=0, \
int "RMax"=255,int "GMax"=255,int "BMax"=255,bool "Precise"=false)

=====
PARAMETERS:

Radius: integer from 1 to 99 (default 2). Defines the radius of how many frames are to be averaged (2 * radius -1). For example, a radius of 7 will instruct the filter to average 13 frames (2 * 7 -1). A value of 1 effectively disables the function (useful if adjusting the values within AVSpmod).

Multiplier: float from >=0.0 to <=10 (default 2.0). Defines the multiplier for the difference between a pixel’s color value (for each of R, G and B values) and its average value (as defined by the Radius). For example, the default value of 2.0 will replace every R, G and B value of every pixel with a value twice as far from the average R, G or B value of that pixel.

Rmin: integer from 0 to 255 (default 0). Cannot be greater than Rmax. The filter will not alter any pixel whose Red color value is below this threshold value.

Gmin: integer from 0 to 255 (default 0). Cannot be greater than Gmax. The filter will not alter any pixel whose Green color value is below this threshold value.

Bmin: integer from 0 to 255 (default 0). Cannot be greater than Bmax. The filter will not alter any pixel whose Blue color value is below this threshold value.

Rmax: integer from 0 to 255 (default 255). Cannot be less than Rmin. The filter will not alter any pixel whose Red color value is above this threshold value.

Gmax: integer from 0 to 255 (default 255). Cannot be less than Gmin. The filter will not alter any pixel whose Ged color value is above this threshold value.

Bmax: integer from 0 to 255 (default 255). Cannot be less than Bmin. The filter will not alter any pixel whose Blue color value is above this threshold value.

Precise: Boolean (default false). If "false" integer math is used for speed. If "true" double precision Floating point is used for accuracy. This is a plug-in setting only.

=====
EXAMPLES:

RgbAmplifier ()
Double pixel color deviations from the average of the current, preceding and postceding frame’s pixel value

RgbAmplifier (Radius=5, Multiplier=0.0)
Temporal averaging using the current frame and the first four (Radius - 1) preceding and postceding frame’s pixel values

RgbAmplifier (Radius=5, Multiplier=3, Rmin=0, GMin=169, Bmin=234, RMax=5, GMax=179, BMax=244)
Triple pixel color deviations from the average of the current frame and the first four preceding and postceding frame’s pixel values, but only for pixels with a color within 2% of pure Cyan (“00AEEF”)

If you would like a free hands-on training workshop, recommend me and this topic to the educational committee of any national group that you belong to. Upcoming training dates are listed in my CV.

=====
OPTIMIZED SCRIPT:

Function RgbAmplifier (clip clip, int "Radius", float "Multiplier",int "Rmin",int "GMin",int "Bmin",int "RMax",int "GMax",int "BMax") {
 myName="RgbAmplifier: "
 Radius=Default(Radius,2) Multiplier=Float(Default(Multiplier,2.0))
 RMin=Default(RMin,0) GMin=Default(GMin,0) BMin=Default(BMin,0)
 RMax=Default(RMax,255) GMax=Default(GMax,255) BMax=Default(BMax,255)
 Assert(Radius>=1 && Radius <= 99,"RgbAmplifier: Radius range 1->99 ("+String(Radius)+")")
 Assert(Multiplier>=0.0 && Multiplier<=10.0,"RgbAmplifier: 0.0 <= Multiplier <= 10.0 ("+String(Multiplier)+")")
 Assert(RMin<=255&&RMin>=0,myName+"RMin range 0 -> 255 ("+String(RMin)+")")
 Assert(GMin<=255&&GMin>=0,myName+"GMin range 0 -> 255 ("+String(GMin)+")")
 Assert(BMin<=255&&BMin>=0,myName+"BMin range 0 -> 255 ("+String(BMin)+")")
 Assert(RMax<=255&&RMax>=RMin,myName+"RMax range RMin -> 255 ("+String(RMax)+")")
 Assert(GMax<=255&&GMax>=GMin,myName+"GMax range GMin -> 255 ("+String(GMax)+")")
 Assert(BMax<=255&&BMax>=BMin,myName+"BMax range BMin -> 255 ("+String(BMax)+")")
 GScript("""
  WW=clip.Width() HH=clip.Height() FC=clip.Framecount()
  clipfinal=0
  for(i=0,FC-1) {
   clip2=clip.trim(i,-1)
   rangemin = Max(i-(Radius-1),0)
   rangemax = Min(i+(Radius-1),FC-1)
   diameter = rangemax - rangemin + 1
   for(y=0,HH-1) {
    for(x=0,WW-1) {
     red=0 green=0 blue=0
     redavg=0 greenavg=0 blueavg=0
     for(j=rangemin,rangemax){
      col = clip.RT_RGB32AsInt(n=j, x=x, y=y)
      b=RT_BitAND(col,$FF) col=RT_BitLSR(col,8)
      g=RT_BitAND(col,$FF) col=RT_BitLSR(col,8)
      r=RT_BitAND(col,$FF)
      redavg = redavg + r
      greenavg = greenavg + g
      blueavg = blueavg + b
      if(j==i) {
       red = r
       green = g
       blue = b
      }
     }
     if(r>=RMin&&r<=RMax && g>=GMin&&g<=GMax && b>=BMin&&b<=BMax) {
      redavg = Float(redavg) / diameter
      newred = Round(redavg + (Multiplier * ( red - redavg)))
      newred = Max(Min(newred,255),0)
      greenavg = Float(greenavg) / diameter
      newgreen = Round(greenavg + (Multiplier * ( green - greenavg)))
      newgreen = Max(Min(newgreen,255),0)
      blueavg = Float(blueavg) / diameter
      newblue = Round(blueavg + (Multiplier * ( blue - blueavg)))
      newblue = Max(Min(newblue,255),0)
      clip2=clip2.Layer(clip2.BlankClip(pixel_type="RGB32", color=(newred*256+newgreen)*256+newblue)
      \ .crop(0,0,1,1), x=x, y=y, level=257)
     }
    }
   }
   clipfinal=(clipfinal.IsClip()) ? clipfinal++clip2 : clip2
  }
  return clipfinal
 """)
}

The above work is the original creation and intellectual property of Doug Carner, president of Forensic Protection, Inc. with special thanks to Gavino and StainlessS for optimizing my script into a plug-in.

Side note: Original introduced in 2010 as "Paranormal filtering" to debunk supernatural videos, it (and I) was publicly ridiculed for its name, ignoring its potential value to the forensic community. It has been renamed "RgbAmplifier" to encourage scientific collaboration and a more civil discussion.

Update: In 2012, realizing the power of this technology, MIT began work on their version, "Eulerian Video Magnification".