OpenCV
4.1.0
Open Source Computer Vision
|
Prev Tutorial: Back Projection
Next Tutorial: Finding contours in your image
In this tutorial you will learn how to:
Template matching is a technique for finding areas of an image that match (are similar) to a template image (patch).
While the patch must be a rectangle it may be that not all of the rectangle is relevant. In such a case, a mask can be used to isolate the portion of the patch that should be used to find the match.
We need two primary components:
our goal is to detect the highest matching area:
To identify the matching area, we have to compare the template image against the source image by sliding it:
For each location of T over I, you store the metric in the result matrix R. Each location \((x,y)\) in R contains the match metric:
the image above is the result R of sliding the patch with a metric TM_CCORR_NORMED. The brightest locations indicate the highest matches. As you can see, the location marked by the red circle is probably the one with the highest value, so that location (the rectangle formed by that point as a corner and width and height equal to the patch image) is considered the match.
If masking is needed for the match, three components are required:
The mask should have a CV_8U or CV_32F depth and the same number of channels as the template image. In CV_8U case, the mask values are treated as binary, i.e. zero and non-zero. In CV_32F case, the values should fall into [0..1] range and the template pixels will be multiplied by the corresponding mask pixel values. Since the input images in the sample have the CV_8UC3 type, the mask is also read as color image.
Good question. OpenCV implements Template matching in the function matchTemplate(). The available methods are 6:
method=TM_SQDIFF
\[R(x,y)= \sum _{x',y'} (T(x',y')-I(x+x',y+y'))^2\]
method=TM_SQDIFF_NORMED
\[R(x,y)= \frac{\sum_{x',y'} (T(x',y')-I(x+x',y+y'))^2}{\sqrt{\sum_{x',y'}T(x',y')^2 \cdot \sum_{x',y'} I(x+x',y+y')^2}}\]
method=TM_CCORR
\[R(x,y)= \sum _{x',y'} (T(x',y') \cdot I(x+x',y+y'))\]
method=TM_CCORR_NORMED
\[R(x,y)= \frac{\sum_{x',y'} (T(x',y') \cdot I(x+x',y+y'))}{\sqrt{\sum_{x',y'}T(x',y')^2 \cdot \sum_{x',y'} I(x+x',y+y')^2}}\]
method=TM_CCOEFF
\[R(x,y)= \sum _{x',y'} (T'(x',y') \cdot I'(x+x',y+y'))\]
where
\[\begin{array}{l} T'(x',y')=T(x',y') - 1/(w \cdot h) \cdot \sum _{x'',y''} T(x'',y'') \\ I'(x+x',y+y')=I(x+x',y+y') - 1/(w \cdot h) \cdot \sum _{x'',y''} I(x+x'',y+y'') \end{array}\]
method=TM_CCOEFF_NORMED
\[R(x,y)= \frac{ \sum_{x',y'} (T'(x',y') \cdot I'(x+x',y+y')) }{ \sqrt{\sum_{x',y'}T'(x',y')^2 \cdot \sum_{x',y'} I'(x+x',y+y')^2} }\]
Testing our program with an input image such as:
and a template image:
The right match is shown below (black rectangle around the face of the guy at the right). Notice that CCORR and CCDEFF gave erroneous best matches, however their normalized version did it right, this may be due to the fact that we are only considering the "highest match" and not the other possible high matches.