Starter Code Using ORB Feature Matching

In this notebook, we will see how we can use ORB to match feature and then try to retrieve the best matches from the images. 🌋

How we are going to solve using feature matching¶

Take the missing image and target images and convert to grayscale
Next step, we initialize the ORB detector
Using ORB, compute the keypoints and the descriptors
Match the keypoints using Brute Force Matcher
Sort the matches according to the distance of the matched features
Now loop across the hundred mini images and sort and store the matches
Since the matches are sorted, the one's with smallest distance will occur in the front, which means the closest matches, so take the first five distances and store as minimum. As you loop through, if you get lesser distances, it means you have a better match, so replace the distance and the image id
Let's see how this works 😋

Downloading Dataset 😎¶

Installing puzzle datasets via aicrowd-cli

In [1]:

!pip install -qq aicrowd-cli
%load_ext aicrowd.magic

     |████████████████████████████████| 50 kB 2.6 MB/s 
     |████████████████████████████████| 170 kB 6.9 MB/s 
     |████████████████████████████████| 63 kB 1.4 MB/s 
     |████████████████████████████████| 1.1 MB 26.9 MB/s 
     |████████████████████████████████| 54 kB 2.5 MB/s 
     |████████████████████████████████| 214 kB 63.6 MB/s 
     |████████████████████████████████| 63 kB 1.6 MB/s 
     |████████████████████████████████| 51 kB 6.1 MB/s 
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
google-colab 1.0.0 requires requests~=2.23.0, but you have requests 2.27.1 which is incompatible.
datascience 0.10.6 requires folium==0.2.1, but you have folium 0.8.3 which is incompatible.

In [2]:

%aicrowd login --api-key xxxxxxxxx # replace with your api key

API Key valid
Gitlab oauth token invalid or absent.
It is highly recommended to simply run `aicrowd login` without passing the API Key.
Saved details successfully!

In [3]:

!rm -rf data
!mkdir data
%aicrowd ds dl -c face-recognition -o data

In [4]:

!unzip data/data.zip -d /content/data > /dev/null

Importing Libraries 🤩¶

In [5]:

import pandas as pd
import os
import numpy as np
import random
from tqdm.notebook import tqdm
from google.colab.patches import cv2_imshow
import cv2

random.seed(2022)

Utils 📰¶

In [6]:

image_ids = os.listdir("data/missing")
len(image_ids)

Out[6]:

In [9]:

def get_target_face(face_no, target_image):
  ''' This function helps to retrieve the individual faces from the main patch of 100 images'''

  x, y = (int(face_no[0]))*216, (int(face_no[1]))*216

  target_face = target_image[x:x+216, y:y+216]

  return target_face

Time to use ORB 🔮¶

In [10]:

################################ This block will take around 11 - 12 minutes on colab #######################################################3

orb = cv2.ORB_create() #initialize orb detector

predictions = {"ImageID":[], "target":[]}

for img_id in tqdm(image_ids):

  missing_image = cv2.imread(os.path.join("data/missing", img_id))
  missing_image = cv2.resize(missing_image, (216, 216))
  missing_image = cv2.cvtColor(missing_image, cv2.COLOR_BGR2GRAY) #convert to gray scale

  kp, missing_des = orb.detectAndCompute(missing_image, None) #keypoint calculation

  target_image = cv2.imread(os.path.join("data/target", img_id))

  lowest = 99999999999999 #variable to store the least distance
  best_id = -99 #stores the id of the best match

  for i, face_no in enumerate(range(100)):

    face_no = str(face_no)
    face_no = face_no.zfill(2)

    target_face = get_target_face(face_no, target_image)
    target_face = cv2.cvtColor(target_face, cv2.COLOR_BGR2GRAY)

    kp, des = orb.detectAndCompute(target_face, None)
    
    try:
      bf = cv2.BFMatcher(cv2.NORM_HAMMING, crossCheck=True) # the brute force matcher
      matches = bf.match(missing_des, des) # matches the features
      matches = sorted(matches, key = lambda x:x.distance) # sorting on distance

      if (matches[0].distance + matches[1].distance + matches[2].distance + matches[3].distance + matches[4].distance + matches[5].distance) < lowest:
        lowest = matches[0].distance + matches[1].distance + matches[2].distance + matches[3].distance + matches[4].distance + matches[5].distance
        best_id = i
    except:
      pass

  predictions['ImageID'].append(img_id.replace(".jpg", ""))
  predictions['target'].append(best_id)

In [11]:

submission = pd.DataFrame(predictions)
submission.head()

Out[11]:

	ImageID	target
0	5jtfj	13
1	k81ao	81
2	tf7t6	26
3	tae8e	46
4	qhd7m	91

Saving the Predictions 🗺️¶

In [12]:

!rm -rf assets
!mkdir assets
submission.to_csv(os.path.join("assets", "submission.csv"), index=False)

Submitting our Predictions 🌗¶

In [13]:

%aicrowd notebook submit -c face-recognition -a assets --no-verify

Using notebook: orb_feature_matching.ipynb for submission...
Scrubbing API keys from the notebook...
Collecting notebook...

                                                   ╭─────────────────────────╮                                                   
                                                   │ Successfully submitted! │                                                   
                                                   ╰─────────────────────────╯

                                                         Important links                                                         
┌──────────────────┬────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│  This submission │ https://www.aicrowd.com/challenges/ai-blitz-xiii/problems/face-recognition/submissions/173971              │
│                  │                                                                                                            │
│  All submissions │ https://www.aicrowd.com/challenges/ai-blitz-xiii/problems/face-recognition/submissions?my_submissions=true │
│                  │                                                                                                            │
│      Leaderboard │ https://www.aicrowd.com/challenges/ai-blitz-xiii/problems/face-recognition/leaderboards                    │
│                  │                                                                                                            │
│ Discussion forum │ https://discourse.aicrowd.com/c/ai-blitz-xiii                                                              │
│                  │                                                                                                            │
│   Challenge page │ https://www.aicrowd.com/challenges/ai-blitz-xiii/problems/face-recognition                                 │
└──────────────────┴────────────────────────────────────────────────────────────────────────────────────────────────────────────┘

Thanks for sticking by and feel free to drop any questions below. I'll do my best to help you out , Good luck ;)

Also here are some good resources to follow on -

Face Recognition

ORB Feature Matching [LB 0.648]