Lidar Car Detection

Keras Classification for Lidar Car Detection (counting)

In this dataset we got 3D points cloud from lidars and the task is to train a classifier to count other cars on the road.

To make things easyer to work with, we are going to remove useless data (over the car, floor informations), flatten the 3D points to obtain 2D images (view from top), augment the quantity of data (x4) by rotating the images and then train a Keras NN classifer on it...

Let's go !

Starter Code for Lidar Car Detection

What we are going to Learn¶

Learning about how lidar works
data normalisation
data augmentation
Using Keras for binary classification.

Note : Create a copy of the notebook and use the copy for submission. Go to File > Save a Copy in Drive to create a new copy

Hi fellas !¶

In this dataset we got 3D points cloud from lidars and the task is to train a classifier to count other cars on the road. To make things easyer to work with, we are going to remove useless data (over the car, floor informations), flatten the 3D points to obtain 2D images (view from top), augment the quantity of data (x4) by rotating the images and then train a Keras NN classifer on it... Let's go !

Downloading Dataset¶

Installing aicrowd-cli

In [1]:

!pip install aicrowd-cli
%load_ext aicrowd.magic

Requirement already satisfied: aicrowd-cli in c:\users\rock_\appdata\local\programs\python\python39\lib\site-packages (0.1.8)
Requirement already satisfied: click<8,>=7.1.2 in c:\users\rock_\appdata\local\programs\python\python39\lib\site-packages (from aicrowd-cli) (7.1.2)
Requirement already satisfied: tqdm<5,>=4.56.0 in c:\users\rock_\appdata\local\programs\python\python39\lib\site-packages (from aicrowd-cli) (4.61.2)
Requirement already satisfied: rich<11,>=10.0.0 in c:\users\rock_\appdata\local\programs\python\python39\lib\site-packages (from aicrowd-cli) (10.6.0)
Requirement already satisfied: requests-toolbelt<1,>=0.9.1 in c:\users\rock_\appdata\local\programs\python\python39\lib\site-packages (from aicrowd-cli) (0.9.1)
Requirement already satisfied: toml<1,>=0.10.2 in c:\users\rock_\appdata\local\programs\python\python39\lib\site-packages (from aicrowd-cli) (0.10.2)
Requirement already satisfied: requests<3,>=2.25.1 in c:\users\rock_\appdata\local\programs\python\python39\lib\site-packages (from aicrowd-cli) (2.25.1)
Requirement already satisfied: GitPython==3.1.18 in c:\users\rock_\appdata\local\programs\python\python39\lib\site-packages (from aicrowd-cli) (3.1.18)
Requirement already satisfied: gitdb<5,>=4.0.1 in c:\users\rock_\appdata\local\programs\python\python39\lib\site-packages (from GitPython==3.1.18->aicrowd-cli) (4.0.7)
Requirement already satisfied: smmap<5,>=3.0.1 in c:\users\rock_\appdata\local\programs\python\python39\lib\site-packages (from gitdb<5,>=4.0.1->GitPython==3.1.18->aicrowd-cli) (4.0.0)
Requirement already satisfied: certifi>=2017.4.17 in c:\users\rock_\appdata\local\programs\python\python39\lib\site-packages (from requests<3,>=2.25.1->aicrowd-cli) (2021.5.30)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in c:\users\rock_\appdata\local\programs\python\python39\lib\site-packages (from requests<3,>=2.25.1->aicrowd-cli) (1.26.5)
Requirement already satisfied: idna<3,>=2.5 in c:\users\rock_\appdata\local\programs\python\python39\lib\site-packages (from requests<3,>=2.25.1->aicrowd-cli) (2.10)
Requirement already satisfied: chardet<5,>=3.0.2 in c:\users\rock_\appdata\local\programs\python\python39\lib\site-packages (from requests<3,>=2.25.1->aicrowd-cli) (4.0.0)
Requirement already satisfied: commonmark<0.10.0,>=0.9.0 in c:\users\rock_\appdata\local\programs\python\python39\lib\site-packages (from rich<11,>=10.0.0->aicrowd-cli) (0.9.1)
Requirement already satisfied: pygments<3.0.0,>=2.6.0 in c:\users\rock_\appdata\local\programs\python\python39\lib\site-packages (from rich<11,>=10.0.0->aicrowd-cli) (2.9.0)
Requirement already satisfied: colorama<0.5.0,>=0.4.0 in c:\users\rock_\appdata\local\programs\python\python39\lib\site-packages (from rich<11,>=10.0.0->aicrowd-cli) (0.4.4)

WARNING: You are using pip version 21.1.3; however, version 21.2.4 is available.
You should consider upgrading via the 'c:\users\rock_\appdata\local\programs\python\python39\python.exe -m pip install --upgrade pip' command.

In [2]:

%aicrowd login

Please login here: https://api.aicrowd.com/auth/FJDaE6YjE__bJBYjYrp4qRGURS94q3xFToMLEbuYi4Q
API Key valid
Saved API Key successfully!

In [3]:

!rm -rf data
!mkdir data
%aicrowd ds dl -c lidar-car-detection -o data

Importing Libraries¶

In [1]:

import pandas as pd
import numpy as np
from sklearn.model_selection import StratifiedKFold
import random
import os
import matplotlib.pyplot as plt
import plotly.graph_objects as go
%matplotlib notebook

import tensorflow as tf
from tensorflow.keras.layers import *
from tensorflow.keras.models import Model, load_model
from tensorflow.keras.optimizers import Adam
from tensorflow.keras.utils import to_categorical
import tensorflow.keras.backend as K

Reading the dataset¶

In [2]:

# Reading the training dataset
train_data = np.load("data/train.npz", allow_pickle=True)
test_data = np.load("data/test.npz", allow_pickle=True)
train_data = train_data['train']
test_data = test_data['test']
train_data.shape, test_data.shape

Out[2]:

((400, 2), (601,))

Visualizing the dataset¶

In this section, we will be visualizing a sample 3D lidar data

In [3]:

# Getting a random 3D lidar sample data
INDEX = random.randint(0, train_data.shape[0]-1)

# Getting the individual x,y and z points.
x = train_data[INDEX][0][:, 0].tolist()
y = train_data[INDEX][0][:, 1].tolist()
z = train_data[INDEX][0][:, 2].tolist()

# Label for the corrosponding sample ( no. of cars )
label  = train_data[INDEX][1]

# Generating the 3D graph
fig = go.Figure(data=[go.Scatter3d(x=x, y=y, z=z,
                                   mode='markers',
                                   marker=dict(
                                   size=1,       
                                   colorscale='Viridis',
                                   opacity=0.8))])
print("No. of cars : ", label)
fig.show()

No. of cars :  4