creative blitz 8_7.1 _linkpreview copy 6.jpg

Getting Started Code for F1 Speed Recognition Challenge on AIcrowd¶

Author : Shubhamai¶

Download Necessary Packages 📚¶

In [1]:

!pip install --upgrade fastai

Collecting fastai
  Downloading https://files.pythonhosted.org/packages/5b/53/edf39e15b7ec5e805a0b6f72adbe48497ebcfa009a245eca7044ae9ee1c6/fastai-2.3.0-py3-none-any.whl (193kB)
     |████████████████████████████████| 194kB 8.3MB/s 
Requirement already satisfied, skipping upgrade: fastprogress>=0.2.4 in /usr/local/lib/python3.7/dist-packages (from fastai) (1.0.0)
Requirement already satisfied, skipping upgrade: scipy in /usr/local/lib/python3.7/dist-packages (from fastai) (1.4.1)
Collecting fastcore<1.4,>=1.3.8
  Downloading https://files.pythonhosted.org/packages/0c/98/60404e2817cff113a6ae4023bc1772e23179408fdf7857fa410551758dfe/fastcore-1.3.19-py3-none-any.whl (53kB)
     |████████████████████████████████| 61kB 6.2MB/s 
Requirement already satisfied, skipping upgrade: matplotlib in /usr/local/lib/python3.7/dist-packages (from fastai) (3.2.2)
Requirement already satisfied, skipping upgrade: requests in /usr/local/lib/python3.7/dist-packages (from fastai) (2.23.0)
Requirement already satisfied, skipping upgrade: pandas in /usr/local/lib/python3.7/dist-packages (from fastai) (1.1.5)
Requirement already satisfied, skipping upgrade: packaging in /usr/local/lib/python3.7/dist-packages (from fastai) (20.9)
Requirement already satisfied, skipping upgrade: pyyaml in /usr/local/lib/python3.7/dist-packages (from fastai) (3.13)
Collecting torchvision<0.9,>=0.8
  Downloading https://files.pythonhosted.org/packages/94/df/969e69a94cff1c8911acb0688117f95e1915becc1e01c73e7960a2c76ec8/torchvision-0.8.2-cp37-cp37m-manylinux1_x86_64.whl (12.8MB)
     |████████████████████████████████| 12.8MB 246kB/s 
Requirement already satisfied, skipping upgrade: scikit-learn in /usr/local/lib/python3.7/dist-packages (from fastai) (0.22.2.post1)
Requirement already satisfied, skipping upgrade: spacy<3 in /usr/local/lib/python3.7/dist-packages (from fastai) (2.2.4)
Requirement already satisfied, skipping upgrade: pip in /usr/local/lib/python3.7/dist-packages (from fastai) (19.3.1)
Collecting torch<1.8,>=1.7.0
  Downloading https://files.pythonhosted.org/packages/90/5d/095ddddc91c8a769a68c791c019c5793f9c4456a688ddd235d6670924ecb/torch-1.7.1-cp37-cp37m-manylinux1_x86_64.whl (776.8MB)
     |████████████████████████████████| 776.8MB 23kB/s 
Requirement already satisfied, skipping upgrade: pillow>6.0.0 in /usr/local/lib/python3.7/dist-packages (from fastai) (7.1.2)
Requirement already satisfied, skipping upgrade: numpy in /usr/local/lib/python3.7/dist-packages (from fastprogress>=0.2.4->fastai) (1.19.5)
Requirement already satisfied, skipping upgrade: python-dateutil>=2.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->fastai) (2.8.1)
Requirement already satisfied, skipping upgrade: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib->fastai) (0.10.0)
Requirement already satisfied, skipping upgrade: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->fastai) (2.4.7)
Requirement already satisfied, skipping upgrade: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->fastai) (1.3.1)
Requirement already satisfied, skipping upgrade: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->fastai) (2.10)
Requirement already satisfied, skipping upgrade: chardet<4,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests->fastai) (3.0.4)
Requirement already satisfied, skipping upgrade: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->fastai) (1.24.3)
Requirement already satisfied, skipping upgrade: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->fastai) (2020.12.5)
Requirement already satisfied, skipping upgrade: pytz>=2017.2 in /usr/local/lib/python3.7/dist-packages (from pandas->fastai) (2018.9)
Requirement already satisfied, skipping upgrade: joblib>=0.11 in /usr/local/lib/python3.7/dist-packages (from scikit-learn->fastai) (1.0.1)
Requirement already satisfied, skipping upgrade: murmurhash<1.1.0,>=0.28.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3->fastai) (1.0.5)
Requirement already satisfied, skipping upgrade: blis<0.5.0,>=0.4.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3->fastai) (0.4.1)
Requirement already satisfied, skipping upgrade: srsly<1.1.0,>=1.0.2 in /usr/local/lib/python3.7/dist-packages (from spacy<3->fastai) (1.0.5)
Requirement already satisfied, skipping upgrade: plac<1.2.0,>=0.9.6 in /usr/local/lib/python3.7/dist-packages (from spacy<3->fastai) (1.1.3)
Requirement already satisfied, skipping upgrade: cymem<2.1.0,>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from spacy<3->fastai) (2.0.5)
Requirement already satisfied, skipping upgrade: tqdm<5.0.0,>=4.38.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3->fastai) (4.41.1)
Requirement already satisfied, skipping upgrade: catalogue<1.1.0,>=0.0.7 in /usr/local/lib/python3.7/dist-packages (from spacy<3->fastai) (1.0.0)
Requirement already satisfied, skipping upgrade: thinc==7.4.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3->fastai) (7.4.0)
Requirement already satisfied, skipping upgrade: wasabi<1.1.0,>=0.4.0 in /usr/local/lib/python3.7/dist-packages (from spacy<3->fastai) (0.8.2)
Requirement already satisfied, skipping upgrade: setuptools in /usr/local/lib/python3.7/dist-packages (from spacy<3->fastai) (54.2.0)
Requirement already satisfied, skipping upgrade: preshed<3.1.0,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from spacy<3->fastai) (3.0.5)
Requirement already satisfied, skipping upgrade: typing-extensions in /usr/local/lib/python3.7/dist-packages (from torch<1.8,>=1.7.0->fastai) (3.7.4.3)
Requirement already satisfied, skipping upgrade: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil>=2.1->matplotlib->fastai) (1.15.0)
Requirement already satisfied, skipping upgrade: importlib-metadata>=0.20; python_version < "3.8" in /usr/local/lib/python3.7/dist-packages (from catalogue<1.1.0,>=0.0.7->spacy<3->fastai) (3.10.1)
Requirement already satisfied, skipping upgrade: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata>=0.20; python_version < "3.8"->catalogue<1.1.0,>=0.0.7->spacy<3->fastai) (3.4.1)
ERROR: torchtext 0.9.1 has requirement torch==1.8.1, but you'll have torch 1.7.1 which is incompatible.
Installing collected packages: fastcore, torch, torchvision, fastai
  Found existing installation: torch 1.8.1+cu101
    Uninstalling torch-1.8.1+cu101:
      Successfully uninstalled torch-1.8.1+cu101
  Found existing installation: torchvision 0.9.1+cu101
    Uninstalling torchvision-0.9.1+cu101:
      Successfully uninstalled torchvision-0.9.1+cu101
  Found existing installation: fastai 1.0.61
    Uninstalling fastai-1.0.61:
      Successfully uninstalled fastai-1.0.61
Successfully installed fastai-2.3.0 fastcore-1.3.19 torch-1.7.1 torchvision-0.8.2

In [2]:

!pip install -U aicrowd-cli

Collecting aicrowd-cli
  Downloading https://files.pythonhosted.org/packages/29/18/2dcc043573e489f6134e4a76644f640874d3fa4d8f3e0593bf54a7c8b53a/aicrowd_cli-0.1.2-py3-none-any.whl (42kB)
     |████████████████████████████████| 51kB 5.0MB/s 
Requirement already satisfied, skipping upgrade: click<8,>=7.1.2 in /usr/local/lib/python3.7/dist-packages (from aicrowd-cli) (7.1.2)
Collecting requests<3,>=2.25.1
  Downloading https://files.pythonhosted.org/packages/29/c1/24814557f1d22c56d50280771a17307e6bf87b70727d975fd6b2ce6b014a/requests-2.25.1-py2.py3-none-any.whl (61kB)
     |████████████████████████████████| 61kB 4.7MB/s 
Collecting tqdm<5,>=4.56.0
  Downloading https://files.pythonhosted.org/packages/72/8a/34efae5cf9924328a8f34eeb2fdaae14c011462d9f0e3fcded48e1266d1c/tqdm-4.60.0-py2.py3-none-any.whl (75kB)
     |████████████████████████████████| 81kB 7.0MB/s 
Collecting requests-toolbelt<1,>=0.9.1
  Downloading https://files.pythonhosted.org/packages/60/ef/7681134338fc097acef8d9b2f8abe0458e4d87559c689a8c306d0957ece5/requests_toolbelt-0.9.1-py2.py3-none-any.whl (54kB)
     |████████████████████████████████| 61kB 8.0MB/s 
Requirement already satisfied, skipping upgrade: toml<1,>=0.10.2 in /usr/local/lib/python3.7/dist-packages (from aicrowd-cli) (0.10.2)
Collecting gitpython<4,>=3.1.12
  Downloading https://files.pythonhosted.org/packages/a6/99/98019716955ba243657daedd1de8f3a88ca1f5b75057c38e959db22fb87b/GitPython-3.1.14-py3-none-any.whl (159kB)
     |████████████████████████████████| 163kB 13.2MB/s 
Collecting rich<11,>=10.0.0
  Downloading https://files.pythonhosted.org/packages/1a/da/2a1f064dc620ab47f3f826ae085384084b71ea05c8c21d67f1dfc29189ab/rich-10.1.0-py3-none-any.whl (201kB)
     |████████████████████████████████| 204kB 15.2MB/s 
Requirement already satisfied, skipping upgrade: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests<3,>=2.25.1->aicrowd-cli) (1.24.3)
Requirement already satisfied, skipping upgrade: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests<3,>=2.25.1->aicrowd-cli) (2020.12.5)
Requirement already satisfied, skipping upgrade: idna<3,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests<3,>=2.25.1->aicrowd-cli) (2.10)
Requirement already satisfied, skipping upgrade: chardet<5,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from requests<3,>=2.25.1->aicrowd-cli) (3.0.4)
Collecting gitdb<5,>=4.0.1
  Downloading https://files.pythonhosted.org/packages/ea/e8/f414d1a4f0bbc668ed441f74f44c116d9816833a48bf81d22b697090dba8/gitdb-4.0.7-py3-none-any.whl (63kB)
     |████████████████████████████████| 71kB 9.8MB/s 
Collecting commonmark<0.10.0,>=0.9.0
  Downloading https://files.pythonhosted.org/packages/b1/92/dfd892312d822f36c55366118b95d914e5f16de11044a27cf10a7d71bbbf/commonmark-0.9.1-py2.py3-none-any.whl (51kB)
     |████████████████████████████████| 51kB 8.4MB/s 
Requirement already satisfied, skipping upgrade: pygments<3.0.0,>=2.6.0 in /usr/local/lib/python3.7/dist-packages (from rich<11,>=10.0.0->aicrowd-cli) (2.6.1)
Requirement already satisfied, skipping upgrade: typing-extensions<4.0.0,>=3.7.4 in /usr/local/lib/python3.7/dist-packages (from rich<11,>=10.0.0->aicrowd-cli) (3.7.4.3)
Collecting colorama<0.5.0,>=0.4.0
  Downloading https://files.pythonhosted.org/packages/44/98/5b86278fbbf250d239ae0ecb724f8572af1c91f4a11edf4d36a206189440/colorama-0.4.4-py2.py3-none-any.whl
Collecting smmap<5,>=3.0.1
  Downloading https://files.pythonhosted.org/packages/68/ee/d540eb5e5996eb81c26ceffac6ee49041d473bc5125f2aa995cf51ec1cf1/smmap-4.0.0-py2.py3-none-any.whl
ERROR: torchtext 0.9.1 has requirement torch==1.8.1, but you'll have torch 1.7.1 which is incompatible.
ERROR: google-colab 1.0.0 has requirement requests~=2.23.0, but you'll have requests 2.25.1 which is incompatible.
ERROR: datascience 0.10.6 has requirement folium==0.2.1, but you'll have folium 0.8.3 which is incompatible.
Installing collected packages: requests, tqdm, requests-toolbelt, smmap, gitdb, gitpython, commonmark, colorama, rich, aicrowd-cli
  Found existing installation: requests 2.23.0
    Uninstalling requests-2.23.0:
      Successfully uninstalled requests-2.23.0
  Found existing installation: tqdm 4.41.1
    Uninstalling tqdm-4.41.1:
      Successfully uninstalled tqdm-4.41.1
Successfully installed aicrowd-cli-0.1.2 colorama-0.4.4 commonmark-0.9.1 gitdb-4.0.7 gitpython-3.1.14 requests-2.25.1 requests-toolbelt-0.9.1 rich-10.1.0 smmap-4.0.0 tqdm-4.60.0

Download Data ⏬¶

The first step is to download out train test data. We will be training a model on the train data and make predictions on test data. We submit our predictions.

In [3]:

API_KEY = '92adadf1ec2b994477a09a6742890318' #Please enter your API Key from [https://www.aicrowd.com/participants/me]
!aicrowd login --api-key $API_KEY

API Key valid
Saved API Key successfully!

In [5]:

!aicrowd dataset download --challenge f1-speed-recognition

sample_submission.csv: 100% 97.8k/97.8k [00:00<00:00, 1.85MB/s]
test.zip: 100% 96.9M/96.9M [00:10<00:00, 9.01MB/s]
train.csv: 100% 407k/407k [00:00<00:00, 3.64MB/s]
train.zip: 100% 385M/385M [00:40<00:00, 9.46MB/s]
val.csv: 100% 36.7k/36.7k [00:00<00:00, 1.35MB/s]
val.zip: 100% 37.8M/37.8M [00:03<00:00, 10.1MB/s]

Below, we create a new directory to put our downloaded data! 🏎

We unzip the ZIP files and move the CSVs.

In [6]:

!rm -rf data
!mkdir data

!unzip -q train.zip  -d data/train
!unzip -q val.zip -d data/val
!unzip -q test.zip  -d data/test

!mv train.csv data/train.csv
!mv val.csv data/val.csv
!mv sample_submission.csv data/sample_submission.csv

Import packages¶

In [7]:

import pandas as pd
from fastai.vision.all import *
from fastai.data.core import *
import os

Load Data¶

We use pandas 🐼 library to load our data.
Pandas loads the data into dataframes and facilitates us to analyse the data.
Learn more about it here 🤓

In [8]:

data_folder = "data"

In [9]:

train_df = pd.read_csv(os.path.join(data_folder, "train.csv"))

Visualize the data 👀¶

Using Pandas and the Matplot Library in Python, we will be viewing the images in our datasets.

In [10]:

train_df

Out[10]:

	ImageID	label
0	0	1528
1	1	929
2	2	1504
3	3	938
4	4	1736
...	...	...
39995	39995	1019
39996	39996	1687
39997	39997	1545
39998	39998	1436
39999	39999	1211

40000 rows × 2 columns

Adding .jpg to all the ImageIDs in "ImageID" column. This will help us with adding the path behind the names of these images.

In [11]:

train_df['ImageID'] = train_df['ImageID'].astype(str)+".jpg"
train_df

Out[11]:

	ImageID	label
0	0.jpg	1528
1	1.jpg	929
2	2.jpg	1504
3	3.jpg	938
4	4.jpg	1736
...	...	...
39995	39995.jpg	1019
39996	39996.jpg	1687
39997	39997.jpg	1545
39998	39998.jpg	1436
39999	39999.jpg	1211

40000 rows × 2 columns

TRAINING PHASE 🏋️¶

Now that we have the dataset is ready, it's time to create a model that we will train on our data!

In [12]:

dls = ImageDataLoaders.from_df(train_df, path=os.path.join(data_folder, "train"), bs=8, y_block=RegressionBlock)
dls.show_batch()

In [13]:

learn = cnn_learner(dls, alexnet, metrics=mse)

Downloading: "https://download.pytorch.org/models/alexnet-owt-4df8aa71.pth" to /root/.cache/torch/hub/checkpoints/alexnet-owt-4df8aa71.pth

Train the Model 🏃🏽‍♂️¶

In [14]:

learn.fine_tune(1)

epoch	train_loss	valid_loss	mse	time
0	285670.531250	280625.312500	280625.312500	01:43

epoch	train_loss	valid_loss	mse	time
0	127313.781250	119547.000000	119547.000000	01:54

Testing Phase 😅¶

We are almost done. We trained and validated on the training data. Now its the time to predict on test set and make a submission.

Load Test Set¶

Load the test data on which final submission is to be made.

In [15]:

test_imgs_name = get_image_files(os.path.join(data_folder, "test"))
test_dls = dls.test_dl(test_imgs_name)

test_img_ids = [re.sub(r"\D", "", str(img_name)) for img_name in test_imgs_name]

In [16]:

test_dls.show_batch()

Predict Test Set¶

Predict on the test set and you are all set to make the submission!

In [17]:

_,_,results = learn.get_preds(dl = test_dls, with_decoded = True)

results = [i[0] for i in results.numpy()]

Save the prediction to csv¶

🚧 Note :¶

Do take a look at the submission format.
The submission file should contain a header.
Follow all submission guidelines strictly to avoid inconvenience.

In [18]:

submission = pd.DataFrame({"ImageID":test_img_ids, "label":results})
submission

Out[18]:

	ImageID	label
0	3392	715.618530
1	1568	1498.146484
2	2166	347.821594
3	231	731.563354
4	4698	1541.324707
...	...	...
9995	3074	1414.569092
9996	4917	849.963074
9997	6775	867.104553
9998	5917	950.143799
9999	700	1757.604614

10000 rows × 2 columns

In [19]:

submission.to_csv("submission.csv", index=False)

In [20]:

!aicrowd submission create -c f1-speed-recognition -f submission.csv

submission.csv ━━━━━━━━━━━━━━━━━━ 100.0% • 228.3/226.7 KB • 502.7 kB/s • 0:00:00
                                                   ╭─────────────────────────╮                                                    
                                                   │ Successfully submitted! │                                                    
                                                   ╰─────────────────────────╯                                                    
                                                         Important links                                                          
┌──────────────────┬─────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│  This submission │ https://www.aicrowd.com/challenges/ai-blitz-8/problems/f1-speed-recognition/submissions/130359              │
│                  │                                                                                                             │
│  All submissions │ https://www.aicrowd.com/challenges/ai-blitz-8/problems/f1-speed-recognition/submissions?my_submissions=true │
│                  │                                                                                                             │
│      Leaderboard │ https://www.aicrowd.com/challenges/ai-blitz-8/problems/f1-speed-recognition/leaderboards                    │
│                  │                                                                                                             │
│ Discussion forum │ https://discourse.aicrowd.com/c/ai-blitz-8                                                                  │
│                  │                                                                                                             │
│   Challenge page │ https://www.aicrowd.com/challenges/ai-blitz-8/problems/f1-speed-recognition                                 │
└──────────────────┴─────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
{'submission_id': 130359, 'created_at': '2021-04-16T18:35:33.779Z'}

In [ ]:

F1 SPEED RECOGNITION

[Baseline] F1 Speed Recognition