[Getting Started Notebook] DIBRD Challange

This is a Baseline Code to get youย started with the challenge.


You can use this code to start understanding the data and create a baseline model for further imporvments.


Baseline for DIBRD Challenge on AIcrowd

Author : Gauransh Kumar, Shubham Sharma

Download Necessary Packages

In [1]:
import sys
!{sys.executable} -m pip install numpy
!{sys.executable} -m pip install pandas
!{sys.executable} -m pip install scikit-learn
!{sys.executable} -m pip install aicrowd-cli
%load_ext aicrowd.magic
Requirement already satisfied: numpy in /home/gauransh/anaconda3/lib/python3.8/site-packages (1.20.3)
Requirement already satisfied: pandas in /home/gauransh/anaconda3/lib/python3.8/site-packages (1.3.2)
Requirement already satisfied: numpy>=1.17.3 in /home/gauransh/anaconda3/lib/python3.8/site-packages (from pandas) (1.20.3)
Requirement already satisfied: python-dateutil>=2.7.3 in /home/gauransh/anaconda3/lib/python3.8/site-packages (from pandas) (2.8.2)
Requirement already satisfied: pytz>=2017.3 in /home/gauransh/anaconda3/lib/python3.8/site-packages (from pandas) (2021.1)
Requirement already satisfied: six>=1.5 in /home/gauransh/anaconda3/lib/python3.8/site-packages (from python-dateutil>=2.7.3->pandas) (1.15.0)
Requirement already satisfied: scikit-learn in /home/gauransh/anaconda3/lib/python3.8/site-packages (0.24.2)
Requirement already satisfied: numpy>=1.13.3 in /home/gauransh/anaconda3/lib/python3.8/site-packages (from scikit-learn) (1.20.3)
Requirement already satisfied: threadpoolctl>=2.0.0 in /home/gauransh/anaconda3/lib/python3.8/site-packages (from scikit-learn) (2.2.0)
Requirement already satisfied: scipy>=0.19.1 in /home/gauransh/anaconda3/lib/python3.8/site-packages (from scikit-learn) (1.6.2)
Requirement already satisfied: joblib>=0.11 in /home/gauransh/anaconda3/lib/python3.8/site-packages (from scikit-learn) (1.0.1)
Requirement already satisfied: aicrowd-cli in /home/gauransh/anaconda3/lib/python3.8/site-packages (0.1.10)
Requirement already satisfied: toml<1,>=0.10.2 in /home/gauransh/anaconda3/lib/python3.8/site-packages (from aicrowd-cli) (0.10.2)
Requirement already satisfied: requests<3,>=2.25.1 in /home/gauransh/anaconda3/lib/python3.8/site-packages (from aicrowd-cli) (2.26.0)
Requirement already satisfied: requests-toolbelt<1,>=0.9.1 in /home/gauransh/anaconda3/lib/python3.8/site-packages (from aicrowd-cli) (0.9.1)
Requirement already satisfied: GitPython==3.1.18 in /home/gauransh/anaconda3/lib/python3.8/site-packages (from aicrowd-cli) (3.1.18)
Requirement already satisfied: click<8,>=7.1.2 in /home/gauransh/anaconda3/lib/python3.8/site-packages (from aicrowd-cli) (7.1.2)
Requirement already satisfied: pyzmq==22.1.0 in /home/gauransh/anaconda3/lib/python3.8/site-packages (from aicrowd-cli) (22.1.0)
Requirement already satisfied: tqdm<5,>=4.56.0 in /home/gauransh/anaconda3/lib/python3.8/site-packages (from aicrowd-cli) (4.60.0)
Requirement already satisfied: rich<11,>=10.0.0 in /home/gauransh/anaconda3/lib/python3.8/site-packages (from aicrowd-cli) (10.15.2)
Requirement already satisfied: gitdb<5,>=4.0.1 in /home/gauransh/anaconda3/lib/python3.8/site-packages (from GitPython==3.1.18->aicrowd-cli) (4.0.9)
Requirement already satisfied: smmap<6,>=3.0.1 in /home/gauransh/anaconda3/lib/python3.8/site-packages (from gitdb<5,>=4.0.1->GitPython==3.1.18->aicrowd-cli) (5.0.0)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/gauransh/anaconda3/lib/python3.8/site-packages (from requests<3,>=2.25.1->aicrowd-cli) (1.26.6)
Requirement already satisfied: charset-normalizer~=2.0.0 in /home/gauransh/anaconda3/lib/python3.8/site-packages (from requests<3,>=2.25.1->aicrowd-cli) (2.0.0)
Requirement already satisfied: certifi>=2017.4.17 in /home/gauransh/anaconda3/lib/python3.8/site-packages (from requests<3,>=2.25.1->aicrowd-cli) (2020.6.20)
Requirement already satisfied: idna<4,>=2.5 in /home/gauransh/anaconda3/lib/python3.8/site-packages (from requests<3,>=2.25.1->aicrowd-cli) (3.1)
Requirement already satisfied: commonmark<0.10.0,>=0.9.0 in /home/gauransh/anaconda3/lib/python3.8/site-packages (from rich<11,>=10.0.0->aicrowd-cli) (0.9.1)
Requirement already satisfied: pygments<3.0.0,>=2.6.0 in /home/gauransh/anaconda3/lib/python3.8/site-packages (from rich<11,>=10.0.0->aicrowd-cli) (2.10.0)
Requirement already satisfied: colorama<0.5.0,>=0.4.0 in /home/gauransh/anaconda3/lib/python3.8/site-packages (from rich<11,>=10.0.0->aicrowd-cli) (0.4.4)

Download dataset

The first step is to download out train test data. We will be training a classifier on the train data and make predictions on test data. We submit our predictions

In [2]:
!rm -rf data
!mkdir data
%aicrowd ds dl -c dibrd -o data

Import packages

In [3]:
import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.svm import SVC
from sklearn.metrics import f1_score,precision_score,recall_score,accuracy_score

Load Data

  • We use pandas ๐Ÿผ library to load our data.
  • Pandas loads the data into dataframes and facilitates us to analyse the data.
  • Learn more about it here ๐Ÿค“
In [4]:
train_data_path = "data/train.csv" #path where data is stored
In [5]:
train_data = pd.read_csv(train_data_path,header=None) #load data in dataframe using pandas

Visualise the Dataset

In [6]:
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
0 1 1 75 63 60 55 48 35 13.195493 4.396967 0.104070 0.000000 0.000000 0.000000 0.000000 0.000000 0.513092 0.123966 0 1
1 1 1 79 76 74 72 69 50 61.559348 28.959444 12.778104 2.045287 0.038016 0.000000 0.000000 0.000000 0.527993 0.101884 0 1
2 1 1 41 41 40 40 38 35 6.090116 0.834492 0.027460 0.000000 0.000000 0.000000 0.000000 0.000000 0.506881 0.091535 1 0
3 1 1 17 16 16 14 12 9 75.438535 20.352500 5.237412 0.206817 0.003884 0.000971 0.000971 0.000971 0.544614 0.089329 1 1
4 1 1 63 63 63 59 57 48 13.558211 5.366467 0.604079 0.051511 0.000000 0.000000 0.000000 0.000000 0.552941 0.112387 0 1

You can see the columns goes from 0 to 19, where columns from 0 to 19 represents features extracted from the image set and last column represents the type of patient i.e 1 if if signs of Diabetic Retinopathy is present else 0.

Split Data into Train and Validation 🔪

  • The next step is to think of a way to test how well our model is performing. we cannot use the test data given as it does not contain the data labels for us to verify.
  • The workaround this is to split the given training data into training and validation. Typically validation sets give us an idea of how our model will perform on unforeseen data. it is like holding back a chunk of data while training our model and then using it to for the purpose of testing. it is a standard way to fine-tune hyperparameters in a model.
  • There are multiple ways to split a dataset into validation and training sets. following are two popular ways to go about it, k-fold, leave one out. ๐Ÿง
  • Validation sets are also used to avoid your model from overfitting on the train dataset.
In [7]:
X_train, X_val= train_test_split(train_data, test_size=0.2, random_state=42)
  • We have decided to split the data with 20 % as validation and 80 % as training.
  • To learn more about the train_test_split function click here. ๐Ÿง
  • This is of course the simplest way to validate your model by simply taking a random chunk of the train set and setting it aside solely for the purpose of testing our train model on unseen data. as mentioned in the previous block, you can experiment ๐Ÿ”ฌ with and choose more sophisticated techniques and make your model better.
  • Now, since we have our data splitted into train and validation sets, we need to get the corresponding labels separated from the data.
  • with this step we are all set move to the next step with a prepared dataset.
In [8]:
X_train,y_train = X_train.iloc[:,:-1],X_train.iloc[:,-1]
X_val,y_val = X_val.iloc[:,:-1],X_val.iloc[:,-1]


Define the Model

  • We have fixed our data and now we are ready to train our model.

  • There are a ton of classifiers to choose from some being Logistic Regression, SVM, Random Forests, Decision Trees, etc.๐Ÿง

  • Remember that there are no hard-laid rules here. you can mix and match classifiers, it is advisable to read up on the numerous techniques and choose the best fit for your solution , experimentation is the key.

  • A good model does not depend solely on the classifier but also on the features you choose. So make sure to analyse and understand your data well and move forward with a clear view of the problem at hand. you can gain important insight from here.๐Ÿง

In [9]:
classifier = LogisticRegression(solver = 'lbfgs',multi_class='auto',max_iter=10)

We have used Logistic Regression as a classifier here and set few of the parameteres. But one can set more parameters and increase the performance. To see the list of parameters visit here.

To read more about other sklearn classifiers visit here ๐Ÿง. Try and use other classifiers to see how the performance of your model changes. Try using Logistic Regression or MLP and compare how the performance changes.

Train the classifier

In [10]:
classifier.fit(X_train, y_train)
/home/gauransh/anaconda3/lib/python3.8/site-packages/sklearn/linear_model/_logistic.py:763: ConvergenceWarning: lbfgs failed to converge (status=1):

Increase the number of iterations (max_iter) or scale the data as shown in:
Please also refer to the documentation for alternative solver options:
  n_iter_i = _check_optimize_result(

Got a warning! Dont worry, its just beacuse the number of iteration is very less(defined in the classifier in the above cell).Increase the number of iterations and see if the warning vanishes and also see how the performance changes.Do remember increasing iterations also increases the running time.( Hint: max_iter=500)

Validation Phase 🤔

Wonder how well your model learned! Lets check it.

Predict on Validation

Now we predict using our trained model on the validation set we created and evaluate our model on unforeseen data.

In [11]:
y_pred = classifier.predict(X_val)

Evaluate the Performance

  • We have used basic metrics to quantify the performance of our model.
  • This is a crucial step, you should reason out the metrics and take hints to improve aspects of your model.
  • Do read up on the meaning and use of different metrics. there exist more metrics and measures, you should learn to use them correctly with respect to the solution,dataset and other factors.
  • F1 score is the metric for this challenge
In [12]:
precision = precision_score(y_val,y_pred,average='micro')
recall = recall_score(y_val,y_pred,average='micro')
accuracy = accuracy_score(y_val,y_pred)
f1 = f1_score(y_val,y_pred,average='macro')
In [13]:
print("Accuracy of the model is :" ,accuracy)
print("Recall of the model is :" ,recall)
print("Precision of the model is :" ,precision)
print("F1 score of the model is :" ,f1)
Accuracy of the model is : 0.6630434782608695
Recall of the model is : 0.6630434782608695
Precision of the model is : 0.6630434782608695
F1 score of the model is : 0.659014825442372

Testing Phase 😅

We are almost done. We trained and validated on the training data. Now its the time to predict on test set and make a submission.

Load Test Set

Load the test data now

In [14]:
final_test_path = "data/test.csv"
final_test = pd.read_csv(final_test_path,header=None)

Predict Test Set

Time for the moment of truth! Predict on test set and time to make the submission.

In [15]:
submission = classifier.predict(final_test)

Save the prediction to csv

In [16]:
# Saving the pandas dataframe
!rm -rf assets
!mkdir assets
submission = pd.DataFrame(submission)

๐Ÿšง Note :

  • Do take a look at the submission format.
  • The submission file should contain a header.
  • Follow all submission guidelines strictly to avoid inconvenience.## Save the prediction to csv

Make a submission using the aicrwd -cli

In [17]:
!aicrowd submission create -c dibrd -f assets/submission.csv
submission.csv โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ”โ” 100.0% โ€ข 2,111/466 bytes โ€ข ? โ€ข 0:00:00
                                                  โ”‚ Successfully submitted! โ”‚                                                  
                                                        Important links                                                        
โ”‚  This submission โ”‚ https://www.aicrowd.com/challenges/aicrowd-blitz-may-2020/problems/dibrd/submissions/174944              โ”‚
โ”‚                  โ”‚                                                                                                          โ”‚
โ”‚  All submissions โ”‚ https://www.aicrowd.com/challenges/aicrowd-blitz-may-2020/problems/dibrd/submissions?my_submissions=true โ”‚
โ”‚                  โ”‚                                                                                                          โ”‚
โ”‚      Leaderboard โ”‚ https://www.aicrowd.com/challenges/aicrowd-blitz-may-2020/problems/dibrd/leaderboards                    โ”‚
โ”‚                  โ”‚                                                                                                          โ”‚
โ”‚ Discussion forum โ”‚ https://discourse.aicrowd.com/c/aicrowd-blitz-may-2020                                                   โ”‚
โ”‚                  โ”‚                                                                                                          โ”‚
โ”‚   Challenge page โ”‚ https://www.aicrowd.com/challenges/aicrowd-blitz-may-2020/problems/dibrd                                 โ”‚
{'submission_id': 174944, 'created_at': '2022-02-21T18:19:02.619Z'}


You must login before you can post a comment.