[Getting Started Notebook] PKHND Challange

This is a Baseline Code to get you started with the challenge.


You can use this code to start understanding the data and create a baseline model for further improvements.

Baseline for PKHND Challenge on AIcrowd

Author : Gauransh Kumar, Ayush Shivani

Download Necessary Packages

In [1]:
import sys
!{sys.executable} -m pip install numpy
!{sys.executable} -m pip install pandas
!{sys.executable} -m pip install scikit-learn
!{sys.executable} -m pip install aicrowd-cli
%load_ext aicrowd.magic
Requirement already satisfied: numpy in /home/gauransh/anaconda3/lib/python3.8/site-packages (1.20.3)
Requirement already satisfied: pandas in /home/gauransh/anaconda3/lib/python3.8/site-packages (1.3.2)
Requirement already satisfied: numpy>=1.17.3 in /home/gauransh/anaconda3/lib/python3.8/site-packages (from pandas) (1.20.3)
Requirement already satisfied: python-dateutil>=2.7.3 in /home/gauransh/anaconda3/lib/python3.8/site-packages (from pandas) (2.8.2)
Requirement already satisfied: pytz>=2017.3 in /home/gauransh/anaconda3/lib/python3.8/site-packages (from pandas) (2021.1)
Requirement already satisfied: six>=1.5 in /home/gauransh/anaconda3/lib/python3.8/site-packages (from python-dateutil>=2.7.3->pandas) (1.15.0)
Requirement already satisfied: scikit-learn in /home/gauransh/anaconda3/lib/python3.8/site-packages (0.24.2)
Requirement already satisfied: numpy>=1.13.3 in /home/gauransh/anaconda3/lib/python3.8/site-packages (from scikit-learn) (1.20.3)
Requirement already satisfied: threadpoolctl>=2.0.0 in /home/gauransh/anaconda3/lib/python3.8/site-packages (from scikit-learn) (2.2.0)
Requirement already satisfied: joblib>=0.11 in /home/gauransh/anaconda3/lib/python3.8/site-packages (from scikit-learn) (1.0.1)
Requirement already satisfied: scipy>=0.19.1 in /home/gauransh/anaconda3/lib/python3.8/site-packages (from scikit-learn) (1.6.2)
Requirement already satisfied: aicrowd-cli in /home/gauransh/anaconda3/lib/python3.8/site-packages (0.1.10)
Requirement already satisfied: tqdm<5,>=4.56.0 in /home/gauransh/anaconda3/lib/python3.8/site-packages (from aicrowd-cli) (4.60.0)
Requirement already satisfied: requests-toolbelt<1,>=0.9.1 in /home/gauransh/anaconda3/lib/python3.8/site-packages (from aicrowd-cli) (0.9.1)
Requirement already satisfied: rich<11,>=10.0.0 in /home/gauransh/anaconda3/lib/python3.8/site-packages (from aicrowd-cli) (10.15.2)
Requirement already satisfied: toml<1,>=0.10.2 in /home/gauransh/anaconda3/lib/python3.8/site-packages (from aicrowd-cli) (0.10.2)
Collecting pyzmq==22.1.0
  Downloading pyzmq-22.1.0-cp38-cp38-manylinux2010_x86_64.whl (1.1 MB)
     |████████████████████████████████| 1.1 MB 1.5 MB/s eta 0:00:01
Requirement already satisfied: requests<3,>=2.25.1 in /home/gauransh/anaconda3/lib/python3.8/site-packages (from aicrowd-cli) (2.26.0)
Requirement already satisfied: GitPython==3.1.18 in /home/gauransh/anaconda3/lib/python3.8/site-packages (from aicrowd-cli) (3.1.18)
Requirement already satisfied: click<8,>=7.1.2 in /home/gauransh/anaconda3/lib/python3.8/site-packages (from aicrowd-cli) (7.1.2)
Requirement already satisfied: gitdb<5,>=4.0.1 in /home/gauransh/anaconda3/lib/python3.8/site-packages (from GitPython==3.1.18->aicrowd-cli) (4.0.9)
Requirement already satisfied: smmap<6,>=3.0.1 in /home/gauransh/anaconda3/lib/python3.8/site-packages (from gitdb<5,>=4.0.1->GitPython==3.1.18->aicrowd-cli) (5.0.0)
Requirement already satisfied: charset-normalizer~=2.0.0 in /home/gauransh/anaconda3/lib/python3.8/site-packages (from requests<3,>=2.25.1->aicrowd-cli) (2.0.0)
Requirement already satisfied: idna<4,>=2.5 in /home/gauransh/anaconda3/lib/python3.8/site-packages (from requests<3,>=2.25.1->aicrowd-cli) (3.1)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/gauransh/anaconda3/lib/python3.8/site-packages (from requests<3,>=2.25.1->aicrowd-cli) (1.26.6)
Requirement already satisfied: certifi>=2017.4.17 in /home/gauransh/anaconda3/lib/python3.8/site-packages (from requests<3,>=2.25.1->aicrowd-cli) (2020.6.20)
Requirement already satisfied: colorama<0.5.0,>=0.4.0 in /home/gauransh/anaconda3/lib/python3.8/site-packages (from rich<11,>=10.0.0->aicrowd-cli) (0.4.4)
Requirement already satisfied: pygments<3.0.0,>=2.6.0 in /home/gauransh/anaconda3/lib/python3.8/site-packages (from rich<11,>=10.0.0->aicrowd-cli) (2.10.0)
Requirement already satisfied: commonmark<0.10.0,>=0.9.0 in /home/gauransh/anaconda3/lib/python3.8/site-packages (from rich<11,>=10.0.0->aicrowd-cli) (0.9.1)
Installing collected packages: pyzmq
  Attempting uninstall: pyzmq
    Found existing installation: pyzmq 22.0.3
    Uninstalling pyzmq-22.0.3:
      Successfully uninstalled pyzmq-22.0.3
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
aicrowd-gym-internal 0.0.4 requires pyzmq==22.0.3, but you have pyzmq 22.1.0 which is incompatible.
Successfully installed pyzmq-22.1.0

Download data

The first step is to download out train test data. We will be training a classifier on the train data and make predictions on test data. We submit our predictions

In [3]:
!rm -rf data
!mkdir data
%aicrowd ds dl -c pkhnd -o data
In [11]:
!unzip data/train.zip -d data/
Archive:  data/train.zip
  inflating: data/train.csv          

Import packages

In [1]:
import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.svm import SVC
from sklearn.metrics import f1_score,precision_score,recall_score,accuracy_score

Load Data

We use pandas library to load our data. Pandas loads them into dataframes which helps us analyze our data easily. Learn more about it here

In [2]:
train_data_path = "data/train.csv" #path where data is stored
In [3]:
train_data = pd.read_csv(train_data_path) #load data in dataframe using pandas

Visualize data

In [4]:
S1 C1 S2 C2 S3 C3 S4 C4 S5 C5 label
0 1 1 1 13 2 4 2 3 1 12 0
1 3 12 3 2 3 11 4 5 2 5 1
2 1 9 4 6 1 4 3 2 3 9 1
3 1 4 3 13 2 13 2 1 3 6 1
4 3 10 2 7 1 2 2 11 4 9 0

We can see there are 11 column where first 10 column contains the cards information and the last one describing the hand it makes. 1st and 2nd column contains suit and rank of first card respectively, 3rd and 4th column suit and rank of 2nd card and so on.

Split Data into Train and Validation

Now we want to see how well our classifier is performing, but we dont have the test data labels with us to check. What do we do ? So we split our dataset into train and validation. The idea is that we test our classifier on validation set in order to get an idea of how well our classifier works. This way we can also ensure that we dont overfit on the train dataset. There are many ways to do validation like k-fold,leave one out, etc

In [5]:
X_train, X_val= train_test_split(train_data, test_size=0.2, random_state=42)

Here we have selected the size of the testing data to be 20% of the total data. You can change it and see what effect it has on the accuracies. To learn more about the train_test_split function click here.

Now, since we have our data splitted into train and validation sets, we need to get the label separated from the data.

In [6]:
X_train,y_train = X_train.iloc[:,:-1],X_train.iloc[:,-1]
X_val,y_val = X_val.iloc[:,:-1],X_val.iloc[:,-1]

Define the Classifier

Now we come to the juicy part. We have fixed our data and now we train a classifier. The classifier will learn the function by looking at the inputs and corresponding outputs. There are a ton of classifiers to choose from some being Logistic Regression, SVM, Random Forests, Decision Trees, etc.
Tip: A good model doesnt depend solely on the classifier but on the features(columns) you choose. So make sure to play with your data and keep only whats important.

In [7]:
classifier = SVC(gamma='auto',max_iter=10)

#from sklearn.linear_model import LogisticRegression
# classifier = LogisticRegression()

We have used Support Vector Machines as a classifier here and set few of the parameteres. But one can set more parameters and increase the performance. To see the list of parameters visit here.

We can also use other classifiers. To read more about sklean classifiers visit here. Try and use other classifiers to see how the performance of your model changes. Try using Logistic Regression or MLP and compare how the performance changes.

Train the classifier

In [8]:
classifier.fit(X_train, y_train)
/home/gauransh/anaconda3/lib/python3.8/site-packages/sklearn/svm/_base.py:255: ConvergenceWarning: Solver terminated early (max_iter=10).  Consider pre-processing your data with StandardScaler or MinMaxScaler.
  warnings.warn('Solver terminated early (max_iter=%i).'
SVC(gamma='auto', max_iter=10)

Got a warning! Dont worry, its just beacuse the number of iteration is very less(defined in the classifier in the above cell).Increase the number of iterations and see if the warning vanishes and also see how the performance changes.Do remember increasing iterations also increases the running time.( Hint: max_iter=500)

Predict on Validation

Now we predict our trained classifier on the validation set and evaluate our model

In [9]:
y_pred = classifier.predict(X_val)

Evaluate the Performance

We use the same metrics as that will be used for the test set.
F1 score are the metrics for this challenge

In [10]:
precision = precision_score(y_val,y_pred,average='micro')
recall = recall_score(y_val,y_pred,average='micro')
accuracy = accuracy_score(y_val,y_pred)
f1 = f1_score(y_val,y_pred,average='macro')
In [11]:
print("Accuracy of the model is :" ,accuracy)
print("Recall of the model is :" ,recall)
print("Precision of the model is :" ,precision)
print("F1 score of the model is :" ,f1)
Accuracy of the model is : 0.49894
Recall of the model is : 0.49894
Precision of the model is : 0.49894
F1 score of the model is : 0.07442511649369188

Prediction on Evaluation Set

Load Test Set

Load the test data now

In [12]:
final_test_path = "data/test.csv"
final_test = pd.read_csv(final_test_path)

Predict Test Set

Time for the moment of truth! Predict on test set and time to make the submission.

In [13]:
submission = classifier.predict(final_test)

Save the prediction to csv

In [14]:
# Saving the pandas dataframe
!rm -rf assets
!mkdir assets
submission = pd.DataFrame(submission)

Note: Do take a look at the submission format.The submission file should contain a header.For eg here it is "label".

Make a submission using the aicrwd -cli

In [15]:
!!aicrowd submission create -c pkhnd -f assets/submission.csv
['submission.csv ━━━━━━━━━━━━━━━━━━━━━ 100.0% • 51.7/50.0 KB • 11.0 MB/s • 0:00:00',
 '                                                  ╭─────────────────────────╮                                                  ',
 '                                                  │ Successfully submitted! │                                                  ',
 '                                                  ╰─────────────────────────╯                                                  ',
 '                                                        Important links                                                        ',
 '│  This submission │ https://www.aicrowd.com/challenges/aicrowd-blitz-may-2020/problems/pkhnd/submissions/174811              │',
 '│                  │                                                                                                          │',
 '│  All submissions │ https://www.aicrowd.com/challenges/aicrowd-blitz-may-2020/problems/pkhnd/submissions?my_submissions=true │',
 '│                  │                                                                                                          │',
 '│      Leaderboard │ https://www.aicrowd.com/challenges/aicrowd-blitz-may-2020/problems/pkhnd/leaderboards                    │',
 '│                  │                                                                                                          │',
 '│ Discussion forum │ https://discourse.aicrowd.com/c/aicrowd-blitz-may-2020                                                   │',
 '│                  │                                                                                                          │',
 '│   Challenge page │ https://www.aicrowd.com/challenges/aicrowd-blitz-may-2020/problems/pkhnd                                 │',
 "{'submission_id': 174811, 'created_at': '2022-02-21T05:45:14.340Z'}"]


You must login before you can post a comment.