This repository contains the code that acheived 39th place in Google AI4Code – Understand Code in Python Notebooks.

• numpy
• omegaconf
• pandas
• pytorch_lightning
• scikit_learn
• torch
• tqdm
• fasttext
• sentencepiece
• transformers
• wandb

Instead of installing the above modules independently, you can simply do at once by using:

$ pip install -f requirements.txt -f https://download.pytorch.org/whl/torch_stable.html

This repository supports NVIDIA Apex. It will automatically detect the apex module and if it is found then some training procedures will be replaced with the highly-optimized and fused operations in the apex module. Run the below codes in the terminal to install apex and enable performance boosting:

$ git clone https://github.com/NVIDIA/apex
$ sed -i "s/or (bare_metal_minor != torch_binary_minor)//g" apex/setup.py
$ pip install -v --disable-pip-version-check --no-cache-dir --global-option="--cpp_ext" --global-option="--cuda_ext" apex/
$ rm -rf apex

Instead, we recommend to use docker and PyTorch NGC Container where apex, optimized cuda driver and faster pytorch kernel are installed:

$ docker run --gpus all -it nvcr.io/nvidia/pytorch:22.07-py3