Skip to content

Commit 608ca25

Browse files
committed
ViLT
ViLT
1 parent 0c0aa4c commit 608ca25

8 files changed

+35840
-0
lines changed

.DS_Store

2 KB
Binary file not shown.

deep-learning/.DS_Store

0 Bytes
Binary file not shown.

deep-learning/Transformer-Tutorials/ViLT/Fine_tuning_ViLT_for_VQA.ipynb

+22,864
Large diffs are not rendered by default.

deep-learning/Transformer-Tutorials/ViLT/Inference_with_ViLT_(visual_question_answering).ipynb

+1,051
Large diffs are not rendered by default.

deep-learning/Transformer-Tutorials/ViLT/Masked_language_modeling_with_ViLT.ipynb

+3,196
Large diffs are not rendered by default.
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
# ViLT notebooks
2+
In this directory, you can find several notebooks that illustrate how to use NAVER AI Lab's [ViLT](https://arxiv.org/abs/2102.03334) both for fine-tuning on custom data as well as inference. It currently includes the following notebooks:
3+
4+
- fine-tuning ViLT for visual question answering (VQA) (based on the [VQAv2 dataset](https://visualqa.org/))
5+
- performing inference with ViLT to illustrate visual question answering (VQA)
6+
- masked language modeling (MLM) with a pre-trained ViLT model
7+
- performing inference with ViLT for image-text retrieval
8+
- performing inference with ViLT to illustrate natural language for visual reasoning (based on the [NLVRv2 dataset](https://lil.nlp.cornell.edu/nlvr/)).
9+
10+
All models can be found on the [hub](https://huggingface.co/models?search=vilt).

0 commit comments

Comments
 (0)