Added a script to fetch models and changed pipeline to use model/tokenizer loaded in from a local file w/o cacheing in

~/.cache in addition to a Dockerfile to build an image to run the question answering API.

Author: odellus

.dockerignore

@@ -0,0 +1 @@
+models

.gitignore

@@ -0,0 +1 @@
+models

Dockerfile

@@ -0,0 +1,33 @@
+# Pull an image of Ubuntu 20.04 as the base for this container.
+FROM ubuntu:20.04
+
+# Set /app to be the root directory of the container.
+WORKDIR /app
+
+# Let's go ahead and copy requirements.txt into container.
+COPY ./requirements.txt /app/requirements.txt
+
+# Now we install our dependencies. Don't need sudo. We're root.
+RUN apt-get update -y && \
+    apt-get install -y python3-pip python3-dev && \
+    pip3 install -r requirements.txt
+
+# Copy the rest of the code in after we've downloaded
+# and installed the dependencies. That way code changes
+# will not require reinstalling PyTorch for every typo.
+COPY . /app
+
+# This is the program that is run as an entrypoint to container
+# I often comment this part out when testing so I can run:
+#   docker run -it image_name:version_tag /bin/bash
+# and take a look around inside the container.
+ENTRYPOINT ["python3"]
+
+# We're going to execute python3 question_answering_api.py
+# like we are in the /app directory whenever we use this container.
+# This also needs to be commented out to run:
+#   docker run -it image_name:version_tag /bin/bash
+# for debugging purposes. Uncommenting and running:
+#    docker build -t image_name:version_tag .
+# will not take long at all unless you change requirements.txt.
+CMD ["question_answering_api.py"]

README.md

@@ -3,17 +3,11 @@
 Hello all! This is a little example of using :hugs: [huggingface transformers](https://github.com/huggingface/transformers) and [Flask-RESTful](https://flask-restful.readthedocs.io/en/latest/index.html) to create a  question answering API.
 
 ### Install
-1. The only requirements are [Git](https://www.digitalocean.com/community/tutorials/how-to-install-git-on-ubuntu-20-04) and [Python3](https://docs.python-guide.org/starting/install3/linux/) with [pip](https://pip.pypa.io/en/stable/installing/) installed in a Linux environment. If you are using Windows I recommend [installing Ubuntu for Windows](https://ubuntu.com/tutorials/ubuntu-on-windows). If you don't have pip installed, you can open a terminal and enter:
-    ```bash
-    curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
-    python3 get-pip.py
-    ```
-2. In the same or a new terminal enter:
-    ```bash
-    cd /path/to/question_answering_api # Where ever you forked it to. I don't know!
-    python3 -m pip install requirements.txt
-    ```
-    and that should install the requirements for the Question Answering API.
+The only real requirement is a Linux environment. If you are using Windows I recommend [installing Ubuntu for Windows](https://ubuntu.com/tutorials/ubuntu-on-windows). To install the needed software dependencies run:
+```bash
+cd /path/to/question_answering_api
+bash install_dependencies.sh
+```
 
 ### Usage
 1. #### Start the API server  
@@ -54,9 +48,43 @@ Context:
     largest and most biodiverse tract of tropical rainforest
     in the world, with an estimated 390 billion individual
     trees divided into 16,000 species.
-
 Question:
   Which name is also used to describe the Amazon rainforest in English?
 Answer:
   Amazonia.
-```
+```  
+
+### Docker
+
+To run the API inside a container you need to take the following steps:
+1. #### Install docker
+  Follow the instructions [here](https://docs.docker.com/engine/install/) to install docker on your system.
+2. #### Download the model and tokenizer
+  We don't want to put large machine learning models inside our containers if we don't have to, so we fetch the models from huggingface.co so we can mount them inside a volume for docker. Open a terminal and run:
+  ```bash
+  cd /path/to/question_answering_api
+  bash fetch_model.sh
+  ```
+  This will pull the model and save it to a directory we can mount as a volume for our container.
+3. #### Build the container
+  In the same or a new terminal, run:
+  ```bash
+  cd /path/to/question_answering_api # Optional if you're in repo root already.
+  # Build container and name image qa-api with version tag v1.
+  docker build -t qa-api:v1
+  ```
+4. #### Start the container
+  In the same terminal, type in:
+  ```bash
+  docker run \
+    -p 5000:5000 \ # Map port 5000 in container to port localhost:5000
+    -v /path/to/question_answering_api/models:/app/models \ # Use abspath!
+    qa-api:v1
+    ```
+5. #### Run the client
+  In a new terminal window (just like before, we need two open), run the following:
+  ```bash
+  # Make sure you're in repo root!
+  python3 question_answering_api.py
+  ```
+And that's it! If you want to host your container in the cloud now it's as easy as saying `docker push`.

install_dependencies.sh

@@ -0,0 +1,23 @@
+#! /bin/bash
+
+echo "Installing software dependencies"
+echo "Requires sudo..."
+sudo apt-get update -y
+sudo apt-get install -y \
+  git-lfs \
+  python3-pip \
+  python3-dev
+pip3 install -r requirements.txt
+
+echo "Fetching huggginface question answering model..."
+mkdir models
+cd models
+
+echo "It complains that git lfs clone is the same as git clone"
+echo "but it isn't"
+MODEL_NAME=distilbert-base-cased-distilled-squad
+git lfs clone https://huggingface.co/$MODEL_NAME
+echo "Model and tokenizer have been downloaded to models/${MODEL_NAME}!"
+echo "Have a great day!"
+# Move back into the repo root
+cd -

question_answering_api.py

@@ -2,7 +2,11 @@
 import time
 from flask import Flask, request
 from flask_restful import Resource, Api
-from transformers import pipeline
+from transformers import (
+    pipeline,
+    DistilBertTokenizerFast,
+    DistilBertForQuestionAnswering
+    )
 
 # Initialize API.
 app = Flask(__name__)
@@ -14,8 +18,13 @@ def load_model():
     Create a transformers pipeline for question answering inference.
     '''
     print(' * Loading model...')
+    model_dir = 'models'
+    model_name = 'distilbert-base-cased-distilled-squad'
+    model_path = f'./{model_dir}/{model_name}'
     start = time.time()
-    nlp = pipeline('question-answering', model='distilbert-base-cased-distilled-squad')
+    tokenizer = DistilBertTokenizerFast.from_pretrained(model_path)
+    model = DistilBertForQuestionAnswering.from_pretrained(model_path)
+    nlp = pipeline('question-answering', model=model, tokenizer=tokenizer)
     print(f' * Model loaded in {time.time()-start} seconds!')
     return nlp
 

requirements.txt

@@ -1,3 +1,4 @@
+numpy
 Flask
 Flask-RESTful
 transformers