What dog breed is that? This AI can tell you

step by step, how to build the neural network using fast.ai and Paperspace

Nov 04, 2020

This is the first edition of @aihelps, where we use artificial intelligence to build something useful for humans. Today we're looking at the story of Don Padre.

Don Padre has been breeding dogs for over 40 years. He can look at any dog and tell you whether it's a labrador, terrier, or poodle. But he wants to retire so we're going to build a machine learning application to help his colleagues at work when he leaves. You’d think that knowing every dog breed and being able to recognize them all from just a photo would be a difficult task, but with modern neural networks, it’s surprisingly easy to build.

In about 10 minutes we can review the code and understand everything we need to build an app that will take a picture of a dog and tell you the breed. So let's get started!

Follow along with the code on GitHub: dog_breeds.ipynb

Follow along with the demonstration on YouTube: What dog breed is that? This AI can tell you

Getting started with Jupyter notebooks on Paperspace

Code on GitHub: dog_breeds.ipynb

Let’s walk through all the code that you need to build and run the machine learning algorithm. The language is Python and it runs in a Jupyter Notebook - a UI for documenting and visualizing data while running Python scripts.

To run this code on your own, you’ll need a Linux server with dedicated GPU for training a neural network. Fortunately, our friends at Paperspace are providing hosted Jupyter notebooks on GPU cloud servers free for fast.ai users - that’s us!

Get $10 credit as a new Paperspace user: console.paperspace.com/signup?R=FN9VJ58

Once inside, choose Gradient. Gradient hosts Jupyter notebooks and even has some free GPU options and increasingly powerful ones for hourly cost.

When creating a new notebook, first choose fast.ai container and you will find below some options GPU’s. Look for the free P5000, leave the other default settings and scroll down to click “Create Notebook”

It will take a few minutes for your notebook to spin up and become ready. When you are able to open it, you will see something like this:

From this screen, create a new Python 3 notebook from the top-right menu
and we'll start scripting!

from pathlib import Path
import re
from fastai.vision.all import *
from fastai.callback.fp16 import *

This imports the libraries and tools we need

!curl -o ./images.tar "http://vision.stanford.edu/aditya86/ImageNetDogs/images.tar"
!tar -xf images.tar
dataset_path = Path('Images')

This downloads the 756MB of dog photos from the Stanford University database of images. Your already on a cloud server it will download quickly.

Next, you can copy the functions and class below. Most of the code here is for parsing name of each breed from the image files. Depending on your familiarity with regex, you can review it.

BREED_NAMES = {}

def get_breed_from_filename(filename):
    return BREED_NAMES[re.findall('(n\d+).+.jpg$', str(filename))[0]]


class DogBreedsNN(ABC):
    RANDOM_SEED = 42  # random.randint(1,100)
    AI_MODEL_FILENAME = 'model_export.pkl'
    SHOW_FIGURES = True
    DATASET_PATH = None

    def parse_breed_name(self, folder_name):
        return re.findall('.+/n\d+-([\w-]+)$', folder_name)[0].replace("_", " ").title()

    def parse_ncode(self, folder_name):
        return re.findall('.+/(n\d+)-[\w-]+$', folder_name)[0]

    def __init__(self, dataset_path):
        self.DATASET_PATH = dataset_path
        for dir_element in dataset_path.ls():
            if "/n0" in str(dir_element):
                BREED_NAMES[self.parse_ncode(str(dir_element))] = self.parse_breed_name(str(dir_element))

    def setup(self):  # set show_figures True if running in Jupyter nb
        if self.SHOW_FIGURES:
            print(f"Some dog breeds: {list(BREED_NAMES.values())[:5]}")

        dogs = DataBlock(
            blocks=(ImageBlock, CategoryBlock),
            get_items=get_image_files,
            splitter=RandomSplitter(seed=self.RANDOM_SEED),
            get_y=get_breed_from_filename,
            item_tfms=Resize(460),
            batch_tfms=aug_transforms(size=224, min_scale=0.75)
        )
        dataloaders = dogs.dataloaders(self.DATASET_PATH)
        if self.SHOW_FIGURES:
            dataloaders.show_batch(nrows=2, ncols=4)
        self.learn = cnn_learner(dataloaders, resnet50, metrics=error_rate).to_fp16()

    def run(self):
        self.learn.fine_tune(6, freeze_epochs=3)

    def save(self):
        self.learn.export(self.AI_MODEL_FILENAME, pickle_protocol=2)

    def load_learner(self, alt_file_path=""):
        return load_learner(alt_file_path or self.AI_MODEL_FILENAME)

    def test(self, image_path):  # pass 'images/some_dog.jpg'
        self.learn = self.load_learner()
        customer_photo = PILImage.create(image_path)
        breed, _, probs = self.learn.predict(customer_photo)
        print(f"{max(probs)*100:.2f}% sure this is a {breed}")

Preparing our Data

For our purposes today, I’ll focus only on describing how we train the neural network.
We will start with an instance of the class, assign our images path, and run setup()

dog_breeds_nn = DogBreedsNN(dataset_path)
Path.BASE_PATH = dog_breeds_nn.DATASET_PATH
print(dog_breeds_nn.DATASET_PATH)
dog_breeds_nn.setup()

your output should look something like this

The script downloads the pre-trained resnet50 neural network for image processing. The resnet50 model has been extensively pre-trained on image data. That means we do not need to teach our AI how to read images in general. We will borrow the knowledge it already has and simply train it on the differences between dog breeds.
To confirm that our data is being properly read and labeled, it displays a random sample of the labeled image data - that is 8 example images with their breed label.

Training Our Model

The data is ready, now we let the neural network start training!

dog_breeds_nn.run()

Looking inside run(), it’s simply calling
self.learn.fine_tune(6, freeze_epochs=3)
fast.ai is doing quite a bit of magic behind the scenes for us. In professional data science, there would be a long list of ‘hyper-parameters’ to set. Fortunately the defaults are good enough and we only provide the number of epochs for training. An epoch is a single round of training over the entire data set. So 6 epochs means that our neural network will review all the training images 6 times. Feel free to change this number, but from my testing, 6 epochs gets us pretty close the minimum error rate. Additional training will likely result in over-fitting and also just take longer to process.

The training took nearly 30 minutes to run and of all the information here, the most important we want to see is a decreasing error_rate in the 4th column after each epoch

dog_breeds_nn.save()

After it’s finished training, we’re going to simply save it straight away. This saves all the weights and parameters of neural network as a single 105MB file. With that file, we can now take this trained AI somewhere else to run it. For example, if you want to use this in a website, then we simply take this file over to the webserver. The file is a .pkl file, that is a “pickle” which is a file type that allows us to package up anything in python memory space, and save it, then and load it elsewhere.

Finally, let’s actually test this to make sure it even works!

Testing Our Model

dog_breeds_nn.test(Path('Images/n02085782-Japanese_spaniel/n02085782_50.jpg'))

63.50% sure this is a Japanese Spaniel

Fantastic! Inspecting the code that printed that number, you’ll see f"{max(probs)*100:.2f}%”. That means the 63.5% printed is the maximum of all probabilities (total add to 1.0). This is reasonably high confidence because the remaining (36%) is the sum of all other possible breed’s probabilities.

Let’s run this test on a random dog photo from the internet. Scrolling through a Google Image search for dog purebreed, I found this one:

To test inside the Jupyter notebook, you’ll need to upload the image file into a directory, like we did with the initial images.

To do so, follow these steps:

First, navigate to the Jupyter Notebook interface home page. ...
Click the “Upload” button to open the file chooser window.
Choose the file you wish to upload. ...
Click “Upload” for each file that you wish to upload.
Wait for the progress bar to finish for each file.

Back in our Jupyter Notebook, test the neural network on your image the same way we did the other test, but edit the path to point to the your uploaded image file.

dog_breeds_nn.test("path_to/random_dog.jpg")

The image above returned:
99.42% sure this is a Yorkshire Terrier

Congratulations! You’ve now completed building a neural network that is probably better than you at guessing a dog’s breed.

Some next steps

We didn’t train this system on non-dogs, I would speculate that sending it a photo of something other than a dog would cause max(probs) to be very low, like under 30%. Try it on your own and see what happens!
How would you adjust the training data so that it could also return with something like this?
95% sure this is a Not A Dog
Send it a selfie and see which breed is most similar looking. This one is fun with friends.
Host the code on a server so that other people can interact with it on a website or chat bot. I chose to host this on Heroku and connect to a Telegram bot: t.me/ai_helps_bot. Feel free to play with it.

AI Helps

Discussion about this post