AWS 電腦視覺入門（第 2 部分）：訓練自訂 CNN 模型

This is the second part of the series "Your first steps in Computer Vision on AWS". If you are just starting, this tutorial will guide you step by step from the initial dataset to a deployed app.

You can read the first part on ASL dataset + preprocessing.

Why training on CPU would take ages

While we used a simple CPU SageMaker notebook instance to work with the data, for training we would need a GPU-powered instance.
A CPU is designed for complex sequential tasks. Training a CNN requires performing millions of simple matrix multiplications and convolutions simultaneously.
A GPU contains thousands of smaller, simpler cores designed to execute thousands of calculations at the exact same time, offering massive parallelism.

As an analogy, you can picture a CPU as a powerful supercomputer which can perform most complex calculations (and it will answer '42' 😂). A GPU in this case would be a fleet of calculators, each performing simple parallel operations.

As you can see on the screenshot, one epoch can take 1.5 hours (while we would need 20-25 epochs):

📖 An epoch in machine learning is one complete pass of the entire training dataset through a model.

Let's create a GPU-powered SageMaker notebook instance

Let's create a SageMaker notebook instance.

The AWS ml.g5.xlarge SageMaker instance (and the corresponding EC2 g5.xlarge instance) has 1 NVIDIA A10G Tensor Core GPU with 24 GiB of VRAM (video memory).

Pricing: ml.g5.xlarge is not as cheap as ml.t3.medium. The exact price can vary depending on the region (e.g., for Frankfurt it is $1.258 per hour); see this page for on-demand pricing reference.

⚠️ Don't forget to stop and terminate your instance once the data is processed and uploaded to S3.

Why CNN model?

Before we get into model training, let's stop for a moment to discuss what a CNN model is and why we selected it.

📖 A Convolutional Neural Network (CNN) is designed specifically to process and understand visual data. In simple terms, it uses filters/kernels to look at small pieces of the image, detecting simple shapes like edges or lines. Then, it uses pooling to reduce the image size while retaining the most useful information. Simple shapes are combined into more complex features like wheels, eyes, etc. Finally, the model decides what the picture is ("car", "person", etc.).

Using a CNN for image classification is a good starting point: the model is lightweight (compared to fully connected neural networks), but still quite performant.

We use different types of layers, stacked together, to build the CNN architecture. As the input, each image has a resolution of 200×200 pixels: raw and filtered RGB images will have 3 channels (200, 200, 3), edge images will have 1 channel (200, 200, 1).

• Convolutional Layer: applies small, learnable filter matrices (kernels) across input data to extract local visual patterns.
• Pooling: each 2×2 pooling layer reduces the image size, e.g. 200 → 100.
• Global Average Pooling: converts the final 25×25×128 feature maps into 128 values.
• Dense layer: 128 neurons combine the extracted features to make the final decision.
• Dropout: randomly turns off 50% of the dense-layer neurons to prevent overfitting.

Finally, the output, where 29 neurons will represent the 29 classes. Softmax converts their outputs into probabilities, and the class with the highest probability is the prediction.

Let's show the model architecture:

In our model, the first convolutional layer learns simple features (e.g., edges); the second learns combinations of those features (e.g., finger shapes), while the third learns more complex hand/sign patterns. The number of filters increases (32 → 64 → 128), so the network can learn more types of features.

You can also see that the model will have 113,501 parameters, whose values will change during training.

📖 A parameter in a CNN is an internal, learnable variable (specifically, weights and biases) that the model automatically adjusts and optimizes during training through backpropagation.

Copy the image data to the instance

Once the SageMaker notebook instance is ready, you can Open Jupyter / Open JupyterLab.

Upload the Raw_images_training.ipynb Jupyter notebook from the GitHub repo.

Run cell by cell.

A useful command to use before any training starts: !nvidia-smi. It would show attached GPUs, temperatures, memory usage, power, and active processes. For example:

⚠️ Don't forget to replace the placeholder and put your S3 bucket name:

BUCKET = "YOUR_BUCKET"

This cell copies the preprocessed data from S3 to the instance. You only need to run it once and then check that the data has been copied successfully.

!aws s3 sync s3://YOUR-BUCKET/raw/asl_alphabet_train/ /home/ec2-user/SageMaker/asl_alphabet_train/ --quiet
!aws s3 sync s3://YOUR-BUCKET/processed/ /home/ec2-user/SageMaker/asl_preprocessed/  --quiet

Let's train 3 CNN models

⚠️ Remember we had 3 versions of data: raw, Gaussian-filtered and Canny edge? Well, you can use this Gaussian_filtered_training.ipynb notebook to train the CNN with Gaussian-filtered images; and this Canny_edges_training.ipynb notebook to train the CNN with Canny edge images.
They don't differ much. I created three separate notebooks mainly to keep the cell output histories separate. If for you it is more comfortable to use a single notebook, feel free to do it. The only thing you would need to change is the REPRESENTATION variable. Set it to raw, filtered, or edges depending on the image version.

In a nutshell, training the CNN model follows this process:
image → prediction → compare with correct answer → calculate error → adjust weights → repeat.

What each notebook does (high-level):

1. Creates train/validation/test dataframes
2. Designs and compiles the CNN
3. Trains the CNN from scratch
4. Evaluates on the held-out test set
5. Saves the model and uploads it to S3

Train/validation/test dataframes

⚠️ Note: In some sources, the final held-out dataset is called 'validation'.
Here, we use the validation set to evaluate model performance after each epoch.
The final held-out dataset is called 'test' (we will do the final metric evaluation against it).

We created and recorded the dataset split in the previous part. We saved it in a CSV file, which we now use to retrieve the corresponding image paths.

Compile the network

We compile the designed CNN model using this code snippet:

model.compile(
    optimizer=tf.keras.optimizers.Adam(
        learning_rate=0.001
    ),
    loss=tf.keras.losses.SparseCategoricalCrossentropy(),
    metrics=[
        tf.keras.metrics.SparseCategoricalAccuracy(
            name="accuracy"
        )
    ]
)

Adam optimizer is a popular optimization algorithm used to train deep learning models. SparseCategoricalCrossentropy is a loss function used for multi-class classification problems where the true labels are provided as integers. Finally, for the metric, we are using SparseCategoricalAccuracy, an evaluation metric used in multi-class classification models to measure how often model predictions match integer-encoded target labels.

📖 A loss function in deep learning is a mathematical formula that measures the difference between what a neural network predicts and the actual ground truth label.

Model training

We train the model for 25 EPOCHS. It would take only several minutes to fully train the CNN:

history = model.fit(
    train_ds,
    validation_data=val_ds,
    epochs=EPOCHS,
    callbacks=callbacks,
    verbose=1
)

We will save both best model and final model.

❓ Why isn't the final model always the best model?
The final model represents the state of your model's weights after the last training epoch has finished. The best model is the state of your model weights from the epoch that achieved the optimal score (lowest validation loss or highest validation accuracy) during training. For saving the best model, we are using ModelCheckpoint with save_best_only=True.

Once training is finished, it is useful to plot how was the accuracy changes across training epochs. It was increasing, which is a good sign.

The same goes for the loss. It was decreasing, which indicates that the model's predictions were becoming more accurate according to the loss function.

Model evaluation

We are evaluating the best model on the held-out test dataset with:

test_loss, test_accuracy = model.evaluate(
    test_ds,
    verbose=1
)

The evaluation results on the test dataset are very good, with an accuracy of 99.34%:

Upload models to S3

We are saving the final model, best model, and training history to S3.
After running all the cells in all three notebooks (one for each dataset version), your bucket should contain a new models/ directory with the following structure:

s3://YOUR-BUCKET-NAME/

models/
├── asl_cnn_raw_best.keras
├── asl_cnn_raw_final.keras
├── training_history_raw.csv
├── asl_cnn_filtered_best.keras
├── asl_cnn_filtered_final.keras
├── training_history_filtered.csv
├── asl_cnn_edges_best.keras
├── asl_cnn_edges_final.keras
└── training_history_edges.csv

Which model is the best?

After running the notebook for 3 dataset versions (raw, Gaussian-filtered and Canny edge), it is interesting to find out which data resulted in the best model performance.

It turns out that the optimal performance (highest accuracy, lowest loss) was achieved using raw RGB images. One possible reason is that raw RGB images retain rich color information, gradients, and skin texture. Gaussian filtering may blur high-frequency finger boundaries, while Canny edge detection may discard useful internal hand details.

Conclusion

In this part, we used three dataset versions to train CNN models from scratch. The best performance was achieved using raw RGB images.

We will keep three best models (asl_cnn_raw_best.keras, asl_cnn_filtered_best.keras, asl_cnn_edges_best.keras) in S3 for now. In the next part, we will deploy these models to use in an ASL prediction app. Stay tuned!