At the end of this course, successful students will understand the logic behind and the high-level structure of widely used neural networks implemented by common Python frameworks (e.g., TensorFlow/Keras, PyTorch) for deep learning tasks (e.g., image and text processing) and be able to present and discuss their process and findings by creating interactive documents using the Jupyter Notebook web environment for reproducible dissemination of results.

Successful students will be able to perform:

• Image processing tasks (e.g., image classification, object detection, instance or semantic segmentation) using common deep learning frameworks (e.g., TensorFlow/Keras, PyTorch), including:
  • Constructing and training models comprising neural networks based on
    • established convolutional architectures (e.g., VGG, U-Net, ResNet); or
    • pre-trained model checkpoints available within online repositories (e.g., TensorFlow, Kaggle, HuggingFace) that serve as the model’s backbone (transfer learning with or without finetuning); or
    • novel convolutional feedforward architectures using specialized neural layers such as (de)convolution, (un)pooling, and dropout.
  • Selecting candidate (backbone) models by assessing an individual model’s performance potential, relying on:
    • common frameworks for data benchmarking (e.g., CIFAR, ImageNet) and measurement (e.g., online leaderboards)
    • the exploration of several model characteristic values (e.g., model card)
    • considerations about model generalization (e.g., bias/error within training/testing dataset)
  • Validating the constructed model’s performance using scoring methods appropriate for the problem/question (e.g., IoU, AUC).
• Text/Natural language processing tasks (e.g., sentence classification, named entity recognition, translation) using common deep learning frameworks (e.g., TensorFlow/Keras, PyTorch), including:
  • Constructing and training models comprising neural networks based on:
    • established transformer/attention architectures (e.g., GPT, BERT, BART, and their derivatives); or
    • pre-trained model checkpoints available within online repositories (e.g., TensorFlow, Kaggle, HuggingFace) that serve as the model’s backbone (transfer learning with or without finetuning); or
    • novel pipelines combining tokenizers (for data pre-processing), embeddings into latent space (for feature selection), and decoders (for post-processing back into text).
  • Selecting candidate (backbone) models by assessing an individual model’s performance potential, relying on:
    • common frameworks for data benchmarking and measurement
    • the exploration of several model characteristic values
    • considerations about model generalization
  • Validating the constructed model’s performance using scoring methods appropriate for the problem/question (e.g., F1, BLEU).