Open Source Generative AI (OSGA)

1. The Mechanics of Deep Learning - Gain an intuitive understanding of the most current generative AI architecture.

• Choosing the latest AI models
  • What is current and what has already gone extinct?
• Neural network architecture essentials
  • Tokenization
  • Embedding
  • Parameters: weights and bias
  • Nodes
  • Fully connected/Partially connected
  • Prompts and prompt engineering

2. The Transformer Model - Develop an intuitive understanding of the transformer model, without math.

• Neural Network Architectures
• Word embeddings
  • Importance in NLP
  • Word2Vec and GloVe embeddings
  • Contextualized embeddings (BERT, ELMo, etc.)
• Self-attention mechanism and multi-head attention
  • Input Representation (Query, Key, and Value)
  • Computing Similarities
  • Attention Weights
  • Weighted Sum
  • Final Output
• Positional encoding for attention
• Transformer layers and stacking
• Quantization
• Effective LLM selection criteria and use-cases
  • Models: size, datasets, quantization, etc
  • Libraries
  • Frameworks

3. Hardware Requirements

• GPUs role in AI performance (CPU vs GPU)
• GPUs architecture history
  • Pascal, Turing, Ampere
• Tensor core vs older GPU architectures
• GPU vocabulary and the transformer
  • SM, tensors, tensor core, CUDA core, FP and INT cores, warp, threads
• Current GPUs and cost vs value
  • Analysis of GPU specification sheets
• GPU selection for models and workload
• Quantizing for hardware performance and cost

4. Pre-trained LLM Model Essentials

• Select and use pre-trained models to produce immediate results
• Model ratings, metrics, leader boards
• Model licensing and commercial use
• Numbers every AI developer should know (cost, time, dataset size, etc.)
• Synthetic data generation for model training.
• Model training through feature extraction
  • Derive features from provided dataset
  • Use annotated data for fine-tuning training

5. Pre-trained LLM hands-on - Run inference on multiple models on diverse prompts

• Frameworks
  • llama.cpp, exllama, GPT4All
• Evaluate multiple models, fine-tunings quantization
  • Falcon, Orca Mini, OpenLlama, Alpaca, MPT
• Parameters
• Fine tuned models
• Prompts
  • Model expectations (instruct, chat, etc)
  • Extension improvement

6. Transformer Model training essentials

• Overfitting
• Regularization (Avoiding Overfitting)
• Backpropagation
• Gradient Descent
• Embedding
• Learning Rate
• Perplexity
• Batch Normalization
• Warm-up and Learning Rate Decay
• Loss Functions
• Data Augmentation
• Training Strategies
• Evaluation Metrics

7. Conversion of Model Formats

• PyTorch to ggml
• JAX to ggml
• F16 to 4bit quantization

8. Hands on with AI programming environments

• PyTorch and transformers
• Transformers low level interactive inspection

9. Model Fine Tuning

• Perform fine-tuning in a hands-on environment
• Demonstration project with good fine-tuning data to highlight frameworks
  • with llama.cpp
  • with PyTorch
• Understanding fine tuning dataset formats
• Data cleaning skills

10. Introduce Application Interfacing to Models

• LangChain
• Guidance for structured LLM output

11. Applications Augmentation with Langchain and Guidance

• LangChain
• Guidance for structured LLM output
• Prompt engineering

12. Advanced Topics

• Optimize your model through fine-tuning and quantization.
• Realize quality and speed inference on specific workload
• Context extension through fine tuning
• LLM deployment

13. Use llama to perform Natural Language Processing tasks

• Rewrite a classic poem of a well-known author in the tone of another well know author
• Use Named Entity Recognition (NER) to identify Cajun food in recipes
• Use domain specific models to improve project accuracy

14. Deploy a Natural Language Model Capstone

• Build a full NLP project
• Download, install and implement a trained NLP model
• Deploy your NLP to perform the following tasks:
  • text classification
  • sentiment analysis
  • machine translation
  • chatbots
  • question answering