Documentation of RapidFire AI Trial Version

Note

This project is under active development.

Contents

• Overview of RapidFire AI
• Trial Version Usage Walkthrough
  • QuickStart Demo Video
  • Full Usage Walkthrough Video
  • Step 1: Access you cluster
  • Step 2: Launch the Notebook
  • Step 3: Data Download
  • Step 4: Create Experiment
  • Step 5: Launch Runs
  • Step 6: Monitor Training with ML Metrics Dashboard
  • Step 7: System Metrics Dashboard
  • Step 8: Interactive Control (IC) Ops: Stop
  • Step 9: IC Ops: Clone-Modify
  • Step 10: Check Results of IC Ops
  • Step 11: Test, Predict, and End Experiment
  • Step 12: Venture Beyond!
• Tutorial Use Cases
  • Use Case Examples
    • Tutorial Use Case: IMDb
      • Dataset
      • Model(s)
      • Config Knobs
      • Step by Step Code in our API
      • Extension: Custom Collate
    • Tutorial Use Case: ImageNet
      • Dataset
      • Model(s)
      • Config Knobs
      • Step by Step Code in our API
    • Tutorial Use Case: News
      • Dataset
      • Model(s)
      • Config Knobs
      • Step by Step Code in our API
      • Custom User-Defined Metrics
      • Extension: Custom Transfer Learning with a Llama model
    • Tutorial Use Case: COCO Detection and Segmentation
      • Dataset
      • Model
      • Config Knobs
      • Step by Step Code in our API
• Jupyter Notebook and File Handling
  • Our use of Jupyter Notebooks
  • About requirements.txt and Custom Code Files
• API: Data Ingestion
  • Example Structure File (ESF)
  • Locators Dictionary and Semantics
• API: Data Handles
  • Overview
  • Data Handle Constructor
    • __init__()
  • Download
    • download()
  • Delete Local
    • delete_local()
  • List
    • list()
  • Get
    • get()
• API: Experiment Ops
  • Overview
  • Experiment Constructor
    • __init__()
  • Run Fit
    • run_fit()
  • Run Test
    • run_test()
  • Run Predict
    • run_predict()
  • Get Results
    • get_results()
  • Get Runs Information
    • get_runs_info()
  • End
    • end()
• API: MLSpec: Training and Inference
  • Overview
  • Initialize Run
    • initialize_run()
  • Row Preprocess
    • row_prep()
  • Custom Collate
  • Create Model
    • create_model()
  • Load Model
    • load_model()
  • Forward Pass
    • compute_forward()
  • Custom Metrics Definition
    • compute_metrics()
  • Custom Metrics Aggregation
    • aggregate_metrics()
  • Convergence Check
    • has_converged()
• API: Config Knobs and AutoML
  • Train Knobs
  • Named Metrics
  • User Knobs
  • Other Knobs
  • Value Set Generators
    • LIST()
    • RANGE()
  • Config Group Generators
    • GridSearch()
    • RandomSearch()
  • AutoML Heuristics
    • SuccessiveHalving()
  • Advanced: Lists of Configs or Config Group Generators
• API: Utility Functions
  • Cancel Current
    • cancel_current()
  • Download Logs
    • download_logs()
  • File Handling Utilities
    • get_object()
    • put_object()
    • list_objects()
    • delete_object()
• Dashboard: MLflow
  • About Our Use of MLflow
  • Tabs in the Dashboard
  • Metrics Plots
  • Message Logs
• Dashboard: Interactive Control (IC) Ops
  • Motivation for IC Ops
  • Semantics of IC Ops
  • Stop
  • Resume
  • Clone-Modify
  • Delete
  • Templated Automation of IC Ops
• Dashboard: File Upload
  • Coming Soon: Bring Your Own S3 Buckets
• Dashboard: Cluster Ops
  • Stop Cluster
  • Restart Cluster
  • Create Cluster
  • Delete Cluster
  • Cluster Logs
  • Failed Clusters
  • Bring Your AWS Account
  • On-Premise and Other Clouds
• Dashboard: Plutono
• Current Limitations and Updates Coming Soon
  • Model Size
  • Cap on Multi-Node Parallelism
  • Script-based Jobs Without Jupyter
  • Semi-Automated IC Ops
  • Named Metrics Roster
  • LLM-Specific Training, Fine-Tuning, and Inference
• Glossary of Key Terms and Concepts
  • Artifacts
  • Cancel Current
  • Cluster Ops
  • Config Dictionary
  • Config Group
  • Config Group Generator
  • Custom Metrics
  • Data Handle
  • Data Locators Dictionary
  • Example Structure File (ESF)
  • Experiment
  • Experiment Ops
  • File Handling Utilities
  • In-Situ Dataset
  • Interactive Control Ops (IC Ops)
  • Knob
  • Logs
  • ML Metrics Dashboard
  • MLSpec
  • Named Metrics
  • Non-Tensor Metrics
  • Object Directories and Files
  • Results
  • Requirements
  • Run
  • System Metrics Dashboard
  • Train Knob
  • User Knob