Real-time collaboration for Jupyter Notebooks, Linux Terminals, LaTeX, VS Code, R IDE, and more,
all in one place.
Real-time collaboration for Jupyter Notebooks, Linux Terminals, LaTeX, VS Code, R IDE, and more,
all in one place.
Path: tree/main/_static/img/distributed
Views: 713
Name | Size | Last Modified |
|---|---|---|
| DDP_memory.gif | 2 MB | |
| DistPyTorch.jpg | 38.7 KB | |
| FSDP_autowrap.gif | 1.7 MB | |
| FSDP_memory.gif | 1.7 MB | |
| all_gather.pdf | 18.8 KB | |
| all_gather.png | 46.4 KB | |
| all_reduce.pdf | 19.2 KB | |
| all_reduce.png | 44.7 KB | |
| broadcast.png | 19.8 KB | |
| device_mesh.png | 38.4 KB | |
| distributed_checkpoint_generated_files.png | 23.1 KB | |
| fsdp_sharding.png | 93.2 KB | |
| fsdp_tp.png | 256.1 KB | |
| fsdp_workflow.png | 568.6 KB | |
| gather.png | 21.9 KB | |
| loss_parallel.png | 296.8 KB | |
| megatron_lm.png | 792.4 KB | |
| reduce.png | 21.9 KB | |
| scatter.png | 21.6 KB | |
| send_recv.png | 8.6 KB | |
| send_recv_big.png | 13.1 KB | |
| tcpstore_barrier_time.png | 497.2 KB | |
| tcpstore_init_time.png | 433.2 KB |
