Quellcode durchsuchen

Append epoch rather than best val. loss to val_loss

**Problem**
Currently, we're val_loss.append(best_val_loss) in each epoch. This is misleading because we're appending the corresponding epoch (not best across epochs) quantities in train_loss, train_prep, and val_prep. This is also inconvenient, as one often would like to plot both train and validation losses as a function of the epochs to look for overfitting.

**Solution**
val_loss.append(eval_epoch_loss)
celestinoalan vor 6 Monaten
Ursprung
Commit
2a94bfff26
1 geänderte Dateien mit 1 neuen und 1 gelöschten Zeilen
  1. 1 1
      src/llama_recipes/utils/train_utils.py

+ 1 - 1
src/llama_recipes/utils/train_utils.py

@@ -288,7 +288,7 @@ def train(model, train_dataloader,eval_dataloader, tokenizer, optimizer, lr_sche
                         print(f"best eval loss on epoch {epoch+1} is {best_val_loss}")
                 else:
                         print(f"best eval loss on epoch {epoch+1} is {best_val_loss}")
-            val_loss.append(float(best_val_loss))
+            val_loss.append(float(eval_epoch_loss))
             val_prep.append(float(eval_ppl))
         if train_config.enable_fsdp:
             if rank==0: