Performance drop with reloaded model

[nanni00]

In my current work, I need to extract tables from spreadsheets like those in VEnron2 datasets, and I'm trying to use ExcelTableCNN for this task.
First, thanks for the project! I'm not familiar with CNNs, but the code is clear and with it I have understood better the core concepts of the original paper TableSense.
However, I am experiencing a strange problem: when training a model with the setting proposed in the readme, or with few epochs and limited training set size (e.g. epochs<60, train_size<100) the training completes and also when I reload the model the performance are similar to those measured right after the training with the model still in memory.
Differently, when epochs=160, train_size=None the model results are significantly lower with the reloaded model:

=============================================== EVALUATE IN-MEMORY MODEL ===============================================
Evaluated 197 sheets, 342 tables:
  EoB-0: precision=0.293 recall=0.301 (tp=103 fp=249 fn=239)
  EoB-2: precision=0.531 recall=0.547 (tp=187 fp=165 fn=155)

=============================================== EVALUATE RELOADED MODEL ================================================
Evaluated 197 sheets, 342 tables:
  EoB-0: precision=0.077 recall=0.006 (tp=2 fp=24 fn=340)
  EoB-2: precision=0.192 recall=0.015 (tp=5 fp=21 fn=337)

Why when reloading the model the performance are lower?
I pass also the script used for my experiments.

Thanks!

Python 3.14
torch: 2.12.1
cuda: 13.0

import time
import logging
from pathlib import Path

import matplotlib.pyplot as plt
import pandas as pd

from excel_table_cnn import (
    NUM_FEATURES,
    SpreadsheetDataset,
    TrainConfig,
    build_model,
    evaluate_model,
    format_report,
    get_train_test,
    load_checkpoint,
    train_model,
)

logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(message)s")

epochs = 160
lr = 0.005

WORK_DIR = Path("..")
DATA_DIR = WORK_DIR / "data"
CHECKPOINT_DIR = DATA_DIR / "checkpoints" / f"cp_ep{epochs}_lr{lr}"
CHECKPOINT_DIR.mkdir(parents=True, exist_ok=True)
HISTORY_LOGS_PATH = CHECKPOINT_DIR / "history.csv"
LOSS_PLOT_PATH = CHECKPOINT_DIR / "losses_plot.png"
logging.info(f"Checkpoint directory: {CHECKPOINT_DIR.resolve().as_posix()}")

train_samples, test_samples = get_train_test(
    data_folder_path=DATA_DIR.as_posix(),
    train_size=None,
    testing_size=None,
)

train_dataset = SpreadsheetDataset(train_samples)
print(f"{len(train_dataset)} training sheets ({len(train_dataset.skipped)} skipped)")

model = build_model(in_channels=NUM_FEATURES)

config = TrainConfig(
    epochs=epochs,
    lr=lr,
    # device="auto" resolves to CUDA when available (else CPU); AMP follows suit.
    checkpoint_dir=CHECKPOINT_DIR.as_posix(),
)

time_start = time.time()
history = train_model(model, train_dataset, config)
time_training = time.time() - time_start

print(f"Training completed in {time_training:.2f} seconds")

history_df = pd.DataFrame(history)
history_df[['loss_total', 'loss_objectness', 'loss_rpn_box_reg',
            'loss_classifier', 'loss_box_reg']].rolling(200).mean().plot(figsize=(10, 4))
plt.savefig(LOSS_PLOT_PATH)

try:
    history_df.to_csv(HISTORY_LOGS_PATH, index=False)
except Exception as e:
    print(f"Failed to save history logs: {e}")
else:
    print(f"History logs saved at {HISTORY_LOGS_PATH}")

print(" EVALUATE IN-MEMORY MODEL ".center(120, "="))
test_dataset = SpreadsheetDataset(test_samples)
report = evaluate_model(model, test_dataset)  # device="auto"
print(format_report(report))
print()

last_checkpoint_path = CHECKPOINT_DIR / "last.pt"
model = load_checkpoint(last_checkpoint_path.as_posix(), device="cuda")

print(" EVALUATE RELOADED MODEL ".center(120, "="))
test_dataset = SpreadsheetDataset(test_samples)
report = evaluate_model(model, test_dataset)  # device="auto"
print(format_report(report))