Cross validation baseline:
python3 -m src.protocols_methodology.exp_cv --dataset adult --algo rf --n-folds 5 --seeds 0 1 2 3 4


Bootstrap baseline:
python3 -m src.protocols_methodology.exp_bootstrap --dataset adult --algo rf --n-bootstrap 30 --seeds 0 1 2 3 4


Noise baseline:
python3 -m src.protocols_methodology.exp_noise --dataset adult --algo rf --n-replicates 30 --noise-std 0.01 --seeds 0 1 2 3 4



Small run: compute 0
tmux new -s small
python3 -m src.protocols_methodology.run_nsga_protocols --dataset adult --generations 3 --pop-size 8 --seed 0
tmux attach-session -t small

Medium run: compute 1
tmux new -s medium
python3 -m src.protocols_methodology.run_nsga_protocols --dataset adult --generations 10 --pop-size 24 --seed 0
tmux attach-session -t medium

Larger run: compute 3
tmux new -s larger
python3 -m src.protocols_methodology.run_nsga_protocols --dataset adult --generations 20 --pop-size 40 --seed 0
tmux attach-session -t larger


Send back:
The three baseline summary csv files:
runs/protocol_cv/__cv_summary.csv

runs/protocol_bootstrap/__bootstrap_summary.csv

runs/protocol_noise/__noise_summary.csv



The three AutoML Pareto fronts:
runs/_protocol_study/cv/pareto_front.csv

runs/_protocol_study/bootstrap/pareto_front.csv

runs/_protocol_study/noise/pareto_front.csv