Visualization¶
Flow supports visualization of both rllab and RLlib experiments.
rllab¶
Call the rllab visualizer with
python ./visualizer_rllab.py /result_dir/itr_XXX.pkl
The rllab visualizer also takes some inputs:
--num_rollouts
--plotname
--use_sumogui
--run_long
--emission_to_csv
The params.pkl
file can be used as well.
RLlib¶
Call the RLlib visualizer with
python ./visualizer_rllib.py /ray_results/result_dir 1
# OR
python ./visualizer_rllib.py /ray_results/result_dir 1 --run PPO
# OR
python ./visualizer_rllib.py /ray_results/result_dir 1 --run PPO \
--module cooperative_merge --flowenv TwoLoopsMergePOEnv \
--exp_tag cooperative_merge_example
The first command-line argument corresponds to the directory containing
experiment results (usually within RLlib’s ray_results
). The second is
the checkpoint number, corresponding to the iteration number you wish to
visualize. The --run
input is optional; the default algorithm used is
PPO. If the experiment module, Flow environment name, and experiment tag
have not been stored automatically (see section below), then those
parameters can be passed in using the flags --module
, --flowenv
,
and --exp_tag
.
Parameter storage¶
RLlib doesn’t automatically store all parameters needed for restoring the state of a Flow experiment upon visualization. As such, Flow experiments in RLlib include code to store relevant parameters. Include the following code snippet in RLlib experiments you will need to visualize
# Logging out flow_params to ray's experiment result folder
json_out_file = alg.logdir + '/flow_params.json'
with open(json_out_file, 'w') as outfile:
json.dump(flow_params, outfile, cls=NameEncoder, sort_keys=True, indent=4)
These lines should be placed after initialization of the PPOAgent RL algorithm as
it relies on alg.logdir
. Store parameters before training, though, so
partially-trained experiments can be visualized.
Another thing to keep in mind is that Flow parameters in RLlib experiments
should be defined outside of the make_create_env
function. This allows
that environment creator function to use other experiment parameters later,
upon visualization.