script

项目架构

.
├── abstract_scenario_builder.py
├── abstract_scenario.py
├── BUILD
├── cache
│   ├── BUILD
│   ├── cached_scenario.py
│   ├── __init__.py
│   ├── __pycache__
│   │   ├── cached_scenario.cpython-39.pyc
│   │   └── __init__.cpython-39.pyc
│   └── test
│       ├── BUILD
│       ├── __init__.py
│       └── test_cached_scenario.py
├── __init__.py
├── nuplan_db
│   ├── BUILD
│   ├── __init__.py
│   ├── nuplan_scenario_builder.py
│   ├── nuplan_scenario_filter_utils.py
│   ├── nuplan_scenario.py
│   ├── nuplan_scenario_utils.py
│   ├── __pycache__
│   │   ├── __init__.cpython-39.pyc
│   │   ├── nuplan_scenario_builder.cpython-39.pyc
│   │   ├── nuplan_scenario.cpython-39.pyc
│   │   ├── nuplan_scenario_filter_utils.cpython-39.pyc
│   │   └── nuplan_scenario_utils.cpython-39.pyc
│   └── test
│       ├── BUILD
│       ├── __init__.py
│       ├── nuplan_scenario_test_utils.py
│       ├── test_nuplan_scenario_builder.py
│       ├── test_nuplan_scenario_filter_utils.py
│       ├── test_nuplan_scenario_integration.py
│       ├── test_nuplan_scenario.py
│       ├── test_nuplan_scenario_utils_integration.py
│       └── test_nuplan_scenario_utils.py
├── __pycache__
│   ├── abstract_scenario_builder.cpython-39.pyc
│   ├── abstract_scenario.cpython-39.pyc
│   ├── __init__.cpython-39.pyc
│   ├── scenario_filter.cpython-39.pyc
│   └── scenario_utils.cpython-39.pyc
├── scenario_filter.py
├── scenario_utils.py
└── test
    ├── BUILD
    ├── __init__.py
    ├── mock_abstract_scenario_builder.py
    ├── mock_abstract_scenario.py
    ├── test
    │   ├── BUILD
    │   ├── __init__.py
    │   ├── test_mock_abstract_scenario_builder.py
    │   └── test_mock_abstract_scenario.py
    └── test_scenario_utils.py

关键文件介绍

Config Group

1. 文件以yaml的形式储存，多级目录文件的嵌套使用defaults参数。其他的直接用key: value的形式表示, config文件下的yaml文件，training, simulation, nuboard分别表示训练过程，评分过程，和可视化过程中需要的配置文件，而从common文件表示三者均需用到的配置文件

   1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44

   ├── common │ ├── default_common.yaml │ ├── default_experiment.yaml │ ├── default_submission.yaml │ ├── model │ ├── scenario_builder │ │ ├── scenario_mapping │ │ └── vehicle_parameters │ ├── scenario_filter │ ├── simulation_metric │ │ ├── ego_in_stop_line │ │ ├── high_level │ │ └── low_level │ ├── splitter │ └── worker ├── nuboard │ └── default_nuboard.yaml ├── simulation │ ├── default_run_metric_aggregator.yaml │ ├── default_simulation.yaml │ ├── default_submission_planner.yaml │ ├── callback │ ├── ego_controller │ │ ├── motion_model │ │ └── tracker │ ├── main_callback │ ├── metric_aggregator │ ├── observation │ ├── planner │ ├── predictor │ └── simulation_time_controller └── training ├── default_training.yaml ├── callbacks ├── data_augmentation ├── data_augmentation_scheduler ├── data_loader ├── lightning ├── lr_scheduler ├── objective ├── optimizer ├── scenario_type_weights ├── training_metric └── warm_up_lr_scheduler

2. config同级目录experiments的文件表示实验过程的参数配置

   1 2 3 4 5 6 7 8 9 10 11 12 13 14

   ├── simulation │ ├── closed_loop_nonreactive_agents.yaml │ ├── closed_loop_reactive_agents.yaml │ ├── open_loop_boxes.yaml │ ├── remote_closed_loop_nonreactive_agents.yaml │ ├── remote_closed_loop_reactive_agents.yaml │ ├── remote_open_loop_boxes.yaml │ └── validation_challenge.yaml └── training │ ├── training_our_model.yaml │ ├── training_raster_model.yaml │ ├── training_simple_vector_model.yaml │ ├── training_urban_driver_open_loop_model.yaml │ └── training_vector_model.yaml

3. 运行脚本run_training.py在所有配置组yaml文件的上层目录，或者同级目录也可以，这实际上是为了Entry python入口传参数方便

   1 2 3 4 5 6 7 8 9

   ├── script │ ├── config │ ├── experiments └── run_training.py # 或者 ├── config ├── experiments └── run_training.py

Entry

将***@hydra.main写在run_training.py***程序入口函数main()的上方，即表示hydra传入cfg参数的入口，其中CONFIG_PATH表示相对于run_training.py的相对路径，而CONFIG_NAME 表示相对路径下yaml文件的名字，即CONFIG_PATH+ CONFIG_NAME中的yaml文件构成参数入口

1
2
3
4

CONFIG_PATH = 'config/training'
CONFIG_NAME = 'default_training'
@hydra.main(config_path=CONFIG_PATH, config_name=CONFIG_NAME)
def main(cfg: DictConfig) -> Optional[TrainingEngine]:

Config Object

我们以default_training.yaml和training_vector_model.yaml为例看配置参数的写法。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44

hydra:
  run:
    dir: ${output_dir}
  output_subdir: ${output_dir}/code/hydra           # Store hydra's config breakdown here for debugging
  searchpath:                                       # Only <exp_dir> in these paths are discoverable
    - pkg://nuplan.planning.script.config.common
    - pkg://nuplan.planning.script.experiments      # Put experiments configs in script/experiments/<exp_dir>

defaults:
  - default_experiment
  - default_common

  # Trainer and callbacks
  - lightning: default_lightning
  - callbacks: default_callbacks

  # Optimizer settings
  - optimizer: adam  # [adam, adamw] supported optimizers
  - lr_scheduler: null  # [one_cycle_lr] supported lr_schedulers
  - warm_up_lr_scheduler: null  # [linear_warm_up, constant_warm_up] supported warm up lr schedulers

  # Data Loading
  - data_loader: default_data_loader
  - splitter: ??? 

  # Objectives and metrics
  - objective: ???
  - training_metric: ???
  - data_augmentation: null
  - data_augmentation_scheduler: null  # [default_augmentation_schedulers, stepwise_augmentation_probability_scheduler, stepwise_noise_parameter_scheduler] supported data augmentation schedulers
  - scenario_type_weights: default_scenario_type_weights

experiment_name: 'training'
objective_aggregate_mode: ??? # How to aggregate multiple objectives, can be 'mean', 'max', 'sum'

# Cache parameters
cache:
  cache_path: ${oc.env:NUPLAN_EXP_ROOT}/cache         # Local/remote path to store all preprocessed artifacts from the data pipeline
  use_cache_without_dataset: false                    # Load all existing features from a local/remote cache without loading the dataset
  force_feature_computation: false                    # Recompute features even if a cache exists
  cleanup_cache: false                                # Cleanup cached data in the cache_path, this ensures that new data are generated if the same cache_path is passed

# Mandatory parameters
py_func: ???                                          # Function to be run inside main (can be "train", "test", "cache")

Search Path

searchpath是hydra的配置参数，一种是以pkg://的方式来找搜索路径，即其文件夹需要是python模块，即在文件夹下添加函数init.py。另一种是以file:// 的形式来找搜索路径。两种方式既可以采用相对路径也可采用绝对路径的模式。若采用相对路径的模式，则当前路径是Vscode等软件当前打开的文件夹目录/工程目录

Defaults

defaults的意思是默认采用的配置文件，其语法是加一个 - key: value 或者 - value 这里的key表示文件夹，value表示文件名，若当前default_training.yaml找不到对应文件则在搜索目录下寻找。

符号 ??? 表示缺失值，若调用对应的key会报错。null表示值为None，赋值给python对象即表示没有值。${oc.env:NUPLAN_EXP_ROOT}表示引用系统环境变量NUPLAN_EXP_ROOT。${output_dir}表示引用变量output_dir(key)的value。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

# @package _global_
job_name: vector_model
py_func: train
objective_aggregate_mode: mean

defaults:
  - override /data_augmentation:
      - kinematic_agent_augmentation
      - agent_dropout_augmentation
  - override /objective:
      - imitation_objective
  - override /splitter: nuplan
  - override /model: vector_model
  - override /scenario_filter: training_scenarios
  - override /training_metric:
      - avg_displacement_error
      - avg_heading_error
      - final_displacement_error
      - final_heading_error

Package

@package _global_表示将training_vector_model.yaml的配置文件的变量放入global变量即default_training.yaml中。更多用法见hydra package

Override

override表示覆盖原有的参数值。以data_augmentation为例，由于data_augmentation等文件夹(key)不在当前文件下，故采用绝对路径的形式，加一个/，表示该文件夹在global文件配置default_training.yaml文件下。而后面字符串有的加 - 表示以字典的形式传入，为了传入多个配置文件。以data_augmentation为例，其参数格式为：

1
2
3
4

cfg.data_augmentation={
"kinematic_agent_augmentation":{}
"agent_dropout_augmentation":{}
}

Running Args

运行时参数传入和修改。这里以vscode调试中的参数为例。

–multirun表示运行多个脚本，这里training和scenario_filter.limit_total_scenarios参数分别给了两个参数，故其将连续运行四个独立不同参数的脚本。

+key=value表示在cfg字典中添加元素，可以添加配置文件也可添加键值对。其中+training表示添加配置文件，而+training_checkpoint表示添加键值对。因为training是searchpath下experiments下的文件夹名，而其值表示的是其中的文件名。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

{
            "name": "Hydra-MultiRun ",
            "type": "python",
            "request": "launch",
            "program": "${workspaceFolder}/nuplan/planning/script/run_training.py",
            "console": "integratedTerminal",
            "justMyCode": true,
            "args": ["--multirun",
                    "py_func=train" ,
                    "+training=training_vector_model,training_raster_model" ,
                    "scenario_builder=nuplan_mini" ,
                    "scenario_filter.limit_total_scenarios=100,200" ,
                    "lightning.trainer.params.max_epochs=1" ,
                    "data_loader.params.batch_size=8" ,
                    "data_loader.params.num_workers=6",
                    "worker=sequential"
                     "+training_checkpoint='/home/user001/Code/nuplan-devkit-v1.1-ours/nuplan/exp/exp/training/vector_model/2023.04.28.21.31.45/best_model/epoch=9-step=199.ckpt'",
                    ]
        },

Hydra Instantiate

Hydra使用yaml文件中的参数将类实例化的函数instantiate

1
2
3
4
5
6

worker: WorkerPool = (
        instantiate(cfg.worker, output_dir=cfg.output_dir)
        if is_target_type(cfg.worker, RayDistributed)
        else instantiate(cfg.worker)
    )
    validate_type(worker, WorkerPool)

其中instantiate定义如下

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

def instantiate(config: Any, *args: Any, **kwargs: Any) -> Any:
    """
    :param config: An config object describing what to call and what params to use.
                   In addition to the parameters, the config must contain:
                   _target_ : target class or callable name (str)
                   And may contain:
                   _args_: List-like of positional arguments to pass to the target
                   _recursive_: Construct nested objects as well (bool).
                                True by default.
                                may be overridden via a _recursive_ key in
                                the kwargs
                   _convert_: Conversion strategy
                        none    : Passed objects are DictConfig and ListConfig, default
                        partial : Passed objects are converted to dict and list, with
                                  the exception of Structured Configs (and their fields).
                        all     : Passed objects are dicts, lists and primitives without
                                  a trace of OmegaConf containers
                   _args_: List-like of positional arguments
    :param args: Optional positional parameters pass-through
    :param kwargs: Optional named parameters to override
                   parameters in the config object. Parameters not present
                   in the config objects are being passed as is to the target.
                   IMPORTANT: dataclasses instances in kwargs are interpreted as config
                              and cannot be used as passthrough
    :return: if _target_ is a class name: the instantiated object
             if _target_ is a callable: the return value of the call
    """

其中cfg.worker假设使用的是ray_distributed.yaml

1
2
3
4
5
6
7

_target_: nuplan.planning.utils.multithreading.worker_ray.RayDistributed
_convert_: 'all'
master_node_ip: null    # Set to a master node IP if you desire to connect to cluster remotely
threads_per_node: null  # Number of CPU threads to use per node, "null" means all threads available
debug_mode: false       # If true all tasks will be executed serially, mainly for testing
log_to_driver: true     # If true, all printouts from ray threads will be displayed in driver
logs_subdir: 'logs'     # Subdirectory to store logs inside the experiment directory

OmegaConf

hydra传入的cfg文件为DictConfig，其使用OmegaConf模块管理多个config文件，但实际上就是字典类型，使用字典的函数来修改，增添或删除值。

1
2
3
4
5
6
7
8
9
10
11

if planners and 'planner' in cfg.keys():
      logger.info('Using pre-instantiated planner. Ignoring planner in config')
      OmegaConf.set_struct(cfg, False)  # 设置cfg可修改
      cfg.pop('planner') # 删除关键字以及其值
      OmegaConf.set_struct(cfg, True) # 设置cfg不可修改
if cfg.lightning.trainer.overfitting.enable:
        OmegaConf.set_struct(cfg, False)
        # 合并两个配置字典DictConfig
        params = OmegaConf.merge(params, cfg.lightning.trainer.overfitting.params)
        params.check_val_every_n_epoch = params.max_epochs + 1
        OmegaConf.set_struct(cfg, True)