Stable Baselines3 RuntimeError：mat1 和 mat2 必须具有相同的 dtype

Question

我正在尝试在 Stable Baselines3 中使用自定义环境实现 SAC，但我不断收到标题中的错误。任何非策略算法都会发生该错误，而不仅仅是 SAC。

追溯：

File "<MY PROJECT PATH>\src\main.py", line 70, in <module>
  main()
File "<MY PROJECT PATH>\src\main.py", line 66, in main
  model.learn(total_timesteps=timesteps, reset_num_timesteps=False, tb_log_name=f"sac_{num_cars}_cars")
File "<MY PROJECT PATH>\venv\lib\site-packages\stable_baselines3\sac\sac.py", line 309, in learn
  return super().learn(
File "<MY PROJECT PATH>\venv\lib\site-packages\stable_baselines3\common\off_policy_algorithm.py", line 375, in learn
  self.train(batch_size=self.batch_size, gradient_steps=gradient_steps)
File "<MY PROJECT PATH>\venv\lib\site-packages\stable_baselines3\sac\sac.py", line 256, in train
  current_q_values = self.critic(replay_data.observations, replay_data.actions)
File "<MY PROJECT PATH>\venv\lib\site-packages\torch\nn\modules\module.py", line 1190, in _call_impl
  return forward_call(*input, **kwargs)
File "<MY PROJECT PATH>\venv\lib\site-packages\stable_baselines3\common\policies.py", line 885, in forward
  return tuple(q_net(qvalue_input) for q_net in self.q_networks)
File "<MY PROJECT PATH>\venv\lib\site-packages\stable_baselines3\common\policies.py", line 885, in <genexpr>
  return tuple(q_net(qvalue_input) for q_net in self.q_networks)
File "<MY PROJECT PATH>\venv\lib\site-packages\torch\nn\modules\module.py", line 1190, in _call_impl
  return forward_call(*input, **kwargs)
File "<MY PROJECT PATH>\venv\lib\site-packages\torch\nn\modules\container.py", line 204, in forward
  input = module(input)
File "<MY PROJECT PATH>\venv\lib\site-packages\torch\nn\modules\module.py", line 1190, in _call_impl
  return forward_call(*input, **kwargs)
File "<MY PROJECT PATH>\venv\lib\site-packages\torch\nn\modules\linear.py", line 114, in forward
  return F.linear(input, self.weight, self.bias)
RuntimeError: mat1 and mat2 must have the same dtype

行动和观察空间：

self.action_space = Box(low=-1., high=1., shape=(2,), dtype=np.float)
self.observation_space = Box(
    np.array(
        [-np.inf] * (9 * 40) + [-np.inf] * 3 + [-np.inf] * 3 + [-np.inf] * 3
        + [0.] + [0.] + [0.] + [-1.] + [0.] * 4 + [0.] * 4 + [0.] * 4,
        dtype=np.float
    ),
    np.array(
        [np.inf] * (9 * 40) + [np.inf] * 3 + [np.inf] * 3 + [np.inf] * 3
        + [np.inf] + [1.] + [1.] + [1.] + [1.] * 4 + [np.inf] * 4 + [np.inf] * 4,
        dtype=np.float
    ),
    dtype=np.float
)

观察结果在步骤和重置方法中作为浮点数的 numpy 数组返回。

是否有我遗漏的东西导致了这个错误？如果我使用健身房附带的环境之一（例如钟摆），它可以正常工作，这就是为什么我认为我的自定义环境存在问题。

预先感谢您的任何帮助，如果需要更多信息，请告诉我。

Answer 1

将输入更改为 float32 ，默认加载程序将类型设置为 float64。

inputs = inputs.to(torch.float32)