如何调整R中h2o.grid中的hidden_​​dropout_ratios

Question

我想在R中使用h2o调整神经网络.这里我提供了一个可重现的虹膜数据集示例.我正在避免调整eta和epsiplon(即ADADELTA超参数),其唯一目的是使计算更快.

require(h2o)
h2o.init()
data(iris)
iris = iris[sample(1:nrow(iris)), ]
irisTrain = as.h2o(iris[1:90, ])
irisValid = as.h2o(iris[91:120, ])
irisTest = as.h2o(iris[121:150, ])
hyper_params <- list(
    input_dropout_ratio = list(0, 0.15, 0.3),
    hidden_dropout_ratios = list(0, 0.15, 0.3, c(0,0), c(0.15,0.15),c(0.3,0.3)),
    hidden = list(64, c(32,32)))
grid = h2o.grid("deeplearning", x=colnames(iris)[1:4], y=colnames(iris)[5],
                training_frame = irisTrain, validation_frame = irisValid,
                hyper_params = hyper_params, adaptive_rate = TRUE,
                variable_importances = TRUE, epochs = 50, stopping_rounds=5,
                stopping_tolerance=0.01, activation=c("RectifierWithDropout"),
                seed=1, reproducible=TRUE)

输出是:

Details: ERRR on field: _hidden_dropout_ratios: Must have 1 hidden layer dropout ratios.

问题在于hidden_dropout_ratios.请注意,我为input_dropout_ratio和hidden_​​dropout_ratios包含0,因为我还想测试激活函数而不会丢失.我知道我可以使用,activation="Rectifier但我认为我的配置应该会导致相同的结果.hidden_dropout_ratios在调整具有不同层数的架构时,如何调整？

尝试1:不成功,我没有调整hidden.

hyper_params <- list(
    input_dropout_ratio = c(0, 0.15, 0.3),
    hidden_dropout_ratios = list(c(0.3,0.3), c(0.5,0.5)),
    hidden = c(32,32))
ERRR on field: _hidden_dropout_ratios: Must have 1 hidden layer dropout ratios.

尝试2:成功,但我没有调整hidden.

hyper_params <- list(
    input_dropout_ratio = c(0, 0.15, 0.3),
    hidden_dropout_ratios = c(0.3,0.3),
    hidden = c(32,32))

Answer 1

如果尝试使用hidden_​​dropout_ratios,则必须修复网格中隐藏层的数量.起初我搞砸了多个网格; 然后,在研究我的H2O书时,我看到有人提到,如果你给它们相同的名字,网格会自动合并.

因此,您仍然需要调用h2o.grid()每个隐藏层数,但它们最终都可以位于同一个网格中.以下是为此修改的示例:

require(h2o)
h2o.init()
data(iris)
iris = iris[sample(1:nrow(iris)), ]
irisTrain = as.h2o(iris[1:90, ])
irisValid = as.h2o(iris[91:120, ])
irisTest = as.h2o(iris[121:150, ])

hyper_params1 <- list(
    input_dropout_ratio = c(0, 0.15, 0.3),
    hidden_dropout_ratios = list(0, 0.15, 0.3),
    hidden = list(64)
    )

hyper_params2 <- list(
    input_dropout_ratio = c(0, 0.15, 0.3),
    hidden_dropout_ratios = list(c(0,0), c(0.15,0.15),c(0.3,0.3)),
    hidden = list(c(32,32))
    )

grid = h2o.grid("deeplearning", x=colnames(iris)[1:4], y=colnames(iris)[5],
    grid_id = "stackoverflow",
    training_frame = irisTrain, validation_frame = irisValid,
    hyper_params = hyper_params1, adaptive_rate = TRUE,
    variable_importances = TRUE, epochs = 50, stopping_rounds=5,
    stopping_tolerance=0.01, activation=c("RectifierWithDropout"),
    seed=1, reproducible=TRUE)

grid = h2o.grid("deeplearning", x=colnames(iris)[1:4], y=colnames(iris)[5],
    grid_id = "stackoverflow",
    training_frame = irisTrain, validation_frame = irisValid,
    hyper_params = hyper_params2, adaptive_rate = TRUE,
    variable_importances = TRUE, epochs = 50, stopping_rounds=5,
    stopping_tolerance=0.01, activation=c("RectifierWithDropout"),
    seed=1, reproducible=TRUE)

当我去打印网格时,我被提醒在使用列表超参数时存在网格输出错误,例如hidden或hidden_​​dropout_ratios.你的代码是一个很好的自包含示例,所以我现在就报告.与此同时,这里有一个单行显示每个对应的超参数的值:

sapply(models, function(m) c(
  paste(m@parameters$hidden, collapse = ","),
  paste(m@parameters$hidden_dropout_ratios, collapse=",")
  ))

这使:

     [,1]    [,2] [,3]        [,4]   [,5]      [,6] 
[1,] "32,32" "64" "32,32"     "64"   "32,32"   "64" 
[2,] "0,0"   "0"  "0.15,0.15" "0.15" "0.3,0.3" "0.3"

也就是说,没有隐藏的辍学比一点好,这比好多了.两个隐藏层比一个好.

顺便说说,

• input_dropout_ratio:控制输入图层和第一个隐藏图层之间的丢失.可以独立于激活功能使用.
• hidden_dropout_ratios:控制每个隐藏层和下一个图层(下一个隐藏图层或输出图层)之间的丢失.如果已指定,则必须指定"WithDropout"激活功能之一.