尝试使用 model.fit() 训练张量流模型时出现“ResourceExhaustedError:图形执行错误”

Aur*_*hie 4 python tensorflow google-colaboratory

几天前,我在第 12 纪元时遇到了同样的错误。这次,它发生在 1 号。我不知道为什么会发生这种情况,因为我没有对模型进行任何更改。X_train.max()我只是在按应有的比例缩放后将输入标准化为 1。

和贴片大小有关系吗?我应该减少它吗?

为什么会出现此错误以及如何修复它?

my_model.summary()

Model: "U-Net"
__________________________________________________________________________________________________
 Layer (type)                   Output Shape         Param #     Connected to                     
==================================================================================================
 input_6 (InputLayer)           [(None, 64, 64, 64,  0           []                               
                                 3)]                                                              
                                                                                                  
 conv3d_95 (Conv3D)             (None, 64, 64, 64,   5248        ['input_6[0][0]']                
                                64)                                                               
                                                                                                  
 batch_normalization_90 (BatchN  (None, 64, 64, 64,   256        ['conv3d_95[0][0]']              
 ormalization)                  64)                                                               
                                                                                                  
 activation_90 (Activation)     (None, 64, 64, 64,   0           ['batch_normalization_90[0][0]'] 
                                64)                                                               
                                                                                                  
 conv3d_96 (Conv3D)             (None, 64, 64, 64,   110656      ['activation_90[0][0]']          
                                64)                                                               
                                                                                                  
 batch_normalization_91 (BatchN  (None, 64, 64, 64,   256        ['conv3d_96[0][0]']              
 ormalization)                  64)                                                               
                                                                                                  
 activation_91 (Activation)     (None, 64, 64, 64,   0           ['batch_normalization_91[0][0]'] 
                                64)                                                               
                                                                                                  
 max_pooling3d_20 (MaxPooling3D  (None, 32, 32, 32,   0          ['activation_91[0][0]']          
 )                              64)                                                               
                                                                                                  
 conv3d_97 (Conv3D)             (None, 32, 32, 32,   221312      ['max_pooling3d_20[0][0]']       
                                128)                                                              
                                                                                                  
 batch_normalization_92 (BatchN  (None, 32, 32, 32,   512        ['conv3d_97[0][0]']              
 ormalization)                  128)                                                              
                                                                                                  
 activation_92 (Activation)     (None, 32, 32, 32,   0           ['batch_normalization_92[0][0]'] 
                                128)                                                              
                                                                                                  
 conv3d_98 (Conv3D)             (None, 32, 32, 32,   442496      ['activation_92[0][0]']          
                                128)                                                              
                                                                                                  
 batch_normalization_93 (BatchN  (None, 32, 32, 32,   512        ['conv3d_98[0][0]']              
 ormalization)                  128)                                                              
                                                                                                  
 activation_93 (Activation)     (None, 32, 32, 32,   0           ['batch_normalization_93[0][0]'] 
                                128)                                                              
                                                                                                  
 max_pooling3d_21 (MaxPooling3D  (None, 16, 16, 16,   0          ['activation_93[0][0]']          
 )                              128)                                                              
                                                                                                  
 conv3d_99 (Conv3D)             (None, 16, 16, 16,   884992      ['max_pooling3d_21[0][0]']       
                                256)                                                              
                                                                                                  
 batch_normalization_94 (BatchN  (None, 16, 16, 16,   1024       ['conv3d_99[0][0]']              
 ormalization)                  256)                                                              
                                                                                                  
 activation_94 (Activation)     (None, 16, 16, 16,   0           ['batch_normalization_94[0][0]'] 
                                256)                                                              
                                                                                                  
 conv3d_100 (Conv3D)            (None, 16, 16, 16,   1769728     ['activation_94[0][0]']          
                                256)                                                              
                                                                                                  
 batch_normalization_95 (BatchN  (None, 16, 16, 16,   1024       ['conv3d_100[0][0]']             
 ormalization)                  256)                                                              
                                                                                                  
 activation_95 (Activation)     (None, 16, 16, 16,   0           ['batch_normalization_95[0][0]'] 
                                256)                                                              
                                                                                                  
 max_pooling3d_22 (MaxPooling3D  (None, 8, 8, 8, 256  0          ['activation_95[0][0]']          
 )                              )                                                                 
                                                                                                  
 conv3d_101 (Conv3D)            (None, 8, 8, 8, 512  3539456     ['max_pooling3d_22[0][0]']       
                                )                                                                 
                                                                                                  
 batch_normalization_96 (BatchN  (None, 8, 8, 8, 512  2048       ['conv3d_101[0][0]']             
 ormalization)                  )                                                                 
                                                                                                  
 activation_96 (Activation)     (None, 8, 8, 8, 512  0           ['batch_normalization_96[0][0]'] 
                                )                                                                 
                                                                                                  
 conv3d_102 (Conv3D)            (None, 8, 8, 8, 512  7078400     ['activation_96[0][0]']          
                                )                                                                 
                                                                                                  
 batch_normalization_97 (BatchN  (None, 8, 8, 8, 512  2048       ['conv3d_102[0][0]']             
 ormalization)                  )                                                                 
                                                                                                  
 activation_97 (Activation)     (None, 8, 8, 8, 512  0           ['batch_normalization_97[0][0]'] 
                                )                                                                 
                                                                                                  
 max_pooling3d_23 (MaxPooling3D  (None, 4, 4, 4, 512  0          ['activation_97[0][0]']          
 )                              )                                                                 
                                                                                                  
 conv3d_103 (Conv3D)            (None, 4, 4, 4, 102  14156800    ['max_pooling3d_23[0][0]']       
                                4)                                                                
                                                                                                  
 batch_normalization_98 (BatchN  (None, 4, 4, 4, 102  4096       ['conv3d_103[0][0]']             
 ormalization)                  4)                                                                
                                                                                                  
 activation_98 (Activation)     (None, 4, 4, 4, 102  0           ['batch_normalization_98[0][0]'] 
                                4)                                                                
                                                                                                  
 conv3d_104 (Conv3D)            (None, 4, 4, 4, 102  28312576    ['activation_98[0][0]']          
                                4)                                                                
                                                                                                  
 batch_normalization_99 (BatchN  (None, 4, 4, 4, 102  4096       ['conv3d_104[0][0]']             
 ormalization)                  4)                                                                
                                                                                                  
 activation_99 (Activation)     (None, 4, 4, 4, 102  0           ['batch_normalization_99[0][0]'] 
                                4)                                                                
                                                                                                  
 conv3d_transpose_20 (Conv3DTra  (None, 8, 8, 8, 512  4194816    ['activation_99[0][0]']          
 nspose)                        )                                                                 
                                                                                                  
 concatenate_20 (Concatenate)   (None, 8, 8, 8, 102  0           ['conv3d_transpose_20[0][0]',    
                                4)                                'activation_97[0][0]']          
                                                                                                  
 conv3d_105 (Conv3D)            (None, 8, 8, 8, 512  14156288    ['concatenate_20[0][0]']         
                                )                                                                 
                                                                                                  
 batch_normalization_100 (Batch  (None, 8, 8, 8, 512  2048       ['conv3d_105[0][0]']             
 Normalization)                 )                                                                 
                                                                                                  
 activation_100 (Activation)    (None, 8, 8, 8, 512  0           ['batch_normalization_100[0][0]']
                                )                                                                 
                                                                                                  
 conv3d_106 (Conv3D)            (None, 8, 8, 8, 512  7078400     ['activation_100[0][0]']         
                                )                                                                 
                                                                                                  
 batch_normalization_101 (Batch  (None, 8, 8, 8, 512  2048       ['conv3d_106[0][0]']             
 Normalization)                 )                                                                 
                                                                                                  
 activation_101 (Activation)    (None, 8, 8, 8, 512  0           ['batch_normalization_101[0][0]']
                                )                                                                 
                                                                                                  
 conv3d_transpose_21 (Conv3DTra  (None, 16, 16, 16,   1048832    ['activation_101[0][0]']         
 nspose)                        256)                                                              
                                                                                                  
 concatenate_21 (Concatenate)   (None, 16, 16, 16,   0           ['conv3d_transpose_21[0][0]',    
                                512)                              'activation_95[0][0]']          
                                                                                                  
 conv3d_107 (Conv3D)            (None, 16, 16, 16,   3539200     ['concatenate_21[0][0]']         
                                256)                                                              
                                                                                                  
 batch_normalization_102 (Batch  (None, 16, 16, 16,   1024       ['conv3d_107[0][0]']             
 Normalization)                 256)                                                              
                                                                                                  
 activation_102 (Activation)    (None, 16, 16, 16,   0           ['batch_normalization_102[0][0]']
                                256)                                                              
                                                                                                  
 conv3d_108 (Conv3D)            (None, 16, 16, 16,   1769728     ['activation_102[0][0]']         
                                256)                                                              
                                                                                                  
 batch_normalization_103 (Batch  (None, 16, 16, 16,   1024       ['conv3d_108[0][0]']             
 Normalization)                 256)                                                              
                                                                                                  
 activation_103 (Activation)    (None, 16, 16, 16,   0           ['batch_normalization_103[0][0]']
                                256)                                                              
                                                                                                  
 conv3d_transpose_22 (Conv3DTra  (None, 32, 32, 32,   262272     ['activation_103[0][0]']         
 nspose)                        128)                                                              
                                                                                                  
 concatenate_22 (Concatenate)   (None, 32, 32, 32,   0           ['conv3d_transpose_22[0][0]',    
                                256)                              'activation_93[0][0]']          
                                                                                                  
 conv3d_109 (Conv3D)            (None, 32, 32, 32,   884864      ['concatenate_22[0][0]']         
                                128)                                                              
                                                                                                  
 batch_normalization_104 (Batch  (None, 32, 32, 32,   512        ['conv3d_109[0][0]']             
 Normalization)                 128)                                                              
                                                                                                  
 activation_104 (Activation)    (None, 32, 32, 32,   0           ['batch_normalization_104[0][0]']
                                128)                                                              
                                                                                                  
 conv3d_110 (Conv3D)            (None, 32, 32, 32,   442496      ['activation_104[0][0]']         
                                128)                                                              
                                                                                                  
 batch_normalization_105 (Batch  (None, 32, 32, 32,   512        ['conv3d_110[0][0]']             
 Normalization)                 128)                                                              
                                                                                                  
 activation_105 (Activation)    (None, 32, 32, 32,   0           ['batch_normalization_105[0][0]']
                                128)                                                              
                                                                                                  
 conv3d_transpose_23 (Conv3DTra  (None, 64, 64, 64,   65600      ['activation_105[0][0]']         
 nspose)                        64)                                                               
                                                                                                  
 concatenate_23 (Concatenate)   (None, 64, 64, 64,   0           ['conv3d_transpose_23[0][0]',    
                                128)                              'activation_91[0][0]']          
                                                                                                  
 conv3d_111 (Conv3D)            (None, 64, 64, 64,   221248      ['concatenate_23[0][0]']         
                                64)                                                               
                                                                                                  
 batch_normalization_106 (Batch  (None, 64, 64, 64,   256        ['conv3d_111[0][0]']             
 Normalization)                 64)                                                               
                                                                                                  
 activation_106 (Activation)    (None, 64, 64, 64,   0           ['batch_normalization_106[0][0]']
                                64)                                                               
                                                                                                  
 conv3d_112 (Conv3D)            (None, 64, 64, 64,   110656      ['activation_106[0][0]']         
                                64)                                                               
                                                                                                  
 batch_normalization_107 (Batch  (None, 64, 64, 64,   256        ['conv3d_112[0][0]']             
 Normalization)                 64)