内存分配、字节耗尽 PHP/LARAVEL

Question

大家好，

我正在使用Laravel Excel/Maatwebsite开发一个系统。我想要实现的是，当用户将 Excel 文件插入系统时，系统将检查一些内容，然后将数据插入数据库。

这是数据库模式的一个实例：

Hadees:
h_id | h_english | r_id | h_last_raavi_id | b_id | s_id | k_id | h_status

Raavi:
r_id | r_english | r_status

Baab:
b_id | b_english | s_id | b_status

Section:
s_id | s_english | k_id | s_status

Kitaab:
k_id | k_english | k_status

我的控制器：

use Illuminate\Http\Request;
use App\Http\Controllers\Controller;
use App\Models\Section;
use App\Models\Raavi;
use App\Imports\HadeesImport;
use Excel;

class ImportHadeesController extends Controller{
    /**
     * Show the application import-hadees page.
     *
     * @return \Illuminate\Http\Response
     */
    public function index(){
        $section = Section::where(['s_status' => 1])->get();

        return view('admin/import-hadees', compact('section'));
    }

    /**
     * This method uses the Excel facade to prep the excel file 
     * and extract data from it and uses App\Imports\HadeesImport 
     * class to insert each row in the database schema accordingly.
     * 
     * @param Illuminate\Http\Request $request
     * @return \Illuminate\Http\Response
     */
    public function importHadees(Request $request){
        $raavi = Raavi::where(['r_status' => 1])->get();

        if($request->file('hadees_sheet')) {
        } else {
           return response()->json(['status'=>'error', 'msg'=> 'No file present!']);
        }
    
        $temp = $request->file('hadees_sheet')->store('temp'); 
        $path=storage_path('app').'/'.$temp;

        $hadees = Excel::import(new HadeesImport($request->s_id, compact('raavi')), $path);

        if($hadees){
            return response()->json(['status'=>'success', 'msg'=> 'Successfully imported all the data from the file to the database!']);
        } else{
            return response()->json(['status'=>'error', 'msg'=> 'Unable to import data from the file to the database!']);
        }
    }
}

Hades导入类：

use Illuminate\Support\Collection;
use Maatwebsite\Excel\Concerns\ToCollection;
use Maatwebsite\Excel\Concerns\WithHeadingRow;
use App\Models\Section;
use App\Models\Baab;
use App\Models\Hadees;
use App\Models\Raavi;

class HadeesImport implements ToCollection, WithHeadingRow{
    /**
     * Global variable for section_id.
     */
    public $s_id;

    /**
     * Global variable for raavi's data.
     */
    public $raavi;

    /**
     * Construction function.
     * 
     * @param int $id
     */
    function __construct($id, $arr) {
        $this->s_id = $id;
        $this->raavi = $arr;
    }

    /**
    * This method is responsible for inserting the excel
    * sheet's rows data to the database schema.
    * 
    * @param Collection $row
    */
    public function collection(Collection $rows){
        $baab = Baab::where(['s_id' => $this->s_id])->get();
        $hissa = Section::where(['s_id' => $this->s_id])->first();
        $kitaab = $hissa->k_id;
        $first_raavi = 0;
        $last_raavi = 0;
        $baab_id = 0;
        $data = array();

        foreach ($rows as $row){
            if($row['hadees_arabic'] != "" && $row['hadees_urdu'] != ""){
                $baab_id = $this->baabCheck($baab, $row);
            
                foreach($this->raavi['raavi'] as $rav){
                    if(trim($rav->r_english) == trim($row['first_raavi_english'])){ 
                        $first_raavi = $rav->r_id; 
                    } else{
                        $first_raavi = 0;
                    }
    
                    $last_raavi = (trim($rav->r_english) == trim($row['last_raavi_english']))? $rav->r_id : 0;
                }

                if($first_raavi == 0){
                    $raavi = Raavi::create([
                        'r_arabic' => trim($row['first_raavi_urdu']),
                        'r_urdu' => trim($row['first_raavi_urdu']),
                        'r_english' => trim($row['first_raavi_english']),
                        'r_nickname' => trim($row['raavi_other_names']),
                        'r_status' => 1,
                    ]);

                    $first_raavi = $raavi->r_id;
                }

                if($last_raavi == 0){
                    $raavi = Raavi::create([
                        'r_arabic' => trim($row['last_raavi_urdu']),
                        'r_urdu' => trim($row['last_raavi_urdu']),
                        'r_english' => trim($row['last_raavi_english']),
                        'r_nickname' => 'n/a',
                        'r_status' => 1,
                    ]);

                    $last_raavi = $raavi->r_id;
                }

                $data = array([
                    'h_arabic' => trim($row['hadees_arabic']),
                    'h_urdu' => trim($row['hadees_urdu']),
                    'h_english' => trim($row['hadees_english']),
                    'h_roman_keywords' => trim($row['roman_keywords']),
                    'h_number' => trim($row['hadees_number']),
                    'r_id' => $first_raavi,
                    'h_last_raavi_id' => $last_raavi,
                    'b_id' => $baab_id,
                    's_id' => $this->s_id,
                    'k_id' => $kitaab,
                    'h_status' => 1
                ]);
            }
        }

        $hadees = Hadees::create($data);
    }

    /**
    * Checks if the baab exists in the database or not.
    * 
    * @param Collection $baab
    * @param Object $row
    * @return int - baad_id or 0
    */
    public function baabCheck($baab, $row){
        foreach($baab as $b){
            if(trim($b->b_arabic) == trim($row['baab_arabic']) || trim($b->b_urdu) == trim($row['baab_urdu']) || trim($b->b_english) == trim($row['baab_english'])){
                return $b->b_id;
            } else{
                return 0;
            }
        }
    }
}

当数据较少时，一切都运行良好。现在， Raavi表中有1400 多行，Baab表中有10,000多行。现在，每当我尝试将工作表导入系统时，都会出现以下错误：

允许的内存大小 268435456 字节已耗尽（尝试分配 37748736 字节）。

我认为这是因为foreach()循环太长。任何形式的帮助将不胜感激。如果你们对糟糕的编码或糟糕的逻辑构建有任何建议，请告诉我。我已经被这个问题困扰了快两天了。提前致谢。

Ps：错误在本地主机和托管上是相同的，只是字节不同。我相信这是由于不同的内存限制设置造成的。

Answer 1

所有发布的解决方案都提到提高 PHP 的内存限制。

它不是那样工作的。你不能只用更多的内存来解决问题。如果您的服务器有 2GB RAM，并且您上传的文件（其中创建的所有阵列）可以使用超过 2GB RAM，该怎么办？下一步是什么？更不用说服务器内存不足并杀死其他进程的风险。例如，如果服务器共享运行 PHP 和 MySQL，并且 PHP 导致服务器内存不足，则 OOM Killer 将启动并可能杀死 MySQL 进程。

问题的解决方案是分块处理该文件。例如，Laravel 6 有一种新的集合类型：Lazy Collections。他们可以帮助您加快速度。您的代码可能必须更改才能使用块处理，但恕我直言，这是解决此问题的唯一方法。

我还会在队列中运行它，而不是根据用户请求运行。

我还知道您使用的包支持读取分块和插入批处理。