Sphinx即使长时间没有活动,我如何保持连接活动？

Question

我是做bulk inserts在RealTime Index使用PHP和禁用AUTOCOMIT,如

// sphinx connection
$sphinxql = mysqli_connect($sphinxql_host.':'.$sphinxql_port,'',''); 

//do some other time consuming work

//sphinx start transaction
mysqli_begin_transaction($sphinxql);

//do 50k updates or inserts

// Commit transaction
mysqli_commit($sphinxql);

在我看到的早晨,让剧本一夜之间保持运行

PHP Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate
212334 bytes) in

所以当我仔细检查nohup.out文件时,我注意到,这些线条,

PHP Warning: mysqli_query(): MySQL server has gone away in /home/script.php on line 502
Warning: mysqli_query(): MySQL server has gone away in /home/script.php on line 502

这些行之前的内存使用率是正常的,但这些行之后的内存使用量开始增加,并且它击中php mem_limit并给了PHP Fatal error并且死了.

in script.php , line 502 is

mysqli_query($sphinxql,$update_query_sphinx);

所以我的猜测是,狮身人面像服务器在几小时/几分钟不活动后关闭/死亡.

我试过在sphinx.conf中设置

client_timeout = 3600

重新开始搜索

systemctl restart searchd

而且我仍面临同样的问题.

那么,如果长时间没有活动,我怎么能不让狮身人面像服务器死在我身上呢？

---

更多信息添加 -

我一次从50k块中获取mysql的数据,并执行while循环以获取每一行并在sphinx RT索引中更新它.像这样

//6mil rows update in mysql, so it takes around 18-20 minutes to complete this then comes this following part.

$subset_count = 50000 ;

$total_count_query = "SELECT COUNT(*) as total_count FROM content WHERE enabled = '1'" ;
$total_count = mysqli_query ($conn,$total_count_query);
$total_count = mysqli_fetch_assoc($total_count);
$total_count = $total_count['total_count'];

$current_count = 0;

while ($current_count <= $total_count){

$get_mysql_data_query = "SELECT record_num, views , comments, votes FROM content WHERE enabled = 1  ORDER BY record_num ASC LIMIT $current_count , $subset_count ";

//sphinx start transaction
mysqli_begin_transaction($sphinxql);

if ($result = mysqli_query($conn, $get_mysql_data_query)) {

    /* fetch associative array */
    while ($row = mysqli_fetch_assoc($result)) {

    //sphinx escape whole array
    $escaped_sphinx = mysqli_real_escape_array($sphinxql,$row);

    //update data in sphinx index
    $update_query_sphinx = "UPDATE $sphinx_index  
    SET 
        views       = ".$escaped_sphinx['views']." , 
        comments    = ".$escaped_sphinx['comments']." , 
        votes   = ".$escaped_sphinx['votes']." 
    WHERE 
        id          = ".$escaped_sphinx['record_num']." ";  

    mysqli_query ($sphinxql,$update_query_sphinx);

    }

    /* free result set */
    mysqli_free_result($result);
}
// Commit transaction
mysqli_commit($sphinxql);

$current_count = $current_count + $subset_count ;
}

Answer 1

您需要在之前重新连接或重新启动数据库会话mysqli_begin_transaction($sphinxql)

像这样的东西。

<?php

//reconnect to spinx if it is disconnected due to timeout or whatever , or force reconnect
function sphinxReconnect($force = false) {
    global $sphinxql_host;
    global $sphinxql_port;
    global $sphinxql;
    if($force){
        mysqli_close($sphinxql);
        $sphinxql = @mysqli_connect($sphinxql_host.':'.$sphinxql_port,'','') or die('ERROR'); 
    }else{
        if(!mysqli_ping($sphinxql)){
            mysqli_close($sphinxql);
            $sphinxql = @mysqli_connect($sphinxql_host.':'.$sphinxql_port,'','') or die('ERROR'); 
        }
    }
}



//10mil+ rows update in mysql, so it takes around 18-20 minutes to complete this then comes this following part.

//reconnect to sphinx
sphinxReconnect(true);

//sphinx start transaction
mysqli_begin_transaction($sphinxql);

//do your otherstuff

// Commit transaction
mysqli_commit($sphinxql);