原文:我最近开始从我的一些旧代码中获取MySQL OperationalErrors,似乎无法追溯问题.由于它之前的工作,我认为它可能是一个破坏了一些东西的软件更新.我正在使用带有nginx的django runfcgi的python 2.7.这是我的原始代码:
views.py
DBNAME = "test"
DBIP = "localhost"
DBUSER = "django"
DBPASS = "password"
db = MySQLdb.connect(DBIP,DBUSER,DBPASS,DBNAME)
cursor = db.cursor()
def list(request):
statement = "SELECT item from table where selected = 1"
cursor.execute(statement)
results = cursor.fetchall()
我尝试了以下内容,但它仍然无效:
views.py
class DB:
conn = None
DBNAME = "test"
DBIP = "localhost"
DBUSER = "django"
DBPASS = "password"
def connect(self):
self.conn = MySQLdb.connect(DBIP,DBUSER,DBPASS,DBNAME)
def cursor(self):
try:
return self.conn.cursor()
except (AttributeError, MySQLdb.OperationalError):
self.connect()
return self.conn.cursor()
db = DB()
cursor = db.cursor() …假设我打开一个文件open().那我是fork()我的节目.
父和子现在会共享文件描述符的相同偏移量吗？
我的意思是如果我在父亲那里写作,那么儿童的偏移量也会改变吗？
或者补偿是否会在fork()？之后独立？
我一直在解决这个问题.我设置了一台全新的机器.我已经安装了postgresql和我所有其他依赖项的新副本.基本上,我随机地将这些数据库断开连接.我可以执行相同的请求,无论是有效还是无效.外观非常不确定.在Postgresql上查看日志,它甚至没有连接.现在,我希望如果它从未连接过,我会在建立连接并获取光标时遇到此问题,但是我在以后尝试实际使用连接时会得到它.鉴于下面的回溯,我希望看到在pg日志中建立连接,然后由于某种原因断开连接.我没有,所以我想知道是否有一些不匹配的线索.
Traceback (most recent call last):
File "/usr/local/lib/python2.6/dist-packages/Django-1.1-py2.6.egg/django/core/handlers/wsgi.py", line 242, in __call__
response = self.get_response(request)
File "/usr/local/lib/python2.6/dist-packages/Django-1.1-py2.6.egg/django/core/handlers/base.py", line 73, in get_response
response = middleware_method(request)
File "/usr/local/lib/python2.6/dist-packages/Django-1.1-py2.6.egg/django/middleware/locale.py", line 16, in process_request
language = translation.get_language_from_request(request)
File "/usr/local/lib/python2.6/dist-packages/Django-1.1-py2.6.egg/django/utils/translation/__init__.py", line 97, in get_language_from_request
return real_get_language_from_request(request)
File "/usr/local/lib/python2.6/dist-packages/Django-1.1-py2.6.egg/django/utils/translation/trans_real.py", line 349, in get_language_from_request
lang_code = request.session.get('django_language', None)
File "/usr/local/lib/python2.6/dist-packages/Django-1.1-py2.6.egg/django/contrib/sessions/backends/base.py", line 63, in get
return self._session.get(key, default)
File "/usr/local/lib/python2.6/dist-packages/Django-1.1-py2.6.egg/django/contrib/sessions/backends/base.py", line 172, in _get_session
self._session_cache = self.load()
File "/usr/local/lib/python2.6/dist-packages/Django-1.1-py2.6.egg/django/contrib/sessions/backends/db.py", line 16, in load
expire_date__gt=datetime.datetime.now()
File "/usr/local/lib/python2.6/dist-packages/Django-1.1-py2.6.egg/django/db/models/manager.py", line 120, …所以,我正在尝试编写一个使用django作为其ORM的应用程序,因为它需要做一些幕后处理和易于使用的前端.它的核心功能是处理数据库中的数据,在高CPU过程中(基本上是monte carlo模拟),我想实现多处理,特别是使用Pool(我得到4个进程).基本上我的代码就是这样运行的,父代大约有20个孩子:
assorted import statements to get the django environment in the script
from multiprocessing import Pool
from random import random
from time import sleep
def test(child):
x=[]
print child.id
for i in range(100):
print child.id, i
x.append(child.parent.id) #just to hit the DB
return x
if __name__ == '__main__':
parent = Parent.objects.get(id=1)
pool = Pool()
results = []
results = pool.map(test,parent.children.all())
pool.close()
pool.join()
print results
使用这样的代码,我得到间歇性的DatabaseErrors或PicklingErrors.前者通常是"格式错误的数据库"或"与MySQL服务器失去连接"的形式,后者通常是"不能泡菜型号.DoesNotExist".它们是随机的,随任何进程发生,当然DB本身没有任何问题.如果我设置pool = Pool(proccesses=1)它然后运行,在一个线程就好了.我还会抛出各种打印语句,以确保它们中的大多数实际上都在运行.
我也一直在test改为:
def test(child):
x=[] …当我使用 Python 多处理访问 MySQL 数据库时，总是出现错误：
OperationalError: (2006, 'MySQL 服务器已经消失')
查询期间与 MySQL 服务器的连接丢失
如果有人能向我解释这一点，我将不胜感激。
这是我的代码：
class MetricSource:
def __init__(self, task):
self.task = task
def get_next_task_time(self):
try:
task_id = self.task.id
next_task = Task.objects.get(id=task_id)
next_time = next_task.last_success_time
except Task.DoesNotExist as e:
print 'Find Exception: %d' % self.task.id
def access_queue(input_queue, output_queue):
while True:
try:
metric_source = input_queue.get(timeout=0.5)
metric_source.get_next_task_time()
output_queue.put(metric_source)
except Queue.Empty:
print "Queue Empty Error"
continue
class Command(BaseCommand):
def handle(self, *args, **options):
self.manager = multiprocessing.Manager()
self.input_queue = self.manager.Queue()
self.output_queue = self.manager.Queue()
self.init_data()
for i …我正在尝试通过我的应用程序的多重处理进行数据库连接。我假设我调用函数的地方发生了错误。这几天我一直在为多处理而苦苦挣扎。我从https://engineering.talentpair.com/django-multiprocessing-153dbcf51dab/找到了代码
我的代码如下
视图.py
from src.conf.multiproces import MultiProcess
class GridSearchAPI(generics.GenericAPIView):
@transaction.atomic
def post(self, request):
with MultiProcess() as mp:
mp.map(request)
returnData = mp.results()
if returnData['result']:
return Response(returnData, status=status.HTTP_200_OK)
else:
returnData["message"] = "no data found"
return Response(result, status=status.HTTP_400_BAD_REQUEST)
多进程.py
from __future__ import print_function, absolute_import, unicode_literals
import time
try:
from Queue import Empty
except ImportError:
from queue import Empty
# from pathos import
from multiprocessing import Process,cpu_count, Manager
import logging
import traceback
from django import db
from django.conf import settings
from django.core.cache …我使用 Jupyter Notebook 来处理我存储在 django/postgres 中的数据。我以这种方式初始化我的项目：
sys.path.append('/srv/gr/prg')
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'prg.settings')
if 'setup' in dir(django):
django.setup()
有许多更新数据的独立进程，我想多线程处理它以加快进程。当我在单个线程中进行更新或使用时，一切正常sqlite.
def extract_org_description(id):
o = models.Organization.objects.get(pk=id)
logging.info("Looking for description for %s" % o.symbol)
try:
content = open('/srv/data/%s.html' % o.symbol)
except FileNotFoundError:
logging.error("HTML file not found for %s" % o.symbol)
return
doc = BeautifulSoup(content, 'html.parser')
desc = doc.select("#cr_description_mod > div.cr_expandBox > div.cr_description_full.cr_expand")
if not desc or not desc[0]:
logging.info("Cannot not find description for %s" % o.symbol)
return
o.description = desc[0].text
o.save(update_fields=['description'])
logging.info("Description for %s found" …