标签: dask-gateway

Dask Gateway,设置worker资源

我正在尝试按照此处的文档为工作人员设置资源,但设置使用 Dask Gateway。具体来说,我希望能够遵循这个问题的答案,但使用 Dask Gateway。

我无法在ClusterConfig 选项中找到对工作线程资源的引用,并且我尝试了以下操作(按照此答案),但这似乎不起作用:

def set_resources(dask_worker):
    dask_worker.set_resources(task_limit=1)
    return dask_worker.available_resources, dask_worker.total_resources

client.run(set_resources)

# output from a 1 worker cluster
> {'tls://255.0.91.211:39302': ({}, {})}


# checking info known by scheduler
cluster.scheduler_info

> {'type': 'Scheduler',
 'id': 'Scheduler-410438c9-6b3a-494d-974a-52d9e9fss121',
 'address': 'tls://255.0.44.161:8786',
 'services': {'dashboard': 8787, 'gateway': 8788},
 'started': 1632434883.9022279,
 'workers': {'tls://255.0.92.232:39305': {'type': 'Worker',
   'id': 'dask-worker-f95c163cf41647c6a6d85da9efa9919b-wvnf6',
   'host': '255.0.91.211',
   'resources': {}, #### still {} empty dict
   'local_directory': '/home/jovyan/dask-worker-space/worker-ir8tpkz_',
   'name': 'dask-worker-f95c157cf41647c6a6d85da9efa9919b-wvnf6',
   'nthreads': 4,
   'memory_limit': 6952476672,
   'services': {'dashboard': 8787},
   'nanny': …
Run Code Online (Sandbox Code Playgroud)

dask dask-kubernetes dask-gateway

5
推荐指数
1
解决办法
491
查看次数

标签 统计

dask ×1

dask-gateway ×1

dask-kubernetes ×1