小编Dan*_*l99的帖子

目标：修改此笔记本以与albert-base-v2模型一起使用

1.3 节中出现错误。

核心：conda_pytorch_p36。我重新启动并运行全部，并刷新了工作目录中的文件视图。

列出了 3 种可能导致此错误的方式。我不确定我的情况属于哪一种情况。

第 1.3 节：

# define the tokenizer
tokenizer = AutoTokenizer.from_pretrained(
        configs.output_dir, do_lower_case=configs.do_lower_case)

追溯：

---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-25-1f864e3046eb> in <module>
    140 # define the tokenizer
    141 tokenizer = AutoTokenizer.from_pretrained(
--> 142         configs.output_dir, do_lower_case=configs.do_lower_case)
    143 
    144 # Evaluate the original FP32 BERT model

~/anaconda3/envs/pytorch_p36/lib/python3.6/site-packages/transformers/models/auto/tokenization_auto.py in from_pretrained(cls, pretrained_model_name_or_path, *inputs, **kwargs)
    548             tokenizer_class_py, tokenizer_class_fast = TOKENIZER_MAPPING[type(config)]
    549             if tokenizer_class_fast and (use_fast or tokenizer_class_py is None):
--> 550 …

并非总是如此，但在运行我的代码时偶尔会出现此错误。

起初，我怀疑这是一个连接问题，而是与兑现问题有关，正如旧的Git Issue中所讨论的那样。

清除缓存对运行时没有帮助：

$ rm ~/.cache/huggingface/transformers/ *

回溯参考：

• NLTK 也得到Error loading stopwords: <urlopen error [Errno -2] Name or service not known.
• 最后 2 行 recached_path和get_from_cache.

缓存（清除之前）：

$ cd ~/.cache/huggingface/transformers/
(sdg) me@PF2DCSXD:~/.cache/huggingface/transformers$ ls
16a2f78023c8dc511294f0c97b5e10fde3ef9889ad6d11ffaa2a00714e73926e.cf2d0ecb83b6df91b3dbb53f1d1e4c311578bfd3aa0e04934215a49bf9898df0
16a2f78023c8dc511294f0c97b5e10fde3ef9889ad6d11ffaa2a00714e73926e.cf2d0ecb83b6df91b3dbb53f1d1e4c311578bfd3aa0e04934215a49bf9898df0.json
16a2f78023c8dc511294f0c97b5e10fde3ef9889ad6d11ffaa2a00714e73926e.cf2d0ecb83b6df91b3dbb53f1d1e4c311578bfd3aa0e04934215a49bf9898df0.lock
4029f7287fbd5fa400024f6bbfcfeae9c5f7906ea97afcaaa6348ab7c6a9f351.723d8eaff3b27ece543e768287eefb59290362b8ca3b1c18a759ad391dca295a.h5
4029f7287fbd5fa400024f6bbfcfeae9c5f7906ea97afcaaa6348ab7c6a9f351.723d8eaff3b27ece543e768287eefb59290362b8ca3b1c18a759ad391dca295a.h5.json
4029f7287fbd5fa400024f6bbfcfeae9c5f7906ea97afcaaa6348ab7c6a9f351.723d8eaff3b27ece543e768287eefb59290362b8ca3b1c18a759ad391dca295a.h5.lock
684fe667923972fb57f6b4dcb61a3c92763ad89882f3da5da9866baf14f2d60f.c7ed1f96aac49e745788faa77ba0a26a392643a50bb388b9c04ff469e555241f
684fe667923972fb57f6b4dcb61a3c92763ad89882f3da5da9866baf14f2d60f.c7ed1f96aac49e745788faa77ba0a26a392643a50bb388b9c04ff469e555241f.json
684fe667923972fb57f6b4dcb61a3c92763ad89882f3da5da9866baf14f2d60f.c7ed1f96aac49e745788faa77ba0a26a392643a50bb388b9c04ff469e555241f.lock
c0c761a63004025aeadd530c4c27b860ec4ecbe8a00531233de21d865a402598.5d12962c5ee615a4c803841266e9c3be9a691a924f72d395d3a6c6c81157788b
c0c761a63004025aeadd530c4c27b860ec4ecbe8a00531233de21d865a402598.5d12962c5ee615a4c803841266e9c3be9a691a924f72d395d3a6c6c81157788b.json
c0c761a63004025aeadd530c4c27b860ec4ecbe8a00531233de21d865a402598.5d12962c5ee615a4c803841266e9c3be9a691a924f72d395d3a6c6c81157788b.lock
fc674cd6907b4c9e933cb42d67662436b89fa9540a1f40d7c919d0109289ad01.7d2e0efa5ca20cef4fb199382111e9d3ad96fd77b849e1d4bed13a66e1336f51
fc674cd6907b4c9e933cb42d67662436b89fa9540a1f40d7c919d0109289ad01.7d2e0efa5ca20cef4fb199382111e9d3ad96fd77b849e1d4bed13a66e1336f51.json
fc674cd6907b4c9e933cb42d67662436b89fa9540a1f40d7c919d0109289ad01.7d2e0efa5ca20cef4fb199382111e9d3ad96fd77b849e1d4bed13a66e1336f51.lock

代码：

from transformers import pipeline, set_seed

generator = pipeline('text-generation', model='gpt2')  # Error
set_seed(42)

追溯：

2022-03-03 10:18:06.803989: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; …

我一直在研究 FastAI 模型，现在准备部署它。然而，我在通过 Docker 在本地运行模型时遇到了困难。

我不知道导致问题的原因以及如何解决它。我对 Docker 的经验很少。创建自己的 Dockerfile 对我来说是个新鲜事；所以我可能遗漏了明显的命令。

Dockerfile：

# Install image and dependencies
FROM python:3.9

# Set working directory
WORKDIR /app

# Copy the application files from the host to the container
COPY app.py /app/

# Install necessary packages
RUN apt-get update && \
    apt-get install -y --fix-missing && \
    pip install pip==20.2

# Install project dependencies
COPY requirements.txt /requirements.txt
RUN pip install -r /requirements.txt

# Download model artefacts from S3
RUN apt-get install -y awscli …

目标：最终，转换.geojson为.wkt. 在这里，我想提取所有coordinates，每个作为一个列表。

在my.geojson中，有n个：{"type":"Polygon","coordinates":...

更新：我已经成功提取了第一个coordinates. 但是，该文件有两个coordinates.

每个.geojson至少有 1 个coordinates，但可能有更多。

如何动态提取许多的所有键值coordinates？

代码：

from pathlib import Path
import os
import geojson
import json
from shapely.geometry import shape


ROOT = Path('path/')

all_files = os.listdir(ROOT)
geojson_files = list(filter(lambda f: f.endswith('.geojson'), all_files))

for gjf in geojson_files:
    with open(f'{str(ROOT)}/{gjf}') as f:
        gj = geojson.load(f)
    o = dict(coordinates = gj['features'][0]['geometry']['coordinates'], type = "Polygon")
    geom = shape(o)
    wkt …

目标：分成随机的小数点100后两位数字。5

def split(x, n):
 
    if(x < n):
        print(-1)
 
    elif (x % n == 0):
        for i in range(n):
            print(x//n, end =" ")
    else:
        zp = n - (x % n)
        pp = x//n
        for i in range(n):
            if(i>= zp):
                print(pp + 1, end =" ")
            else:
                print(pp, end =" ")
       
split(100, 5)
>>> 20 20 20 20 20 

[10.50, 22.98, 13.23, 40.33, 12.96]