macOS安装sentencepiece模块失败

Posted on In Symbols count in article: 4.1k Reading time ≈ 4 mins.
macOS安装sentencepiece模块失败

环境

1
2
3
4
5
6
7
macOS Monterey 12.2.1，Apple M1 Pro版本

$ python --version
Python 3.8.9

$ pip --version
pip 22.0.4 from /Users/gavin/Library/Python/3.8/lib/python/site-packages/pip (python 3.8)

问题

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
# 安装模块时报错，信息如下
$ pip install sentencepiece

Running setup.py install for sentencepiece ... error
  error: subprocess-exited-with-error

  × Running setup.py install for sentencepiece did not run successfully.
  │ exit code: 1
  ╰─> [25 lines of output]
      /Users/gavin/Library/Python/3.8/lib/python/site-packages/setuptools/dist.py:757: UserWarning: Usage of dash-separated 'description-file' will not be supported in future versions. Please use the underscore name 'description_file' instead
        warnings.warn(
      running install
      /Users/gavin/Library/Python/3.8/lib/python/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
        warnings.warn(
      running build
      running build_py
      creating build
      creating build/lib.macosx-10.14-arm64-cpython-38
      creating build/lib.macosx-10.14-arm64-cpython-38/sentencepiece
      copying src/sentencepiece/__init__.py -> build/lib.macosx-10.14-arm64-cpython-38/sentencepiece
      copying src/sentencepiece/sentencepiece_model_pb2.py -> build/lib.macosx-10.14-arm64-cpython-38/sentencepiece
      copying src/sentencepiece/sentencepiece_pb2.py -> build/lib.macosx-10.14-arm64-cpython-38/sentencepiece
      running build_ext
      /bin/sh: pkg-config: command not found
      mkdir: bundled: File exists
      fatal: destination path 'sentencepiece' already exists and is not an empty directory.
      fatal: destination path 'sentencepiece' already exists and is not an empty directory.
      mkdir: build: File exists
      ./build_bundled.sh: line 15: cmake: command not found
      ./build_bundled.sh: line 16: nproc: command not found
      make: *** No targets specified and no makefile found.  Stop.
      make: *** No rule to make target `install'.  Stop.
      env: pkg-config: No such file or directory
      Failed to find sentencepiece pkg-config
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

解决

既然无法在线安装，遂尝试下载安装包，本地安装。

下载安装包

登录官方下载页面，找到最新版本sentencepiece-0.1.96-cp39-cp39-macosx_10_6_x86_64.whl，下载，然后执行命令安装：

1
2
3
4
$ pip install sentencepiece-0.1.96-cp39-cp39-macosx_10_6_x86_64.whl
Defaulting to user installation because normal site-packages is not writeable
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
ERROR: sentencepiece-0.1.96-cp39-cp39-macosx_10_6_x86_64.whl is not a supported wheel on this platform.

报错平台不支持，而且过程中会到仓库中查找 index。因为我用的是清华的源，第一反应是清华的源仓库里没有这个版本，遂到清华仓库找了一下，有这个版本，名称也一模一样。

查了一下 cp39 对应的 python 版本，3.9才能安装。

我电脑安装的是 3.8，所以重新下载sentencepiece-0.1.96-cp38-cp38-macosx_10_6_x86_64.whl 版本。

重新执行pip install sentencepiece-0.1.96-cp38-cp38-macosx_10_6_x86_64.whl，仍然报错平台不支持。

解决版本问题

报错平台不支持，当前思路是查看当前电脑上 python 支持版本，查到一个方法，但是执行报错：

1
2
3
4
5
6
7
8
9
# python 环境中执行
import pip._internal
print(pip._internal.pep425tags.get_supported())

# 报错信息，模块中没有这个属性
Traceback (most recent call last):
  File "/Users/gavin/Downloads/PycharmProjects/pythonProject/version.py", line 2, in <module>
    print(pip._internal.pep425tags.get_supported())
AttributeError: module 'pip._internal' has no attribute 'pep425tags'

后来发现这个方法，查看当前 python 版本适配的标签：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
$ pip debug --verbose

# 输出
······
Compatible tags: 334
  cp38-cp38-macosx_12_0_arm64
  cp38-cp38-macosx_12_0_universal2
  cp38-cp38-macosx_11_0_arm64
  cp38-cp38-macosx_11_0_universal2
  cp38-cp38-macosx_10_16_universal2
  cp38-cp38-macosx_10_15_universal2
  cp38-cp38-macosx_10_14_universal2
  cp38-cp38-macosx_10_13_universal2
  cp38-cp38-macosx_10_12_universal2
  cp38-cp38-macosx_10_11_universal2
  cp38-cp38-macosx_10_10_universal2
  cp38-cp38-macosx_10_9_universal2
  cp38-cp38-macosx_10_8_universal2
  cp38-cp38-macosx_10_7_universal2
  cp38-cp38-macosx_10_6_universal2
  cp38-cp38-macosx_10_5_universal2
  cp38-cp38-macosx_10_4_universal2
  cp38-abi3-macosx_12_0_arm64
  cp38-abi3-macosx_12_0_universal2
  cp38-abi3-macosx_11_0_arm64
  cp38-abi3-macosx_11_0_universal2
  cp38-abi3-macosx_10_16_universal2
······

从中发现cp38-cp38-macosx_10_6_universal2标签，尝试将下载的包名修改为该标签：

1
2
3
$ cp sentencepiece-0.1.96-cp38-cp38-macosx_10_6_x86_64.whl sentencepiece-0.1.96-cp38-cp38-macosx_10_6_universal2.whl

$ pip install sentencepiece-0.1.96-cp38-cp38-macosx_10_6_universal2.whl

安装成功。