2021年9月26日日曜日

【Keras2.6】ImportError: cannot import name 'Adam' from 'keras.optimizers'

keras2.6でAdamをインポートすると以下のエラーがでました

ImportError: cannot import name 'Adam' from 'keras.optimizers'

以前のバージョンから変わったようです。

ProgrammerAHのブログを参考して以下の通りに修正

(修正前)
from keras.optimizers import Adam

(修正後)
from keras.optimizers import adam_v2

実際Adamを使用しているところも合わせて修正

(修正前)
model.compile(loss='categorical_crossentropy', optimizer=Adam(), metrics=['accuracy']) 

(修正後)
model.compile(loss='categorical_crossentropy', optimizer=adam_v2.Adam(), metrics=['accuracy']) 

参考書とのバージョン差異がポツリポツリと・・・勉強中のみとしてはこういうのは辛いかも。

2021年9月21日火曜日

keras2.6でAttributeError: module 'keras.optimizers' has no attribute 'SGD'

参考書通りにコーディングしてみたところエラー・・・・。
たしかに環境は違うけれど困る。 

【エラーメッセージ】
>>> sgd = optimizers.SGD(lr=0.01, clipnorm=1.)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: module 'keras.optimizers' has no attribute 'SGD'


>>> import keras
>>> keras.__version__
'2.6.0'>>> from keras import optimizers

dirコマンドでoptimizersを確認すると確かにSGDはなく、仕様が変更されたのかな・・・。
検索しても意外に情報が少なくもしかしてあたりまえ?常識

とりあえず無事別の方法を発見、ついでにオプションのlrは非推奨になったようで変わりにlearning_rateを使用するらしい。

【代替え?】
>>> sgd = optimizers.gradient_descent_v2.SGD(lr=0.001, clipnorm=1.)
/home/xx/.local/lib/python3.7/site-packages/keras/optimizer_v2/optimizer_v2.py:356: UserWarning: The `lr` argument is deprecated, use `learning_rate` instead.
  "The `lr` argument is deprecated, use `learning_rate` instead.")

--------------------------------------------

【追記】

kerasはtensorflow.kerasとしてTensorFlow2にで組み込まれたようです。単独のkerasの使い方が変わっているようです。

2021年9月19日日曜日

【TensorFlow】Chromebook(ARM64)のLinux(Debian)にPython3.7+TensorFlow2.6.0+ keras

 Pandasのインストールも問題なく終わったので早速TensorFlowをインストールしようとpipコマンドで実行してみると。

pip install tensorflow

Defaulting to user installation because normal site-packages is not writeable

ERROR: Could not find a version that satisfies the requirement tensorflow (from versions: none)

ERROR: No matching distribution found for tensorflow

対象のバージョンがない?

調べてみるとこういうケースはあるようでPtyhon3.7は以前は対応していなかったらしいですが調べていくうちに現在は対応していることは判明、一安心。

どうやら自分の環境に合うパッケージ(whl)ファイルをダウンロードしてインストールすれば良さそうということが分かりました。

■端末構成

ASUS Chromebook Detachable CM3

$cat /etc/issue
Debian GNU/Linux 10 \n \l

コードネームはbuster

$lscpu
Architecture:        aarch64
Byte Order:          Little Endian
CPU(s):              8
On-line CPU(s) list: 0-7
Thread(s) per core:  1
Core(s) per socket:  4
Socket(s):           2
Vendor ID:           ARM
Model:               4
Model name:          Cortex-A53
Stepping:            r0p4
BogoMIPS:            26.00
Flags:               fp asimd evtstrm aes pmull sha1 sha2 crc32 cpuid

Python 3.7.3 (default, Jan 22 2021, 20:04:44)

■numpyの最新化

ダウンロードするパッケージを選ぶ際にnumpyのバージョンらしい数字で分れていたので事前に最新化
・バージョン確認
$pip list --outdated
Package           Version Latest Type
----------------- ------- ------ -----
numpy             1.20.3  1.21.2 wheel

・バージョンアップ
$pip install --upgrade numpy

■ TensorFlowパッケージファイルダウンロード

ダウンロードサイト:https://github.com/PINTO0309/Tensorflow-bin

事前に調べたArchitectureとPythonのバージョンとnumpyのバージョンからtensorflow-2.6.0-cp37-none-linux_aarch64_numpy1211_download.shを選択

リンクからwhlファイルをダウンロードできるわけではなさそうで上記のshファイルをダウンロードするらしい。
ただダウンロード方法が分からずリンク先にあるshファイルのソースをコピーして適当に作成したローカルファイル(/tmp/download.sh)にソースをペースト

(https://github.com/PINTO0309/Tensorflow-binから引用) 

#!/bin/bash

curl -sc /tmp/cookie "https://drive.google.com/uc?export=download&id=1XnV8W38HGW4PmENmfQA5-5NrQ84SsTqX" > /dev/null
CODE="$(awk '/_warning_/ {print $NF}' /tmp/cookie)"
curl -Lb /tmp/cookie "https://drive.google.com/uc?export=download&confirm=${CODE}&id=1XnV8W38HGW4PmENmfQA5-5NrQ84SsTqX" -o tensorflow-2.6.0-cp37-none-linux_aarch64.whl
echo Download finished.


download.shに実行権を付与した後download.shを実行して無事tensorflow-2.6.0-cp37-none-linux_aarch64.whlを入手

■TensorFlowインストール

さっそくインストールしてみると
pip install tensorflow-2.6.0-cp37-none-linux_aarch64.whl

(エラー)
Defaulting to user installation because normal site-packages is not writeable
Processing ./tensorflow-2.6.0-cp37-none-linux_aarch64.whl
Collecting h5py~=3.1.0
  Downloading h5py-3.1.0.tar.gz (371 kB)
     |████████████████████████████████| 371 kB 1.5 MB/s
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
    Preparing wheel metadata ... done
Collecting tensorflow-estimator~=2.6
  Downloading tensorflow_estimator-2.6.0-py2.py3-none-any.whl (462 kB)
     |████████████████████████████████| 462 kB 2.3 MB/s
Collecting wheel~=0.35
  Using cached wheel-0.37.0-py2.py3-none-any.whl (35 kB)
Collecting six~=1.15.0
  Downloading six-1.15.0-py2.py3-none-any.whl (10 kB)
Collecting clang~=5.0
  Downloading clang-5.0.tar.gz (30 kB)
Collecting keras-preprocessing~=1.1.2
  Downloading Keras_Preprocessing-1.1.2-py2.py3-none-any.whl (42 kB)
     |████████████████████████████████| 42 kB 284 kB/s
Collecting flatbuffers~=1.12.0
  Downloading flatbuffers-1.12-py2.py3-none-any.whl (15 kB)
Collecting keras~=2.6
  Downloading keras-2.6.0-py2.py3-none-any.whl (1.3 MB)
     |████████████████████████████████| 1.3 MB 7.3 MB/s
Collecting gast==0.4.0
  Downloading gast-0.4.0-py3-none-any.whl (9.8 kB)
Collecting grpcio<2.0,>=1.37.0
  Downloading grpcio-1.40.0-cp37-cp37m-manylinux_2_24_aarch64.whl (39.2 MB)
     |████████████████████████████████| 39.2 MB 40 kB/s
Collecting termcolor~=1.1.0
  Downloading termcolor-1.1.0.tar.gz (3.9 kB)
Requirement already satisfied: numpy~=1.21.1 in /home/hoto/.local/lib/python3.7/site-packages (from tensorflow==2.6.0) (1.21.2)
Collecting opt-einsum~=3.3.0
  Downloading opt_einsum-3.3.0-py3-none-any.whl (65 kB)
     |████████████████████████████████| 65 kB 846 kB/s
Collecting absl-py~=0.10
  Downloading absl_py-0.13.0-py3-none-any.whl (132 kB)
     |████████████████████████████████| 132 kB 12.6 MB/s
Collecting typing-extensions~=3.7.4
  Downloading typing_extensions-3.7.4.3-py3-none-any.whl (22 kB)
Collecting google-pasta~=0.2
  Downloading google_pasta-0.2.0-py3-none-any.whl (57 kB)
     |████████████████████████████████| 57 kB 1.3 MB/s
Collecting tensorboard~=2.6
  Downloading tensorboard-2.6.0-py3-none-any.whl (5.6 MB)
     |████████████████████████████████| 5.6 MB 10.0 MB/s
Collecting wrapt~=1.12.1
  Downloading wrapt-1.12.1.tar.gz (27 kB)
Collecting protobuf>=3.9.2
  Downloading protobuf-3.18.0-cp37-cp37m-manylinux2014_aarch64.whl (924 kB)
     |████████████████████████████████| 924 kB 10.7 MB/s
Collecting astunparse~=1.6.3
  Downloading astunparse-1.6.3-py2.py3-none-any.whl (12 kB)
Collecting cached-property
  Downloading cached_property-1.5.2-py2.py3-none-any.whl (7.6 kB)
Collecting google-auth<2,>=1.6.3
  Downloading google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
     |████████████████████████████████| 152 kB 11.5 MB/s
Collecting tensorboard-data-server<0.7.0,>=0.6.0
  Downloading tensorboard_data_server-0.6.1-py3-none-any.whl (2.4 kB)
Collecting setuptools>=41.0.0
  Using cached setuptools-58.0.4-py3-none-any.whl (816 kB)
Collecting tensorboard-plugin-wit>=1.6.0
  Downloading tensorboard_plugin_wit-1.8.0-py3-none-any.whl (781 kB)
     |████████████████████████████████| 781 kB 8.2 MB/s
Collecting werkzeug>=0.11.15
  Downloading Werkzeug-2.0.1-py3-none-any.whl (288 kB)
     |████████████████████████████████| 288 kB 8.9 MB/s
Collecting google-auth-oauthlib<0.5,>=0.4.1
  Downloading google_auth_oauthlib-0.4.6-py2.py3-none-any.whl (18 kB)
Collecting requests<3,>=2.21.0
  Downloading requests-2.26.0-py2.py3-none-any.whl (62 kB)
     |████████████████████████████████| 62 kB 154 kB/s
Collecting markdown>=2.6.8
  Downloading Markdown-3.3.4-py3-none-any.whl (97 kB)
     |████████████████████████████████| 97 kB 1.1 MB/s
Collecting pyasn1-modules>=0.2.1
  Downloading pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
     |████████████████████████████████| 155 kB 8.5 MB/s
Collecting rsa<5,>=3.1.4
  Downloading rsa-4.7.2-py3-none-any.whl (34 kB)
Collecting cachetools<5.0,>=2.0.0
  Downloading cachetools-4.2.2-py3-none-any.whl (11 kB)
Collecting requests-oauthlib>=0.7.0
  Downloading requests_oauthlib-1.3.0-py2.py3-none-any.whl (23 kB)
Collecting importlib-metadata
  Downloading importlib_metadata-4.8.1-py3-none-any.whl (17 kB)
Collecting pyasn1<0.5.0,>=0.4.6
  Downloading pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
     |████████████████████████████████| 77 kB 1.3 MB/s
Collecting certifi>=2017.4.17
  Downloading certifi-2021.5.30-py2.py3-none-any.whl (145 kB)
     |████████████████████████████████| 145 kB 8.2 MB/s
Collecting charset-normalizer~=2.0.0
  Downloading charset_normalizer-2.0.6-py3-none-any.whl (37 kB)
Collecting idna<4,>=2.5
  Downloading idna-3.2-py3-none-any.whl (59 kB)
     |████████████████████████████████| 59 kB 1.6 MB/s
Collecting urllib3<1.27,>=1.21.1
  Downloading urllib3-1.26.6-py2.py3-none-any.whl (138 kB)
     |████████████████████████████████| 138 kB 15.9 MB/s
Collecting oauthlib>=3.0.0
  Downloading oauthlib-3.1.1-py2.py3-none-any.whl (146 kB)
     |████████████████████████████████| 146 kB 1.6 MB/s
Collecting zipp>=0.5
  Downloading zipp-3.5.0-py3-none-any.whl (5.7 kB)
Building wheels for collected packages: clang, h5py, termcolor, wrapt
  Building wheel for clang (setup.py) ... done
  Created wheel for clang: filename=clang-5.0-py3-none-any.whl size=30705 sha256=965264867ac952ffe0cdcd7e9e4ec815662a7d5e09bd44b5542bf4283fab3ac9
  Stored in directory: /home/hoto/.cache/pip/wheels/98/91/04/971b4c587cf47ae952b108949b46926f426c02832d120a082a
  Building wheel for h5py (PEP 517) ... error
  ERROR: Command errored out with exit status 1:
   command: /usr/bin/python3 /home/hoto/.local/lib/python3.7/site-packages/pip/_vendor/pep517/in_process/_in_process.py build_wheel /tmp/tmpjazixgh6
       cwd: /tmp/pip-install-r_317f6s/h5py_3678e278ecf34daf8f166a7050ac0992
  Complete output (69 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-aarch64-3.7
  creating build/lib.linux-aarch64-3.7/h5py
  copying h5py/__init__.py -> build/lib.linux-aarch64-3.7/h5py
  copying h5py/h5py_warnings.py -> build/lib.linux-aarch64-3.7/h5py
  copying h5py/ipy_completer.py -> build/lib.linux-aarch64-3.7/h5py
  copying h5py/version.py -> build/lib.linux-aarch64-3.7/h5py
  creating build/lib.linux-aarch64-3.7/h5py/_hl
  copying h5py/_hl/__init__.py -> build/lib.linux-aarch64-3.7/h5py/_hl
  copying h5py/_hl/attrs.py -> build/lib.linux-aarch64-3.7/h5py/_hl
  copying h5py/_hl/base.py -> build/lib.linux-aarch64-3.7/h5py/_hl
  copying h5py/_hl/compat.py -> build/lib.linux-aarch64-3.7/h5py/_hl
  copying h5py/_hl/dataset.py -> build/lib.linux-aarch64-3.7/h5py/_hl
  copying h5py/_hl/datatype.py -> build/lib.linux-aarch64-3.7/h5py/_hl
  copying h5py/_hl/dims.py -> build/lib.linux-aarch64-3.7/h5py/_hl
  copying h5py/_hl/files.py -> build/lib.linux-aarch64-3.7/h5py/_hl
  copying h5py/_hl/filters.py -> build/lib.linux-aarch64-3.7/h5py/_hl
  copying h5py/_hl/group.py -> build/lib.linux-aarch64-3.7/h5py/_hl
  copying h5py/_hl/selections.py -> build/lib.linux-aarch64-3.7/h5py/_hl
  copying h5py/_hl/selections2.py -> build/lib.linux-aarch64-3.7/h5py/_hl
  copying h5py/_hl/vds.py -> build/lib.linux-aarch64-3.7/h5py/_hl
  creating build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/__init__.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/common.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/conftest.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_attribute_create.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_attrs.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_attrs_data.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_base.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_big_endian_file.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_completions.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_dataset.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_dataset_getitem.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_dataset_swmr.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_datatype.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_dimension_scales.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_dims_dimensionproxy.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_dtype.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_errors.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_file.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_file2.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_file_image.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_filters.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_group.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_h5.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_h5d_direct_chunk.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_h5f.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_h5p.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_h5pl.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_h5t.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_objects.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_selections.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_slicing.py -> build/lib.linux-aarch64-3.7/h5py/tests
  creating build/lib.linux-aarch64-3.7/h5py/tests/data_files
  copying h5py/tests/data_files/__init__.py -> build/lib.linux-aarch64-3.7/h5py/tests/data_files
  creating build/lib.linux-aarch64-3.7/h5py/tests/test_vds
  copying h5py/tests/test_vds/__init__.py -> build/lib.linux-aarch64-3.7/h5py/tests/test_vds
  copying h5py/tests/test_vds/test_highlevel_vds.py -> build/lib.linux-aarch64-3.7/h5py/tests/test_vds
  copying h5py/tests/test_vds/test_lowlevel_vds.py -> build/lib.linux-aarch64-3.7/h5py/tests/test_vds
  copying h5py/tests/test_vds/test_virtual_source.py -> build/lib.linux-aarch64-3.7/h5py/tests/test_vds
  copying h5py/tests/data_files/vlen_string_dset.h5 -> build/lib.linux-aarch64-3.7/h5py/tests/data_files
  copying h5py/tests/data_files/vlen_string_dset_utc.h5 -> build/lib.linux-aarch64-3.7/h5py/tests/data_files
  copying h5py/tests/data_files/vlen_string_s390x.h5 -> build/lib.linux-aarch64-3.7/h5py/tests/data_files
  running build_ext
  Building h5py requires pkg-config unless the HDF5 path is explicitly specified
  error: pkg-config probably not installed: FileNotFoundError(2, "No such file or directory: 'pkg-config'")
  ----------------------------------------
  ERROR: Failed building wheel for h5py
  Building wheel for termcolor (setup.py) ... done
  Created wheel for termcolor: filename=termcolor-1.1.0-py3-none-any.whl size=4831 sha256=49219dc8d2a0984da0c4841b0c5e83788f3a3c4f55832a1f160a0443442de651
  Stored in directory: /home/hoto/.cache/pip/wheels/3f/e3/ec/8a8336ff196023622fbcb36de0c5a5c218cbb24111d1d4c7f2
  Building wheel for wrapt (setup.py) ... done
  Created wheel for wrapt: filename=wrapt-1.12.1-cp37-cp37m-linux_aarch64.whl size=72882 sha256=7d42f58dc8673df65fb0f65475d3cc0c5d9b5465a8410228b7998e619e588a89
  Stored in directory: /home/hoto/.cache/pip/wheels/62/76/4c/aa25851149f3f6d9785f6c869387ad82b3fd37582fa8147ac6
Successfully built clang termcolor wrapt
Failed to build h5py
ERROR: Could not build wheels for h5py which use PEP 517 and cannot be installed directly

失敗・・・。
どうやらh5py関連でエラーのようなのでエラーメッセージで検索し同じ様なケースの対応をとりあえず真似してみました。

これ以降はどれが正解だったのか分からずとりあえずやった内容を列挙


setuptoolsとwheelが古いとなる情報がありpip install --upgrade setuptools wheelでアップデート
→失敗

h5pyを単体で色々なオプションで試行
pip install --no-use-pep517 h5py
→失敗
pip install --no-binary h5py
→失敗
pip install --no-binary=h5py h5py
→失敗

pkg-configをインストール
#sudo apt list --installed | grep pkg-config
→失敗

debinのパッケージにh5pyがあるようなのでインストール
#sudo apt install python3-h5py
→バージョンが2.8.0-3となんか古そう

再度TensorFlowのインストールを試行

pip install tensorflow-2.6.0-cp37-none-linux_aarch64.whl

Defaulting to user installation because normal site-packages is not writeable
Processing ./tensorflow-2.6.0-cp37-none-linux_aarch64.whl
Collecting keras-preprocessing~=1.1.2
  Using cached Keras_Preprocessing-1.1.2-py2.py3-none-any.whl (42 kB)
Collecting clang~=5.0
  Using cached clang-5.0-py3-none-any.whl
Collecting wrapt~=1.12.1
  Using cached wrapt-1.12.1-cp37-cp37m-linux_aarch64.whl
Collecting opt-einsum~=3.3.0
  Using cached opt_einsum-3.3.0-py3-none-any.whl (65 kB)
Collecting astunparse~=1.6.3
  Using cached astunparse-1.6.3-py2.py3-none-any.whl (12 kB)
Collecting tensorflow-estimator~=2.6
  Using cached tensorflow_estimator-2.6.0-py2.py3-none-any.whl (462 kB)
Collecting tensorboard~=2.6
  Using cached tensorboard-2.6.0-py3-none-any.whl (5.6 MB)
Collecting termcolor~=1.1.0
  Using cached termcolor-1.1.0-py3-none-any.whl
Collecting gast==0.4.0
  Using cached gast-0.4.0-py3-none-any.whl (9.8 kB)
Collecting flatbuffers~=1.12.0
  Using cached flatbuffers-1.12-py2.py3-none-any.whl (15 kB)
Requirement already satisfied: wheel~=0.35 in /home/hoto/.local/lib/python3.7/site-packages (from tensorflow==2.6.0) (0.37.0)
Collecting six~=1.15.0
  Using cached six-1.15.0-py2.py3-none-any.whl (10 kB)
Collecting keras~=2.6
  Using cached keras-2.6.0-py2.py3-none-any.whl (1.3 MB)
Collecting absl-py~=0.10
  Using cached absl_py-0.13.0-py3-none-any.whl (132 kB)
Collecting google-pasta~=0.2
  Using cached google_pasta-0.2.0-py3-none-any.whl (57 kB)
Collecting grpcio<2.0,>=1.37.0
  Using cached grpcio-1.40.0-cp37-cp37m-manylinux_2_24_aarch64.whl (39.2 MB)
Collecting protobuf>=3.9.2
  Using cached protobuf-3.18.0-cp37-cp37m-manylinux2014_aarch64.whl (924 kB)
Collecting h5py~=3.1.0
  Using cached h5py-3.1.0.tar.gz (371 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
    Preparing wheel metadata ... done
Collecting typing-extensions~=3.7.4
  Using cached typing_extensions-3.7.4.3-py3-none-any.whl (22 kB)
Requirement already satisfied: numpy~=1.21.1 in /home/hoto/.local/lib/python3.7/site-packages (from tensorflow==2.6.0) (1.21.2)
Collecting cached-property
  Using cached cached_property-1.5.2-py2.py3-none-any.whl (7.6 kB)
Requirement already satisfied: setuptools>=41.0.0 in /home/hoto/.local/lib/python3.7/site-packages (from tensorboard~=2.6->tensorflow==2.6.0) (58.0.4)
Collecting google-auth<2,>=1.6.3
  Using cached google_auth-1.35.0-py2.py3-none-any.whl (152 kB)
Collecting requests<3,>=2.21.0
  Using cached requests-2.26.0-py2.py3-none-any.whl (62 kB)
Collecting google-auth-oauthlib<0.5,>=0.4.1
  Using cached google_auth_oauthlib-0.4.6-py2.py3-none-any.whl (18 kB)
Collecting werkzeug>=0.11.15
  Using cached Werkzeug-2.0.1-py3-none-any.whl (288 kB)
Collecting tensorboard-data-server<0.7.0,>=0.6.0
  Using cached tensorboard_data_server-0.6.1-py3-none-any.whl (2.4 kB)
Collecting markdown>=2.6.8
  Using cached Markdown-3.3.4-py3-none-any.whl (97 kB)
Collecting tensorboard-plugin-wit>=1.6.0
  Using cached tensorboard_plugin_wit-1.8.0-py3-none-any.whl (781 kB)
Collecting cachetools<5.0,>=2.0.0
  Using cached cachetools-4.2.2-py3-none-any.whl (11 kB)
Collecting rsa<5,>=3.1.4
  Using cached rsa-4.7.2-py3-none-any.whl (34 kB)
Collecting pyasn1-modules>=0.2.1
  Using cached pyasn1_modules-0.2.8-py2.py3-none-any.whl (155 kB)
Collecting requests-oauthlib>=0.7.0
  Using cached requests_oauthlib-1.3.0-py2.py3-none-any.whl (23 kB)
Collecting importlib-metadata
  Using cached importlib_metadata-4.8.1-py3-none-any.whl (17 kB)
Collecting pyasn1<0.5.0,>=0.4.6
  Using cached pyasn1-0.4.8-py2.py3-none-any.whl (77 kB)
Collecting urllib3<1.27,>=1.21.1
  Using cached urllib3-1.26.6-py2.py3-none-any.whl (138 kB)
Collecting charset-normalizer~=2.0.0
  Using cached charset_normalizer-2.0.6-py3-none-any.whl (37 kB)
Collecting idna<4,>=2.5
  Using cached idna-3.2-py3-none-any.whl (59 kB)
Collecting certifi>=2017.4.17
  Using cached certifi-2021.5.30-py2.py3-none-any.whl (145 kB)
Collecting oauthlib>=3.0.0
  Using cached oauthlib-3.1.1-py2.py3-none-any.whl (146 kB)
Collecting zipp>=0.5
  Using cached zipp-3.5.0-py3-none-any.whl (5.7 kB)
Building wheels for collected packages: h5py
  Building wheel for h5py (PEP 517) ... error
  ERROR: Command errored out with exit status 1:
   command: /usr/bin/python3 /home/hoto/.local/lib/python3.7/site-packages/pip/_vendor/pep517/in_process/_in_process.py build_wheel /tmp/tmpqu4eq5l4
       cwd: /tmp/pip-install-r22datm0/h5py_c83ebf6719484a6cb76ecd27971abef2
  Complete output (70 lines):
  running bdist_wheel
  running build
  running build_py
  creating build
  creating build/lib.linux-aarch64-3.7
  creating build/lib.linux-aarch64-3.7/h5py
  copying h5py/__init__.py -> build/lib.linux-aarch64-3.7/h5py
  copying h5py/h5py_warnings.py -> build/lib.linux-aarch64-3.7/h5py
  copying h5py/ipy_completer.py -> build/lib.linux-aarch64-3.7/h5py
  copying h5py/version.py -> build/lib.linux-aarch64-3.7/h5py
  creating build/lib.linux-aarch64-3.7/h5py/_hl
  copying h5py/_hl/__init__.py -> build/lib.linux-aarch64-3.7/h5py/_hl
  copying h5py/_hl/attrs.py -> build/lib.linux-aarch64-3.7/h5py/_hl
  copying h5py/_hl/base.py -> build/lib.linux-aarch64-3.7/h5py/_hl
  copying h5py/_hl/compat.py -> build/lib.linux-aarch64-3.7/h5py/_hl
  copying h5py/_hl/dataset.py -> build/lib.linux-aarch64-3.7/h5py/_hl
  copying h5py/_hl/datatype.py -> build/lib.linux-aarch64-3.7/h5py/_hl
  copying h5py/_hl/dims.py -> build/lib.linux-aarch64-3.7/h5py/_hl
  copying h5py/_hl/files.py -> build/lib.linux-aarch64-3.7/h5py/_hl
  copying h5py/_hl/filters.py -> build/lib.linux-aarch64-3.7/h5py/_hl
  copying h5py/_hl/group.py -> build/lib.linux-aarch64-3.7/h5py/_hl
  copying h5py/_hl/selections.py -> build/lib.linux-aarch64-3.7/h5py/_hl
  copying h5py/_hl/selections2.py -> build/lib.linux-aarch64-3.7/h5py/_hl
  copying h5py/_hl/vds.py -> build/lib.linux-aarch64-3.7/h5py/_hl
  creating build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/__init__.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/common.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/conftest.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_attribute_create.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_attrs.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_attrs_data.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_base.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_big_endian_file.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_completions.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_dataset.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_dataset_getitem.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_dataset_swmr.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_datatype.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_dimension_scales.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_dims_dimensionproxy.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_dtype.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_errors.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_file.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_file2.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_file_image.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_filters.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_group.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_h5.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_h5d_direct_chunk.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_h5f.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_h5p.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_h5pl.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_h5t.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_objects.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_selections.py -> build/lib.linux-aarch64-3.7/h5py/tests
  copying h5py/tests/test_slicing.py -> build/lib.linux-aarch64-3.7/h5py/tests
  creating build/lib.linux-aarch64-3.7/h5py/tests/data_files
  copying h5py/tests/data_files/__init__.py -> build/lib.linux-aarch64-3.7/h5py/tests/data_files
  creating build/lib.linux-aarch64-3.7/h5py/tests/test_vds
  copying h5py/tests/test_vds/__init__.py -> build/lib.linux-aarch64-3.7/h5py/tests/test_vds
  copying h5py/tests/test_vds/test_highlevel_vds.py -> build/lib.linux-aarch64-3.7/h5py/tests/test_vds
  copying h5py/tests/test_vds/test_lowlevel_vds.py -> build/lib.linux-aarch64-3.7/h5py/tests/test_vds
  copying h5py/tests/test_vds/test_virtual_source.py -> build/lib.linux-aarch64-3.7/h5py/tests/test_vds
  copying h5py/tests/data_files/vlen_string_dset.h5 -> build/lib.linux-aarch64-3.7/h5py/tests/data_files
  copying h5py/tests/data_files/vlen_string_dset_utc.h5 -> build/lib.linux-aarch64-3.7/h5py/tests/data_files
  copying h5py/tests/data_files/vlen_string_s390x.h5 -> build/lib.linux-aarch64-3.7/h5py/tests/data_files
  running build_ext
  Loading library to get version: libhdf5.so
  error: Unable to load dependency HDF5, make sure HDF5 is installed properly
  error: libhdf5.so: cannot open shared object file: No such file or directory
  ----------------------------------------
  ERROR: Failed building wheel for h5py
Failed to build h5py
ERROR: Could not build wheels for h5py which use PEP 517 and cannot be installed directly

失敗・・・でもエラーメッセージが変わっているようで今度はlibhdf5関連

似た事象をインターネットで調べてversioned-hdf5をインストールしてみたけれど失敗・・・。

よくxx.soファイルがないのはコンパイルに必要なライブラリがない場合がおおいのでdebianパッケージのlibhdf5-devをインストールしてみる

#sudo apt install libhdf5-dev

TensorFlowのインストールを試行

pip install tensorflow-2.6.0-cp37-none-linux_aarch64.whl
Defaulting to user installation because normal site-packages is not writeable
Processing ./tensorflow-2.6.0-cp37-none-linux_aarch64.whl
Requirement already satisfied: six~=1.15.0 in /home/hoto/.local/lib/python3.7/site-packages (from tensorflow==2.6.0) (1.15.0)
Requirement already satisfied: grpcio<2.0,>=1.37.0 in /home/hoto/.local/lib/python3.7/site-packages (from tensorflow==2.6.0) (1.40.0)
Requirement already satisfied: protobuf>=3.9.2 in /home/hoto/.local/lib/python3.7/site-packages (from tensorflow==2.6.0) (3.18.0)
Requirement already satisfied: clang~=5.0 in /home/hoto/.local/lib/python3.7/site-packages (from tensorflow==2.6.0) (5.0)
Requirement already satisfied: keras~=2.6 in /home/hoto/.local/lib/python3.7/site-packages (from tensorflow==2.6.0) (2.6.0)
Requirement already satisfied: google-pasta~=0.2 in /home/hoto/.local/lib/python3.7/site-packages (from tensorflow==2.6.0) (0.2.0)
Requirement already satisfied: h5py~=3.1.0 in /home/hoto/.local/lib/python3.7/site-packages (from tensorflow==2.6.0) (3.1.0)
Requirement already satisfied: numpy~=1.21.1 in /home/hoto/.local/lib/python3.7/site-packages (from tensorflow==2.6.0) (1.21.2)
Requirement already satisfied: gast==0.4.0 in /home/hoto/.local/lib/python3.7/site-packages (from tensorflow==2.6.0) (0.4.0)
Requirement already satisfied: keras-preprocessing~=1.1.2 in /home/hoto/.local/lib/python3.7/site-packages (from tensorflow==2.6.0) (1.1.2)
Requirement already satisfied: wrapt~=1.12.1 in /home/hoto/.local/lib/python3.7/site-packages (from tensorflow==2.6.0) (1.12.1)
Requirement already satisfied: absl-py~=0.10 in /home/hoto/.local/lib/python3.7/site-packages (from tensorflow==2.6.0) (0.13.0)
Requirement already satisfied: opt-einsum~=3.3.0 in /home/hoto/.local/lib/python3.7/site-packages (from tensorflow==2.6.0) (3.3.0)
Requirement already satisfied: tensorflow-estimator~=2.6 in /home/hoto/.local/lib/python3.7/site-packages (from tensorflow==2.6.0) (2.6.0)
Requirement already satisfied: wheel~=0.35 in /home/hoto/.local/lib/python3.7/site-packages (from tensorflow==2.6.0) (0.37.0)
Requirement already satisfied: astunparse~=1.6.3 in /home/hoto/.local/lib/python3.7/site-packages (from tensorflow==2.6.0) (1.6.3)
Requirement already satisfied: flatbuffers~=1.12.0 in /home/hoto/.local/lib/python3.7/site-packages (from tensorflow==2.6.0) (1.12)
Requirement already satisfied: termcolor~=1.1.0 in /home/hoto/.local/lib/python3.7/site-packages (from tensorflow==2.6.0) (1.1.0)
Requirement already satisfied: typing-extensions~=3.7.4 in /home/hoto/.local/lib/python3.7/site-packages (from tensorflow==2.6.0) (3.7.4.3)
Requirement already satisfied: tensorboard~=2.6 in /home/hoto/.local/lib/python3.7/site-packages (from tensorflow==2.6.0) (2.6.0)
Requirement already satisfied: cached-property in /home/hoto/.local/lib/python3.7/site-packages (from h5py~=3.1.0->tensorflow==2.6.0) (1.5.2)
Requirement already satisfied: tensorboard-data-server<0.7.0,>=0.6.0 in /home/hoto/.local/lib/python3.7/site-packages (from tensorboard~=2.6->tensorflow==2.6.0) (0.6.1)
Requirement already satisfied: setuptools>=41.0.0 in /home/hoto/.local/lib/python3.7/site-packages (from tensorboard~=2.6->tensorflow==2.6.0) (58.0.4)
Requirement already satisfied: requests<3,>=2.21.0 in /home/hoto/.local/lib/python3.7/site-packages (from tensorboard~=2.6->tensorflow==2.6.0) (2.26.0)
Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /home/hoto/.local/lib/python3.7/site-packages (from tensorboard~=2.6->tensorflow==2.6.0) (0.4.6)
Requirement already satisfied: werkzeug>=0.11.15 in /home/hoto/.local/lib/python3.7/site-packages (from tensorboard~=2.6->tensorflow==2.6.0) (2.0.1)
Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in /home/hoto/.local/lib/python3.7/site-packages (from tensorboard~=2.6->tensorflow==2.6.0) (1.8.0)
Requirement already satisfied: markdown>=2.6.8 in /home/hoto/.local/lib/python3.7/site-packages (from tensorboard~=2.6->tensorflow==2.6.0) (3.3.4)
Requirement already satisfied: google-auth<2,>=1.6.3 in /home/hoto/.local/lib/python3.7/site-packages (from tensorboard~=2.6->tensorflow==2.6.0) (1.35.0)
Requirement already satisfied: cachetools<5.0,>=2.0.0 in /home/hoto/.local/lib/python3.7/site-packages (from google-auth<2,>=1.6.3->tensorboard~=2.6->tensorflow==2.6.0) (4.2.2)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /home/hoto/.local/lib/python3.7/site-packages (from google-auth<2,>=1.6.3->tensorboard~=2.6->tensorflow==2.6.0) (0.2.8)
Requirement already satisfied: rsa<5,>=3.1.4 in /home/hoto/.local/lib/python3.7/site-packages (from google-auth<2,>=1.6.3->tensorboard~=2.6->tensorflow==2.6.0) (4.7.2)
Requirement already satisfied: requests-oauthlib>=0.7.0 in /home/hoto/.local/lib/python3.7/site-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard~=2.6->tensorflow==2.6.0) (1.3.0)
Requirement already satisfied: importlib-metadata in /home/hoto/.local/lib/python3.7/site-packages (from markdown>=2.6.8->tensorboard~=2.6->tensorflow==2.6.0) (4.8.1)
Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /home/hoto/.local/lib/python3.7/site-packages (from pyasn1-modules>=0.2.1->google-auth<2,>=1.6.3->tensorboard~=2.6->tensorflow==2.6.0) (0.4.8)
Requirement already satisfied: certifi>=2017.4.17 in /home/hoto/.local/lib/python3.7/site-packages (from requests<3,>=2.21.0->tensorboard~=2.6->tensorflow==2.6.0) (2021.5.30)
Requirement already satisfied: idna<4,>=2.5 in /home/hoto/.local/lib/python3.7/site-packages (from requests<3,>=2.21.0->tensorboard~=2.6->tensorflow==2.6.0) (3.2)
Requirement already satisfied: charset-normalizer~=2.0.0 in /home/hoto/.local/lib/python3.7/site-packages (from requests<3,>=2.21.0->tensorboard~=2.6->tensorflow==2.6.0) (2.0.6)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/hoto/.local/lib/python3.7/site-packages (from requests<3,>=2.21.0->tensorboard~=2.6->tensorflow==2.6.0) (1.26.6)
Requirement already satisfied: oauthlib>=3.0.0 in /home/hoto/.local/lib/python3.7/site-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard~=2.6->tensorflow==2.6.0) (3.1.1)
Requirement already satisfied: zipp>=0.5 in /home/hoto/.local/lib/python3.7/site-packages (from importlib-metadata->markdown>=2.6.8->tensorboard~=2.6->tensorflow==2.6.0) (3.5.0)
Installing collected packages: tensorflow
Successfully installed tensorflow-2.6.0

無事インストール成功!!

■kerasのインストール


pip install keras

Defaulting to user installation because normal site-packages is not writeable
Collecting keras
  Using cached keras-2.6.0-py2.py3-none-any.whl (1.3 MB)
Installing collected packages: keras
Successfully installed keras-2.6.0

すんなりインストール完了

とりあえずインストールは完了したけれど本当に使用できるか試してみないと。

【追記】

kerasを調べていてITmediaの記事を読んでやっと理解、kerasはtensorflow.kerasとしてTensorFlow2にで組み込まれたようです。なのでわざわざkerasをインストールする必要はなかったようです。







2021年9月17日金曜日

【scikit-learn】Linux(ベータ版)にpandasとscikit-learnのインストール

 numpyまではインストールしていましたが参考書を読み進みそろそろ機械学習のライブラリをしようすることになりそうなのでpandasとscikit-learnをインストールしました。

intel系のマシンならあまり心配ではないのですが使用しているChromeBook(Detachable CM3)で使用されているCPUはARM系なのでこういう計算系のライブラリのインストールが心配でした。

ただ、結果からいうととくに問題なくインストール完了です。


pipのupgradeしろと警告メッセージが表示していますがpandasとしては1.3.3がインストールされました。


scikit-learnも1.20.3が問題なくインストール完了

これで安心して勉強できます。

Chromebook の Linux コンテナ (Crostini) のバックアップ

Linux(ベータ版)は PythonやVisualStudioなどインストールして学習にかなり重宝しています。

さらに色々とPythonのライブラリのインストールもするのでいったんバックアップすることにしました。


バックアップ方法を紹介されているサイトを参考に実施、作業はいたって簡単


設定メニューからデベロッパーから「バックアップと復元」を選択しバックアップボタンをクリック


保存先と保存ファイル名を選択して保存ボタンをクリックするだけ


まだたいしてインストールをしていないからかバックアップファイルは1.6G程度でおおよそ10分ぐらいでバックアップ完了

Chromebookのストレージサイズはけして多くはないので自宅のNASに退避したのち削除、思っていた以上に簡単でした。

Linux自体は起動したままの状態だったのでちょっと気になりましたがなんの問題もなくバックアップもできその後もターミナルを起動してみましたが普通に使用できました。




2021年9月13日月曜日

Windows10+IE11でSSL3.0の有効化して自己証明書サイトへのhttpsアクセス

クローズな環境でWindows10+IE11からSSL3.0でのhttps通信必要になりIE11の詳細設定でSSL3.0の使用にチェックしたにも関わらず接続エラー・・・。

色々調べてみた所レジストリを変更することで接続出来ることが判明

HKLM SYSTEM\CurrentControlSet\Control\SecurityProviders\SCHANNEL\Protocols\SSL 3.0 \Clientに"Enabled" という名前の dword レジストリ値に1を設定

情報はMicrosoftのトランスポート層セキュリティ (TLS) のレジストリ設定を参考にしました 。

説明の中に"DisabledByDefault"と言う項目もあり"Enabled"との違いが気になっていたのですが"DisabledByDefault"はアプリケーションがSSLを使用する際にバージョンを指定しなかった場合に使用される規定値であり"DisabledByDefault"の値が0でも該当のバージョンが有効であればアプリケーションが明示的に該当のバージョンを指定すれば使用可能らしい。

ただOSの規定値が無効だったり"Enabled"という名前のレジストリ値が0の場合はアプリケーションから使用不可ということ。(IE11の詳細設定でSSL3.0にチェックを入れても使用できない)

実際レジストリを確認した時にSSL 3.0についてのレジストリ値がなくOSの規定値が気になったのですがTLS/SSL のプロトコル (Schannel SSP)に記載がありWindows 10、バージョン1607、Windows Server 2016 StandardからSSL3.0は無効のようです。

補足として自己証明書を使用したhttps通信の場合、上記設定を有効にしてもセキュリティ上アクセス出来ずアクセスサイトを信頼済サイトに登録する必要もありました。(詳細設定で証明書の確認をしないようにすれば登録は不要なのかも)