virtualenv+tensorflow初体验

Logs and Questions|2017-07-07中,我便对virtualenv和anaconda两个虚拟环境管理工具做过简单的调查23。今天由于想安装Tensorflow,不得不进一步了解virtualenv。

检查系统及环境:

  • 检查操作系统:OS X EI Capitan Version 10.11.6
  • python: version: Python 2.7.10
  • pip: version: 8.1.2
  • virtualenv: version: 15.1.0

virtualenv

检查virtualenv的版本1

➜  virtualenv --version
15.1.0

如果你的PC上还没有安装virtualenv,建议你按照官方的安装说明进行安装,不是很难。

virtualenvwrapper

➜  ~ sudo pip install virtualenvwrapper
OSError: [Errno 1] Operation not permitted: '/tmp/pip-hvi6CP-uninstall/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/six-1.4.1-py2.7.egg-info'

上网查阅了一些相关资料后,在Six issue when installing package #3165,我找到了解决方案。

原来是因为我的操作系统是OS X EI Capitan 10.11.6, 而Capitan已经安装了six 1.4.1,所以当我们使用pip安装基于six>1.5的插件时,pip会尝试uninstallsix 1.4.1。而这一操作会由于MAC OS的System Integrity Protection而禁止。所以每次遇到此类情况时,最好再命令行后面加上:

--ignore-installed six

例如我遇到的情况,我则需要改为输入

➜  ~ sudo pip install virtualenvwrapper --upgrade --ignore-installed six

Bingo, done!

检查virtualenvwrapper

➜  ~ virtualenvwrapper --version
virtualenvwrapper is a set of extensions to Ian Bicking's virtualenv
tool.  The extensions include wrappers for creating and deleting
virtual environments and otherwise managing your development workflow,
making it easier to work on more than one project at a time without
introducing conflicts in their dependencies.

For more information please refer to the documentation:

http://virtualenvwrapper.readthedocs.org/en/latest/command_ref.html

Commands available:

  add2virtualenv: add directory to the import path
  ---------------------------------
  <...omission...>
  ---------------------------------
  workon: list or change working virtualenvs

接下来我们进行一个简单的测试,创建一个名为test的虚拟环境

➜  ~ mkvirtualenv test
New python executable in /Users/huwei/Envs/test/bin/python
Installing setuptools, pip, wheel...done.
---------------------------------
<...omission...>
---------------------------------
(test) ➜  ~

运行以下命令,则可以关闭虚拟环境:

(test) ➜  ~ deactivate

当我需要列举系统里所有的虚拟环境时,virtualenvwrapper里的lsvirtualenv则会很有帮助

➜  ~ lsvirtualenv
test
====

tensorflow

tensorflow官方安装指南Installing TensorFlow on Mac OS X中推荐四种安装方法,我选择使用virtualenv。

由于我已经安装了pip和virtualenv,所以我直接跳到Installing with virtualenv - Step3.

3.通过以下命令创建一个新的viertualenv环境:

➜  ~ virtualenv --system-site-packages ~/tensorflow # for Python 2.7
New python executable in /Users/huwei/targetDirectory/bin/python
Installing setuptools, pip, wheel...done.

这样系统的根目录下就会新建一个文件夹tensorflow

4.激活virtualenv环境:

➜  ~ source ~/tensorflow/bin/activate      # If using bash, sh, ksh, or zsh
(tensorflow) ➜  ~

这时,terminal命令行前的提示会变成(tensorflow)

5.确保pip>8.1已经安装:

(tensorflow) ➜  ~ easy_install -U pip

6.安装tensorflow

(tensorflow) ➜  ~ pip install --upgrade tensorflow      # for Python 2.7

安装成功。

测试

我们用Youtube红人Siraj Raval的视频How to Do Sentiment Analysis - Intro to Deep Learning #3来进行测试。

将源代码Github|demo.py下载到自己的PC上。

但是我们还需要下载tflearn包。按照官方指南TFLearn-Installation进行安装。

由于我已经安装了tensorflow,那么如何检查tensorflow的版本呢?以下命令可能可以帮到你4

(tensorflow) ➜  pip list | grep tensorflow
tensorflow (1.1.0)
tensorflow-tensorboard (0.1.8)

接下来我们开始安装latest stable version的tflearn:

(tensorflow) ➜  pip install tflearn

通过以下命令检查tflearn的版本:

(tensorflow) ➜  pip list | grep tflearn
tflearn (0.3.2)

刚刚下载的情感分析Sentiment_demo.py我放在了文件夹How_To_Do_Sentimment_Analysis里,cd到这个文件夹,然后运行Sentiment_demo.py

(tensorflow) ➜  How_To_Do_Sentimment_Analysis python Sentiment_demo.py
hdf5 is not supported on this machine (please install/reinstall h5py for optimal experience)
---------------------------------
<...omission...>
---------------------------------
Training samples: 22500
Validation samples: 2500
--
Training Step: 704  | total loss: 0.67773 | time: 216.987s
| Adam | epoch: 001 | loss: 0.67773 - acc: 0.5703 | val_loss: 0.67720 - val_acc: 0.5648 -- iter: 22500/22500
--
Training Step: 1408  | total loss: 0.40822 | time: 181.420s
| Adam | epoch: 002 | loss: 0.40822 - acc: 0.8382 | val_loss: 0.53255 - val_acc: 0.7776 -- iter: 22500/22500
--
Training Step: 2112  | total loss: 0.35289 | time: 239.820s
| Adam | epoch: 003 | loss: 0.35289 - acc: 0.8555 | val_loss: 0.48913 - val_acc: 0.7860 -- iter: 22500/22500
--
Training Step: 2816  | total loss: 0.23383 | time: 170.344s
| Adam | epoch: 004 | loss: 0.23383 - acc: 0.9100 | val_loss: 0.50868 - val_acc: 0.8128 -- iter: 22500/22500
--
Training Step: 3520  | total loss: 0.19371 | time: 167.473s
| Adam | epoch: 005 | loss: 0.19371 - acc: 0.9220 | val_loss: 0.52981 - val_acc: 0.8128 -- iter: 22500/22500
--
Training Step: 4224  | total loss: 0.13586 | time: 168.919s
| Adam | epoch: 006 | loss: 0.13586 - acc: 0.9640 | val_loss: 0.55186 - val_acc: 0.8088 -- iter: 22500/22500
--
Training Step: 4928  | total loss: 0.08376 | time: 188.105s
| Adam | epoch: 007 | loss: 0.08376 - acc: 0.9722 | val_loss: 0.66951 - val_acc: 0.8148 -- iter: 22500/22500
--
Training Step: 5632  | total loss: 0.09493 | time: 180.897s
| Adam | epoch: 008 | loss: 0.09493 - acc: 0.9746 | val_loss: 0.64003 - val_acc: 0.7996 -- iter: 22500/22500
--
Training Step: 6336  | total loss: 0.03989 | time: 175.506s
| Adam | epoch: 009 | loss: 0.03989 - acc: 0.9845 | val_loss: 0.84167 - val_acc: 0.8024 -- iter: 22500/22500
--
Training Step: 7040  | total loss: 0.09154 | time: 175.630s
| Adam | epoch: 010 | loss: 0.09154 - acc: 0.9788 | val_loss: 0.68026 - val_acc: 0.7940 -- iter: 22500/22500
--

这个测试是使用IMDB数据集,22500个训练集,2500个验证集。使用DNN算法进行情感分析,由于我目前的电脑是Macbook Air,而且没有外带GPU,所以训练时间比较长,花了近30min。但是最终结果是令人满意的,最终准确率可以达到97.88%。证明virtualenv+virtualwrapper+tensorflow的安装是成功的。

而Siraj的视频中,他推荐在AWS云端运行。有兴趣的朋友可以查查相关资料。

1. Pipenv & 虚拟环境¶
2. Python—Virtualenv简明教程
3. Anacodna之conda与 virtualenv对比使用教程,创建虚拟环境
4. How to find which version of TensorFlow is installed in my system?
-------------End of postThanks for your time-------------
BaoDuGe_飽蠹閣 wechat
Enjoy it? Subscribe to my blog by scanning my public wechat account