Child pages
  • Python
Skip to end of metadata
Go to start of metadata



Learning Python

http://learnpythonthehardway.org/book/

http://www.codecademy.com/en/tracks/python

Python User Group - Richmond, VA - https://www.meetup.com/PyRVAUserGroup/ 

Showing all available python modules

Three different ways of showing all available python modules

1] pip freeze

2] >>>help("modules")

3]>>>import sys

>>>print sys.modules


pip

pip is a package management tool for python.

pip is installed by default in many linux distributions.  To verify that pip is installed use the command:

which pip

If it is not installed, you need to manually install pip do the following:

http://www.pip-installer.org/en/latest/installing.html

Some python modules are installed by default by the operating system, but there are many more that can be installed. Not all modules are compatible with other modules, so they are not installed by default.
A user can use pip to install modules into a user's home directory without the need for administrator privileges:

pip install --user <module>

modules will be downloaded, compiled, and installed under ~/.local

To show all installed python modules use:

pip freeze

To search for  python modules available for installation use:

pip search <module>


pyenv

pyenv allows for multiple version of python to be installed in a users home directory and they are able to switch from one to another.  If a user needs a version of python that is not installed by default, they can use pyenv to any almost any version they wish in their home directory.  They can then use pip that comes with that version to install the modules they want. Read the documentation for pyenv before using these direction.

Documentation


Install pyenv

( also installs easy_install and pip )


cd
git clone git://github.com/yyuu/pyenv.git .pyenv


Create a file to set the correct environmental variables for pyenv

Create a file called ~/enable_pyenv.sh with the following lines:


export PYENV_ROOT="$HOME/.pyenv"
export PATH="$PYENV_ROOT/bin:$PATH"
eval "$(pyenv init -)"


If a newer version of gcc is required, add the following line to enable_pyenv.sh:

source /opt/rh/devtoolset-7/enable


If a newer version of openssl is required, add the following line to enable_pyenv.sh:

source /usr/global/openssl/enable_openssl.sh



At this point, on the godel cluster, use qrsh to get a shell on a compute node.


activate these variables with the source command:


source ~/enable_pyenv.sh


If you want these to be set with every login, add "source ~/enable_pyenv.sh" to your ~/.bash_profile


Verify pyenv is installed:

which pyenv

pyenv --version

Show all available python version that can be installed

If you need a version of python3, its recommended to use the latest version of 3.6 since its likely to be used on future servers. But, not all python modules are available for every version of python, research supported version for a desired module.

pyenv install --list 

Output should look like below
pyenv versions  Expand source
$ pyenv install --list
Available versions:
  2.1.3
  2.2.3
  2.3.7
  2.4.0
  2.4.1
  2.4.2
  2.4.3
  2.4.4
  2.4.5
  2.4.6
  2.5.0
  2.5.1
  2.5.2
  2.5.3
  2.5.4
  2.5.5
  2.5.6
  2.6.6
  2.6.7
  2.6.8
  2.6.9
  2.7.0
  2.7-dev
  2.7.1
  2.7.2
  2.7.3
  2.7.4
  2.7.5
  2.7.6
  2.7.7
  2.7.8
  2.7.9
  2.7.10
  2.7.11
  2.7.12
  2.7.13
  2.7.14
  2.7.15
  2.7.16
  3.0.1
  3.1.0
  3.1.1
  3.1.2
  3.1.3
  3.1.4
  3.1.5
  3.2.0
  3.2.1
  3.2.2
  3.2.3
  3.2.4
  3.2.5
  3.2.6
  3.3.0
  3.3.1
  3.3.2
  3.3.3
  3.3.4
  3.3.5
  3.3.6
  3.3.7
  3.4.0
  3.4-dev
  3.4.1
  3.4.2
  3.4.3
  3.4.4
  3.4.5
  3.4.6
  3.4.7
  3.4.8
  3.4.9
  3.4.10
  3.5.0
  3.5-dev
  3.5.1
  3.5.2
  3.5.3
  3.5.4
  3.5.5
  3.5.6
  3.5.7
  3.6.0
  3.6-dev
  3.6.1
  3.6.2
  3.6.3
  3.6.4
  3.6.5
  3.6.6
  3.6.7
  3.6.8
  3.6.9
  3.7.0
  3.7-dev
  3.7.1
  3.7.2
  3.7.3
  3.7.4
  3.8.0b4
  3.8-dev
  3.9-dev
  activepython-2.7.14
  activepython-3.5.4
  activepython-3.6.0
  anaconda-1.4.0
  anaconda-1.5.0
  anaconda-1.5.1
  anaconda-1.6.0
  anaconda-1.6.1
  anaconda-1.7.0
  anaconda-1.8.0
  anaconda-1.9.0
  anaconda-1.9.1
  anaconda-1.9.2
  anaconda-2.0.0
  anaconda-2.0.1
  anaconda-2.1.0
  anaconda-2.2.0
  anaconda-2.3.0
  anaconda-2.4.0
  anaconda-4.0.0
  anaconda2-2.4.0
  anaconda2-2.4.1
  anaconda2-2.5.0
  anaconda2-4.0.0
  anaconda2-4.1.0
  anaconda2-4.1.1
  anaconda2-4.2.0
  anaconda2-4.3.0
  anaconda2-4.3.1
  anaconda2-4.4.0
  anaconda2-5.0.0
  anaconda2-5.0.1
  anaconda2-5.1.0
  anaconda2-5.2.0
  anaconda2-5.3.0
  anaconda2-5.3.1
  anaconda2-2018.12
  anaconda2-2019.03
  anaconda2-2019.07
  anaconda3-2.0.0
  anaconda3-2.0.1
  anaconda3-2.1.0
  anaconda3-2.2.0
  anaconda3-2.3.0
  anaconda3-2.4.0
  anaconda3-2.4.1
  anaconda3-2.5.0
  anaconda3-4.0.0
  anaconda3-4.1.0
  anaconda3-4.1.1
  anaconda3-4.2.0
  anaconda3-4.3.0
  anaconda3-4.3.1
  anaconda3-4.4.0
  anaconda3-5.0.0
  anaconda3-5.0.1
  anaconda3-5.1.0
  anaconda3-5.2.0
  anaconda3-5.3.0
  anaconda3-5.3.1
  anaconda3-2018.12
  anaconda3-2019.03
  anaconda3-2019.07
  ironpython-dev
  ironpython-2.7.4
  ironpython-2.7.5
  ironpython-2.7.6.3
  ironpython-2.7.7
  jython-dev
  jython-2.5.0
  jython-2.5-dev
  jython-2.5.1
  jython-2.5.2
  jython-2.5.3
  jython-2.5.4-rc1
  jython-2.7.0
  jython-2.7.1
  micropython-dev
  micropython-1.9.3
  micropython-1.9.4
  micropython-1.10
  micropython-1.11
  miniconda-latest
  miniconda-2.2.2
  miniconda-3.0.0
  miniconda-3.0.4
  miniconda-3.0.5
  miniconda-3.3.0
  miniconda-3.4.2
  miniconda-3.7.0
  miniconda-3.8.3
  miniconda-3.9.1
  miniconda-3.10.1
  miniconda-3.16.0
  miniconda-3.18.3
  miniconda2-latest
  miniconda2-3.18.3
  miniconda2-3.19.0
  miniconda2-4.0.5
  miniconda2-4.1.11
  miniconda2-4.3.14
  miniconda2-4.3.21
  miniconda2-4.3.27
  miniconda2-4.3.30
  miniconda3-latest
  miniconda3-2.2.2
  miniconda3-3.0.0
  miniconda3-3.0.4
  miniconda3-3.0.5
  miniconda3-3.3.0
  miniconda3-3.4.2
  miniconda3-3.7.0
  miniconda3-3.8.3
  miniconda3-3.9.1
  miniconda3-3.10.1
  miniconda3-3.16.0
  miniconda3-3.18.3
  miniconda3-3.19.0
  miniconda3-4.0.5
  miniconda3-4.1.11
  miniconda3-4.2.12
  miniconda3-4.3.11
  miniconda3-4.3.14
  miniconda3-4.3.21
  miniconda3-4.3.27
  miniconda3-4.3.30
  pypy-c-jit-latest
  pypy-c-nojit-latest
  pypy-dev
  pypy-stm-2.3
  pypy-stm-2.5.1
  pypy-1.5-src
  pypy-1.5
  pypy-1.6
  pypy-1.7
  pypy-1.8
  pypy-1.9
  pypy-2.0-src
  pypy-2.0
  pypy-2.0.1-src
  pypy-2.0.1
  pypy-2.0.2-src
  pypy-2.0.2
  pypy-2.1-src
  pypy-2.1
  pypy-2.2-src
  pypy-2.2
  pypy-2.2.1-src
  pypy-2.2.1
  pypy-2.3-src
  pypy-2.3
  pypy-2.3.1-src
  pypy-2.3.1
  pypy-2.4.0-src
  pypy-2.4.0
  pypy-2.5.0-src
  pypy-2.5.0
  pypy-2.5.1-src
  pypy-2.5.1
  pypy-2.6.0-src
  pypy-2.6.0
  pypy-2.6.1-src
  pypy-2.6.1
  pypy-4.0.0-src
  pypy-4.0.0
  pypy-4.0.1-src
  pypy-4.0.1
  pypy-5.0.0-src
  pypy-5.0.0
  pypy-5.0.1-src
  pypy-5.0.1
  pypy-5.1-src
  pypy-5.1
  pypy-5.1.1-src
  pypy-5.1.1
  pypy-5.3-src
  pypy-5.3
  pypy-5.3.1-src
  pypy-5.3.1
  pypy-5.4-src
  pypy-5.4
  pypy-5.4.1-src
  pypy-5.4.1
  pypy-5.6.0-src
  pypy-5.6.0
  pypy-5.7.0-src
  pypy-5.7.0
  pypy-5.7.1-src
  pypy-5.7.1
  pypy2-5.3-src
  pypy2-5.3
  pypy2-5.3.1-src
  pypy2-5.3.1
  pypy2-5.4-src
  pypy2-5.4
  pypy2-5.4.1-src
  pypy2-5.4.1
  pypy2-5.6.0-src
  pypy2-5.6.0
  pypy2-5.7.0-src
  pypy2-5.7.0
  pypy2-5.7.1-src
  pypy2-5.7.1
  pypy2.7-5.8.0-src
  pypy2.7-5.8.0
  pypy2.7-5.9.0-src
  pypy2.7-5.9.0
  pypy2.7-5.10.0-src
  pypy2.7-5.10.0
  pypy2.7-6.0.0-src
  pypy2.7-6.0.0
  pypy2.7-7.0.0-src
  pypy2.7-7.0.0
  pypy2.7-7.1.0-src
  pypy2.7-7.1.0
  pypy2.7-7.1.1-src
  pypy2.7-7.1.1
  pypy3-dev
  pypy3-2.3.1-src
  pypy3-2.3.1
  pypy3-2.4.0-src
  pypy3-2.4.0
  pypy3.3-5.2-alpha1-src
  pypy3.3-5.2-alpha1
  pypy3.3-5.5-alpha-src
  pypy3.3-5.5-alpha
  pypy3.5-c-jit-latest
  pypy3.5-5.7-beta-src
  pypy3.5-5.7-beta
  pypy3.5-5.7.1-beta-src
  pypy3.5-5.7.1-beta
  pypy3.5-5.8.0-src
  pypy3.5-5.8.0
  pypy3.5-5.9.0-src
  pypy3.5-5.9.0
  pypy3.5-5.10.0-src
  pypy3.5-5.10.0
  pypy3.5-5.10.1-src
  pypy3.5-5.10.1
  pypy3.5-6.0.0-src
  pypy3.5-6.0.0
  pypy3.5-7.0.0-src
  pypy3.5-7.0.0
  pypy3.6-7.0.0-src
  pypy3.6-7.0.0
  pypy3.6-7.1.0-src
  pypy3.6-7.1.0
  pypy3.6-7.1.1-src
  pypy3.6-7.1.1
  pyston-0.5.1
  pyston-0.6.0
  pyston-0.6.1
  stackless-dev
  stackless-2.7-dev
  stackless-2.7.2
  stackless-2.7.3
  stackless-2.7.4
  stackless-2.7.5
  stackless-2.7.6
  stackless-2.7.7
  stackless-2.7.8
  stackless-2.7.9
  stackless-2.7.10
  stackless-2.7.11
  stackless-2.7.12
  stackless-2.7.14
  stackless-3.2.2
  stackless-3.2.5
  stackless-3.3.5
  stackless-3.3.7
  stackless-3.4-dev
  stackless-3.4.1
  stackless-3.4.2
  stackless-3.4.7
  stackless-3.5.4


There are many versions of python supported by pyenv. Most users want the latest version of 2.7.x or 3.6.x

Note: Python 2.7 will reach the end of its life on January 1st, 2020

Install Python 2.7.16

pyenv install 2.7.16 -v 2>&1 | tee pyenv-install-2.7.16.log


The install takes several minutes to complete after the "Installing ..." message appears.

Installing openssl 1.0.2

This is necessary for Centos 6 systems in order to install python 3.7.x  . This has already been done for godel  and is found at /usr/global/openssl

mkdir src/openssl-1.0.2t
cd src/openssl-1.0.2t
wget https://www.openssl.org/source/openssl-1.0.2t.tar.gz -O openssl-1.0.2t.tar.gz
tar -zxf openssl-1.0.2t.tar.gz
cd openssl-1.0.2t
./config --prefix=/usr/global/openssl-1.0.2t --openssldir=/usr/global/openssl-1.0.2t shared zlib 2>&1 | tee config-$(date +%F-%H%M).log
make 2>&1 | tee make-$(date +%F-%H%M).log
make test 2>&1 | tee make-test-$(date +%F-%H%M).log
make install 2>&1 | tee make-install-$(date +%F-%H%M).log

Install Python 3.7.2


On godel first do: qrsh

On servers with Centos 6 ( like godel ) an updated version of openssl must be used with python 3.7.x, do the following to install python 3.7.x with pyenv:


source ~/enable_pyenv.sh
source /usr/global/openssl/enable_openssl.sh
CONFIGURE_OPTS="--with-openssl=/usr/global/openssl/" pyenv install 3.7.2 -v 2>&1 | tee pyenv-install-3.7.2.log


The install takes several minutes to complete after the "Installing ..." message appears.


Show current python

which python
pyenv versions
python -V

 

Switch to different version of python and show it


pyenv local 2.7.16
python -V

Note: when using pip from within the pyenv environment, there is no need to use the --user switch, because you have access to default python directories for version you have installed.


When trying to install modules with pip, if the following error comes up:


$ pip install numpy 
Collecting numpy
/home/cgchildr/.pyenv/versions/2.7.14/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
  SNIMissingWarning
  Could not find a version that satisfies the requirement numpy (from versions: )
No matching distribution found for numpy 


You'll need to first install updated version of openssl using the following command:


$ pip install --trusted-host pypi.python.org pyOpenSSL ndg-httpsclient pyasn1 
Collecting pyOpenSSL
/home/cgchildr/.pyenv/versions/2.7.14/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
  SNIMissingWarning
  Downloading pyOpenSSL-17.5.0-py2.py3-none-any.whl (53kB)
    100% |████████████████████████████████| 61kB 6.0MB/s
Collecting ndg-httpsclient
  Downloading ndg_httpsclient-0.4.4-py2-none-any.whl
Collecting pyasn1
  Downloading pyasn1-0.4.2-py2.py3-none-any.whl (71kB)
    100% |████████████████████████████████| 71kB 17.8MB/s
Collecting six>=1.5.2 (from pyOpenSSL)
  Downloading six-1.11.0-py2.py3-none-any.whl
Collecting cryptography>=2.1.4 (from pyOpenSSL)
  Downloading cryptography-2.2.2-cp27-cp27mu-manylinux1_x86_64.whl (2.2MB)
    100% |████████████████████████████████| 2.2MB 18.8MB/s
Collecting cffi>=1.7; platform_python_implementation != "PyPy" (from cryptography>=2.1.4->pyOpenSSL)
  Downloading cffi-1.11.5-cp27-cp27mu-manylinux1_x86_64.whl (407kB)
    100% |████████████████████████████████| 409kB 16.3MB/s
Collecting enum34; python_version < "3" (from cryptography>=2.1.4->pyOpenSSL)
  Downloading enum34-1.1.6-py2-none-any.whl
Collecting asn1crypto>=0.21.0 (from cryptography>=2.1.4->pyOpenSSL)
  Downloading asn1crypto-0.24.0-py2.py3-none-any.whl (101kB)
    100% |████████████████████████████████| 102kB 16.9MB/s
Collecting idna>=2.1 (from cryptography>=2.1.4->pyOpenSSL)
  Downloading idna-2.6-py2.py3-none-any.whl (56kB)
    100% |████████████████████████████████| 61kB 14.8MB/s
Collecting ipaddress; python_version < "3" (from cryptography>=2.1.4->pyOpenSSL)
  Downloading ipaddress-1.0.19.tar.gz
Collecting pycparser (from cffi>=1.7; platform_python_implementation != "PyPy"->cryptography>=2.1.4->pyOpenSSL)
  Downloading pycparser-2.18.tar.gz (245kB)
    100% |████████████████████████████████| 256kB 18.0MB/s
Installing collected packages: six, pycparser, cffi, enum34, asn1crypto, idna, ipaddress, cryptography, pyOpenSSL, pyasn1, ndg-httpsclient
  Running setup.py install for pycparser ... done
  Running setup.py install for ipaddress ... done
Successfully installed asn1crypto-0.24.0 cffi-1.11.5 cryptography-2.2.2 enum34-1.1.6 idna-2.6 ipaddress-1.0.19 ndg-httpsclient-0.4.4 pyOpenSSL-17.5.0 pyasn1-0.4.2 pycparser-2.18 six-1.11.0 


After that, python modules with install normally:


$ pip install numpy 
Collecting numpy
/home/cgchildr/.pyenv/versions/2.7.14/lib/python2.7/site-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
  SNIMissingWarning
  Downloading numpy-1.14.2-cp27-cp27mu-manylinux1_x86_64.whl (12.1MB)
    100% |████████████████████████████████| 12.1MB 66kB/s
Installing collected packages: numpy
Successfully installed numpy-1.14.2 



Remove pyenv


remove ( or move ) the following

~/.pyenv

~/.python-version

~/enable_pyenv.sh


pipenv


research




pythonbrew

PYTHONBREW IS NOW DEPRECATED

THIS DOCUMENTATION IS LEFT HERE FOR HISTORICAL REASONS

The developer encourages the use of pyenv instead: https://github.com/yyuu/pyenv

Homepage: [

install instructions:

curl -kLO http://github.com/utahta/pythonbrew/raw/master/pythonbrew-install
chmod +x pythonbrew-install
./pythonbrew-install
source /home/cgchildr/.pythonbrew/etc/bashrc

or

curl -kL http://xrl.us/pythonbrewinstall | bash
source ~/.pythonbrew/etc/bashrc
$ pythonbrew help
usage: pythonbrew COMMAND [OPTIONS]

options:
  --version   show program's version number and exit
  -h, --help  Show help

Commands available:
  buildout: Runs the buildout with specified or current using python
  cleanup: Remove stale source folders and archives
  help: Show available commands
  install: Build and install the given version of python
  list: List the installed all pythons
  off: Disable pythonbrew
  py: Runs a named python file against specified and/or all pythons
  switch: Permanently use the specified python as default
  symlink: Create/Remove a symbolic link on your $PATH
  uninstall: Uninstall the given version of python
  update: Update the pythonbrew to the latest version
  use: Use the specified python in current shell
  venv: Create isolated python environments
  version: Show version

Further Instructions:
  https://github.com/utahta/pythonbrew

Show available version of python to install:

$ pythonbrew  list -k
# Pythons
Python-1.5.2
Python-1.6.1
Python-2.0.1
Python-2.1.3
Python-2.2.3
Python-2.3.7
Python-2.4.6
Python-2.5.6
Python-2.6.8
Python-2.7.3
Python-3.0.1
Python-3.1.4
Python-3.2.3
Python-3.3.0

Install a version of python: ( I've only been able to install successfully by using the --force switch

pythonbrew available
pythonbrew help
pythonbrew list -k
pythonbrew install 2.7.3
which python
python -V

Using a version of python under pythonbrew:

source ~/.pythonbrew/etc/bashrc
pythonbrew list
pythonbrew switch 2.7.3
which python



echo 'export PYENV_ROOT="$HOME/.pyenv"' >> ~/.bash_profile
echo 'export PATH="$PYENV_ROOT/bin:$PATH"' >> ~/.bash_profile
echo 'eval "$(pyenv init -)"' >> ~/.bash_profile

exec $SHELL

( or log out and back in )


  • No labels
{"serverDuration": 62, "requestCorrelationId": "db208bd517aabd22"}View graphic versionView graphic versionView graphic version