{"id":137,"date":"2017-07-07T21:34:32","date_gmt":"2017-07-07T21:34:32","guid":{"rendered":"https:\/\/xiaohaionline.com\/?p=137"},"modified":"2017-07-08T01:45:32","modified_gmt":"2017-07-08T01:45:32","slug":"understanding-the-python-world","status":"publish","type":"post","link":"https:\/\/xiaohaionline.com\/index.php\/2017\/07\/07\/understanding-the-python-world\/","title":{"rendered":"Understanding the Python world"},"content":{"rendered":"

There are two classes of Python users, those who use Python for management scripting and web development, and those who use Python for scientific computing. According to many, Python is winning the war against R in the field of machine learning. As machine learning becomes ever more important and popular, so is Python.<\/p>\n

This article is for those just got into the Python world. For those experienced Python developers, you are excused to leave now. Python is extremely simple to start with, the traditional Hello World<\/em> program takes only one line. Before no time, you would be writing classes, methods, system calls, and you really enjoy the convenience and power of the language. But quickly, you also realize that, there are many strange codes floating around. And you hear about lots of unacquainted jargons. You could feel a little puzzled, and a little confused about the big picture of the Python world.<\/p>\n

That is where this article comes to help. I intend to draw a broad picture of the Python world, with pointers for you to explore in depth. At the end of the article, you won’t become an expert on anything with Python. But, hopefully, you will be satisfied in understanding the big picture, and know where to go for further information and to gain expertise.
\n<\/p>\n

Installation and environments<\/h3>\n

All software packages start with installations. As of the writing of this post, there are two major versions of Python that are prevalent, i.e. Python 2<\/a> and Python 3<\/a>. Usually, Python 2<\/a> refers to Python with version 2.7.x, and Python 3<\/a> refers to Python with version 3.5+. Python 2 and Python 3 are incompatible<\/a> on many occasions. Which one to choose? The simplified answer is to choose Python 3 for new projects and use whatever version your old project was using for the old projects.<\/p>\n

Chances are, in the near future, you will need to work in both Python 2 and Python 3 worlds for different projects. So you install both. But then, how do you switch between the two? When you run python -m xxx<\/em> command, how do you know which version is started? Do you need to manually set up your environment variables for switching? That would be too complicated and error prone, and you need tools. That is when Virtualenv<\/a> and Anaconda<\/a> come into play.<\/p>\n

Virtualenv<\/a> and Anaconda<\/a> solve more problems than Python versions. Remember, for any real projects, you would need support from many libraries, right? Different projects usually rely on different versions of the same libraries, i.e. it is incorrect to assume all of your projects can always use the latest version of the same libraries. For different projects could be at different lifecycles. So the industry’s best practice is to isolate the python version + specific libraries versions<\/em> into one bundle called: environments<\/em><\/strong>.<\/p>\n

The difference between Virtualenv<\/a> and Anaconda<\/a>? Again, the simplest answer is<\/p>\n

Anaconda = Virtualenv + Pip<\/div><\/div>\n

functionality wise. Wait, what is Pip? Be patient, I will talk about it. But before that, I’d like to show some codes:<\/p>\n

On MacOS, with Virtualenv, commands to install and setup both Python 2 & 3 environments are:<\/p>\n

\n\n
brew install python
\nbrew install python3
\nvirtualenv env2
\nvirtualenv --python=python3 env3<\/div><\/div>\n\n<\/pre>\n
In the above codes, env2 and env3 and simply subfolders (that you can change to your favorite folder names) where the environments will be setup, i.e. all python binaries and libraries will be install and copied into those subfolders.<\/p>\n
Environments need to be activated before invoking Python commands:<\/p>\n
\n\n
source env2\/bin\/activate
\ndeactivate
\nsource env3\/bin\/activate
\ndeactivate<\/div><\/div>\n\n<\/pre>\n
For Anaconda:<\/p>\n
\n\n
conda create --prefix ~\/bio-env python=3.4
\nsource activate ~\/bio-env
\nsource deactivate<\/div><\/div>\n\n<\/pre>\n
In the above codes, you can change to the Python version to use Python 2.<\/p>\n
Package installer<\/h3>\n
As promised, I will talk about Pip<\/a>. As you would expect, any programmer needs libraries. In Python, libraries are called packages. Like in Java world, you pull and install packages from some central repositories called PyPI – the Python Package Index<\/a>. Most likely you don’t have to install Pip<\/a>. For Pip<\/a> is installed when you install your Python environments.<\/p>\n
Some sample pip commands for installing packages:<\/p>\n
\n\n
pip install SomePackage            # latest version
\npip install SomePackage==1.0.4     # specific version
\npip install 'SomePackage&gt;=1.0.4'     # minimum version
\npip install -r requirements.txt . # "Requirements files" are files containing a list of items to be installed
\npip list # list installed packages
\npip search "query" # search for packages
\npip install --upgrade SomePackage # upgrade packages if needed<\/div><\/div>\n\n<\/pre>\n
Another concept you need to understand is Python Wheels<\/a>.<\/p>\n
Pip install<\/div><\/div>\n
, by default, installs from source files involving compilation. This process could be demanding at times, especially when non-python libraries are involved. So the invention of the Python Wheels<\/a> is to allow others to pre-built a package for you, so that you can quickly install without worrying about how to compile the package. Make sense, right?<\/p>\n
\n\n
pip install SomePackage-1.0-py2.py3-none-any.whl # installing from wheels
\npip install wheel
\npip wheel --wheel-dir=\/local\/wheels -r requirements.txt # To build wheels to a local directory<\/div><\/div>\n\n<\/pre>\n
Remember Anaconda<\/a>? It gets its own package installer tool. It is called Conda<\/a>. Its usage patttern is similar to pip, but using a different central repository, the one from Continnum<\/a>.<\/p>\n
\n\n
conda create --name snakes python=3 # create python 3 environment called snakes
\nconda info --envs # display the list of environments
\nsource activate snowflakes # activate snowflakes environment
\nconda search beautiful-soup # search packages
\nconda install package_name # install a package
\nconda list # list installed packages
\nconda update package_name # update a package
\nconda update conda # update conda itself
\nconda update python # update to the latest 2.x or 3.x version of python
\nconda remove packge_name # uninstall a package<\/div><\/div>\n\n<\/pre>\n
In addition to the central repository, Anaconda also supports “private” repositories hosted on anaconda.org<\/a>. The “private” repositories are called channels<\/a>. In install from a channel<\/a>, you can use the -c option, e.g.<\/p>\n
\n\n
conda install -c pandas bottleneck<\/div><\/div>\n\n<\/pre>\n
Don’t worry, with Anaconda, you can still use Pip.<\/p>\n
I personally prefer Anaconda because it has more integrated support cross platforms and it has enough packages for scientific computing.<\/p>\n
Your own modules and packages<\/h3>\n
With the installation and environments taken care of, you are ready to do some serious coding.<\/p>\n
You write a python file with the .py as the extension. Congratulations, you’ve got your first module<\/a> developed. Of course, you will want to write your own libraries for others to use. Well it is quite somple:<\/p>\n
    \n
  1. Create a folder named after your library name<\/li>\n
  2. Within that folder, put an empty __init__.py in it<\/li>\n<\/ol>\n
    Congratulations, you got your first package ready. Of course, it has no meat in it. For you have not done any real coding.<\/p>\n
    If you want some classes visible to others when they write<\/p>\n
    import your_package_name<\/div><\/div>\n
    , you define the class in the __init__.py object. If you prefer other to write<\/p>\n
    import your_package_name.your_module<\/div><\/div>\n
    or<\/p>\n
    from your_package_name import your_module<\/div><\/div>\n
    , you keep the __init__.py empty and create<\/p>\n
    your_module.py<\/div><\/div>\n
    .<\/p>\n
    Don’t expect me to talk about how to write Python classes. It should be another of your Google searches.<\/p>\n
    Install your own packages<\/h3>\n
    Time to share your work with others. Visit this<\/a> page to equip yourself with the full knowledge. Basically you need to provide a set of meta information to inform the packager how to package your stuff. Briefly:<\/p>\n