GitHub Repos

Sadly by necessity some of my repos are private. Those that are private are clearly marked. For those that are, please don’t ask me to share the code, because I can’t. They’re listed here purely for my reference.

Misc

  • Type python at command line to write python live

    • Python has a repl - presumably that’s what this is?

  • To run a Python script: python script-name.py

  • To start up your virtual env (venv), source the activate file that will have been put in a bin folder in your venv folder:

    • Cmd: cd venv

    • Cmd: source bin/activate

  • Use yum provides python3 to find which package will give us python3

    • We want python 3.7 for Alex’s script - gives really good string interpolation. yum only has Python 3.4

    • yum is locked down in CentOS, everything super secure, so doesn’t have stuff as up to date as brew does (eg python3)

    • You might have to be on your laptop and install python3 using brew

      • But apparently (according to Ben Smith) you can manually add the newer python repo for yum on the command line - but you have to accept the keys for the repo not shipped with the distribution

  • Book: Python the hard way - type everything out, no c+p

  • David is using SCL to manage his Python - something collection - something to do with yum - kind of like a virtual env of his own

  • To install the dependencies you have set up in your requirements.txt: pip install -U -r requirements.txt

  • To start your repl: python

    • In the repl, there are library functions that will work on any object - like dir(var) that will tell you what attributes are available on your object called var. There’s no distinction between attributes and functions but if they’re prefixed with __, that means they’re private.

  • Syntax error at end of file: check for unclosed parentheses

  • Filter a list: http://book.pythontips.com/en/latest/map_filter.html

  • Check if element exists in list: https://thispointer.com/python-how-to-check-if-an-item-exists-in-list-search-by-value-or-condition/

  • Elastic Search python library: https://elasticsearch-py.readthedocs.io/en/master/api.html

Packages - pip

Repl

  • Type python at the command line to get a repl / Python interpreter

  • For multiline code, type \ at the end of the first line and it will keep asking you for more

    • Hit Enter twice to finish typing and execute all lines

    • Remember you still need to use indentation! Just use the tab key.

Mocking

Catching exceptions from third party libraries

  • In our closed_indices script (in observability_toolbox, called something like es_indices_snapshot.py, or es_snapshot_handler.py), we wanted to catch an excpetion from the elasticsearch python library.

    • In the repl it was showing as elasticsearch.exceptions.NotFoundError, so we had to add this line at the top of our file:

    • from elasticsearch.exceptions import NotFoundError

    • and then we could catch it like this: except NotFoundError as e:

Virtual environment / venv

  • David’s zshrc sources several other files, for instance a bash function for cd that runs bin/activate whenever you cd into a directory that has a Python venv in it:

    • function cd() {
       if [[ -d ./venv ]] ; then
         deactivate
       fi

       builtin cd $1

       if [[ -d ./venv ]] ; then
         . ./venv/bin/activate
       fi
      }

Getting Started with a new Python project

  • Python scaffolding:

    • Cmd: mkdir [project name]

    • Guide that David uses: https://docs.python-guide.org/

    • venv [folder name]

      • Or if you have two versions of Python (python –version and python3 –version), you will need this to get python3: python3 -m venv venv

      • That’s creating a venv called “venv” which will then be created in a folder in whatever directory you’re currently in

    • To start up your virtual env, source the activate file that will have been put in a bin folder in your venv folder:

      • Cmd: cd venv

      • Cmd: source bin/activate

      • To prove you’re now in a venv which has a different version of python running:

        • Cmd: python –version

        • - you now have a different version of python

    • Scaffolding a new Python project:

      • David likes to have the following five files:

        • Cmd: touch setup.py

        • Cmd: touch [parent folder name].py

          • Some people say this should be called main.py and should be in your bin

        • Cmd: touch requirements.txt

          • This will contain all the pip requiremtns so you can do pip install and pass this file in

        • Cmd: touch readme.md

        • Cmd: touch makefile

          • contents of makefile

          • init section, test section

      • Also a tests folder: mkdir tests

  • To install the dependencies you have set up in your requirements.txt:

    • Cmd: pip install -U -r requirements.txt

    • To get rid of the warning about your version of pip, add pip as the first line in requirements.txt

  • To start your repl: python

    • Then you can start typing commands in repl

    • Once you see them working you can copy them into your actual Python file

    • There are various library functions available