Install multiple versions of python i OS X

If you need install multi python versions in the host, and use different version of python for different application, it is very useful to use pyenv to manage different version installed in the host. For example, I have python 3.6 and 2.7 installed in my Mac, where python –version shows default version is 3.6

mac-c02t6npagtfj:ansible_test grayin$ python –version

Python 3.6.4

But I want to change my default python to version 2.7 in my Mac, and use pip in package 2,7 to install new packages in python2.7.

What I have done is to install pyenv first, then use pyenv to reinstall version 2,7 and 3.6, after that, change version of python used in the host:

$ brew update
$ brew install pyenv

To enable pyenv in your Bash shell, you need to run:

$ eval "$(pyenv init -)"
After this, we can use pyenv to reinstall python
$ pyenv install 2.7.14
$ pyenv install 3.6.4

 

And you can switch between python versions with the command:

$ pyenv global 2.7.14

Also you can set a python version for the current directory with:

$ pyenv local 3.6.4
mac-c02t6npagtfj:ansible_test grayin$ pyenv versions
  system
  2.7
* 2.7.14 (set by /Users/grayin/.pyenv/version)
  3.6.4
After this:
mac-c02t6npagtfj:ansible_test grayin$ python --version
Python 2.7.14
mac-c02t6npagtfj:ansible_test grayin$ pip --version
pip 9.0.1 from /Users/grayin/.pyenv/versions/2.7.14/lib/python2.7/site-packages (python 2.7)

Even pip as default is attached to python2.7 now.

Pyenv can be helpful to avoid home-brew lost track of python

Sometime people may feel very confused that he/she has installed a package already but when importing it in the script, it would show that package is not installed or can not find the module.  This is because that when pip is attached to version 2.7, all package installed using pip will be installed for version 2.7, when pip is attached to version 3.6, all package installed using pip will be for version 3.6.  If version 2.7 is used but actually pip is attached to version 3.6, package will be installed in version 3.6.

Another alternative to address this issue is to use different /path/pip to install the package for the different version of python. Anyhow I have not tested this option.

Some also recommend the easier way for using pip in different version installation:

$ pip2.6 install otherpackage
$ pip2.7 install mybarpackage

However, this does not work for me:

mac-c02t6npagtfj:ansible_test grayin$ pip2.7 --version
pip 9.0.1 from /Users/grayin/.pyenv/versions/2.7.14/lib/python2.7/site-packages (python 2.7)
mac-c02t6npagtfj:ansible_test grayin$ pip3.6 --version
pyenv: pip3.6: command not found
The `pip3.6' command exists in these Python versions:
  3.6.4
mac-c02t6npagtfj:ansible_test grayin$ pip3.6.4 --version
-bash: pip3.6.4: command not found

 

Advertisements

setup An Ansible Test Lab For Windows Managed hosts

I use Mac, and have setup a virtual window 10 host via VMware fusion. The target of the lab is to use ansible to restart windows 10 virtual host, after this, I have created a playbook to finish several tasks in this virtual host: test connection using win_ping, restart host using win_reboot, and visit a website using win_uri. The lab can be extended by adding new task into the playbook:

1, create host file (host inventory) in the folder /etc/ansible/hosts

#mkdir /etc/ansible

#touch /etc/ansible/hosts

#vi /etc/ansible/hosts

add IP address of the virtual host that will be connected remotely

2, create inventory file to save the other attributes. for example, create “windows.yml” to save username/password, the connection method used to manage windows hosts:

$ pwd

/users/username/ansible_test/group_vars

 

:group_vars grayin$ vi windows.yml

#It is suggested that these be encrypted with ansible-vault:

# ansible-vault edit group_vars/windows.yml

ansible_ssh_user: username

ansible_ssh_pass: password

ansible_ssh_port: 5986

ansible_connection: winrm

# The following is necessary for Python 2.7.9+ (or any older Python that has backported SSLContext, eg, Python 2.7.5 on RHEL7) when using default WinRM self-signed certificates:

ansible_winrm_server_cert_validation: ignore

Normally ansible uses ssh to communicate managed hosts, that means in order to manage windows host, windows need have ssh server enabled first. Here is a indication about the ways to enable ssh on windows 10

However ansible recommended to use Winrm to remotely manage windows hosts. Here is ansible document about winrm setup. In short, I did the following in my virtual window 10 machine, and then set ansbile_connection attribute to “winrm” in my above windows.yml file.

I did download file “ConfigureRemotingForAnsible.ps1” to the windows hosts. and run the following command in powershell.

$url = "https://raw.githubusercontent.com/ansible/ansible/devel/examples/scripts/ConfigureRemotingForAnsible.ps1"
$file = "$env:SystemDrive\temp\ConfigureRemotingForAnsible.ps1"

(New-Object -TypeName System.Net.WebClient).DownloadFile($url, $file)

powershell.exe -ExecutionPolicy ByPass -File $file

After this, I got 2 listener for http port 5895 and https 5896.

winrm enumerate winrm/config/Listener

This will output something like the following:

Listener
    Address = *
    Transport = HTTP
    Port = 5985
    Hostname
    Enabled = true
    URLPrefix = wsman
    CertificateThumbprint
    ListeningOn = 10.0.2.15, 127.0.0.1, 192.168.56.155, ::1, fe80::5efe:10.0.2.15%6, fe80::5efe:192.168.56.155%8, fe80::
ffff:ffff:fffe%2, fe80::203d:7d97:c2ed:ec78%3, fe80::e8ea:d765:2c69:7756%7

Listener
    Address = *
    Transport = HTTPS
    Port = 5986
    Hostname = SERVER2016
    Enabled = true
    URLPrefix = wsman
    CertificateThumbprint = E6CDAA82EEAF2ECE8546E05DB7F3E01AA47D76CE
    ListeningOn = 10.0.2.15, 127.0.0.1, 192.168.56.155, ::1, fe80::5efe:10.0.2.15%6, fe80::5efe:192.168.56.155%8, fe80::
ffff:ffff:fffe%2, fe80::203d:7d97:c2ed:ec78%3, fe80::e8ea:d765:2c69:7756%7

3, run testto window 10 using winrm connection. I use win_ping module, which is doing the similar thing as module “ping” towards linux/unix server

:ansible_test grayin$ ansible windows -i host -m win_ping

192.168.2.129 | SUCCESS => {

    “changed”: false, 

    “failed”: false, 

    “ping”: “pong”

}

4, test restart of windows 10 host by using module “win_reboot”

mac-c02t6npagtfj:ansible_test grayin$ ansible windows -i host -m win_reboot

192.168.2.129 | SUCCESS => {

    “changed”: true, 

    “elapsed”: 162, 

    “failed”: false, 

    “rebooted”: true

}

Now we have seen that virtual host is reboot and up again. Below I will create a playbook to implement several tasks one by one

5,create a file named as “test.yaml”

mac-c02t6npagtfj:ansible_test grayin$ vi test.yaml


- name: Hit a Specific Host on the Server

  hosts: 192.168.2.129

  vars:

    user: "username"

    password: "passoword"

  tasks:

 - name: ping host ok

    win_ping:

  - name: restart the host

    win_reboot:

   - name: get url

    win_uri:

      url: "http://www.google.com/"

      method: GET

  - name: Perform a HEAD on an Endpoint

    win_uri:

      url: http://www.google.se/

      method: HEAD

6, run playbook and get the following result:

mac-c02t6npagtfj:ansible_test grayin$ ansible-playbook -i group_vars/windows.yml -i host task.yaml

PLAY [Hit a Specific Host on the Server] **********************************************************************************************************************

TASK [Gathering Facts] ****************************************************************************************************************************************

ok: [192.168.2.129]

TASK [ping host ok] *******************************************************************************************************************************************

ok: [192.168.2.129]

TASK [restart the host] ***************************************************************************************************************************************

changed: [192.168.2.129]

TASK [get url] ************************************************************************************************************************************************

ok: [192.168.2.129]

TASK [Perform a HEAD on an Endpoint] **************************************************************************************************************************

ok: [192.168.2.129]

PLAY RECAP ****************************************************************************************************************************************************

192.168.2.129              : ok=5    changed=1    unreachable=0    failed=0

7, extend the playbook

We can add more management tasks into the playbook, ansible documents has listed all the available modules that can be used to manage windows hosts. More info can be found here.

Here is the blog regarding “setup An Ansible Test Lab For Windows Managed Nodes & Custom Windows Modules”

pandas, csv, xlrd, which one is best for csv and xls files?

 

1, .csv vs .xls:

csv hold plain text as a series of values (cells) separated by commas (,) in a series of lines (rows). However, when I tested in Mac, it seems that values are separated by (;), not (,) as default.  While xls is an MS Excel workbook binary file.

2, python modules: csv & pandas & xlrd:

Both csv and pandas can be used to read .csv file.csv can be used to do simple work with the data that stores in .csv file, while pandas enable you to perform analysis or intense data manipulations.

Both pandas and xlrd can be used to read .xls files. Actually, pandas has used module xlrd when reading or writing .xls files in lower level. So it is xlrd that gave the possibility for reading .xls. However pandas provide calculation and analysis capabities on top of that. The evidence is that if you install pandas but without installing xlrd module, it will shows error when running pandas.read_excel function.

In summary, pandas is a good tool to use for reading both .csv or xls file

3, example codes using pandas to read a .xls file:

Below is a simple code example which can be used to read a .xls files and read the index value or column value accordingly

import sys
import pandas as pd
from pandas import ExcelWriter
from pandas import ExcelFile

filename = sys.argv[1]
sheet_name = sys.argv[2]
df = pd.read_excel(filename, sheet_name = sheet_name)

print(“Column headings:”)
print(df.columns) # print all index value, the first row in excel
print(df[‘$IPv4 Network’]) #print column value whose index is “$IPv4 Network”

4, code example to use csv module to read a csv file and save the data into dictionary:

import csv
import sys
#coding=utf-8

filename = sys.argv[1]

with open(filename,’rU’) as csvfile:
reader = csv.DictReader(csvfile,delimiter= ‘;’)
for line in reader:
print(line)

with .csv file example:

A B C
qer ewr tr
ewe dfe cbv

The output of the above .csv are:

$ python3 Documents/python/readcsvfile.py Documents/test.csv

OrderedDict([(‘\ufeffA’, ‘qer’), (‘B’, ‘ewr’), (‘C’, ‘tr’)])

OrderedDict([(‘\ufeffA’, ‘ewe’), (‘B’, ‘dfe’), (‘C’, ‘cbv’)])

$ python Documents/python/readcsvfile.py Documents/test.csv

{‘\xef\xbb\xbfA’: ‘qer’, ‘B’: ‘ewr’, ‘C’: ‘tr’}

{‘\xef\xbb\xbfA’: ‘ewe’, ‘B’: ‘dfe’, ‘C’: ‘cbv’}

Especially \xef\xbb\xbf  is byte order mark (BOM) of utf-8 unicode, because my test.csv is using utf-8. Normally .csv is encoded with utf-8 or ASCII. And in my example, delimiter = ‘;’ (default is ‘ , ‘) in order to get the correct result.

The result shows that in dictionary there are 3 keywords: A, B and C, each row shows values for the key word, that is ‘qer’, ‘ewe’ for A, ‘ewe’ and ‘die’ for B, etc.

5, Unicode

It is quite often to meet encoding/decoding problem when you are reading a file using unknown unicode. The best way is to find out what unicode used in the original file, then covert it according to the needs. defaultcodevalue in python 2 is ASCII, when reading a utf-8 file, it may generate some error, but not always, we need keep in mind of this.

Some people use the following script to solve above problem, but this is not recommended. More discussion can be found here:

import sys
reload(sys)
sys.setdefaultencoding("utf-8")

Catalyst 2960S bootloop/keep reloading during upgrading

2960 switch went to a booting loop due to bug CSCvf46629 when doing an upgrade from 15.2(2)E7 from 15.0(2)SE7.  That is when VTP mode set as client on the switch, switch will go into reloadloop when trying to upgrade switch to 12.2.2E7.

To recover the switch we need to move the switch to old IOS then boot switch up, then change the vtp mode to transparent.

If you have old IOS image in the flash, then just need to boot switch during pressing “mode” button until switch prompt shows up, then doing flash_init and boot from the old image.

In my case the old IOS image was removed during new image installation, so I have to download the old IOS image to the switch via console port.

Below is the steps to follow:

In rommon

-flash_init

-Set BAUD 115200

-copy xmodem: flash:OLD_IOS_image

Transfer the IOS file from computer to the switch with Serial tool in Mac or Hyperterminal in window. Once old IOS is copied:

-Boot flash:OLD_IOS_image

Once switch boots up in old IOS customer needs to boot up the switch and change the vtp mode, Then do the upgrade from 15.2(2)E7 from 15.0(2)SE7Then change the vtp mode back.

To set Baud as 115200 is because using the default Baud rate 9600 for file transfering will take 3-4 hours to download the ios image to the switch. Once switched to rate 11520 I can download image within half hour. Some other people also tried a different rate, like 57600, which can work too with a little bit longer download time.

One another alternative workaround might be to rename config.text to config.backup in rommon, then tried to boot up the switch with new image. By removing config.text the switch will not have vtp mode client configured, it should be able to bypass VTP client mode bug and boot the switch without need to go back to old image.  After switch is booted with new image, we can just to the following to recover the configuration:

-rename config.backup config.text

-copy config.text running-config

The process is the same as the process of password recovery. This alternative work-around haven’t been verified but is worth testing.

upload file to ftp server via mac os x

mac os has inbuild ftp client function and tftp server function. Click here to check how to activate and connect tftp server in mac.

Below is the step to upload file to ftp server via mac os x

1, open terminal

2, #ftp serverurl

input username and password

3, use dir to browse folder and file list in ftp server, use cd to enter into the folder where you want to put you file in

4, change local directory path

!list local directory

#lcd

!change local directory to /private/

#lcd /private/

5, upload the file

#put filenamn.name

 

two vtp server in a same domain

When vtp is configured, the configured vlans are stored in vlan.dat file, not in running-configure or startup-configure file.

It is common that two vtp server in a same domain is running, this is for vlans database redudant. When there is a new change generated in one VTP server, the update will be sent out inside the vtp domain, the other vtp server in the same domain will update its vlan.dat also.

Be careful to setup the same vtp version on both servers, otherwise the server with higher version will become master.

When replacing a switch that is acting as VTP server in the network, the best way is to change the new switch into client mode first, in order to get vlans updated from other vtp servers, then change the new switch back to server mode.

Configure OS x as tftp server and download from vrf enabled 6800 chassis

Below is the steps that I used to download files from vrf enabled 6800 chassis to my mac laptop:

1, Enable tftp server in OS x:

sudo launchctl load -F /System/Library/LaunchDaemons/tftp.plist
sudo launchctl start com.apple.tftpd

By default use private/tftpboot/ filefolder for tftp download and upload:

sudo chmod 777 /private/tftpboot
sudo chmod 777 /private/tftpboot/*

2, setup tftp route in 6800

I want to download/upload from/to mgmt port, mgmt port belongs to a separated management vrf. In order to make tftp server IP routed correctly, we need add the following configuration into the chassis:

cat6k#ip tftp source-interface mgmt0

3, Now it is ready for tftp download/upload to OS x from/to vrf enabled Chassis.

cat6k#copy running-config tftp://172.27.x.x

Address or name of remote host [172.27.x.x]?

Destination filename [xxx-confg]? running-config

!!

85918 bytes copied in 1.084 secs (79260 bytes/sec)