+++ /dev/null
-# FD.io CSIT migration Python 2.7 to Python 3
-
-## Python 3 version
-
-There is a pre-agreement to migrate to Python 3 version used by
-Ubuntu 18.04-LTS - currently it is version [3.6.8](https://docs.python.org/3.6/whatsnew/changelog.html#python-3-6-8-final).
-
-CentOS7 version 1810 that is used in [FD.io](https://fd.io/) also contains
-Python 3.6.
-
-## Dependency libs
-
-There was used *[caniusepython3](https://pypi.org/project/caniusepython3/)*
-tool to check readiness of current version of csit external libraries for
-Python 3. It identified one external library that needs to be updated to
-support Python 3:
- ```
- (env) vpp@vpp-VirtualBox:~/Documents/csit$ caniusepython3 -r requirements.txt
- Finding and checking dependencies ...
- You need 1 project to transition to Python 3.
- Of that 1 project, 1 has no direct dependencies blocking its transition:
- pypcap
- (env) vpp@vpp-VirtualBox:~/Documents/csit$ caniusepython3 -r tox-requirements.txt
- Finding and checking dependencies ...
- You have 0 projects blocking you from using Python 3!
- (env) vpp@vpp-VirtualBox:~/Documents/csit$
- ```
-
-The latest released version of *[pypcap](https://pypi.org/project/pypcap/)* is
-version 1.2.3 (Python 3 support implemented in version 1.2.0).
-
-Packages were checked for Python 3.6.8 support too and here are proposed
-package versions:
-
-- directly needed packages
- - ecdsa==0.13.3
- - paramiko==2.6.0
- - pycrypto==2.6.1
- - pypcap==1.2.3 # min. v1.2.0 for Python 3.6 support
- - PyYAML==5.1
- - requests==2.22.0 # min. v2.14.0 for Python 3.6 support
- - robotframework==3.1.2
- - scapy==2.4.3 # min. v2.4.0 for Python 3.6 support
- - scp==0.13.2
-
-- directly needed packages for PLRSearch
- - dill==0.3.1.1
- - numpy==1.17.3 # v1.14.5 - compatibility with Python 3.6.2, possible
- incompatibility with Python 3.6.8; v1.14.6 should be compatible with
- Python 3.6.8
- - scipy==1.3.1
-
-- directly needed packages for PAL
- - hdrhistogram==0.6.1
- - pandas==0.25.3
- - plotly==4.1.1
- - PTable==0.9.2
- - Sphinx==2.2.1
- - sphinx-rtd-theme==0.4.0
- - sphinxcontrib-programoutput==0.15
-
-- packages needed by paramiko package
- - bcrypt==3.1.7
- - cffi==1.13.1
- - cryptography==2.8
- - pycparser==2.19
- - PyNaCl==1.3.0
- - six==1.12.0
-
-- packages needed by request package
- - certifi==2019.9.11
- - chardet==3.0.4
- - idna==2.8
- - urllib3==1.25.6
-
-- not needed anymore
- - aenum - enum module in Python 3.6 already contains needed enum types
- - ipaddress - module already included in Python 3.6
- - pexpect - can be removed when corresponding unused code is removed from
- ssh.py
- - pykwalify + docpot + python-dateutil - can be removed if virl not used
- anymore
-
-After discussion there is an agreement to use pip freeze for indirect
-dependencies when all direct dependency versions are resolved - see example of
-*[requirements.txt](https://gerrit.fd.io/r/c/csit/+/23207/17/requirements.txt)*
-file in CSIT gerrit commit
-[Python3: PIP requirement](https://gerrit.fd.io/r/c/csit/+/23207).
-
-## Required CSIT code changes
-
-There were identified following code changes that need to be addressed during
-Python 2.7 to Python 3 migration in CSIT:
-- imports relative to package
- - `import submodul1` => `from . import submodule1`
- - `from csv import my_csv` => `from .csv import my_csv`
-- StringIO
- - `import StringIO` => `from io import StringIO`
-- `StandardError` -=> `Exception`
-- raising exceptions - should be ready
- - `raise ValueError, "wrong value"` => `raise ValueError("wrong value")`
-- catching exceptions - should be ready
- - `except ValueError, e:` => `except ValueError as e:`
-- integers
- - `long` => `int`
-- strings and bytes
- - `unicode` => `str`
- - `basestring` => `str`
- - `str` => `bytes` - not generally, only if bytes type really required
- - use following string style conventions:
- - `u"a unicode string literal"`
- - `b"a bytes string literal"`
- - `f"a formatted unicode string literal"` - `f"He said his name is {name}"`
- instead of `"He said his name is {n}".format(n=name)`
-- integer division with rounding down
- - `2 / 3` => `2 // 3`
-- metaclasses - use only new style
- - `class Form(BaseForm, metaclass=FormType):`
-- for-loop variables and the global namespace leak
- - for-loop variables don't leak into the global namespace anymore
-- returning iterable objects instead of lists
- - `xrange` => `range`
- - `range` => `list(range())`
- - `map` => `list(map())`
- - `zip` => `list(zip())`
- - `filter` => `list(filter())`
- - dictionaries
- - `.iteritems()` => `.items()`
- - `.iterkeys()` => `.keys()`
- - `.itervalues()` => `.values()`
- - `.viewitems()` => `.items()`
- - `.viewkeys()` => `.keys()`
- - `.viewvalues()` => `.values()`
- - `.items()`=> `list(.items())`
- - `.keys()` => `list(.keys())`
- - `.values()` => `list(.values())`
- - `dict.has_key(key)` => `key in dict`
- - lists
- - `L = list(some_iterable); L.sort()` => `L = sorted(some_iterable)`
- - parenthesis in list comprehensions
- - `[... for var in item1, item2, ...]` => `[... for var in (item1, item2, ...)]`
-- file IO with `open`
- - `f = open('myfile.txt') # f.read() returns byte string` =>
- `from io import open` plus
- - `f = open('myfile.txt', 'rb') # f.read() should return bytes`
- - `f = open('myfile.txt', 'rt') # f.read() should return unicode text`
-- reduce()
- - `reduce()` => `from functools import reduce; reduce()`
-
-- python files in following directories:
- - resources/libraries/python
- - resources/tools
- - resources/traffic_profiles/trex
- - resources/traffic_scripts
-
-- check python calls in bash files:
- - resources/libraries/bash/
- - csit root directory
-
-## Migration steps
-
-1. Update all external libraries - week(s) before the week W
-1. Install agreed Python 3 version to all servers used by CSIT for test
- execution - week(s) before the week W
- 1. vpp device servers - already done
- 1. performance testbeds - already done
- 1. jenkins executors - already done
-1. Freeze the CSIT master branch for one week for commits other then Python 2 to
- Python 3 migration - week W
- 1. Create back up branch of actual master
- 1. Migrate libraries - work split between all available CSIT developers. Each
- one will submit separate commit for review - csit-vpp-xxx verify jobs will
- be failing at this phase so committers will need to overwrite verify
- voting to be able to merged these commits.
-
- TODO: provide separate spread sheet with listed libraries to be migrated
- with the name of CSIT developer responsible for the migration of this
- library.
- 1. Run jobs and tests of all of types when all libraries migrated to confirm
- functionality or to catch bugs that needs to be fixed - iterate until
- successful execution of all tests.
-1. Unfreeze the CSIT master branch.
Code Review / csit.git / blobdiff