+++ /dev/null
-.. _csit-design:\r
-\r
-CSIT Design\r
-===========\r
-\r
-FD.io CSIT system design needs to meet continuously expanding\r
-requirements of FD.io projects including VPP, related sub-systems (e.g.\r
-plugin applications, DPDK drivers) and FD.io applications (e.g. DPDK\r
-applications), as well as growing number of compute platforms running\r
-those applications. With CSIT project scope and charter including both\r
-FD.io continuous testing AND performance trending/comparisons, those\r
-evolving requirements further amplify the need for CSIT framework\r
-modularity, flexibility and usability.\r
-\r
-Design Hierarchy\r
-----------------\r
-\r
-CSIT follows a hierarchical system design with SUTs and DUTs at the\r
-bottom level of the hierarchy, presentation level at the top level and a\r
-number of functional layers in-between. The current CSIT system design\r
-including CSIT framework is depicted in the figure below.\r
-\r
-.. raw:: latex\r
-\r
- \begin{figure}[H]\r
- \centering\r
- \includesvg[width=0.90\textwidth]{csit_design_picture}\r
- \label{fig:csit_design_picture}\r
- \end{figure}\r
-\r
-.. figure:: csit_design_picture.svg\r
- :alt: FD.io CSIT system design\r
- :align: center\r
-\r
- *FD.io CSIT system design*\r
-\r
-A brief bottom-up description is provided here:\r
-\r
-#. SUTs, DUTs, TGs\r
-\r
- - SUTs - Systems Under Test;\r
- - DUTs - Devices Under Test;\r
- - TGs - Traffic Generators;\r
-\r
-#. Level-1 libraries - Robot and Python\r
-\r
- - Lowest level CSIT libraries abstracting underlying test environment, SUT,\r
- DUT and TG specifics;\r
- - Used commonly across multiple L2 KWs;\r
- - Performance and functional tests:\r
-\r
- - L1 KWs (KeyWords) are implemented as RF libraries and Python\r
- libraries;\r
-\r
- - Performance TG L1 KWs:\r
-\r
- - All L1 KWs are implemented as Python libraries:\r
-\r
- - Support for TRex only today;\r
- - CSIT IXIA drivers in progress;\r
-\r
- - Performance data plane traffic profiles:\r
-\r
- - TG-specific stream profiles provide full control of:\r
-\r
- - Packet definition – layers, MACs, IPs, ports, combinations thereof\r
- e.g. IPs and UDP ports;\r
- - Stream definitions - different streams can run together, delayed,\r
- one after each other;\r
- - Stream profiles are independent of CSIT framework and can be used\r
- in any T-rex setup, can be sent anywhere to repeat tests with\r
- exactly the same setup;\r
- - Easily extensible – one can create a new stream profile that meets\r
- tests requirements;\r
- - Same stream profile can be used for different tests with the same\r
- traffic needs;\r
-\r
- - Functional data plane traffic scripts:\r
-\r
- - Scapy specific traffic scripts;\r
-\r
-#. Level-2 libraries - Robot resource files:\r
-\r
- - Higher level CSIT libraries abstracting required functions for executing\r
- tests;\r
- - L2 KWs are classified into the following functional categories:\r
-\r
- - Configuration, test, verification, state report;\r
- - Suite setup, suite teardown;\r
- - Test setup, test teardown;\r
-\r
-#. Tests - Robot:\r
-\r
- - Test suites with test cases;\r
- - Functional tests using VIRL environment:\r
-\r
- - VPP;\r
- - Honeycomb;\r
- - NSH_SFC;\r
-\r
- - Performance tests using physical testbed environment:\r
-\r
- - VPP;\r
- - DPDK-Testpmd;\r
- - DPDK-L3Fwd;\r
-\r
- - Tools:\r
-\r
- - Documentation generator;\r
- - Report generator;\r
- - Testbed environment setup ansible playbooks;\r
- - Operational debugging scripts;\r
-\r
-Test Lifecycle Abstraction\r
---------------------------\r
-\r
-A well coded test must follow a disciplined abstraction of the test\r
-lifecycles that includes setup, configuration, test and verification. In\r
-addition to improve test execution efficiency, the commmon aspects of\r
-test setup and configuration shared across multiple test cases should be\r
-done only once. Translating these high-level guidelines into the Robot\r
-Framework one arrives to definition of a well coded RF tests for FD.io\r
-CSIT. Anatomy of Good Tests for CSIT:\r
-\r
-#. Suite Setup - Suite startup Configuration common to all Test Cases in suite:\r
- uses Configuration KWs, Verification KWs, StateReport KWs;\r
-#. Test Setup - Test startup Configuration common to multiple Test Cases: uses\r
- Configuration KWs, StateReport KWs;\r
-#. Test Case - uses L2 KWs with RF Gherkin style:\r
-\r
- - prefixed with {Given} - Verification of Test setup, reading state: uses\r
- Configuration KWs, Verification KWs, StateReport KWs;\r
- - prefixed with {When} - Test execution: Configuration KWs, Test KWs;\r
- - prefixed with {Then} - Verification of Test execution, reading state: uses\r
- Verification KWs, StateReport KWs;\r
-\r
-#. Test Teardown - post Test teardown with Configuration cleanup and\r
- Verification common to multiple Test Cases - uses: Configuration KWs,\r
- Verification KWs, StateReport KWs;\r
-#. Suite Teardown - Suite post-test Configuration cleanup: uses Configuration\r
- KWs, Verification KWs, StateReport KWs;\r
-\r
-RF Keywords Functional Classification\r
--------------------------------------\r
-\r
-CSIT RF KWs are classified into the functional categories matching the test\r
-lifecycle events described earlier. All CSIT RF L2 and L1 KWs have been grouped\r
-into the following functional categories:\r
-\r
-#. Configuration;\r
-#. Test;\r
-#. Verification;\r
-#. StateReport;\r
-#. SuiteSetup;\r
-#. TestSetup;\r
-#. SuiteTeardown;\r
-#. TestTeardown;\r
-\r
-RF Keywords Naming Guidelines\r
------------------------------\r
-\r
-Readability counts: "..code is read much more often than it is written."\r
-Hence following a good and consistent grammar practice is important when\r
-writing :abbr:`RF (Robot Framework)` KeyWords and Tests. All CSIT test cases\r
-are coded using Gherkin style and include only L2 KWs references. L2 KWs are\r
-coded using simple style and include L2 KWs, L1 KWs, and L1 python references.\r
-To improve readability, the proposal is to use the same grammar for both\r
-:abbr:`RF (Robot Framework)` KW styles, and to formalize the grammar of English\r
-sentences used for naming the :abbr:`RF (Robot Framework)` KWs. :abbr:`RF (Robot\r
-Framework)` KWs names are short sentences expressing functional description of\r
-the command. They must follow English sentence grammar in one of the following\r
-forms:\r
-\r
-#. **Imperative** - verb-object(s): *"Do something"*, verb in base form.\r
-#. **Declarative** - subject–verb–object(s): *"Subject does something"*, verb in\r
- a third-person singular present tense form.\r
-#. **Affirmative** - modal_verb-verb-object(s): *"Subject should be something"*,\r
- *"Object should exist"*, verb in base form.\r
-#. **Negative** - modal_verb-Not-verb-object(s): *"Subject should not be\r
- something"*, *"Object should not exist"*, verb in base form.\r
-\r
-Passive form MUST NOT be used. However a usage of past participle as an\r
-adjective is okay. See usage examples provided in the Coding guidelines\r
-section below. Following sections list applicability of the above\r
-grammar forms to different :abbr:`RF (Robot Framework)` KW categories. Usage\r
-examples are provided, both good and bad.\r
-\r
-Coding guidelines\r
------------------\r
-\r
-Coding guidelines can be found on `Design optimizations wiki page\r
-<https://wiki.fd.io/view/CSIT/Design_Optimizations>`_.\r
Code Review / csit.git / blobdiff