OpentestTeeVatf

Background
VATF was originally develop in 1Q 2007. It was developed to automate Digital Video Software Development Kit (DVSDK) System Test cycle, hence the name Video Automation Test Framework (VATF). However, VATF is generic enough and can/has been used to automate test suites on other products such as Linux, Android and Wince Platform Support Packages or Development kits

VATF was written using the Ruby programming language. Why Ruby? Test automation involves a lot of string handling operations. Scripting languages typically supports string operations much better than other languages such as Java and C. Ruby is an object-oriented language that combines some of the best features from Perl and Java and adds some on its own like module mixin and code closures. Ruby is a very dynamic language and has strong reflection capabilities. From Ruby's main site "Ruby is A dynamic, open source programming language with a focus on simplicity and productivity. It has an elegant syntax that is natural to read and easy to write"

Services provided by VATF
The VATF Core engine provides the following services:

1) Read test parameters from a Test Suite Database.

2) Initialize all equipment required to run a test

3) Calls the appropriate test script and pass to it: a) a representation of the test parameter that abstract the Testlink DB (@test_params object), b) a generic representation of the test equipment (@equipment object), c) a generic results form that can be used in the script to provide detailed results data (@html_result_form object), and d) a method to set the test result back in the test suite database (set_result method)

4) Save results back to the Test Suite Database and create an HTML test execution report

VATF tries to be as generic and open as possible. It encourages test script reuse because the equipment and database details are hidden from the user. As an example see the test script snippet in the figure below



Hardware Management
VATF abstracts the hardware details from the test script by relying on 2 user-specified pieces of information:
 * The Bench file: The user defines the equipment he/she has on his/her bench (aka test station) by creating a bench file.

The Bench file describes all the equipment available in a given test station.

All the equipment parameters, including the driver class that controls it, are specified in the bench file exclusively.

This provides a clean separation or interface between the equipment and VATF.

The bench file shown below defines the following equipment: one multimeter, one AM335x-evm, one power controller, one linux PC and one USB switch.

minfo = EquipmentInfo.new("multimeter") minfo.serial_port = '/dev/ttyUSB0' minfo.serial_params = {"baud" => 19200, "data_bits" => 8, "stop_bits" => 1, "parity" => SerialPort::NONE} minfo.driver_class_name = 'KeithleyMultiMeterDriver' minfo.params = {'number_of_channels' => 40}
 * 1) Sample Bench File w/ five pieces of equipment:

dut = EquipmentInfo.new("am335x-evm","linux_sd_usbhostmsc_usbhosthid") dut.driver_class_name = "LinuxEquipmentDriver" dut.serial_port = '/dev/ttyS0' dut.serial_params = {"baud" => 115200, "data_bits" => 8, "stop_bits" => 1, "parity" => SerialPort::NONE} dut.prompt = /[\w\d]+@.+[@:#]+/ dut.login = 'root' dut.telnet_login = 'root' dut.prompt = /#/ dut.boot_prompt = /U-Boot#/ dut.login_prompt = /login:/ dut.board_id='20100720' dut.power_port = {'apc.158.218.103.33' => 8} dut.params = {'usb_port' => 1, 'multimeter1' => minfo}

pwr = EquipmentInfo.new("power_controller", "apc.158.218.103.33") pwr.telnet_ip = '158.218.103.33' pwr.telnet_port = 23 pwr.driver_class_name = 'StafApcPowerController' pwr.params = {'staf_ip' => 'local'}

svr = EquipmentInfo.new("linux_server") svr.tftp_path = '/tftpboot' svr.driver_class_name = 'LinuxLocalHostDriver' svr.telnet_login = 'a0850405local' svr.prompt = /@@/

usb = EquipmentInfo.new("usb_switch", 0) usb.serial_port = '/dev/ttyUSB1' usb.serial_params = {"baud" => 9600, "data_bits" => 8, "stop_bits" => 1, "parity" => SerialPort::NONE} usb.driver_class_name = 'ExtronUsbSwitch'


 * The HW Assets field in TestLink: Identifies the equipment required to run a test case and provides a mapping between the generic equipment names used in the test script and the real hardware available in the test station.

Sample test setup file or HW Assets:

dut1 = ["am335x-evm",linux_sd]; server1 = ["linux_server]

For more information about HW assets field information, please see HW Resource Manager

The HW assets field and the bench file described above allow writing test scripts in a generic hardware-independent way. For instance in the above example the test scripts will use syntax like @equipment['dut1'].do_something or  @equipment['server1'].do_something without knowing or caring about the actual platform type (i.e. am335x, am37x, dm365x, etc.)

Steps required to automate a new area
To automate a given area, the user needs to create the following components: The test script MUST implement 3 methods: setup, run and clean. These methods are the interface or entry points between VATF and the test script.
 * 1) Test Script: This the actual script that contains the logic to run the test.
 * 1) Test Equipment Driver: Optionally the user might need to write a new Test Equipment driver class if one, for the hardware he/she wants to use, doesn't exist

List of available DUT/Test Equipment Drivers
Drivers for the following Hardware has already been written and validated:
 * 1) Generic Linux host driver
 * 2) APC Power Controller
 * 3) Devantech Realy Controller
 * 4) Digital Loggers Relay Controller
 * 5) HP Wage Generator
 * 6) Kvaser CAN controller
 * 7) Extron Video/Audio Switches
 * 8) Tascam DVD players
 * 9) Pionner DVD players
 * 10) Q-Master's Video Quality System
 * 11) Video Clarity's Video Quality System
 * 12) Extron USB Switch
 * 13) Keithley Multimeter