Difference between revisions of "Data Gather Framework"

From MAGEEC
Jump to: navigation, search
(Added a more detailed description of the data structures and the APIs of each module.)
Line 12: Line 12:
  
 
The DBi is just an interface between the database and the rest of the framework. It's designed to abstract the database server and the database's schema from the rest of the framework. This means that if the database schema changes, or what database server is being used, then this module can just be swapped out for another.
 
The DBi is just an interface between the database and the rest of the framework. It's designed to abstract the database server and the database's schema from the rest of the framework. This means that if the database schema changes, or what database server is being used, then this module can just be swapped out for another.
 +
 +
== Data Structures ==
 +
=== Benchmark ===
 +
* name: The name of the benchmark (eg "blowfish").
 +
* build_flags: A dictionary of (non-optimising) build flags. The key should be the compiler name, and the value should be a string of build flags. (eg { 'gcc481' : 'bf.c -o blowfish' }). Currently this string get's split by whitespace and doesn't accept quotes or escape characters (This restriction exists in the builder as build_flags.split()).
 +
 +
=== Platform ===
 +
* name: The name of the platform (eg "arm").
 +
* compilers: A dictionary of compilers and the location or name of their binary. The key should be the compiler name and the value should be the location or name of the binary. (eg { 'gcc481' : 'arm-none-eabi-gcc' })
 +
* build_flags: A dictionary of (non-optimising) build flags. The key should be the compiler name, and the value should be a string of build flags. (eg { 'gcc481' : '-g -DARM -I../platformcode' }). Currently this string gets split by whitespace and doesn't accept quotes or escape characters (This restriction exists in the builder as build_flags.split()).
 +
* runner_name: The PYRONAME of the GDB runner for this platform (eg "runners.arm").
 +
 +
=== Results ===
 +
* run: The run data structure that indicates what test run these results are for.
 +
* results: A list of tuples of the timestamp and power.
 +
* return_code: The return code of the benchmark ran.
 +
 +
=== Run ===
 +
* benchmark: The benchmark data structure associated with this run of the test.
 +
* compiler: The name of the compiler associated with this run of the test (eg "gcc481").
 +
* platform: The platform data structure associated with this run of the test.
 +
* flags: An array of optimization flags associated with this run of the test.
 +
* id: The ID of this run of the test. It should be identical to that of the run_id in the database (and can be generated via the DBi with add_run).
 +
 +
== API ==
 +
=== Builder ===
 +
* init:
 +
* build_and_run: Returns None on error, and a Results structure on success.
 +
 +
=== ConfigReader ===
 +
* read: Returns a tuple of a list of compiler names, a list of platform structures, a list of benchmark structures, and the location of the database (eg "db/development.sqlite3")
 +
 +
=== DBi ===
 +
* init:
 +
* fetch: Returns a list of tuples of the timestamp and power for the given run_id.
 +
* record: Records the given list of timestamp/power tuples into the database.
 +
* add_run: Returns a generated run id.
 +
 +
=== GdbManager ===
 +
* init:
 +
* read_energy: Runs the specified binary on the platform in the given run structure.
 +
* sanity: Return True if the GDB manager can contact all configured GDB runners, and False otherwise.
 +
 +
=== Logger ===
 +
* init:
 +
* log_info, log_warn, log_erro: Records different severity log messages into the log file.
 +
* add_run: See DBi.add_run.
 +
* record_results: See DBi.record.
 +
 +
=== Runner ===
 +
* init:
 +
* run_binary: See GdbManager.read_energy
 +
 +
=== GDB Runners ===
 +
* init:
 +
* run_binary:
 +
* ping:
 +
* stop:

Revision as of 09:00, 29 August 2013

Framework-diagram.png

The builder takes build flags, a compiler, and platform, builds the binary, passes it to the runner, and returns the energy results (with the binary's return code).

The runner takes a binary, compiler, and platform, runs the binary on the specified platform, and returns the energy results (with the binary's return code).

The GDB manager is responsible for running a binary on the required platform, and abstracts the interface with the GDB runners.

Each GDB runner is a GDB client, which connects to it's relevant GDB server. Currently it's talking to the GDB manager via RPC (using the Python library Pyro4). In the future this will be changed to GDB/MI. It's also written in Python, and in the future this will be changed to expect.

The logger just stores information about what's going on into a log file. It also acts as an interface to the database interface.

The DBi is just an interface between the database and the rest of the framework. It's designed to abstract the database server and the database's schema from the rest of the framework. This means that if the database schema changes, or what database server is being used, then this module can just be swapped out for another.

Data Structures

Benchmark

  • name: The name of the benchmark (eg "blowfish").
  • build_flags: A dictionary of (non-optimising) build flags. The key should be the compiler name, and the value should be a string of build flags. (eg { 'gcc481' : 'bf.c -o blowfish' }). Currently this string get's split by whitespace and doesn't accept quotes or escape characters (This restriction exists in the builder as build_flags.split()).

Platform

  • name: The name of the platform (eg "arm").
  • compilers: A dictionary of compilers and the location or name of their binary. The key should be the compiler name and the value should be the location or name of the binary. (eg { 'gcc481' : 'arm-none-eabi-gcc' })
  • build_flags: A dictionary of (non-optimising) build flags. The key should be the compiler name, and the value should be a string of build flags. (eg { 'gcc481' : '-g -DARM -I../platformcode' }). Currently this string gets split by whitespace and doesn't accept quotes or escape characters (This restriction exists in the builder as build_flags.split()).
  • runner_name: The PYRONAME of the GDB runner for this platform (eg "runners.arm").

Results

  • run: The run data structure that indicates what test run these results are for.
  • results: A list of tuples of the timestamp and power.
  • return_code: The return code of the benchmark ran.

Run

  • benchmark: The benchmark data structure associated with this run of the test.
  • compiler: The name of the compiler associated with this run of the test (eg "gcc481").
  • platform: The platform data structure associated with this run of the test.
  • flags: An array of optimization flags associated with this run of the test.
  • id: The ID of this run of the test. It should be identical to that of the run_id in the database (and can be generated via the DBi with add_run).

API

Builder

  • init:
  • build_and_run: Returns None on error, and a Results structure on success.

ConfigReader

  • read: Returns a tuple of a list of compiler names, a list of platform structures, a list of benchmark structures, and the location of the database (eg "db/development.sqlite3")

DBi

  • init:
  • fetch: Returns a list of tuples of the timestamp and power for the given run_id.
  • record: Records the given list of timestamp/power tuples into the database.
  • add_run: Returns a generated run id.

GdbManager

  • init:
  • read_energy: Runs the specified binary on the platform in the given run structure.
  • sanity: Return True if the GDB manager can contact all configured GDB runners, and False otherwise.

Logger

  • init:
  • log_info, log_warn, log_erro: Records different severity log messages into the log file.
  • add_run: See DBi.add_run.
  • record_results: See DBi.record.

Runner

  • init:
  • run_binary: See GdbManager.read_energy

GDB Runners

  • init:
  • run_binary:
  • ping:
  • stop:
Retrieved from "http://mageec.org/w/index.php?title=Data_Gather_Framework&oldid=190"