Meeting 8 July 2014

MAGEEC Meeting – Bristol – 08 July 2014 Present: KIE, GC, GF, SG, A Burgess, JB, SC, JP, SH

Note: Actions denoted by people's initials (e.g. [SH])

Agenda

Review of progress of Greg, George and Stoil.
Discussion of Greg's report on Machine Learning, with follow-up discussions on
Automatic test generation
Use of feature vectors throughout the MAGEEC flow. Do we need it and if so, how do we adapt the plugins?
How to classify 'low' energy reading.
Clarification of framework interfaces and decisions on Expect and integration approaches - George
Case study candidate(s) - Stoil
BEEBS
Evaluation of status of our software and hardware capabilities
Do we need to buy anything else?
Making hardware boards available
Blog updates and other publicity
Setting of objectives for the next fortnight.
Who goes to TSB Energy efficient computing meeting – 28^th July 2014.

Machine Learning

The 50+ size of the Milepost feature vector includes many features that may or may not be missing/appropriate
Feature vector is flattened.
Pass order will be set as outside the scope
We have a good idea about what new features could be added, but this is outside the MAGEEC scope.
Programs will assume input data-independence i.e. fix inputs across
[GC] Long-term Summer goal – Run PCA on a data set of BEEBS with input data variation. To feed into paper on BEEBS with SG. Work with Craig on this too.
[OR] Short blog post on what aspects of the features we are and are not doing
[GC] How long would it take the ML to learn based on 100 programs.
[GC+SC] Set up a communal database for sharing run data.

Framework

MAGEEC framework has working ML inside it now.
Framework can extract feature vector between passes if wanted, until the representation is lowered towards the ISA.
The lowest energy to compare MAGEEC's performance against can be the 1000 random compilations, as used by MILEPOST
Lowest possible energy could be predicted from mathematical extreme limits from random samples.
[AB+GF] Long-term Goal: Develop the software framework to be able to use DejaGnu to run and correlate returned energy value with run program. DejaGnu to be used for test build, pass fail detection. This approach gives long-term support to architectures and platforms. Problem with lack of documentation.
[GF] Documentation for DejaGnu-based framework for MAGEEC. Onto mageec.org.
[GF+SC] MAGEEC has an interface for the database. This needs to be integrated and tested. Any alterations that need to occur to the database schema to be completed.

BEEBS

Up to 91 tests
Fix int length assumptions
Ensure copyright notice on all
Ensure -wall compilation has no warnings
DejaGNU needs integrating
BEEBS V2.0 release goal: end of August
[SC] Register beebs.eu
[SG] Long term Summer goal: Review BEEBS for suitability now they are expanded. Paper on BEEBS V2.0
[GC] Csmith can generate C programs. Can we use auto-generation to cover holes in the feature space w.r.t. the programs we have.
Given BEEBS, we create a set of points in the feature space.
[AB] BEEBS branch user guide on wiki
** [JP] Calibration. How to calibrate board-to-board variation. Test program set

BEEBS workers ('beebsv2' branch)

Case studies

[SG] In 2 weeks: Tinkerman Weather station – ATMEGA328. Get running on Embecosm's extensive AVR hardware.
[SG] For 6 weeks later. Get working the OSSI Cubesat code-base. Needs to be very energy efficient.
[SG] MSP430 target needs to be brought up.
OSCirrus weather station – ATMega128.
Novena Open Laptop running battery controller STMF32 with Chibios.
[SG] Explore further cases where RTOS are working.

Evaluation capabilities and scaling

225 GCC passes needs 64,000 runs. 100 tests in BEEBS. 2 seconds to flash a chip. Total test 5-10s each. ~800 days of execution time. How to address this problem? H/W parallelism lowers this, but not enough.
[JP] FFD tool needs expanding to more than its current level of factors to >=225
What level of flag coverage is needed?
[JP/OR] Could choose e.g. 12 most important flags and be exhaustive in that space and reduce the number of runs this way.
[GF+SG] Long-term Summer Aim: Use Plackett-Burman Design approach to design to run the most important of 255 tests per architecture. Aim: a <= one-week run on the number of boards we have.

Hardware needed

Making hardware boards available

[A Back] Yes to Andrew's plan to make 50 available to the general public on an at-cost basis.
[SH] Get quotations and determine cost sweet-spots

Publicity

2 week objectives

Stoil: Get the weather station assembled and running. Co-ordinate with Simon Hollis and James Pallister.
Greg: Demonstrate BEEBS working with same tests working with different data. Then start report on characterisation of BEEBS. Coordinate with Andrew Burgess.
George: Head around Plackett-Burman and Deja-GNU. Create a plan for running and integrating Plackett-Burman into Deja-GNU. Co-ordinate with Simon Cook and James Pallister.

Follow-up research list

Pass-to-pass feature extraction and ML based on the transformations between individual passes.
RQ: Assume BEEBS representative of class or programs. Are MILEPOST features valid for classifying these. Or, do we assume the features are useful. How useful are BEEBS for covering this space. When we find holes in the feature space, is it because programs do not exist, or because we need to find input programs in that part of the space.

Navigation menu