Meeting 8 July 2014

MAGEEC Meeting – Bristol – 08 July 2014
Present: KIE, GC, GF, SG, A Burgess, JB, SC, JP, SH


Note: Actions denoted by people's initials (e.g. [SH])


Agenda

  • Review of progress of Greg, George and Stoil.
  • Discussion of Greg's report on Machine Learning, with follow-up discussions on
  • Automatic test generation
  • Use of feature vectors throughout the MAGEEC flow. Do we need it and if so, how do we adapt the plugins?
  • How to classify 'low' energy reading.
  • Clarification of framework interfaces and decisions on Expect and integration approaches - George
  • Case study candidate(s) - Stoil
  • BEEBS
  • Evaluation of status of our software and hardware capabilities
  • Do we need to buy anything else?
  • Making hardware boards available
  • Blog updates and other publicity
  • Setting of objectives for the next fortnight.
  • Who goes to TSB Energy efficient computing meeting – 28th July 2014.

Machine Learning

  • The 50+ size of the Milepost feature vector includes many features that may or may not be missing/appropriate
  • Feature vector is flattened.
  • Pass order will be set as outside the scope
  • We have a good idea about what new features could be added, but this is outside the MAGEEC scope.
  • Programs will assume input data-independence i.e. fix inputs across
  • [GC] Long-term Summer goal – Run PCA on a data set of BEEBS with input data variation. To feed into paper on BEEBS with SG. Work with Craig on this too.
  • [OR] Short blog post on what aspects of the features we are and are not doing
  • [GC] How long would it take the ML to learn based on 100 programs.
  • [GC+SC] Set up a communal database for sharing run data.

Framework

  • MAGEEC framework has working ML inside it now.
  • Framework can extract feature vector between passes if wanted, until the representation is lowered towards the ISA.
  • The lowest energy to compare MAGEEC's performance against can be the 1000 random compilations, as used by MILEPOST
  • Lowest possible energy could be predicted from mathematical extreme limits from random samples.
  • [AB+GF] Long-term Goal: Develop the software framework to be able to use DejaGnu to run and correlate returned energy value with run program. DejaGnu to be used for test build, pass fail detection. This approach gives long-term support to architectures and platforms. Problem with lack of documentation.
  • [GF] Documentation for DejaGnu-based framework for MAGEEC. Onto mageec.org.
  • [GF+SC] MAGEEC has an interface for the database. This needs to be integrated and tested. Any alterations that need to occur to the database schema to be completed.

BEEBS

  • Up to 91 tests
  • Fix int length assumptions
  • Ensure copyright notice on all
  • Ensure -wall compilation has no warnings
  • DejaGNU needs integrating
  • BEEBS V2.0 release goal: end of August
  • [SC] Register beebs.eu
  • [SG] Long term Summer goal: Review BEEBS for suitability now they are expanded. Paper on BEEBS V2.0
  • [GC] Csmith can generate C programs. Can we use auto-generation to cover holes in the feature space w.r.t. the programs we have.
  • Given BEEBS, we create a set of points in the feature space.
  • [AB] BEEBS branch user guide on wiki
  • ** [JP] Calibration. How to calibrate board-to-board variation. Test program set

BEEBS workers ('beebsv2' branch)

  • Andrew Burgess lead
  • Greg characterisation
  • George – Plackett-Burman
  • Simon C – User
  • Jeremy - User
  • James – Add more benchmarks
  • Stoil - Contributor

Case studies

  • [SG] In 2 weeks: Tinkerman Weather station – ATMEGA328. Get running on Embecosm's extensive AVR hardware.
  • [SG] For 6 weeks later. Get working the OSSI Cubesat code-base. Needs to be very energy efficient.
  • [SG] MSP430 target needs to be brought up.
  • OSCirrus weather station – ATMega128.
  • Novena Open Laptop running battery controller STMF32 with Chibios.
  • [SG] Explore further cases where RTOS are working.

Evaluation capabilities and scaling

  • 225 GCC passes needs 64,000 runs. 100 tests in BEEBS. 2 seconds to flash a chip. Total test 5-10s each. ~800 days of execution time. How to address this problem? H/W parallelism lowers this, but not enough.
  • [JP] FFD tool needs expanding to more than its current level of factors to >=225
  • What level of flag coverage is needed?
  • [JP/OR] Could choose e.g. 12 most important flags and be exhaustive in that space and reduce the number of runs this way.
  • [GF+SG] Long-term Summer Aim: Use Plackett-Burman Design approach to design to run the most important of 255 tests per architecture. Aim: a <= one-week run on the number of boards we have.

Hardware needed

  • 10 discovery boards needed
  • 5 MSP430 to order.
  • Components for weather station needed
  • Shrimp / Cuttlefish Embecosm → UoB

Making hardware boards available

  • [A Back] Yes to Andrew's plan to make 50 available to the general public on an at-cost basis.
  • [SH] Get quotations and determine cost sweet-spots

Publicity

  • [GF+SG] Complete blogs from George and Stoil. Send to Andrew Back.
  • [KE] ICT Energy Summer school talk.

2 week objectives

  • Stoil: Get the weather station assembled and running. Co-ordinate with Simon Hollis and James Pallister.
  • Greg: Demonstrate BEEBS working with same tests working with different data. Then start report on characterisation of BEEBS. Coordinate with Andrew Burgess.
  • George: Head around Plackett-Burman and Deja-GNU. Create a plan for running and integrating Plackett-Burman into Deja-GNU. Co-ordinate with Simon Cook and James Pallister.

Follow-up research list

  • Pass-to-pass feature extraction and ML based on the transformations between individual passes.
  • RQ: Assume BEEBS representative of class or programs. Are MILEPOST features valid for classifying these. Or, do we assume the features are useful. How useful are BEEBS for covering this space. When we find holes in the feature space, is it because programs do not exist, or because we need to find input programs in that part of the space.