[MAGEEC] BEEBS analysis

[Stoil Ganev]

Hi Simon,

Sorry, I should have given some description of the files in my earlier
e-mail. On the AVR binaries I perform 2 types of analysis - counting the
number and type of instructions in the binary itself and then counting the
instructions during runtime in simulavr. Thats is what I call static and
dynamic respectively. By raw I mean that I have just listed the occurences
of each instruction. In files with .analysis I have just grouped the
numbers of different instructions in categories. As for the gimple stuff I
wasn't sure how to categorise the statements so I ended up just counting
different type of nodes.

The scripts I used to generate these files are on my fork. Specifically:

AVR analysis
https://github.com/Stalekon/beebs/blob/stoil/analysis/beebsv2_analysis/avr-classify.py

Gimple Tree analysis:
https://github.com/Stalekon/beebs/blob/stoil/analysis/beebsv2_analysis/gimple-analyse.py

As for the failed benchmarks,I simulated them on atmega328 and for most of
them the simulator was giving warnings about invalid reads and writes.
However, 2 or 3 of them didn't get any warning but never reached the
stop_trigger function but still ended up in the __stop_program one which is
just an infinite loop. Also, some benchmarks also gave me a few read and
write warnings but still ended up in stop_trigger so I deemed them
successful. If that is of interest I could run the simulations again and
note down everything that looks out of the ordinary.

Regards,
Stoil Ganev



On Fri, Aug 8, 2014 at 12:03 PM, Simon Cook <simon.cook at embecosm.com> wrote:

> Hi Stoil,
>
> Could you give me a description on what the different files are? For
> example, I'm looking at the 2dfir directory, it looks to me the gimpdump
> files are a result of a plugin looking at the gimple structure, so counting
> the different types of nodes, but its not immediately clear what the
> .raw.static and .raw.dynamic files are, are these the instructions found in
> a binary and then those executed, or is there a more detailed analysis
> going on?
>
> For the list of benchmarks you have listed as failed, some of these may be
> due to them being too large to fit on the target hardware, are you
> simulating/targetting an ATmega328 or some other AVR device?
>
> In Lymington I've been running my own experiments with BEEBS with a newer
> toolchain (it looks to me like the linker hasn't been correctly identifying
> when tests don't fit in memory), and as such have a larger list of tests
> which don't fit on an ATmega328. I've noticed a few of these match up with
> your failed list, perhaps you are seeing issues due to out of memory issues
> only picked up when running simulavr. For reference, my ignore list is now:
>
>  adpcm blowfish compress dtoa fir int_matmult miniz nettle-cast128
> nettle-des ns picojpeg rijndael stringsearch1
>
>
> Thanks,
> Simon
>
>
> On Fri, Aug 8, 2014 at 11:31 AM, Stoil Ganev <
> sg13286.2013 at my.bristol.ac.uk> wrote:
>
>> Hi all,
>>
>> So I extracted some data for the benchmarks based on the gimple tree
>> structure and AVR instruction traces. Nothing ARM based as of yet.
>> Currently it is located in this repositry :
>>
>> https://github.com/Stalekon/beebsv2-analysis-data
>>
>> I am looking for feedback on the format of the data and whether I should
>> move the repositry under MAGEEC or just make it part of the beebs one.
>>
>> Some of the benchmarks I consider to have failed during AVR simulation
>> due to timeouts or weird behavior. These are listed in 'fails'. Also,
>> during the gimple analysis i got data per C file so I have combined that in
>> a *-combined.gimpdump file.
>>
>> Regards,
>> Stoil Ganev
>>
>> _______________________________________________
>> mageec mailing list
>> mageec at mageec.org
>> http://mageec.org/cgi-bin/mailman/listinfo/mageec
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mageec.org/pipermail/mageec/attachments/20140808/7ad5c90f/attachment.html>