<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>Geek in progress &#187; analysis</title>
	<atom:link href="http://www.itkovian.net/base/tag/analysis/feed/" rel="self" type="application/rss+xml" />
	<link>http://www.itkovian.net/base</link>
	<description>I am not yet done.</description>
	<lastBuildDate>Thu, 20 Oct 2011 20:56:59 +0000</lastBuildDate>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.org/?v=3.2.1</generator>
		<item>
		<title>No more lies, just benchmarks</title>
		<link>http://www.itkovian.net/base/no-more-lies-just-benchmarks/</link>
		<comments>http://www.itkovian.net/base/no-more-lies-just-benchmarks/#comments</comments>
		<pubDate>Thu, 20 Oct 2011 16:04:10 +0000</pubDate>
		<dc:creator>Itkovian</dc:creator>
				<category><![CDATA[Uncategorized]]></category>
		<category><![CDATA[analysis]]></category>
		<category><![CDATA[benchmarks]]></category>
		<category><![CDATA[performance]]></category>
		<category><![CDATA[presentation]]></category>

		<guid isPermaLink="false">http://www.itkovian.net/base/no-more-lies-just-benchmarks/</guid>
		<description><![CDATA[I was asked by the people from Zeus WPI to give a talk on benchmarking, since students need to both test and measure the code they write for several courses and apparently there still is no course which teaches people how to conduct experiments. Being the yes-saying fool that I am, I agreed and figured [...]]]></description>
			<content:encoded><![CDATA[<p>I was asked by the people from <a href="http://zeus.ugent.be">Zeus WPI</a> to give a talk on benchmarking, since students need to both test and measure the code they write for several courses and apparently there still is no course which teaches people how to conduct experiments. Being the yes-saying fool that I am, I agreed and figured out what to talk about. It turned out there are a lot of things I would like students to become more familiar with. In the end I ended up dismissing a number of more technical things, preferring to talk about as many subjects as I could.</p>
<p>I have made the <a href="http://itkovian.net/base/files/presentations/zeuswpi.20111020.pdf">slides</a> (in Dutch) available in PDF format. Ignore the jokes if you do not grok them.</p>
<p>Thanks Zeus, for the Grimbergen bottles. I will make them count.</p>
]]></content:encoded>
			<wfw:commentRss>http://www.itkovian.net/base/no-more-lies-just-benchmarks/feed/</wfw:commentRss>
		<slash:comments>7</slash:comments>
		</item>
		<item>
		<title>WCET analysis</title>
		<link>http://www.itkovian.net/base/wcet-analysis/</link>
		<comments>http://www.itkovian.net/base/wcet-analysis/#comments</comments>
		<pubDate>Tue, 14 Jul 2009 09:27:11 +0000</pubDate>
		<dc:creator>Itkovian</dc:creator>
				<category><![CDATA[2009]]></category>
		<category><![CDATA[acaces]]></category>
		<category><![CDATA[analysis]]></category>
		<category><![CDATA[barcelona]]></category>
		<category><![CDATA[summerschool]]></category>
		<category><![CDATA[wcet]]></category>

		<guid isPermaLink="false">http://www.itkovian.net/base/?p=238</guid>
		<description><![CDATA[One of the courses I am attending deals with worst case execution time analysis or WCET analysis for short.

<div class="figure">
<a href="http://www.flickr.com/photos/itkovian/3715683959/" title="Peter Puschner by Itkovian, on Flickr"><img src="http://farm3.static.flickr.com/2586/3715683959_8e96126ef4_b.jpg" width="500" height="333" alt="Peter Puschner" /></a>
</div>]]></description>
			<content:encoded><![CDATA[<p>One of the courses I am attending deals with worst case execution time analysis or WCET analysis for short.</p>
<div class="figure">
<a href="http://www.flickr.com/photos/itkovian/3715683959/" title="Peter Puschner by Itkovian, on Flickr"><img src="http://farm3.static.flickr.com/2586/3715683959_8e96126ef4_b.jpg" width="500" height="333" alt="Peter Puschner" /></a>
</div>
<p><!--break--></p>
<p>I think that the following are the main points of interest <a<br />
href="http://www.vmars.tuwien.ac.at/people/puschner.html">Peter Puschner</a><br />
tried to get drilled into our skulls. The course focuses on the worst case<br />
analysis, and for that there are three major items one needs to take into<br />
account: the piece of code to analyse, the input it consumes and the hardware<br />
it runs on. Giving the fact the most code consumes input means that there is<br />
some state associated with it, that determines the paths takes in the static<br />
code graph. Peter clearly stated that the input should be part of the problem<br />
specification, which often complicates the analysis. Additionally, the hardware<br />
has some associated state as well (think caches, branch prediciton, etc.) that<br />
influences the timing of each task we want to analyse. Different instances of<br />
actions in the task have different durations, and as such the sequence of the<br />
actions carries out during each task becomes the important factor. It was<br />
clearly illustrated that measurements are not adequatefor dealing with worst<br />
case scenario&#8217;s.  I think the lesson to take away from the first class is that<br />
there is no way to measure WCET, one must therefore analyse the program.</p>
<p>The second lesson dealt with a number of approaches for determining the WCET:<br />
tree-based (from the program graph), path-based (potential dynamic execution<br />
paths), and IPET based. Tree-based WCET does not scale well, path=based does,<br />
but that has the disadavantage of getting quite complex as the program size<br />
increases. To deal with this, the IPET technqiue can be used, in which we<br />
desribe the flow of control by using a number of equations that constrain the<br />
possibilities: program flow going ionto a node, must come out of it, so this<br />
yields a number of equations that state how many times an edge can be taken.<br />
The drawback here is that solving the equation using ILP is NP hard in general.<br />
Another technique discussed was the modelling of execution time by mapping a<br />
sequence of instructions to an execution time. This however requires<br />
information on the hardware timing to deal with the various hardware<br />
enhancements to the processor (think pipelines, caches, out=of-order execution,<br />
&#8230;). I think this seems to have quite some non-deterministic aspects.  Peter<br />
then moved on to show how one can model pipelines, to be used in the WCET<br />
analysis, as well as how one can model caches within the IPET framework.  The<br />
takeway lesson here is that there is no straighforward technique to estimate<br />
the WCET, and the most powerful techniques must use hardware timing<br />
information, which is a hard problem in itself.</p>
<p>The third lesson dealt almost exclusively with the modeling of the caches,<br />
moving from a concrete state to an abstract state, which can be reasoned about.<br />
Peter identified 4 essential categories for the analysis in a cache model:<br />
always hit, always miss, globally persistent, locally persistent, plus one<br />
leftover category. For each of these categories, there is a different semantics<br />
for the start state, the update function and the join function on the abstarct<br />
cache model. Finally, we received an introduction to potential timing anolalies<br />
that can occur and how to deal with those.</p>
<p>In the fourth lesson, the focus moved to measuring execution times. Here the<br />
key takeway was that it is very unlikely that the hardware will be set to the<br />
worst case state for executing the application, leading to (at times gross)<br />
underestimates of the WCET. For a WCET bound, more systematic techniques are<br />
required. One of the best performing WCET measurements tools relies on genetic<br />
algorithms for generating the input that drives the WC scenario. Still, for the<br />
benchmarks shown, there can be quite a large discrepancy between the<br />
analytically obtained WCET and the WCET obtained through use of a GA. Other<br />
potential techniques include a probabilisitc approach and program segmentation<br />
with a path based analysis.</p>
<p>In the final class on Friday, Peter discussed some issues with time-predicatble<br />
sofware and hardware. Predicates, transforming execution paths, etc. can be<br />
used to get more reliable WCET bounds. The major takeaway here was that the<br />
classical analysis where one focuses on the mean execution time (with its<br />
distribution, like we did in <a<br />
href="http://www.itkovian.net/base/statistically-rigorous-java-performance-evaluation">our<br />
Java stats paper</a> at <a href="http://www.ooplsa.org">OOPSLA</a>) is not<br />
going to work at all for a WCET analysis. Peter also argued for better WCET<br />
bounded hardware, or at least hardware that could make the WCET analysis<br />
easier.</p>
<div class="figure">
<a href="http://www.flickr.com/photos/itkovian/3729256016/" title="Final words by Itkovian, on Flickr"><img src="http://farm3.static.flickr.com/2564/3729256016_fbe8a59c3e.jpg" width="333" height="500" alt="Final words" /></a>
</div>
]]></content:encoded>
			<wfw:commentRss>http://www.itkovian.net/base/wcet-analysis/feed/</wfw:commentRss>
		<slash:comments>2</slash:comments>
		</item>
	</channel>
</rss>

