blob: de3b4d7f7ae605924a19b7e8b6f6763f7ec32fc2 [file] [log] [blame]
<?xml version="1.0" encoding="iso-8859-1"?>
<!DOCTYPE html
PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
<html xmlns="" xml:lang="en" lang="en">
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1" />
OSADL - The Open Source Automation Development Lab
This website is powered by TYPO3 - inspiring people to share!
TYPO3 is a free open source Content Management Framework initially created by Kasper Skaarhoj and licensed under GNU/GPL.
TYPO3 is copyright 1998-2006 of Kasper Skaarhoj. Extensions are copyright of their respective owners.
Information and contribution at and
<base href="" />
<link rel="SHORTCUT ICON" href="fileadmin/favicon.ico" />
<link rel="stylesheet" type="text/css" href="typo3temp/stylesheet_de96f05ce7.css" />
<script type="text/javascript" src="fileadmin/scripts/cookies.js"></script><script type="text/javascript" src="fileadmin/scripts/efa_fontsize.js"></script>
<link href="fileadmin/template/../css/layout_3col_fixed.css" rel="stylesheet" type="text/css" />
<!--[if lte IE 7]>
<link href="fileadmin/template/../css/explorer/iehacks_3col_fixed.css" rel="stylesheet" type="text/css" />
<link rel="alternate" type="application/rss+xml" title="RSS feed: OSADL News" href="" />
<link rel="alternate" type="application/rss+xml" title="RSS feed: OSADL Last Changes" href="" /><meta name="description" content="OSADL promotes and supports the use of Open Source software in the automation and machine industry." />
<meta name="keywords" content="Realtime,Linux,Real-time,Realtime Linux,Real-time Linux,Open Source,Software,OSS,FOSS,Kernel,Automation,Cooperative,Member,Membership,Academic Member,Purchase Community" />
<meta name="robots" content="index,follow" />
<meta name="copyright" content="Open Source Automation Development Lab, (c)2007-2009. All rights reserved." />
<meta http-equiv="content-language" content="en" />
<link rev="made" href="mailto:info at" />
<meta http-equiv="reply-to" content="info at" />
<meta name="author" content="Open Source Automation Development Lab contributors" />
<meta name="revisit-after" content="1 days" />
<meta name="DC.Description" content="OSADL promotes and supports the use of Open Source software in the automation and machine industry." />
<meta name="DC.Subject" content="Realtime,Linux,Real-time,Realtime Linux,Real-time Linux,Open Source,Software,OSS,FOSS,Kernel,Automation,Cooperative,Member,Membership,Academic Member,Purchase Community" />
<meta name="DC.Rights" content="Open Source Automation Development Lab, (c)2007-2009. All rights reserved." />
<meta name="DC.Language" scheme="NISOZ39.50" content="en" />
<meta name="DC.Creator" content="Open Source Automation Development Lab contributors" />
<link rel="schema.dc" href="" />
<title>You can tune a piano - and you can tuna Linux system, too!: OSADL - Open Source Automation Development Lab eG</title>
<meta name="generator" content="TYPO3 4.1 CMS" />
<script type="text/javascript" src="typo3temp/javascript_f52d1a094f.js"></script>
<script type="text/javascript">
function openPic(url,winName,winParams) { //
var theWindow =,winName,winParams);
if (theWindow) {theWindow.focus();}
// -->
<script type="text/javascript">if (efa_fontSize) efa_fontSize.efaInit();</script>
<div id="page_margins"><!--###page_margins### begin -->
<div id="page" class="hold_floats"><!--###page### begin -->
<table class="toptable" id="toptable">
<td class="topleft" id="topleft"><img src="fileadmin/dam/logos/osadl_logo.png" width="160" height="70" border="0" alt="" title="" /></td>
<td class="topcenter" id="topcenter">Open Source Automation Development Lab</td>
<div id="col1_content" class="clearfix"></div>
<!--###col1### end --></div>
<!-- #col1: - Ende -->
<!-- #col2: zweite Flaot-Spalte des Inhaltsbereiches -->
<div id="col2"><!--###col2### begin -->
<div id="col2_content" class="clearfix"></div>
<hr class="hide" />
<!--###col2### end --></div>
<!-- #col2: - Ende -->
<a id="content" name="content"></a> <!-- Skiplink-Anker: Content -->
<!-- #col3: Statische Spalte des Inhaltsbereiches -->
<div id="col3"><!--###col3### begin -->
<div id="col3_content" class="clearfix"><!--###col3_content### begin -->
<div id="col3_innen" class="floatbox"><a id="c508"></a>
<div class="news-single-item">
<div class="news-single-timedata">2008-07-27 12:00 Age: 351 Days </div>
<h2>You can tune a piano - and you can tuna Linux system, too!</h2>
<p class="news-single-author">By: Carsten Emde (Used by permission)</p>
<h3>New versatile tools are helping us to manage real-time challenges under Linux (but you still can't tuna fish).</h3>
<div class="news-single-img"><img src="uploads/pics/spike.gif" width="144" height="107" border="0" alt="" title="" /><p class="news-single-imgcaption">The "CyclictestoSCOPE" in action</p></div>
<p class="bodytext">Do you wish you had a built-in oscilloscope that continuously displays the latency of your system? Here is one - thanks to Arnaldo Carvalho de Melo who not only wrote the <strong>oscilloscope</strong> but also the graphical process inspection and modification tool <strong>tuna</strong> that is part of the same tool package. In addition, these tools are not only very versatile tools that help us to manage real-time challenges - they also are excellent examples of the power and the beauty of the Python script language and its libraries. <br /> </p>
<p class="bodytext">However, before we can install and run these tools, we must learn a little bit about <strong>cyclictest</strong> and its command line arguments, since this is where oscilloscope gets its data from. </p>
<h5>Cyclictest<br /></h5>
<p class="bodytext">Cyclictest was initially written by Thomas Gleixner as an in-house tool to investigate possible regressions while developing the RT Preempt patches. Today, it is the state-of-the-art measurement tool to determine the internal worst-case latency of a Linux real-time system. No new version of the RT Preempt patches is ever released without having run cyclictest successfully on the new kernel. The git tree that contains the cyclictest tool can be cloned </p>
<pre>KERNELGIT=git://<br />git clone $KERNELGIT/clrkwllms/rt-tests.git</pre>
<p class="bodytext">but the software is also available as tarball. On an RPM package system such as Redhat Enterprise Linux or Fedora, simply type </p>
<pre>cd rt-tests<br />make release<br />mv rt-tests-*.tar.gz releases<br />make rpm<br />rpm -Uvh RPMS/i386/rt-tests-0*.i386.rpm</pre>
<p class="bodytext">to build and install it. Mandatory arguments of cyclictest are <strong>-n</strong> (use clock_nanosleep) and <strong>-p99</strong> (RT priority) on a uniprocessor and additional <strong>-a -t</strong> (one thread per CPU each on its own processor) on a multi-processor system. If in doubt, the <strong>-m</strong> option may be used to prevent the cyclictest memory from being paged to the swap area. In order to run as many consecutive threads as possible and to achieve the highest probability to meet a latency, the thread interval (<strong>-i</strong> option) should be about twice as long as the expected worst-case latency, and the thread delay (<strong>-d</strong> option) should be set to the interval divided by the number of CPUs. Here is an example of a cyclictest run on a Core 2 Duo, 64-bit, 2.4-GHz board:<br /> </p>
<pre># cyclictest -a -t -n -p99 -i100 -d50<br />560.44 586.11 606.12 211/1160 3727<br />T: 0 (18617) P:99 I:100 C:1011846111 Min:&nbsp; 2 Act:&nbsp; 4 Avg:&nbsp; 5 Max: 39<br />T: 1 (18618) P:98 I:150 C: 708641019 Min:&nbsp; 2 Act:&nbsp; 5 Avg: 11 Max: 57</pre>
<p class="bodytext">The test was running on a linux- kernel in parallel with repeated loops of &quot;hackbench 25&quot;,&quot;ls -Ral /&quot;, and flood ping from outside. After having executed more than one billion cyclictest loops, the internal worst-case latency is still only 39 µs!<br /> </p>
<h5>Tuna<br /></h5>
<p class="bodytext">The tuna tool is downloaded from the git repository and installed using the command sequence (<strong>updated</strong> on February 2, 2009): </p>
<pre>KERNELGIT=git://<br />git clone $KERNELGIT/acme/python-linux-procfs.git<br />cd python-linux-procfs <br />make rpm<br />rpm -Uvh rpm/RPMS/noarch/python-linux-procfs-*.noarch.rpm<br />cd -<br /></pre>
<pre>git clone $KERNELGIT/acme/tuna.git<br />cd tuna<br />make rpm<br />rpm -Uvh rpm/RPMS/noarch/tuna-*.noarch.rpm \<br /> rpm/RPMS/noarch/oscilloscope-*.noarch.rpm</pre>
<p class="bodytext">Start tuna simply by running it in an X Window environment. Now, what can you get from tuna? <img src="" alt="See tuna in action" title="See tuna in action" style="padding: 12px; width: 300px; height: 99px; float: left;" /></a>First, the output is very similar to the output of the well-known command line tool <strong>top</strong>. In addition, tuna has a much easier interface to arrange the display, to show threads etc. - <img style="padding: 12px; width: 128px; height: 95px; float: right;" src="" alt="tuna's CU options" title="tuna's CU options" /></a>but most importantly, it lets you modify many process settings such as the scheduling priority and policy (click on the right mouse button while the cursor is above one of the process lines). It also has a special menu (click on the right mouse button while the cursor is above the CPU Usage frame) to include or isolate the available processors.<br /> </p>
<h5>Oscilloscope<br /></h5>
<p class="bodytext">Finally, here comes the oscilloscope tool. It uses cyclictest's special <strong>-v</strong> option that causes the data to be continuously written to standard output path so they can be evaluated by another program connected via pipe. In this case, we can only run a single thread and, unfortunately, we cannot use the <strong>-i</strong> option explained above, since the oscilloscope has only a limited capacity (will be covered later). To get a first impression, run oscilloscope with the following command: </p>
<pre>cyclictest -t1 -n -p99 -v | oscilloscope &gt;/dev/null<br /></pre>
<p class="bodytext">Cool, isn't it? You may even increase the number of samples on screen using the <strong>-s</strong> option such as </p>
<pre>cyclictest -t1 -n -p99 -v | oscilloscope -s1000 &gt;/dev/null</pre>
<p class="bodytext"><img src="" alt="The GUI of the oscilloscope" title="The GUI of the oscilloscope" style="padding: 12px; width: 300px; height: 186px; float: left;" />Unfortunately, this will not capture every possible latency, since the cyclictest thread only runs once per millisecond. If we run it faster, for example by specifying <strong>-i100</strong>, our oscilloscope will be unable to follow. We have, therefore, implemented cyclictest's <strong>-o</strong> option (oscilloscope mode). This option lets cyclictest reduce the output by the given factor and write only the maximum value that occurred during the specified period of time. The command </p>
<pre>cyclictest -t1 -n -p99 -i100 -o10 -v | oscilloscope -s1000 &gt;/dev/null</pre>
<p class="bodytext">will then probably be able to provide a much better picture of the worst-case latency of a given system - as a disadvantage, however, the statistics given in the oscilloscope display are incorrect: The numbers no longer reflect the original raw samples but are based on preprocessed reduced data. However, since this is in any case a more pessimistic view, there is no danger that we underestimate the worst-case latency of the system. </p>
<hr class="clearer" />
<pre>Online Article:</pre>
<!--###col3_content### end --></div>
<!--###col3### end --></div>
<!-- #col3: - Ende -->
<!--###main### end --></div>
<!-- #main: - Ende -->
<!-- #Footer: Beginn Fußzeile -->
<div id="footer"><!--###footer### begin -->
<table width="100%">
<td id="footerleft">&nbsp;</td>
<td id="footercenter">&nbsp;</td>
<td id="footerright">&nbsp;</td>
<td id="subfooterleft">&nbsp;</td>
<td id="last_modified">This page was last modified on &nbsp;2008-01-11 - 16:15</td>
<td id="subfooterright">&nbsp;</td>
<hr class="hide" />
<!--###footer### end --></div>
<!-- #Footer: Ende -->
<!--###page### end --></div>
<!--###page_margins### end --></div>