trunk/lttv/QUICKSTART.html

   1 <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
   2 <html>
   3 <head>
   4   <title>Linux Trace Toolkit Next Generation User Documentation</title>
   5 </head>
   6   <body>
   7
   8 <h1>Linux Trace Toolkit Next Generation User Documentation</h1>
   9
  10 Author : Mathieu Desnoyers, September 2005<br>
  11 Last update : January 21st, 2009<br>
  12 (originally known as the LTTng QUICKSTART guide)
  13
  14 <h2>Table of Contents</h2>
  15 <ul>
  16
  17 <li><a href="#intro" name="TOCintro">Introduction</a></li>
  18 <li><a href="#section1" name="TOCsection1">Installing LTTng and LTTV from
  19 sources</a></li>
  20 <ul>
  21 <li><a href="#prerequisites" name="TOCprerequisites">Prerequisistes</li>
  22 <li><a href="#getlttng" name="TOCgetlttng">Getting the LTTng packages</li>
  23 <li><a href="#getlttngsrc" name="TOCgetlttngsrc">Getting the LTTng kernel sources</li>
  24 <li><a href="#installlttng" name="TOCinstalllttng">Installing a LTTng kernel</li>
  25 <li><a href="#editconfig" name="TOCeditconfig">Editing the system wide
  26 configuration</a>
  27 <li><a href="#getlttctl" name="TOCgetlttctl">Getting and installing the
  28 ltt-control package</li>
  29 <li><a href="#userspacetracing" name="TOCuserspacetracing">Userspace Tracing</li>
  30 <li><a href="#getlttv" name="TOCgetlttv">Getting and installing the LTTV package</ul>
  31 <li><a href="#section2" name="TOCsection2">Using LTTng and LTTV</a></li>
  32 <li><a href="#section3" name="TOCsection3">Adding kernel and user-space
  33 instrumentation</a>
  34 <ul>
  35 <li><a href="#kerneltp" name="TOCkerneltp">Adding kernel instrumentation</a></li>
  36 <li><a href="#usertp" name="TOCusertp">Adding userspace instrumentation</a></li>
  37 </ul>
  38 <li><a href="#section4" name="TOCsection4">Creating Debian and RPM packages
  39 from LTTV</a></li>
  40 <ul>
  41 <li><a href="#pkgdebian" name="TOCpkgdebian">Create custom LTTV Debian
  42 <li><a href="#pkglttng" name="TOCpkglttng">Create custom LTTng packages</a></li>
  43
  44 </ul>
  45
  46 </ul>
  47
  48 <hr />
  49
  50 <h2><a href="#TOCintro" name="intro">Introduction</a></h2>
  51 <p>
  52 This document is made of four parts : the first one explains how
  53 to install LTTng and LTTV from sources, the second one describes the steps
  54 to follow to trace a system and view it. The third part explains
  55 briefly how to add a new trace point to the kernel and to user space
  56 applications. The fourth and last part explains how to create Debian or RPM
  57 packages from the LTTng and LTTV sources.
  58 <p>
  59 These operations are made for installing the LTTng 0.86 tracer on a linux 2.6.X
  60 kernel. You will also find instructions for installation of LTTV 0.12.x : the
  61 Linux Trace Toolkit Viewer.
  62 To see the list of compatibilities between LTTng, ltt-control, LTTV, please
  63 refer to :
  64 <a
  65 href="http://ltt.polymtl.ca/svn/trunk/lttv/doc/developer/lttng-lttv-compatibility.html">LTTng+LTTV versions compatibility</a>
  66 The lttng patch is necessary to have the tracing hooks in the kernel.
  67
  68 <br>
  69 <br>
  70 Supported architectures :
  71 br>
  72 LTTng :<br>
  73 <li> x86 32/64 bits
  74 <li> PowerPC 32 and 64 bits
  75 <li> ARM (with limited timestamping precision, e.g. 1HZ. Need
  76 architecture-specific support for better precision)
  77 <li> MIPS
  78 <br>
  79 <br>
  80 LTTV :<br>
  81 <li> Intel 32/64 bits
  82 <li> PowerPC 32 and 64 bits
  83 <li> Possibly others. Takes care of endianness and type size difference between
  84 the LTTng traces and the LTTV analysis tool.
  85
  86 <hr />
  87
  88
  89 <h2><a href="#TOCsection1" name="section1">Installation from sources</a></h2>
  90 <p>
  91
  92 <h3><a href="#TOCprerequisites" name="prerequisites">Prerequisites</a></h3>
  93 <ul>
  94 <p>
  95 Tools needed to follow the package download steps :
  96
  97 <li>wget
  98 <li>bzip2
  99 <li>gzip
 100 <li>tar
 101
 102 <p>
 103 You have to install the standard development libraries and programs necessary
 104 to compile a kernel :
 105
 106 <PRE>
 107 (from Documentation/Changes in the Linux kernel tree)
 108 Gnu C                2.95.3                # gcc --version
 109 Gnu make             3.79.1                # make --version
 110 binutils             2.12                  # ld -v
 111 util-linux           2.10o                 # fdformat --version
 112 module-init-tools    0.9.10                # depmod -V
 113 </PRE>
 114
 115 <p>
 116 You might also want to have libncurses5 to have the text mode kernel
 117 configuration menu, but there are alternatives.
 118
 119 <p>
 120 Prerequisites for LTTV 0.x.x installation are :
 121
 122 <PRE>
 123 gcc 3.2 or better
 124 gtk 2.4 or better development libraries
 125   (Debian : libgtk2.0, libgtk2.0-dev)
 126   (Fedora : gtk2, gtk2-devel)
 127   note : For Fedora users : this might require at least core 3 from Fedora,
 128   or you might have to compile your own GTK2 library.
 129 glib 2.4 or better development libraries
 130   (Debian : libglib2.0-0, libglib2.0-dev)
 131   (Fedora : glib2, glib2-devel)
 132 libpopt development libraries
 133   (Debian : libpopt0, libpopt-dev)
 134   (Fedora : popt)
 135 libpango development libraries
 136   (Debian : libpango1.0, libpango1.0-dev)
 137   (Fedora : pango, pango-devel)
 138 libc6 development librairies
 139   (Debian : libc6, libc6-dev)
 140   (Fedora : glibc, glibc)
 141 </PRE>
 142 </ul>
 143
 144 <li>Reminder</li>
 145
 146 <p>
 147 See the list of compatibilities between LTTng, ltt-control and LTTV at :
 148 <a
 149 href="http://ltt.polymtl.ca/svn/trunk/lttv/doc/developer/lttng-lttv-compatibility.html">LTTng+LTTV
 150 versions compatibility</a>.
 151
 152
 153 <h3><a href="#TOCgetlttng" name="getlttng">Getting the LTTng packages</a></h3>
 154
 155 <PRE>
 156 su -
 157 mkdir /usr/src/lttng
 158 cd /usr/src/lttng
 159 (see http://ltt.polymtl.ca/lttng for package listing)
 160 wget http://ltt.polymtl.ca/lttng/patch-2.6.X-lttng-0.x.xx.tar.bz2
 161 bzip2 -cd patch-2.6.X-lttng-0.x.xx.tar.bz2 | tar xvof -
 162 </PRE>
 163
 164
 165 <h3><a href="#TOCgetlttngsrc" name="getlttngsrc">Getting LTTng kernel sources</a></h3>
 166
 167 <PRE>
 168 su -
 169 cd /usr/src
 170 wget http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.X.tar.bz2
 171 bzip2 -cd linux-2.6.X.tar.bz2 | tar xvof -
 172 cd linux-2.6.X
 173 - For LTTng 0.9.4- cat /usr/src/lttng/patch*-2.6.X-lttng-0.x.xx* | patch -p1
 174 - For LTTng 0.9.5+ apply the patches in the order specified in the series file,
 175   or use quilt
 176 cd ..
 177 mv linux-2.6.X linux-2.6.X-lttng-0.x.xx
 178 </PRE>
 179
 180
 181 <h3><a href="#TOCinstalllttng" name="installlttng">Installing a LTTng kernel</a></h3>
 182
 183 <PRE>
 184 su -
 185 cd /usr/src/linux-2.6.X-lttng-0.x.xx
 186 make menuconfig (or make xconfig or make config)
 187     Select the < Help > button if you are not familiar with kernel
 188     configuration.
 189     Items preceded by [*] means they has to be built into the kernel.
 190     Items preceded by [M] means they has to be built as modules.
 191     Items preceded by [ ] means they should be removed.
 192   go to the "General setup" section
 193     Select the following options :
 194     [*] Prompt for development and/or incomplete code/drivers
 195     [*] Activate markers
 196     [*] Activate userspace markers ABI (experimental, optional)
 197     [*] Immediate value optimization (optional)
 198     [*] Linux Trace Toolkit Next Generation (LTTng) --->
 199     <M> or <*> Compile lttng tracing probes
 200     <M> or <*> Linux Trace Toolkit High-speed Lockless Data Relay
 201     <M> or <*> Linux Trace Toolkit Lock-Protected Data Relay
 202     <M> or <*> Linux Trace Toolkit Serializer
 203     <M> or <*>   Linux Trace Toolkit Marker Control
 204     <M> or <*>     Linux Trace Toolkit Tracer
 205     [*] Align Linux Trace Toolkit Traces
 206     <M> or <*> Support logging events from userspace
 207     [*] Support trace extraction from crash dump
 208     <M> or <*> Linux Trace Toolkit Trace Controller
 209     <M> or <*> Linux Trace Toolkit State Dump
 210   Select <Exit>
 211   Select <Exit>
 212   Select <Yes>
 213 make
 214 make modules_install
 215 (if necessary, create a initrd with mkinitrd or your preferate alternative)
 216 (mkinitrd -o /boot/initrd.img-2.6.X-lttng-0.x.xx 2.6.X-lttng-0.x.xx)
 217
 218 -- on X86, X86_64
 219 make install
 220 reboot
 221 Select the Linux 2.6.X-lttng-0.x.xx kernel in your boot loader.
 222
 223 -- on PowerPC
 224 cp vmlinux.strip /boot/vmlinux-2.6.X-lttng-0.x.xx
 225 cp System.map /boot/System.map-2.6.X-lttng-0.x.xx
 226 cp .config /boot/config-2.6.X-lttng-0.x.xx
 227 depmod -ae -F /boot/System.map-2.6.X-lttng-0.x.xx 2.6.X-lttng-0.x.xx
 228 mkinitrd /boot/initrd.img-2.6.X-lttng-0.x.xx 2.6.X-lttng-0.x.xx
 229 (edit /etc/yaboot.conf to add a new entry pointing to your kernel : the entry
 230 that comes first is the default kernel)
 231 ybin
 232 select the right entry at the yaboot prompt (see choices : tab, select : type
 233 the kernel name followed by enter)
 234 Select the Linux 2.6.X-lttng-0.x.xx kernel in your boot loader.
 235 --
 236 </PRE>
 237
 238 <h3><a href="#TOCeditconfig" name="editconfig">Editing the system wide
 239 configuration</a></h3>
 240
 241 <p>
 242 You must activate debugfs and specify a mount point. This is typically done in
 243 fstab such that it happens at boot time.  If you have never used DebugFS before,
 244 these operation would do this for you :
 245
 246 <PRE>
 247 mkdir /mnt/debugfs
 248 cp /etc/fstab /etc/fstab.lttng.bkp
 249 echo "debugfs         /mnt/debugfs    debugfs rw              0       0"  >> /etc/fstab
 250 </PRE>
 251
 252 <p>
 253 then, rebooting or issuing the following command will activate debugfs :
 254 <PRE>
 255 mount /mnt/debugfs
 256 </PRE>
 257
 258 <p>
 259 You need to load the LTT modules to be able to control tracing from user
 260 space. This is done by issuing the following commands. Note however
 261 these commands load all LTT modules. Depending on what options you chose to
 262 compile statically, you may not need to issue all these commands.
 263
 264 <PRE>
 265 modprobe ltt-trace-control
 266 modprobe ltt-marker-control
 267 modprobe ltt-tracer
 268 modprobe ltt-serialize
 269 modprobe ltt-relay
 270 modprobe ipc-trace
 271 modprobe kernel-trace
 272 modprobe mm-trace
 273 modprobe net-trace
 274 modprobe fs-trace
 275 modprobe jbd2-trace
 276 modprobe ext4-trace
 277 modprobe syscall-trace
 278 modprobe trap-trace
 279 #if locking tracing is wanted, uncomment the following
 280 #modprobe lockdep-trace
 281 </PRE>
 282
 283 <p>
 284 If you want to have complete information about the kernel state (including all
 285 the process names), you need to load the ltt-statedump module. This is done by
 286 issuing the command :
 287
 288 <PRE>
 289 modprobe ltt-statedump
 290 </PRE>
 291 <p>
 292 You can automate at boot time loading the ltt-control module by :
 293
 294 <PRE>
 295 cp /etc/modules /etc/modules.bkp
 296 echo ltt-trace-control >> /etc/modules
 297 echo ltt-marker-control >> /etc/modules
 298 echo ltt-tracer >> /etc/modules
 299 echo ltt-serialize >> /etc/modules
 300 echo ltt-relay >> /etc/modules
 301 echo ipc-trace >> /etc/modules
 302 echo kernel-trace >> /etc/modules
 303 echo mm-trace >> /etc/modules
 304 echo net-trace >> /etc/modules
 305 echo fs-trace >> /etc/modules
 306 echo jbd2-trace >> /etc/modules
 307 echo ext4-trace >> /etc/modules
 308 echo syscall-trace >> /etc/modules
 309 echo trap-trace >> /etc/modules
 310 #if locking tracing is wanted, uncomment the following
 311 #echo lockdep-trace >> /etc/modules
 312 </PRE>
 313
 314
 315 <h3><a href="#TOCgetlttctl" name="getlttctl">Getting and installing the
 316 ltt-control package (on the traced machine)</a></h3>
 317 <p>
 318 (note : the ltt-control package contains lttd and lttctl. Although it has the
 319 same name as the ltt-control kernel module, they are *not* the same thing.)
 320
 321 <PRE>
 322 su -
 323 cd /usr/src
 324 wget http://ltt.polymtl.ca/lttng/ltt-control-0.x-xxxx2006.tar.gz
 325 gzip -cd ltt-control-0.x-xxxx2008.tar.gz | tar xvof -
 326 cd ltt-control-0.x-xxxx2006
 327 (refer to README to see the development libraries that must be installed on you
 328 system)
 329 ./configure
 330 make
 331 make install
 332 </PRE>
 333
 334 <h3><a href="#TOCuserspacetracing" name="userspacetracing">Userspace tracing</a></h3>
 335
 336 <PRE>
 337 Make sure you selected the kernel menuconfig option :
 338     <M> or <*> Support logging events from userspace
 339 And that the ltt-userspace-event kernel module is loaded if selected as a
 340 module.
 341
 342 Simple userspace tracing is available through
 343 echo "some text to record" > /mnt/debugfs/ltt/write_event
 344
 345 It will appear in the trace under event :
 346 channel : userspace
 347 event name : event
 348 </PRE>
 349
 350 <h3><a href="#TOCgetlttv" name="getlttv">Getting and installing the LTTV package
 351 (on the visualisation machine, same
 352 or different from the visualisation machine)</a></h3>
 353
 354 <PRE>
 355 su -
 356 cd /usr/src
 357 wget http://ltt.polymtl.ca/packages/lttv-0.x.xx-xxxx2008.tar.gz
 358 gzip -cd lttv-0.x.xx-xxxx2008.tar.gz | tar xvof -
 359 cd lttv-0.x.xx-xxxx2008
 360 (refer to README to see the development libraries that must be installed on your
 361 system)
 362 ./configure
 363 make
 364 make install
 365 </PRE>
 366
 367 <hr />
 368
 369
 370 <h2><a href="#TOCsection2" name="section2">Using LTTng and LTTV</a></h2>
 371
 372 <li>IMPORTANT : Arm Linux Kernel Markers after each boot</li>
 373 <PRE>
 374 ltt-armall
 375 </PRE>
 376
 377 <li>Use graphical LTTV to control tracing and analyse traces</li>
 378 <PRE>
 379 lttv-gui (or /usr/local/bin/lttv-gui)
 380   - Spot the "Tracing Control" icon : click on it
 381       (it's a traffic light icon)
 382     - enter the root password
 383     - click "start"
 384     - click "stop"
 385     - Yes
 386       * You should now see a trace
 387 </PRE>
 388
 389 <li>Use text mode LTTng to control tracing</li>
 390 <PRE>
 391 The tracing can be controlled from a terminal by using the lttctl command (as
 392 root).
 393
 394 Start tracing :
 395
 396 lttctl -C -w /tmp/trace1 trace1
 397
 398 Stop tracing and destroy trace channels :
 399
 400 lttctl -D trace1
 401
 402 see lttctl --help for details.
 403 </PRE>
 404 <p>
 405 (note : to see if the buffers has been filled, look at the dmesg output after
 406 lttctl -R or after stopping tracing from the GUI, it will show an event lost
 407 count. If it is the case, try using larger buffers. See lttctl --help to learn
 408 how. lttv now also shows event lost messages in the console when loading a trace
 409 with missing events or lost subbuffers.)
 410
 411 <li>Use text mode LTTV</li>
 412 <p>
 413 Feel free to look in /usr/local/lib/lttv/plugins to see all the text and
 414 graphical plugins available.
 415 <p>
 416 For example, a simple trace dump in text format is available with :
 417 <PRE>
 418 lttv -m textDump -t /tmp/trace
 419 </PRE>
 420 <p>
 421 See lttv -m textDump --help for detailed command line options of textDump.
 422 <p>
 423 It is, in the current state of the project, very useful to use "grep" on the
 424 text output to filter by specific event fields. You can later copy the timestamp
 425 of the events to the clipboard and paste them in the GUI by clicking on the
 426 bottom right label "Current time". Support for this type of filtering should
 427 be added to the filter module soon.
 428
 429 <li>Hybrid mode</li>
 430 <p>
 431 Starting from LTTng 0.5.105 and ltt-control 0.20, a new mode can be used :
 432 hybrid. It can be especially useful when studying big workloads on a long period
 433 of time.
 434 <p>
 435 When using this mode, the most important, low rate control information will be
 436 recorded during all the trace by lttd (i.e. process creation/exit). The high
 437 rate information (i.e. interrupt/traps/syscall entry/exit) will be kept in a
 438 flight recorder buffer (now named flight-channelname_X).
 439 <p>
 440 The following lttctl commands take an hybrid trace :
 441 <p>
 442 Create trace channel, start lttd on normal channels, start tracing:
 443 <PRE>
 444 lttctl -C -w /tmp/trace2 -o channel.kernel.overwrite=1 trace2
 445 </PRE>
 446 <p>
 447 Stop tracing, start lttd on flight recorder channels, destroy trace channels :
 448 <PRE>
 449 lttctl -D -w /tmp/trace2 trace2
 450 </PRE>
 451 <p>
 452 Each "overwrite" channel is flight recorder channel.
 453
 454 <li>Flight recorder mode</li>
 455 <p>
 456 The flight recorder mode writes data into overwritten buffers for all channels,
 457 including control channels, except for the facilities tracefiles. It consists of
 458 setting all channels to "overwrite".
 459 <p>
 460 The following lttctl commands take a flight recorder trace :
 461 <PRE>
 462 lttctl -C -w /tmp/trace3 -o channel.all.overwrite=1 trace3
 463 ...
 464 lttctl -D -w /tmp/trace3 trace3
 465 </PRE>
 466
 467 <hr />
 468
 469
 470 <h2><a href="#TOCsection3" name="section3">Adding new instrumentations with the
 471 markers</a></h2>
 472 <p>
 473
 474 <h3><a href="#TOCkerneltp" name="kerneltp">Adding kernel
 475 instrumentation</a></h3>
 476
 477 <p>
 478 See <a
 479 href="http://git.kernel.org/?p=linux/kernel/git/compudj/linux-2.6-lttng.git;a=blob;f=Documentation/markers.txt">Documentation/markers.txt</a>
 480 and <a
 481 href="http://git.kernel.org/?p=linux/kernel/git/compudj/linux-2.6-lttng.git;a=blob;f=Documentation/tracepoints.txt">Documentation/tracepoints.txt</a> in your kernel
 482 tree.
 483 <p>
 484 Also see <a
 485 href="http://git.kernel.org/?p=linux/kernel/git/compudj/linux-2.6-lttng.git;a=tree;f=ltt/probes">ltt/probes/</a>
 486 for LTTng probe examples.
 487
 488 <h3><a href="#TOCusertp" name="usertp">Adding userspace instrumentation</a></h3>
 489
 490 Add new events to userspace programs with
 491 <a href="http://ltt.polymtl.ca/packages/">userspace markers packages</a>.
 492 Get the latest markers-userspace-*.tar.bz2 and see the Makefile and examples. It
 493 allows inserting markers in executables and libraries, currently only on x86_32
 494 and x86_64.
 495 See <a
 496 href="http://ltt.polymtl.ca/packages/markers-userspace-0.5.tar.bz2">markers-userspace-0.5.tar.bz2</a> or more recent.
 497
 498 <p>
 499 Note that a new design document for a 3rd generation of tracepoint/marker-based
 500 userspace tracing is available at <a
 501 href="http://ltt.polymtl.ca/svn/trunk/lttv/doc/developer/ust.html">LTTng User-space Tracing
 502 Design</a>. This new infrastructure is not yet implemented.
 503
 504 <p>
 505 The easy quick-and-dirty way to perform userspace tracing is currently to write
 506 an string to /mnt/debugfs/ltt/write_event. See <a
 507 href="#userspacetracing">Userspace tracing</a> in the
 508 installation for sources section of this document.
 509
 510 <hr />
 511
 512 <h2><a href="#TOCsection4" name="section4">Creating Debian or RPM packages</a></h2>
 513 <p>
 514
 515 <h3><a href="#TOCpkgdebian" name="pkgdebian">Create custom LTTV Debian packages</a></h3>
 516
 517 <PRE>
 518 Use : dpkg-buildpackage -rfakeroot
 519 </PRE>
 520 <p>
 521 You should then have your LTTV .deb files created for your architecture.
 522
 523 <h3><a href="#TOCpkglttng" name="pkglttng">Create custom LTTng packages</a></h3>
 524 <p>
 525 For building LTTng Debian packages :
 526 get the build tree with patches applies as explained in section 2.
 527
 528 <PRE>
 529 make menuconfig (or xconfig or config) (customize your configuration)
 530 make-kpkg kernel_image
 531 </PRE>
 532 <p>
 533 You will then see your freshly created .deb in /usr/src. Install it with
 534 <PRE>
 535 dpkg -i /usr/src/(image-name).deb
 536 </PRE>
 537 <p>
 538 Then, follow the section "Editing the system wide configuration" in section 2.
 539
 540
 541
 542   </body>
 543 </html>