trunk/lttv/QUICKSTART.html

   1 <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
   2 <html>
   3 <head>
   4   <title>Linux Trace Toolkit Next Generation User Documentation</title>
   5 </head>
   6   <body>
   7
   8 <h1>Linux Trace Toolkit Next Generation User Documentation</h1>
   9
  10 Author : Mathieu Desnoyers, September 2005<br>
  11 Last update : January 21st, 2009<br>
  12 (originally known as the LTTng QUICKSTART guide)
  13
  14 <h2>Table of Contents</h2>
  15 <ul>
  16
  17 <li><a href="#intro" name="TOCintro">Introduction</a></li>
  18 <li><a href="#section1" name="TOCsection1">Installing LTTng and LTTV from
  19 <ul>
  20 <li><a href="#userspacetracing" name="TOCuserspacetracing">Userspace Tracing</ul>
  21 sources</a></li>
  22 <li><a href="#section2" name="TOCsection2">Using LTTng and LTTV</a></li>
  23 <li><a href="#section3" name="TOCsection3">Adding kernel and user-space
  24 tracepoints</a></li>
  25 <li><a href="#section4" name="TOCsection4">Creating Debian and RPM packages
  26 from LTTV</a>
  27
  28 </ul>
  29
  30 <hr />
  31
  32 <h2><a href="#TOCintro" name="intro">Introduction</a></h2>
  33 <p>
  34 This document is made of four parts : the first one explains how
  35 to install LTTng and LTTV from sources, the second one describes the steps
  36 to follow to trace a system and view it. The third part explains
  37 briefly how to add a new trace point to the kernel and to user space
  38 applications. The fourth and last part explains how to create Debian or RPM
  39 packages from the LTTng and LTTV sources.
  40 <p>
  41 These operations are made for installing the LTTng 0.86 tracer on a linux 2.6.X
  42 kernel. You will also find instructions for installation of LTTV 0.12.x : the
  43 Linux Trace Toolkit Viewer.
  44 To see the list of compatibilities between LTTng, ltt-control, LTTV, please
  45 refer to :
  46 <a
  47 href="http://ltt.polymtl.ca/svn/trunk/lttv/doc/developer/lttng-lttv-compatibility.html">LTTng+LTTV versions compatibility</a>
  48 The lttng patch is necessary to have the tracing hooks in the kernel.
  49
  50 <br>
  51 <br>
  52 Supported architectures :
  53 <br>
  54 LTTng :<br>
  55 <li> x86 32/64 bits
  56 <li> PowerPC 32 and 64 bits
  57 <li> ARM (with limited timestamping precision, e.g. 1HZ. Need
  58 architecture-specific support for better precision)
  59 <li> MIPS
  60 <br>
  61 <br>
  62 LTTV :<br>
  63 <li> Intel 32/64 bits
  64 <li> PowerPC 32 and 64 bits
  65 <li> Possibly others. Takes care of endianness and type size difference between
  66 the LTTng traces and the LTTV analysis tool.
  67
  68 <hr />
  69
  70
  71 <h2><a href="#TOCsection1" name="section1">Installation from sources</a></h2>
  72 <p>
  73
  74 <li>Prerequisites</li>
  75 <ul>
  76 <p>
  77 Tools needed to follow the package download steps :
  78
  79 <li>wget
  80 <li>bzip2
  81 <li>gzip
  82 <li>tar
  83
  84 <p>
  85 You have to install the standard development libraries and programs necessary
  86 to compile a kernel :
  87
  88 <PRE>
  89 (from Documentation/Changes in the Linux kernel tree)
  90 Gnu C                2.95.3                # gcc --version
  91 Gnu make             3.79.1                # make --version
  92 binutils             2.12                  # ld -v
  93 util-linux           2.10o                 # fdformat --version
  94 module-init-tools    0.9.10                # depmod -V
  95 </PRE>
  96
  97 <p>
  98 You might also want to have libncurses5 to have the text mode kernel
  99 configuration menu, but there are alternatives.
 100
 101 <p>
 102 Prerequisites for LTTV 0.x.x installation are :
 103
 104 <PRE>
 105 gcc 3.2 or better
 106 gtk 2.4 or better development libraries
 107   (Debian : libgtk2.0, libgtk2.0-dev)
 108   (Fedora : gtk2, gtk2-devel)
 109   note : For Fedora users : this might require at least core 3 from Fedora,
 110   or you might have to compile your own GTK2 library.
 111 glib 2.4 or better development libraries
 112   (Debian : libglib2.0-0, libglib2.0-dev)
 113   (Fedora : glib2, glib2-devel)
 114 libpopt development libraries
 115   (Debian : libpopt0, libpopt-dev)
 116   (Fedora : popt)
 117 libpango development libraries
 118   (Debian : libpango1.0, libpango1.0-dev)
 119   (Fedora : pango, pango-devel)
 120 libc6 development librairies
 121   (Debian : libc6, libc6-dev)
 122   (Fedora : glibc, glibc)
 123 </PRE>
 124 </ul>
 125
 126 <li>Reminder</li>
 127
 128 <p>
 129 See the list of compatibilities between LTTng, ltt-control and LTTV at :
 130 <a
 131 href="http://ltt.polymtl.ca/svn/trunk/lttv/doc/developer/lttng-lttv-compatibility.html">LTTng+LTTV
 132 versions compatibility</a>.
 133
 134
 135 <li>Getting the LTTng packages</li>
 136
 137 <PRE>
 138 su -
 139 mkdir /usr/src/lttng
 140 cd /usr/src/lttng
 141 (see http://ltt.polymtl.ca/lttng for package listing)
 142 wget http://ltt.polymtl.ca/lttng/patch-2.6.X-lttng-0.x.xx.tar.bz2
 143 bzip2 -cd patch-2.6.X-lttng-0.x.xx.tar.bz2 | tar xvof -
 144 </PRE>
 145
 146
 147 <li>Getting LTTng kernel sources</li>
 148
 149 <PRE>
 150 su -
 151 cd /usr/src
 152 wget http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.X.tar.bz2
 153 bzip2 -cd linux-2.6.X.tar.bz2 | tar xvof -
 154 cd linux-2.6.X
 155 - For LTTng 0.9.4- cat /usr/src/lttng/patch*-2.6.X-lttng-0.x.xx* | patch -p1
 156 - For LTTng 0.9.5+ apply the patches in the order specified in the series file,
 157   or use quilt
 158 cd ..
 159 mv linux-2.6.X linux-2.6.X-lttng-0.x.xx
 160 </PRE>
 161
 162
 163 <li>Installing a LTTng kernel</li>
 164
 165 <PRE>
 166 su -
 167 cd /usr/src/linux-2.6.X-lttng-0.x.xx
 168 make menuconfig (or make xconfig or make config)
 169     Select the < Help > button if you are not familiar with kernel
 170     configuration.
 171     Items preceded by [*] means they has to be built into the kernel.
 172     Items preceded by [M] means they has to be built as modules.
 173     Items preceded by [ ] means they should be removed.
 174   go to the "General setup" section
 175     Select the following options :
 176     [*] Prompt for development and/or incomplete code/drivers
 177     [*] Activate markers
 178     [*] Activate userspace markers ABI (experimental, optional)
 179     [*] Immediate value optimization (optional)
 180     [*] Linux Trace Toolkit Next Generation (LTTng) --->
 181     <M> or <*> Compile lttng tracing probes
 182     <M> or <*> Linux Trace Toolkit High-speed Lockless Data Relay
 183     <M> or <*> Linux Trace Toolkit Lock-Protected Data Relay
 184     <M> or <*> Linux Trace Toolkit Serializer
 185     <M> or <*>   Linux Trace Toolkit Marker Control
 186     <M> or <*>     Linux Trace Toolkit Tracer
 187     [*] Align Linux Trace Toolkit Traces
 188     <M> or <*> Support logging events from userspace
 189     [*] Support trace extraction from crash dump
 190     <M> or <*> Linux Trace Toolkit Trace Controller
 191     <M> or <*> Linux Trace Toolkit State Dump
 192   Select <Exit>
 193   Select <Exit>
 194   Select <Yes>
 195 make
 196 make modules_install
 197 (if necessary, create a initrd with mkinitrd or your preferate alternative)
 198 (mkinitrd -o /boot/initrd.img-2.6.X-lttng-0.x.xx 2.6.X-lttng-0.x.xx)
 199
 200 -- on X86, X86_64
 201 make install
 202 reboot
 203 Select the Linux 2.6.X-lttng-0.x.xx kernel in your boot loader.
 204
 205 -- on PowerPC
 206 cp vmlinux.strip /boot/vmlinux-2.6.X-lttng-0.x.xx
 207 cp System.map /boot/System.map-2.6.X-lttng-0.x.xx
 208 cp .config /boot/config-2.6.X-lttng-0.x.xx
 209 depmod -ae -F /boot/System.map-2.6.X-lttng-0.x.xx 2.6.X-lttng-0.x.xx
 210 mkinitrd /boot/initrd.img-2.6.X-lttng-0.x.xx 2.6.X-lttng-0.x.xx
 211 (edit /etc/yaboot.conf to add a new entry pointing to your kernel : the entry
 212 that comes first is the default kernel)
 213 ybin
 214 select the right entry at the yaboot prompt (see choices : tab, select : type
 215 the kernel name followed by enter)
 216 Select the Linux 2.6.X-lttng-0.x.xx kernel in your boot loader.
 217 --
 218 </PRE>
 219
 220 <li>Editing the system wide configuration</li>
 221
 222 <p>
 223 You must activate debugfs and specify a mount point. This is typically done in
 224 fstab such that it happens at boot time.  If you have never used DebugFS before,
 225 these operation would do this for you :
 226
 227 <PRE>
 228 mkdir /mnt/debugfs
 229 cp /etc/fstab /etc/fstab.lttng.bkp
 230 echo "debugfs         /mnt/debugfs    debugfs rw              0       0"  >> /etc/fstab
 231 </PRE>
 232
 233 <p>
 234 then, rebooting or issuing the following command will activate debugfs :
 235 <PRE>
 236 mount /mnt/debugfs
 237 </PRE>
 238
 239 <p>
 240 You need to load the LTT modules to be able to control tracing from user
 241 space. This is done by issuing the following commands. Note however
 242 these commands load all LTT modules. Depending on what options you chose to
 243 compile statically, you may not need to issue all these commands.
 244
 245 <PRE>
 246 modprobe ltt-trace-control
 247 modprobe ltt-marker-control
 248 modprobe ltt-tracer
 249 modprobe ltt-serialize
 250 modprobe ltt-relay
 251 modprobe ipc-trace
 252 modprobe kernel-trace
 253 modprobe mm-trace
 254 modprobe net-trace
 255 modprobe fs-trace
 256 modprobe jbd2-trace
 257 modprobe ext4-trace
 258 modprobe syscall-trace
 259 modprobe trap-trace
 260 #if locking tracing is wanted, uncomment the following
 261 #modprobe lockdep-trace
 262 </PRE>
 263
 264 <p>
 265 If you want to have complete information about the kernel state (including all
 266 the process names), you need to load the ltt-statedump module. This is done by
 267 issuing the command :
 268
 269 <PRE>
 270 modprobe ltt-statedump
 271 </PRE>
 272 <p>
 273 You can automate at boot time loading the ltt-control module by :
 274
 275 <PRE>
 276 cp /etc/modules /etc/modules.bkp
 277 echo ltt-trace-control >> /etc/modules
 278 echo ltt-marker-control >> /etc/modules
 279 echo ltt-tracer >> /etc/modules
 280 echo ltt-serialize >> /etc/modules
 281 echo ltt-relay >> /etc/modules
 282 echo ipc-trace >> /etc/modules
 283 echo kernel-trace >> /etc/modules
 284 echo mm-trace >> /etc/modules
 285 echo net-trace >> /etc/modules
 286 echo fs-trace >> /etc/modules
 287 echo jbd2-trace >> /etc/modules
 288 echo ext4-trace >> /etc/modules
 289 echo syscall-trace >> /etc/modules
 290 echo trap-trace >> /etc/modules
 291 #if locking tracing is wanted, uncomment the following
 292 #echo lockdep-trace >> /etc/modules
 293 </PRE>
 294
 295 <li>Getting and installing the ltt-control package (on the traced machine)</li>
 296 <p>
 297 (note : the ltt-control package contains lttd and lttctl. Although it has the
 298 same name as the ltt-control kernel module, they are *not* the same thing.)
 299
 300 <PRE>
 301 su -
 302 cd /usr/src
 303 wget http://ltt.polymtl.ca/lttng/ltt-control-0.x-xxxx2006.tar.gz
 304 gzip -cd ltt-control-0.x-xxxx2008.tar.gz | tar xvof -
 305 cd ltt-control-0.x-xxxx2006
 306 (refer to README to see the development libraries that must be installed on you
 307 system)
 308 ./configure
 309 make
 310 make install
 311 </PRE>
 312
 313 <h3><a href="#TOCuserspacetracing" name="userspacetracing">Userspace tracing</a></h3>
 314
 315 <PRE>
 316 Make sure you selected the kernel menuconfig option :
 317     <M> or <*> Support logging events from userspace
 318 And that the ltt-userspace-event kernel module is loaded if selected as a
 319 module.
 320
 321 Simple userspace tracing is available through
 322 echo "some text to record" > /mnt/debugfs/ltt/write_event
 323
 324 It will appear in the trace under event :
 325 channel : userspace
 326 event name : event
 327 </PRE>
 328
 329 <li>Getting and installing the LTTV package (on the visualisation machine, same
 330 or different from the visualisation machine)</li>
 331
 332 <PRE>
 333 su -
 334 cd /usr/src
 335 wget http://ltt.polymtl.ca/packages/lttv-0.x.xx-xxxx2008.tar.gz
 336 gzip -cd lttv-0.x.xx-xxxx2008.tar.gz | tar xvof -
 337 cd lttv-0.x.xx-xxxx2008
 338 (refer to README to see the development libraries that must be installed on your
 339 system)
 340 ./configure
 341 make
 342 make install
 343 </PRE>
 344
 345 <hr />
 346
 347
 348 <h2><a href="#TOCsection2" name="section2">Using LTTng and LTTV</a></h2>
 349
 350 <li>IMPORTANT : Arm Linux Kernel Markers after each boot</li>
 351 <PRE>
 352 ltt-armall
 353 </PRE>
 354
 355 <li>Use graphical LTTV to control tracing and analyse traces</li>
 356 <PRE>
 357 lttv-gui (or /usr/local/bin/lttv-gui)
 358   - Spot the "Tracing Control" icon : click on it
 359       (it's a traffic light icon)
 360     - enter the root password
 361     - click "start"
 362     - click "stop"
 363     - Yes
 364       * You should now see a trace
 365 </PRE>
 366
 367 <li>Use text mode LTTng to control tracing</li>
 368 <PRE>
 369 The tracing can be controlled from a terminal by using the lttctl command (as
 370 root).
 371
 372 Start tracing :
 373
 374 lttctl -C -w /tmp/trace1 trace1
 375
 376 Stop tracing and destroy trace channels :
 377
 378 lttctl -D trace1
 379
 380 see lttctl --help for details.
 381 </PRE>
 382 <p>
 383 (note : to see if the buffers has been filled, look at the dmesg output after
 384 lttctl -R or after stopping tracing from the GUI, it will show an event lost
 385 count. If it is the case, try using larger buffers. See lttctl --help to learn
 386 how. lttv now also shows event lost messages in the console when loading a trace
 387 with missing events or lost subbuffers.)
 388
 389 <li>Use text mode LTTV</li>
 390 <p>
 391 Feel free to look in /usr/local/lib/lttv/plugins to see all the text and
 392 graphical plugins available.
 393 <p>
 394 For example, a simple trace dump in text format is available with :
 395 <PRE>
 396 lttv -m textDump -t /tmp/trace
 397 </PRE>
 398 <p>
 399 See lttv -m textDump --help for detailed command line options of textDump.
 400 <p>
 401 It is, in the current state of the project, very useful to use "grep" on the
 402 text output to filter by specific event fields. You can later copy the timestamp
 403 of the events to the clipboard and paste them in the GUI by clicking on the
 404 bottom right label "Current time". Support for this type of filtering should
 405 be added to the filter module soon.
 406
 407 <li>Hybrid mode</li>
 408 <p>
 409 Starting from LTTng 0.5.105 and ltt-control 0.20, a new mode can be used :
 410 hybrid. It can be especially useful when studying big workloads on a long period
 411 of time.
 412 <p>
 413 When using this mode, the most important, low rate control information will be
 414 recorded during all the trace by lttd (i.e. process creation/exit). The high
 415 rate information (i.e. interrupt/traps/syscall entry/exit) will be kept in a
 416 flight recorder buffer (now named flight-channelname_X).
 417 <p>
 418 The following lttctl commands take an hybrid trace :
 419 <p>
 420 Create trace channel, start lttd on normal channels, start tracing:
 421 <PRE>
 422 lttctl -C -w /tmp/trace2 -o channel.kernel.overwrite=1 trace2
 423 </PRE>
 424 <p>
 425 Stop tracing, start lttd on flight recorder channels, destroy trace channels :
 426 <PRE>
 427 lttctl -D -w /tmp/trace2 trace2
 428 </PRE>
 429 <p>
 430 Each "overwrite" channel is flight recorder channel.
 431
 432 <li>Flight recorder mode</li>
 433 <p>
 434 The flight recorder mode writes data into overwritten buffers for all channels,
 435 including control channels, except for the facilities tracefiles. It consists of
 436 setting all channels to "overwrite".
 437 <p>
 438 The following lttctl commands take a flight recorder trace :
 439 <PRE>
 440 lttctl -C -w /tmp/trace3 -o channel.all.overwrite=1 trace3
 441 ...
 442 lttctl -D -w /tmp/trace3 trace3
 443 </PRE>
 444
 445 <hr />
 446
 447
 448 <h2><a href="#TOCsection3" name="section3">Adding new instrumentations with the
 449 markers</a></h2>
 450 <p>
 451
 452 <p>
 453 See Documentation/markers.txt and Documentation/tracepoints.txt in your kernel
 454 tree.
 455
 456 <li>Add new events to userspace programs with
 457 <a href="http://ltt.polymtl.ca/packages/">userspace markers packages</a></li>
 458
 459 <p>
 460 Get the latest markers-userspace-*.tar.bz2 and see the Makefile and examples. It
 461 allows inserting markers in executables and libraries, currently only on x86_32
 462 and x86_64.
 463 See <a
 464 href="http://ltt.polymtl.ca/packages/markers-userspace-0.5.tar.bz2">markers-userspace-0.5.tar.bz2</a> or more recent.
 465
 466 <p>
 467 Note that a new design document for a 3rd generation of tracepoint/marker-based
 468 userspace tracing is available at <a
 469 href="http://ltt.polymtl.ca/svn/trunk/lttv/doc/developer/ust.html">LTTng User-space Tracing
 470 Design"</a>. This new infrastructure is not yet implemented.
 471
 472 <p>
 473 The easy quick-and-dirty way to perform userspace tracing is currently to write
 474 an string to /mnt/debugfs/ltt/write_event. See <a
 475 href="#userspacetracing">Userspace tracing</a> in the
 476 installation for sources section of this document.
 477
 478 <hr />
 479
 480 <h2><a href="#TOCsection4" name="section4">Creating Debian or RPM packages</a></h2>
 481 <p>
 482
 483 <li>Create custom LTTV Debian packages</li>
 484
 485 <PRE>
 486 Use : dpkg-buildpackage -rfakeroot
 487 </PRE>
 488 <p>
 489 You should then have your LTTV .deb files created for your architecture.
 490
 491 <li>Create custom LTTng packages</li>
 492 <p>
 493 For building LTTng Debian packages :
 494 get the build tree with patches applies as explained in section 2.
 495
 496 <PRE>
 497 make menuconfig (or xconfig or config) (customize your configuration)
 498 make-kpkg kernel_image
 499 </PRE>
 500 <p>
 501 You will then see your freshly created .deb in /usr/src. Install it with
 502 <PRE>
 503 dpkg -i /usr/src/(image-name).deb
 504 </PRE>
 505 <p>
 506 Then, follow the section "Editing the system wide configuration" in section 2.
 507
 508
 509
 510   </body>
 511 </html>