trunk/lttv/QUICKSTART.html

   1 <!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
   2 <html>
   3 <head>
   4   <title>Linux Trace Toolkit Quickstart</title>
   5 </head>
   6   <body>
   7
   8 <h1>Linux Trace Toolkit Quickstart</h1>
   9
  10
  11 Author : Mathieu Desnoyers, September 2005<br>
  12 Last update : January 21st, 2009<br>
  13
  14 <h2>Table of Contents</h2>
  15 <ul>
  16
  17 <li><a href="#intro" name="TOCintro">Introduction</a></li>
  18 <li><a href="#section1" name="TOCsection1">Installing LTTng and LTTV from
  19 sources</a></li>
  20 <li><a href="#section2" name="TOCsection2">Using LTTng and LTTV</a></li>
  21 <li><a href="#section3" name="TOCsection3">Adding kernel and user-space
  22 tracepoints</a></li>
  23
  24 </ul>
  25
  26 <hr />
  27
  28 <h2><a href="#TOCintro" name="intro">Introduction</a></h2>
  29 <p>
  30 This document is made of three parts : the first one explains how
  31 to install LTTng and LTTV from sources, the second one describes the steps
  32 to follow to trace a system and view it. The third and last part explains
  33 briefly how to add a new trace point to the kernel and to user space
  34 applications.
  35
  36 <p>
  37 These operations are made for installing the LTTng 0.86 tracer on a linux 2.6.X
  38 kernel. You will also find instructions for installation of LTTV 0.12.x : the
  39 Linux Trace Toolkit Viewer.
  40
  41 <p>
  42 To see the list of compatibilities between LTTng, ltt-control, LTTV, please
  43 refer to :
  44 <a
  45 href="http://ltt.polymtl.ca/svn/trunk/lttv/doc/developer/lttng-lttv-compatibility.html">LTTng+LTTV versions compatibility</a>
  46
  47 <p>
  48 The lttng patch is necessary to have the tracing hooks in the kernel.
  49
  50 <br>
  51 <br>
  52 ** Current development status **<br>
  53 <br>
  54 <br>
  55 LTTng :<br>
  56 Supported architectures :
  57 <li> x86 32/64 bits
  58 <li> PowerPC 32 and 64 bits
  59 <li> ARM (with limited timestamping precision, e.g. 1HZ. Need
  60 architecture-specific support for better precision)
  61 <li> MIPS
  62 <br>
  63 <br>
  64 LTTV :<br>
  65 supported architectures :<br>
  66 <li> Intel 32/64 bits
  67 <li> PowerPC 32 and 64 bits
  68 <li> Possibly others. Takes care of endianness and type size difference between
  69 the LTTng traces and the LTTV analysis tool.
  70
  71 <hr />
  72
  73 ***********************************************************
  74 ** Section 1 * Installation from Debian or RPM packages  **
  75 ***********************************************************
  76
  77 * Create custom LTTV Debian packages
  78
  79 Use : dpkg-buildpackage -rfakeroot
  80
  81 You should then have your LTTV .deb files created for your architecture.
  82
  83 * Create custom LTTng packages
  84
  85 For building LTTng Debian packages :
  86
  87 Get the build tree with patches applies as explained in section 2.
  88
  89 make menuconfig (or xconfig or config) (customize your configuration)
  90 make-kpkg kernel_image
  91
  92 You will then see your freshly created .deb in /usr/src. Install it with
  93 dpkg -i /usr/src/(image-name).deb
  94
  95 Then, follow the section "Editing the system wide configuration" in section 2.
  96
  97
  98 ***********************************************************
  99 ** Section 2 * Installation from sources                 **
 100 ***********************************************************
 101
 102 * Prerequisites
 103
 104 Tools needed to follow the package download steps :
 105
 106 o wget
 107 o bzip2
 108 o gzip
 109 o tar
 110
 111 You have to install the standard development libraries and programs necessary
 112 to compile a kernel :
 113
 114 (from Documentation/Changes in the Linux kernel tree)
 115 o  Gnu C                  2.95.3                  # gcc --version
 116 o  Gnu make               3.79.1                  # make --version
 117 o  binutils               2.12                    # ld -v
 118 o  util-linux             2.10o                   # fdformat --version
 119 o  module-init-tools      0.9.10                  # depmod -V
 120
 121 You might also want to have libncurses5 to have the text mode kernel
 122 configuration menu, but there are alternatives.
 123
 124 Prerequisites for LTTV 0.x.x installation are :
 125
 126 gcc 3.2 or better
 127 gtk 2.4 or better development libraries
 128   (Debian : libgtk2.0, libgtk2.0-dev)
 129   (Fedora : gtk2, gtk2-devel)
 130   note : For Fedora users : this might require at least core 3 from Fedora,
 131   or you might have to compile your own GTK2 library.
 132 glib 2.4 or better development libraries
 133   (Debian : libglib2.0-0, libglib2.0-dev)
 134   (Fedora : glib2, glib2-devel)
 135 libpopt development libraries
 136   (Debian : libpopt0, libpopt-dev)
 137   (Fedora : popt)
 138 libpango development libraries
 139   (Debian : libpango1.0, libpango1.0-dev)
 140   (Fedora : pango, pango-devel)
 141 libc6 development librairies
 142   (Debian : libc6, libc6-dev)
 143   (Fedora : glibc, glibc)
 144
 145 * Reminder
 146
 147 See the list of compatibilities between LTTng, ltt-control, LTTV and
 148 markers-userspace at :
 149 http://ltt.polymtl.ca > LTTng+LTTV versions compatibility
 150
 151
 152
 153 * Getting the LTTng packages
 154
 155 su -
 156 mkdir /usr/src/lttng
 157 cd /usr/src/lttng
 158 (see http://ltt.polymtl.ca/lttng for package listing)
 159 wget http://ltt.polymtl.ca/lttng/patch-2.6.X-lttng-0.x.xx.tar.bz2
 160 bzip2 -cd patch-2.6.X-lttng-0.x.xx.tar.bz2 | tar xvof -
 161
 162
 163 * Getting LTTng kernel sources
 164
 165 su -
 166 cd /usr/src
 167 wget http://kernel.org/pub/linux/kernel/v2.6/linux-2.6.X.tar.bz2
 168 bzip2 -cd linux-2.6.X.tar.bz2 | tar xvof -
 169 cd linux-2.6.X
 170 - For LTTng 0.9.4- cat /usr/src/lttng/patch*-2.6.X-lttng-0.x.xx* | patch -p1
 171 - For LTTng 0.9.5+ apply the patches in the order specified in the series file,
 172   or use quilt
 173 cd ..
 174 mv linux-2.6.X linux-2.6.X-lttng-0.x.xx
 175
 176
 177 * Installing a LTTng kernel
 178
 179 su -
 180 cd /usr/src/linux-2.6.X-lttng-0.x.xx
 181 make menuconfig (or make xconfig or make config)
 182     Select the < Help > button if you are not familiar with kernel
 183     configuration.
 184     Items preceded by [*] means they has to be built into the kernel.
 185     Items preceded by [M] means they has to be built as modules.
 186     Items preceded by [ ] means they should be removed.
 187   go to the "General setup" section
 188     Select the following options :
 189     [*] Prompt for development and/or incomplete code/drivers
 190     [*] Activate markers
 191     [*] Activate userspace markers ABI (experimental, optional)
 192     [*] Immediate value optimization (optional)
 193     [*] Linux Trace Toolkit Next Generation (LTTng) --->
 194     <M> or <*> Compile lttng tracing probes
 195     <M> or <*> Linux Trace Toolkit High-speed Lockless Data Relay
 196     <M> or <*> Linux Trace Toolkit Lock-Protected Data Relay
 197     <M> or <*> Linux Trace Toolkit Serializer
 198     <M> or <*>   Linux Trace Toolkit Marker Control
 199     <M> or <*>     Linux Trace Toolkit Tracer
 200     [*] Align Linux Trace Toolkit Traces
 201     <M> or <*> Support logging events from userspace
 202     [*] Support trace extraction from crash dump
 203     <M> or <*> Linux Trace Toolkit Trace Controller
 204     <M> or <*> Linux Trace Toolkit State Dump
 205   Select <Exit>
 206   Select <Exit>
 207   Select <Yes>
 208 make
 209 make modules_install
 210 (if necessary, create a initrd with mkinitrd or your preferate alternative)
 211 (mkinitrd -o /boot/initrd.img-2.6.X-lttng-0.x.xx 2.6.X-lttng-0.x.xx)
 212
 213 -- on X86, X86_64
 214 make install
 215 reboot
 216 Select the Linux 2.6.X-lttng-0.x.xx kernel in your boot loader.
 217
 218 -- on PowerPC
 219 cp vmlinux.strip /boot/vmlinux-2.6.X-lttng-0.x.xx
 220 cp System.map /boot/System.map-2.6.X-lttng-0.x.xx
 221 cp .config /boot/config-2.6.X-lttng-0.x.xx
 222 depmod -ae -F /boot/System.map-2.6.X-lttng-0.x.xx 2.6.X-lttng-0.x.xx
 223 mkinitrd /boot/initrd.img-2.6.X-lttng-0.x.xx 2.6.X-lttng-0.x.xx
 224 (edit /etc/yaboot.conf to add a new entry pointing to your kernel : the entry
 225 that comes first is the default kernel)
 226 ybin
 227 select the right entry at the yaboot prompt (see choices : tab, select : type
 228 the kernel name followed by enter)
 229 Select the Linux 2.6.X-lttng-0.x.xx kernel in your boot loader.
 230 --
 231
 232
 233
 234 * Editing the system wide configuration
 235
 236 You must activate debugfs and specify a mount point. This is typically done in
 237 fstab such that it happens at boot time.
 238
 239 If you have never used DebugFS before, these operation would do this for you :
 240
 241 mkdir /mnt/debugfs
 242 cp /etc/fstab /etc/fstab.lttng.bkp
 243 echo "debugfs         /mnt/debugfs    debugfs rw              0       0"  >> /etc/fstab
 244
 245 then, rebooting or issuing the following command will activate debugfs :
 246
 247 mount /mnt/debugfs
 248
 249 You need to load the LTT modules to be able to control tracing from user
 250 space. This is done by issuing the following commands. Note however
 251 these commands load all LTT modules. Depending on what options you chose to
 252 compile statically, you may not need to issue all these commands.
 253
 254 modprobe ltt-trace-control
 255 modprobe ltt-marker-control
 256 modprobe ltt-tracer
 257 modprobe ltt-serialize
 258 modprobe ltt-relay
 259 modprobe ipc-trace
 260 modprobe kernel-trace
 261 modprobe mm-trace
 262 modprobe net-trace
 263 modprobe fs-trace
 264 modprobe jbd2-trace
 265 modprobe ext4-trace
 266 modprobe syscall-trace
 267 modprobe trap-trace
 268 #if locking tracing is wanted, uncomment the following
 269 #modprobe lockdep-trace
 270
 271 If you want to have complete information about the kernel state (including all
 272 the process names), you need to load the ltt-statedump module. This is done by
 273 issuing the command :
 274
 275 modprobe ltt-statedump
 276
 277 You can automate at boot time loading the ltt-control module by :
 278
 279 cp /etc/modules /etc/modules.bkp
 280 echo ltt-trace-control >> /etc/modules
 281 echo ltt-marker-control >> /etc/modules
 282 echo ltt-tracer >> /etc/modules
 283 echo ltt-serialize >> /etc/modules
 284 echo ltt-relay >> /etc/modules
 285 echo ipc-trace >> /etc/modules
 286 echo kernel-trace >> /etc/modules
 287 echo mm-trace >> /etc/modules
 288 echo net-trace >> /etc/modules
 289 echo fs-trace >> /etc/modules
 290 echo jbd2-trace >> /etc/modules
 291 echo ext4-trace >> /etc/modules
 292 echo syscall-trace >> /etc/modules
 293 echo trap-trace >> /etc/modules
 294 #if locking tracing is wanted, uncomment the following
 295 #echo lockdep-trace >> /etc/modules
 296
 297
 298 * Getting and installing the ltt-control package (on the traced machine)
 299 (note : the ltt-control package contains lttd and lttctl. Although it has the
 300 same name as the ltt-control kernel module, they are *not* the same thing.)
 301 su -
 302 cd /usr/src
 303 wget http://ltt.polymtl.ca/lttng/ltt-control-0.x-xxxx2006.tar.gz
 304 gzip -cd ltt-control-0.x-xxxx2008.tar.gz | tar xvof -
 305 cd ltt-control-0.x-xxxx2006
 306 (refer to README to see the development libraries that must be installed on you
 307 system)
 308 ./configure
 309 make
 310 make install
 311
 312
 313 * Userspace tracing
 314
 315 Make sure you selected the kernel menuconfig option :
 316     <M> or <*> Support logging events from userspace
 317 And that the ltt-userspace-event kernel module is loaded if selected as a
 318 module.
 319
 320 Simple userspace tracing is available through
 321 echo "some text to record" > /mnt/debugfs/ltt/write_event
 322
 323 It will appear in the trace under event :
 324 channel : userspace
 325 event name : event
 326
 327
 328 * Getting and installing the LTTV package (on the visualisation machine, same or
 329   different from the visualisation machine)
 330
 331 su -
 332 cd /usr/src
 333 wget http://ltt.polymtl.ca/packages/lttv-0.x.xx-xxxx2008.tar.gz
 334 gzip -cd lttv-0.x.xx-xxxx2008.tar.gz | tar xvof -
 335 cd lttv-0.x.xx-xxxx2008
 336 (refer to README to see the development libraries that must be installed on your
 337 system)
 338 ./configure
 339 make
 340 make install
 341
 342
 343 * Getting and installing the markers-userspace package for user space tracing
 344 (experimental)
 345 See http://ltt.polymtl.ca/packages/markers-userspace-0.5.tar.bz2 or more recent.
 346
 347
 348
 349 ***********************************************************
 350 ** Section 3 * Using LTTng and LTTV                      **
 351 ***********************************************************
 352
 353 * IMPORTANT : Arm Linux Kernel Markers after each boot
 354
 355 ltt-armall
 356
 357 * Use graphical LTTV to control tracing and analyse traces
 358
 359 lttv-gui (or /usr/local/bin/lttv-gui)
 360   - Spot the "Tracing Control" icon : click on it
 361       (it's a traffic light icon)
 362     - enter the root password
 363     - click "start"
 364     - click "stop"
 365     - Yes
 366       * You should now see a trace
 367
 368 * Use text mode LTTng to control tracing
 369
 370 The tracing can be controlled from a terminal by using the lttctl command (as
 371 root).
 372
 373 Start tracing :
 374
 375 lttctl -C -w /tmp/trace1 trace1
 376
 377 Stop tracing and destroy trace channels :
 378
 379 lttctl -D trace1
 380
 381 see lttctl --help for details.
 382
 383 (note : to see if the buffers has been filled, look at the dmesg output after
 384 lttctl -R or after stopping tracing from the GUI, it will show an event lost
 385 count. If it is the case, try using larger buffers. See lttctl --help to learn
 386 how. lttv now also shows event lost messages in the console when loading a trace
 387 with missing events or lost subbuffers.)
 388
 389 * Use text mode LTTV
 390
 391 Feel free to look in /usr/local/lib/lttv/plugins to see all the text and
 392 graphical plugins available.
 393
 394 For example, a simple trace dump in text format is available with :
 395
 396 lttv -m textDump -t /tmp/trace
 397
 398 see lttv -m textDump --help for detailed command line options of textDump.
 399
 400 It is, in the current state of the project, very useful to use "grep" on the
 401 text output to filter by specific event fields. You can later copy the timestamp
 402 of the events to the clipboard and paste them in the GUI by clicking on the
 403 bottom right label "Current time". Support for this type of filtering should
 404 be added to the filter module soon.
 405
 406 * Hybrid mode
 407
 408 Starting from LTTng 0.5.105 and ltt-control 0.20, a new mode can be used :
 409 hybrid. It can be especially useful when studying big workloads on a long period
 410 of time.
 411
 412 When using this mode, the most important, low rate control information will be
 413 recorded during all the trace by lttd (i.e. process creation/exit). The high
 414 rate information (i.e. interrupt/traps/syscall entry/exit) will be kept in a
 415 flight recorder buffer (now named flight-channelname_X).
 416
 417 The following lttctl commands take an hybrid trace :
 418
 419 Create trace channel, start lttd on normal channels, start tracing:
 420 lttctl -C -w /tmp/trace2 -o channel.kernel.overwrite=1 trace2
 421
 422 Stop tracing, start lttd on flight recorder channels, destroy trace channels :
 423 lttctl -D -w /tmp/trace2 trace2
 424
 425 Each "overwrite" channel is flight recorder channel.
 426
 427 * Flight recorder mode
 428
 429 The flight recorder mode writes data into overwritten buffers for all channels,
 430 including control channels, except for the facilities tracefiles. It consists of
 431 setting all channels to "overwrite".
 432
 433 The following lttctl commands take a flight recorder trace :
 434
 435 lttctl -C -w /tmp/trace3 -o channel.all.overwrite=1 trace3
 436 ...
 437 lttctl -D -w /tmp/trace3 trace3
 438
 439
 440 **************************************************************
 441 ** Section 4 * Adding new instrumentations with the markers **
 442 **************************************************************
 443
 444 See Documentation/markers.txt and Documentation/tracepoints.txt in your kernel
 445 tree.
 446
 447 * Add new events to userspace programs with userspace markers
 448 http://ltt.polymtl.ca/packages/
 449
 450 Get the latest markers-userspace-*.tar.bz2 and see the Makefile and examples. It
 451 allows inserting markers in executables and libraries, currently only on x86_32
 452 and x86_64.
 453
 454   </body>
 455 </html>