The LTTng Documentation
=======================
Philippe Proulx <pproulx@efficios.com>
-v2.8, 25 October 2016
+v2.8, 8 December 2016
include::../common/copyright.txt[]
[[whats-new]]
== What's new in LTTng {revision}?
+LTTng{nbsp}{revision} bears the name _Isseki Nicho_. The result of a
+collaboration between http://www.dieuduciel.com/[Dieu du Ciel!] and
+Nagano-based Shiga Kogen,
+https://www.beeradvocate.com/beer/profile/1141/53111/[_**Isseki
+Nicho**_] is a strong Imperial Dark Saison offering a rich roasted malt
+flavor combined with a complex fruity finish typical of Saison yeasts.
+
+New features and changes in LTTng{nbsp}{revision}:
+
* **Tracing control**:
** You can attach <<java-application-context,Java application-specific
context fields>> to a <<channel,channel>> with the
+
See man:lttng-status(1).
-** New `lttng metadata regenerate` command to regenerate the metadata
- file of an LTTng trace at any moment. This command is meant to be
- used to resample the wall time following a major
+** New `lttng metadata regenerate` command to
+ <<metadata-regenerate,regenerate the metadata file of an LTTng
+ trace>> at any moment. This command is meant to be used to resample
+ the wall time following a major
https://en.wikipedia.org/wiki/Network_Time_Protocol[NTP] correction
so that a system which boots with an incorrect wall time can be
traced before its wall time is NTP-corrected.
applications.
[role="growable"]
-.Availability of LTTng{nbsp}{revision} for major Linux distributions.
+.Availability of LTTng{nbsp}{revision} for major Linux distributions as of 2 December 2016.
|====
|Distribution |Available in releases |Alternatives
-|Ubuntu
-|<<ubuntu,Ubuntu{nbsp}16.10 _Yakkety Yak_>>
+|https://www.ubuntu.com/[Ubuntu]
+|<<ubuntu,Ubuntu{nbsp}16.10 _Yakkety Yak_>>.
|LTTng{nbsp}{revision} for Ubuntu{nbsp}14.04 _Trusty Tahr_
and Ubuntu{nbsp}16.04 _Xenial Xerus_:
<<ubuntu-ppa,use the LTTng Stable{nbsp}{revision} PPA>>.
+LTTng{nbsp}2.9 for Ubuntu{nbsp}14.04 _Trusty Tahr_
+and Ubuntu{nbsp}16.04 _Xenial Xerus_:
+link:/docs/v2.9#doc-ubuntu-ppa[use the LTTng Stable{nbsp}2.9 PPA].
+
<<building-from-source,Build LTTng{nbsp}{revision} from source>> for
other Ubuntu releases.
-|Fedora
-|_Not available_
-|LTTng{nbsp}{revision} for Fedora{nbsp}25 and Fedora{nbsp}26 (not
-released yet).
-
-<<building-from-source,Build LTTng{nbsp}{revision} from source>> for
+|https://getfedora.org/[Fedora]
+|<<fedora,Fedora{nbsp}25>>.
+|<<building-from-source,Build LTTng{nbsp}{revision} from source>> for
other Fedora releases.
-|Debian
-|<<debian,Debian "stretch" (testing)>>
-|
-<<building-from-source,Build LTTng{nbsp}{revision} from source>> for
+|https://www.debian.org/[Debian]
+|<<debian,Debian "stretch" (testing)>>.
+|<<building-from-source,Build LTTng{nbsp}{revision} from source>> for
previous Debian releases.
-|openSUSE
+|https://www.opensuse.org/[openSUSE]
|_Not available_
-|LTTng{nbsp}2.7 for openSUSE Leap{nbsp}42.1.
+|<<building-from-source,Build LTTng{nbsp}{revision} from source>>.
-<<building-from-source,Build LTTng{nbsp}{revision} from source>> for
-other openSUSE releases.
-
-|Arch Linux
-|Latest AUR packages.
-|
+|https://www.archlinux.org/[Arch Linux]
+|_Not available_
+|link:/docs/v2.9#doc-arch-linux[LTTng{nbsp}2.9 from the AUR].
-|Alpine Linux
-|<<alpine-linux,Alpine Linux "edge">>
+|https://alpinelinux.org/[Alpine Linux]
+|<<alpine-linux,Alpine Linux "edge">>.
|LTTng{nbsp}{revision} for Alpine Linux{nbsp}3.5 (not released yet).
<<building-from-source,Build LTTng{nbsp}{revision} from source>> for
other Alpine Linux releases.
-|RHEL and SLES
+|https://www.redhat.com/[RHEL] and https://www.suse.com/[SLES]
|See http://packages.efficios.com/[EfficiOS Enterprise Packages].
|
-|Buildroot
-|_Not available_
-|LTTng{nbsp}{revision} for Buildroot{nbsp}2016.11 (not released yet).
-
-LTTng{nbsp}2.7 for Buildroot{nbsp}2016.02, Buildroot{nbsp}2016.05,
-and Buildroot{nbsp}2016.08.
-
-<<building-from-source,Build LTTng{nbsp}{revision} from source>> for
+|https://buildroot.org/[Buildroot]
+|<<buildroot,Buildroot 2016.11>>.
+|<<building-from-source,Build LTTng{nbsp}{revision} from source>> for
other Buildroot releases.
-|OpenEmbedded and Yocto
-|<<oe-yocto,Yocto Project{nbsp}2.2 _Morty_>> (`openembedded-core` layer)
+|http://www.openembedded.org/wiki/Main_Page[OpenEmbedded] and
+https://www.yoctoproject.org/[Yocto]
+|<<oe-yocto,Yocto Project{nbsp}2.2 _Morty_>> (`openembedded-core` layer).
|<<building-from-source,Build LTTng{nbsp}{revision} from source>> for
other OpenEmbedded releases.
|====
For previous releases of Ubuntu, <<ubuntu-ppa,use the LTTng
Stable{nbsp}{revision} PPA>>.
-To install LTTng{nbsp}{revision} on Ubuntu 16.10{nbsp}_Yakkety Yak_:
+To install LTTng{nbsp}{revision} on Ubuntu{nbsp}16.10 _Yakkety Yak_:
. Install the main LTTng{nbsp}{revision} packages:
+
--
. **If you need to instrument and trace
- <<python-application,Python applications>>**, install the
+ <<python-application,Python{nbsp}3 applications>>**, install the
LTTng-UST Python agent:
+
--
--
. **If you need to instrument and trace
- <<python-application,Python applications>>**, install the
+ <<python-application,Python{nbsp}3 applications>>**, install the
LTTng-UST Python agent:
+
--
--
+[[fedora]]
+=== Fedora
+
+To install LTTng{nbsp}{revision} on Fedora{nbsp}25:
+
+. Install the LTTng-tools{nbsp}{revision} and LTTng-UST{nbsp}{revision}
+ packages:
++
+--
+[role="term"]
+----
+sudo yum install lttng-tools
+sudo yum install lttng-ust
+----
+--
+
+. Download, build, and install the latest LTTng-modules{nbsp}{revision}:
++
+--
+[role="term"]
+----
+cd $(mktemp -d) &&
+wget http://lttng.org/files/lttng-modules/lttng-modules-latest-2.8.tar.bz2 &&
+tar -xf lttng-modules-latest-2.8.tar.bz2 &&
+cd lttng-modules-2.8.* &&
+make &&
+sudo make modules_install &&
+sudo depmod -a
+----
+--
+
+[IMPORTANT]
+.Java and Python application instrumentation and tracing
+====
+If you need to instrument and trace <<java-application,Java
+applications>> on openSUSE, you need to build and install
+LTTng-UST{nbsp}{revision} <<building-from-source,from source>> and pass
+the `--enable-java-agent-jul`, `--enable-java-agent-log4j`, or
+`--enable-java-agent-all` options to the `configure` script, depending
+on which Java logging framework you use.
+
+If you need to instrument and trace <<python-application,Python
+applications>> on openSUSE, you need to build and install
+LTTng-UST{nbsp}{revision} from source and pass the
+`--enable-python-agent` option to the `configure` script.
+====
+
+
[[debian]]
=== Debian
[[alpine-linux]]
=== Alpine Linux
-To install LTTng{nbsp}{revision} (tracing control and user space
-tracing) on Alpine Linux "edge":
+To install LTTng-tools{nbsp}{revision} and LTTng-UST{nbsp}{revision} on
+Alpine Linux "edge":
. Make sure your system is
https://wiki.alpinelinux.org/wiki/Edge[configured for "edge"].
--
+[[enterprise-distributions]]
+=== RHEL, SUSE, and other enterprise distributions
+
+To install LTTng on enterprise Linux distributions, such as Red Hat
+Enterprise Linux (RHEL) and SUSE Linux Enterprise Server (SUSE), please
+see http://packages.efficios.com/[EfficiOS Enterprise Packages].
+
+
+[[buildroot]]
+=== Buildroot
+
+To install LTTng{nbsp}{revision} on Buildroot{nbsp}2016.11:
+
+. Launch the Buildroot configuration tool:
++
+--
+[role="term"]
+----
+make menuconfig
+----
+--
+
+. In **Kernel**, check **Linux kernel**.
+. In **Toolchain**, check **Enable WCHAR support**.
+. In **Target packages**{nbsp}→ **Debugging, profiling and benchmark**,
+ check **lttng-modules** and **lttng-tools**.
+. In **Target packages**{nbsp}→ **Libraries**{nbsp}→
+ **Other**, check **lttng-libust**.
+
+
[[oe-yocto]]
=== OpenEmbedded and Yocto
====
-[[enterprise-distributions]]
-=== RHEL, SUSE, and other enterprise distributions
-
-To install LTTng on enterprise Linux distributions, such as Red Hat
-Enterprise Linux (RHEL) and SUSE Linux Enterprise Server (SUSE), please
-see http://packages.efficios.com/[EfficiOS Enterprise Packages].
-
-
[[building-from-source]]
=== Build from source
The following command lines start with cmd:sudo because you need root
privileges to trace the Linux kernel. You can avoid using cmd:sudo if
-your Unix user is a member of the <<lttng-sessiond,tracing group>>.
+your Unix user is a member of the <<tracing-group,tracing group>>.
-. Create a <<tracing-session,tracing session>>:
+. Create a <<tracing-session,tracing session>> which writes its traces
+ to dir:{/tmp/my-kernel-trace}:
+
--
[role="term"]
----
-sudo lttng create my-kernel-session
+sudo lttng create my-kernel-session --output=/tmp/my-kernel-trace
----
--
[role="term"]
----
lttng list --kernel
+lttng list --kernel --syscall
----
--
-. Create an <<event,event rule>> which matches the desired event names,
- for example `sched_switch` and `sched_process_fork`:
+. Create <<event,event rules>> which match the desired instrumentation
+ point names, for example the `sched_switch` and `sched_process_fork`
+ tracepoints, and the man:open(2) and man:close(2) system calls:
+
--
[role="term"]
----
sudo lttng enable-event --kernel sched_switch,sched_process_fork
+sudo lttng enable-event --kernel --syscall open,close
----
--
+
-You can also create an event rule which _matches_ all the Linux kernel
+You can also create an event rule which matches _all_ the Linux kernel
tracepoints (this will generate a lot of data when tracing):
+
--
----
--
-. Start tracing:
+. <<basic-tracing-session-control,Start tracing>>:
+
--
[role="term"]
. Do some operation on your system for a few seconds. For example,
load a website, or list the files of a directory.
-. Stop tracing and destroy the tracing session:
+. <<basic-tracing-session-control,Stop tracing>> and destroy the
+ tracing session:
+
--
[role="term"]
The man:lttng-destroy(1) command does not destroy the trace data; it
only destroys the state of the tracing session.
-By default, LTTng saves the traces in
-+$LTTNG_HOME/lttng-traces/__name__-__date__-__time__+,
-where +__name__+ is the tracing session name. Note that the
-env:LTTNG_HOME environment variable defaults to `$HOME` if not set.
+. For the sake of this example, make the recorded trace accessible to
+ the non-root users:
++
+--
+[role="term"]
+----
+sudo chown -R $(whoami) /tmp/my-kernel-trace
+----
+--
See <<viewing-and-analyzing-your-traces,View and analyze the
recorded events>> to view the recorded events.
----
--
-. Start tracing:
+. <<basic-tracing-session-control,Start tracing>>:
+
--
[role="term"]
. Go back to the running `hello` application and press Enter. The
program executes all `tracepoint()` instrumentation points and exits.
-. Stop tracing and destroy the tracing session:
+. <<basic-tracing-session-control,Stop tracing>> and destroy the
+ tracing session:
+
--
[role="term"]
By default, LTTng saves the traces in
+$LTTNG_HOME/lttng-traces/__name__-__date__-__time__+,
-where +__name__+ is the tracing session name. Note that the
+where +__name__+ is the tracing session name. The
env:LTTNG_HOME environment variable defaults to `$HOME` if not set.
See <<viewing-and-analyzing-your-traces,View and analyze the
NOTE: This section assumes that the traces recorded during the previous
tutorials were saved to their default location, in the
-dir:{$LTTNG_HOME/lttng-traces} directory. Note that the env:LTTNG_HOME
+dir:{$LTTNG_HOME/lttng-traces} directory. The env:LTTNG_HOME
environment variable defaults to `$HOME` if not set.
[role="term"]
----
-babeltrace ~/lttng-traces/my-kernel-session* | grep sys_
+babeltrace /tmp/my-kernel-trace | grep _switch
----
You can pipe the output of cmd:babeltrace into a tool like man:wc(1) to
[role="term"]
----
-babeltrace ~/lttng-traces/my-kernel-session* | grep sys_read | wc --lines
+babeltrace /tmp/my-kernel-trace | grep _open | wc --lines
----
[role="term"]
----
-python3 top5proc.py ~/lttng-traces/my-kernel-session*/kernel
+python3 top5proc.py /tmp/my-kernel-trace/kernel
----
Output example:
Once you <<tpp-header,create a tracepoint provider header file>>, you
can use the `tracepoint()` macro in your application's
source code to insert the tracepoints that this header
-<<defining-tracepoints,defined>> defines.
+<<defining-tracepoints,defines>>.
The `tracepoint()` macro takes at least two parameters: the tracepoint
provider name and the tracepoint name. The corresponding tracepoint
`libemon.so`::
User library shared object file.
-The red star indicates that this object file is instrumented
-(contains code which uses the `tracepoint()` macro). The spring
-symbol between the application and a library means the application is
-linked with the library at build time.
+We use the following symbols in the diagrams of table below:
+
+[role="img-100"]
+.Symbols used in the build scenario diagrams.
+image::ust-sit-symbols.png[]
We assume that path:{.} is part of the env:LD_LIBRARY_PATH environment
variable in the following instructions.
To build the instrumented user library:
. In path:{emon.c}, before including path:{tpp.h}, add the
- following line:
+ following lines:
+
--
[source,c]
To build the instrumented user library:
. In path:{emon.c}, before including path:{tpp.h}, add the
- following line:
+ following lines:
+
--
[source,c]
To build the instrumented user library:
. In path:{emon.c}, before including path:{tpp.h}, add the
- following line:
+ following lines:
+
--
[source,c]
To build the instrumented user library:
. In path:{emon.c}, before including path:{tpp.h}, add the
- following line:
+ following lines:
+
--
[source,c]
tar -xf lttng-tools-latest-2.8.tar.bz2 &&
cd lttng-tools-2.8.* &&
./configure --libdir=/usr/local/lib32 CFLAGS=-m32 CXXFLAGS=-m32 \
- LDFLAGS='-L/usr/local/lib32 -L/usr/lib32' &&
+ LDFLAGS='-L/usr/local/lib32 -L/usr/lib32' \
+ --disable-bin-lttng --disable-bin-lttng-crash \
+ --disable-bin-lttng-relayd --disable-bin-lttng-sessiond &&
make &&
cd src/bin/lttng-consumerd &&
sudo make install &&
entry is enough to create a call graph, since an event record always
contains the ID of the CPU that generated it.
+
-You can use a tool like
-https://sourceware.org/binutils/docs/binutils/addr2line.html[cmd:addr2line]
-to convert function addresses back to source file names and
-line numbers.
+You can use a tool like man:addr2line(1) to convert function addresses
+back to source file names and line numbers.
* **path:{liblttng-ust-cyg-profile.so}** is a more robust variant
which also works in use cases where event records might get discarded or
----
====
+In the resulting trace, an <<event,event record>> generated by a Java
+application using `java.util.logging` is named `lttng_jul:event` and
+has the following fields:
+
+`msg`::
+ Log record's message.
+
+`logger_name`::
+ Logger name.
+
+`class_name`::
+ Name of the class in which the log statement was executed.
+
+`method_name`::
+ Name of the method in which the log statement was executed.
+
+`long_millis`::
+ Logging time (timestamp in milliseconds).
+
+`int_loglevel`::
+ Log level integer value.
+
+`int_threadid`::
+ ID of the thread in which the log statement was executed.
+
You can use the opt:lttng-enable-event(1):--loglevel or
opt:lttng-enable-event(1):--loglevel-only option of the
man:lttng-enable-event(1) command to target a range of JUL log levels
----
====
+In the resulting trace, an <<event,event record>> generated by a Java
+application using log4j is named `lttng_log4j:event` and
+has the following fields:
+
+`msg`::
+ Log record's message.
+
+`logger_name`::
+ Logger name.
+
+`class_name`::
+ Name of the class in which the log statement was executed.
+
+`method_name`::
+ Name of the method in which the log statement was executed.
+
+`filename`::
+ Name of the file in which the executed log statement is located.
+
+`line_number`::
+ Line number at which the log statement was executed.
+
+`timestamp`::
+ Logging timestamp.
+
+`int_loglevel`::
+ Log level integer value.
+
+`thread_name`::
+ Name of the Java thread in which the log statement was executed.
+
You can use the opt:lttng-enable-event(1):--loglevel or
opt:lttng-enable-event(1):--loglevel-only option of the
man:lttng-enable-event(1) command to target a range of Apache log4j log levels
----
====
+In the resulting trace, an <<event,event record>> generated by a Python
+application is named `lttng_python:event` and has the following fields:
+
+`asctime`::
+ Logging time (string).
+
+`msg`::
+ Log record's message.
+
+`logger_name`::
+ Logger name.
+
+`funcName`::
+ Name of the function in which the log statement was executed.
+
+`lineno`::
+ Line number at which the log statement was executed.
+
+`int_loglevel`::
+ Log level integer value.
+
+`thread`::
+ ID of the Python thread in which the log statement was executed.
+
+`threadName`::
+ Name of the Python thread in which the log statement was executed.
+
You can use the opt:lttng-enable-event(1):--loglevel or
opt:lttng-enable-event(1):--loglevel-only option of the
man:lttng-enable-event(1) command to target a range of Python log levels
belongs to the Linux kernel <<domain,tracing domain>>. However, unlike
other instrumentation points in the kernel tracing domain, **any Unix
user** can <<enabling-disabling-events,create an event rule>> which
-matches its event name, not only the root user or users in the tracing
-group.
+matches its event name, not only the root user or users in the
+<<tracing-group,tracing group>>.
To use the LTTng logger:
<<domain,tracing domain>>::
A namespace for event sources.
-tracing group::
+<<tracing-group,tracing group>>::
The Unix group in which a Unix user can be to be allowed to trace the
Linux kernel.