The LTTng Documentation
=======================
Philippe Proulx <pproulx@efficios.com>
-v2.9, 6 December 2016
+v2.9, 22 January 2018
include::../common/copyright.txt[]
--
[role="term"]
----
-lttng add-context --kernel --type=perf:cpu:raw:r0013c:x86unhalted
+$ lttng add-context --kernel --type=perf:cpu:raw:r0013c:x86unhalted
----
--
+
applications.
[role="growable"]
-.Availability of LTTng{nbsp}{revision} for major Linux distributions as of 2 December 2016.
+.Availability of LTTng{nbsp}{revision} for major Linux distributions as of 22 January 2018.
|====
|Distribution |Available in releases |Alternatives
|https://www.ubuntu.com/[Ubuntu]
-|Ubuntu{nbsp}14.04 _Trusty Tahr_ and Ubuntu{nbsp}16.04 _Xenial Xerus_:
-<<ubuntu-ppa,use the LTTng Stable{nbsp}{revision} PPA>>.
-|link:/docs/v2.8#doc-ubuntu[LTTng{nbsp}2.8 for Ubuntu{nbsp}16.10 _Yakkety Yak_].
+|<<ubuntu,Ubuntu{nbsp}17.04 _Zesty Zapus_ and Ubuntu{nbsp}17.10 _Artful Aardvark_>>.
-<<building-from-source,Build LTTng{nbsp}{revision} from source>> for
+Ubuntu{nbsp}14.04 _Trusty Tahr_ and Ubuntu{nbsp}16.04 _Xenial Xerus_:
+<<ubuntu-ppa,use the LTTng Stable{nbsp}{revision} PPA>>.
+|<<building-from-source,Build LTTng{nbsp}{revision} from source>> for
other Ubuntu releases.
|https://getfedora.org/[Fedora]
-|_Not available_
-|<<building-from-source,Build LTTng-modules{nbsp}{revision} from
-source>>.
+|<<fedora,Fedora{nbsp}26>>.
+|link:/docs/v2.10#doc-fedora[LTTng{nbsp}2.10 for Fedora{nbsp}27].
+
+<<building-from-source,Build LTTng{nbsp}{revision} from source>> for
+other Fedora releases.
|https://www.debian.org/[Debian]
-|_Not available_
-|<<building-from-source,Build LTTng-modules{nbsp}{revision} from
-source>>.
+|<<debian,Debian "stretch" (stable)>>.
+|link:/docs/v2.10#doc-debian[LTTng{nbsp}2.10 for Debian "buster" (testing)
+and Debian "sid" (unstable)].
-|https://www.opensuse.org/[openSUSE]
-|_Not available_
-|<<building-from-source,Build LTTng-modules{nbsp}{revision} from
-source>>.
+
+<<building-from-source,Build LTTng{nbsp}{revision} from source>> for
+other Debian releases.
|https://www.archlinux.org/[Arch Linux]
-|<<arch-linux,Latest AUR packages>>.
-|
+|_Not available_
+|link:/docs/v2.10#doc-arch-linux[LTTng{nbsp}2.10 for the current Arch Linux build].
+
+<<building-from-source,Build LTTng{nbsp}{revision} from source>>.
|https://alpinelinux.org/[Alpine Linux]
|_Not available_
-|link:/docs/v2.8#doc-alpine-linux[LTTng{nbsp}2.8 for Alpine Linux{nbsp}"edge"].
+|link:/docs/v2.10#doc-alpine-linux[LTTng{nbsp}2.10 for Alpine Linux{nbsp}3.7
+and Alpine Linux{nbsp}"edge"].
-LTTng{nbsp}2.8 for Alpine Linux{nbsp}3.5 (not released yet).
-
-<<building-from-source,Build LTTng{nbsp}{revision} from source>> for
-other Alpine Linux releases.
+<<building-from-source,Build LTTng{nbsp}{revision} from source>>.
|https://www.redhat.com/[RHEL] and https://www.suse.com/[SLES]
|See http://packages.efficios.com/[EfficiOS Enterprise Packages].
|
|https://buildroot.org/[Buildroot]
-|_Not available_
-|link:/docs/v2.8#doc-buildroot[LTTng{nbsp}2.8 for Buildroot{nbsp}2016.11].
-
-<<building-from-source,Build LTTng{nbsp}{revision} from source>> for
+|<<"buildroot", "Buildroot{nbsp}2017.02, Buildroot{nbsp}2017.05, Buildroot{nbsp}2017.08, and Buildroot{nbsp}2017.11">>.
+|<<building-from-source,Build LTTng{nbsp}{revision} from source>> for
other Buildroot releases.
|http://www.openembedded.org/wiki/Main_Page[OpenEmbedded] and
https://www.yoctoproject.org/[Yocto]
-|_Not available_
-|link:/docs/v2.8#doc-oe-yocto[LTTng{nbsp}2.8 for Yocto Project{nbsp}2.2 _Morty_]
+|<<oe-yocto,Yocto Project{nbsp}2.3 _Pyro_ and Yocto Project{nbsp}2.4 _Rocko_>>
(`openembedded-core` layer).
-
-<<building-from-source,Build LTTng{nbsp}{revision} from source>> for
-other OpenEmbedded releases.
+|<<building-from-source,Build LTTng{nbsp}{revision} from source>> for
+other Yocto/OpenEmbedded releases.
|====
[[ubuntu]]
=== [[ubuntu-official-repositories]]Ubuntu
+LTTng{nbsp}{revision} is available on Ubuntu{nbsp}17.04 _Zesty Zapus_
+and Ubuntu{nbsp}17.10 _Artful Aardvark_. For previous releases of
+Ubuntu, <<ubuntu-ppa,use the LTTng Stable{nbsp}{revision} PPA>>.
+
+To install LTTng{nbsp}{revision} on Ubuntu{nbsp}17.04 _Zesty Zapus_:
+
+. Install the main LTTng{nbsp}{revision} packages:
++
+--
+[role="term"]
+----
+# apt-get install lttng-tools
+# apt-get install lttng-modules-dkms
+# apt-get install liblttng-ust-dev
+----
+--
+
+. **If you need to instrument and trace
+ <<java-application,Java applications>>**, install the LTTng-UST
+ Java agent:
++
+--
+[role="term"]
+----
+# apt-get install liblttng-ust-agent-java
+----
+--
+
+. **If you need to instrument and trace
+ <<python-application,Python{nbsp}3 applications>>**, install the
+ LTTng-UST Python agent:
++
+--
+[role="term"]
+----
+# apt-get install python3-lttngust
+----
+--
+
+
[[ubuntu-ppa]]
==== noch:{LTTng} Stable {revision} PPA
--
[role="term"]
----
-sudo apt-add-repository ppa:lttng/stable-2.9
-sudo apt-get update
+# apt-add-repository ppa:lttng/stable-2.9
+# apt-get update
----
--
--
[role="term"]
----
-sudo apt-get install lttng-tools
-sudo apt-get install lttng-modules-dkms
-sudo apt-get install liblttng-ust-dev
+# apt-get install lttng-tools
+# apt-get install lttng-modules-dkms
+# apt-get install liblttng-ust-dev
----
--
--
[role="term"]
----
-sudo apt-get install liblttng-ust-agent-java
+# apt-get install liblttng-ust-agent-java
----
--
--
[role="term"]
----
-sudo apt-get install python3-lttngust
+# apt-get install python3-lttngust
+----
+--
+
+
+[[fedora]]
+=== Fedora
+
+To install LTTng{nbsp}{revision} on Fedora{nbsp}26:
+
+. Install the LTTng-tools{nbsp}{revision} and LTTng-UST{nbsp}{revision}
+ packages:
++
+--
+[role="term"]
+----
+# yum install lttng-tools
+# yum install lttng-ust
+----
+--
+
+. Download, build, and install the latest LTTng-modules{nbsp}{revision}:
++
+--
+[role="term"]
+----
+$ cd $(mktemp -d) &&
+wget http://lttng.org/files/lttng-modules/lttng-modules-latest-2.9.tar.bz2 &&
+tar -xf lttng-modules-latest-2.9.tar.bz2 &&
+cd lttng-modules-2.9.* &&
+make &&
+sudo make modules_install &&
+sudo depmod -a
----
--
+[IMPORTANT]
+.Java and Python application instrumentation and tracing
+====
+If you need to instrument and trace <<java-application,Java
+applications>> on Fedora, you need to build and install
+LTTng-UST{nbsp}{revision} <<building-from-source,from source>> and pass
+the `--enable-java-agent-jul`, `--enable-java-agent-log4j`, or
+`--enable-java-agent-all` options to the `configure` script, depending
+on which Java logging framework you use.
+
+If you need to instrument and trace <<python-application,Python
+applications>> on Fedora, you need to build and install
+LTTng-UST{nbsp}{revision} from source and pass the
+`--enable-python-agent` option to the `configure` script.
+====
-[[arch-linux]]
-=== Arch Linux
-To install LTTng{nbsp}{revision} on Arch Linux using
-https://archlinux.fr/yaourt-en[Yaourt]:
+[[debian]]
+=== Debian
+
+To install LTTng{nbsp}{revision} on Debian "stretch" (stable):
. Install the main LTTng{nbsp}{revision} packages:
+
--
[role="term"]
----
-yaourt -S lttng-tools
-yaourt -S lttng-ust
-yaourt -S lttng-modules
+# apt-get install lttng-modules-dkms
+# apt-get install liblttng-ust-dev
+# apt-get install lttng-tools
+----
+--
+
+. **If you need to instrument and trace <<java-application,Java
+ applications>>**, install the LTTng-UST Java agent:
++
+--
+[role="term"]
+----
+# apt-get install liblttng-ust-agent-java
----
--
--
[role="term"]
----
-yaourt -S python-lttngust
-yaourt -S python2-lttngust
+# apt-get install python3-lttngust
----
--
see http://packages.efficios.com/[EfficiOS Enterprise Packages].
+[[buildroot]]
+=== Buildroot
+
+To install LTTng{nbsp}{revision} on Buildroot{nbsp}2017.02,
+Buildroot{nbsp}2017.05, Buildroot{nbsp}2017.08, or
+Buildroot{nbsp}2017.11:
+
+. Launch the Buildroot configuration tool:
++
+--
+[role="term"]
+----
+$ make menuconfig
+----
+--
+
+. In **Kernel**, check **Linux kernel**.
+. In **Toolchain**, check **Enable WCHAR support**.
+. In **Target packages**{nbsp}→ **Debugging, profiling and benchmark**,
+ check **lttng-modules** and **lttng-tools**.
+. In **Target packages**{nbsp}→ **Libraries**{nbsp}→
+ **Other**, check **lttng-libust**.
+
+
+[[oe-yocto]]
+=== OpenEmbedded and Yocto
+
+LTTng{nbsp}{revision} recipes are available in the
+http://layers.openembedded.org/layerindex/branch/master/layer/openembedded-core/[`openembedded-core`]
+layer for Yocto Project{nbsp}2.3 _Pyro_ and Yocto Project{nbsp}2.4 _Rocko_
+under the following names:
+
+* `lttng-tools`
+* `lttng-modules`
+* `lttng-ust`
+
+With BitBake, the simplest way to include LTTng recipes in your target
+image is to add them to `IMAGE_INSTALL_append` in path:{conf/local.conf}:
+
+----
+IMAGE_INSTALL_append = " lttng-tools lttng-modules lttng-ust"
+----
+
+If you use Hob:
+
+. Select a machine and an image recipe.
+. Click **Edit image recipe**.
+. Under the **All recipes** tab, search for **lttng**.
+. Check the desired LTTng recipes.
+
+[IMPORTANT]
+.Java and Python application instrumentation and tracing
+====
+If you need to instrument and trace <<java-application,Java
+applications>> on Yocto/OpenEmbedded, you need to build and install
+LTTng-UST{nbsp}{revision} <<building-from-source,from source>> and pass
+the `--enable-java-agent-jul`, `--enable-java-agent-log4j`, or
+`--enable-java-agent-all` options to the `configure` script, depending
+on which Java logging framework you use.
+
+If you need to instrument and trace <<python-application,Python
+applications>> on Yocto/OpenEmbedded, you need to build and install
+LTTng-UST{nbsp}{revision} from source and pass the
+`--enable-python-agent` option to the `configure` script.
+====
+
+
[[building-from-source]]
=== Build from source
--
[role="term"]
----
-cd $(mktemp -d) &&
+$ cd $(mktemp -d) &&
wget http://lttng.org/files/lttng-modules/lttng-modules-latest-2.9.tar.bz2 &&
tar -xf lttng-modules-latest-2.9.tar.bz2 &&
cd lttng-modules-2.9.* &&
--
[role="term"]
----
-cd $(mktemp -d) &&
+$ cd $(mktemp -d) &&
wget http://lttng.org/files/lttng-ust/lttng-ust-latest-2.9.tar.bz2 &&
tar -xf lttng-ust-latest-2.9.tar.bz2 &&
cd lttng-ust-2.9.* &&
--
[role="term"]
----
-cd $(mktemp -d) &&
+$ cd $(mktemp -d) &&
wget http://lttng.org/files/lttng-tools/lttng-tools-latest-2.9.tar.bz2 &&
tar -xf lttng-tools-latest-2.9.tar.bz2 &&
cd lttng-tools-2.9.* &&
[[tracing-the-linux-kernel]]
=== Trace the Linux kernel
-The following command lines start with cmd:sudo because you need root
-privileges to trace the Linux kernel. You can avoid using cmd:sudo if
-your Unix user is a member of the <<tracing-group,tracing group>>.
+The following command lines start with the `#` prompt because you need
+root privileges to trace the Linux kernel. You can also trace the kernel
+as a regular user if your Unix user is a member of the
+<<tracing-group,tracing group>>.
. Create a <<tracing-session,tracing session>> which writes its traces
to dir:{/tmp/my-kernel-trace}:
--
[role="term"]
----
-sudo lttng create my-kernel-session --output=/tmp/my-kernel-trace
+# lttng create my-kernel-session --output=/tmp/my-kernel-trace
----
--
--
[role="term"]
----
-lttng list --kernel
-lttng list --kernel --syscall
+# lttng list --kernel
+# lttng list --kernel --syscall
----
--
--
[role="term"]
----
-sudo lttng enable-event --kernel sched_switch,sched_process_fork
-sudo lttng enable-event --kernel --syscall open,close
+# lttng enable-event --kernel sched_switch,sched_process_fork
+# lttng enable-event --kernel --syscall open,close
----
--
+
--
[role="term"]
----
-sudo lttng enable-event --kernel --all
+# lttng enable-event --kernel --all
----
--
--
[role="term"]
----
-sudo lttng start
+# lttng start
----
--
--
[role="term"]
----
-sudo lttng stop
-sudo lttng destroy
+# lttng stop
+# lttng destroy
----
--
+
--
[role="term"]
----
-sudo chown -R $(whoami) /tmp/my-kernel-trace
+# chown -R $(whoami) /tmp/my-kernel-trace
----
--
--
[role="term"]
----
-gcc -c -I. hello-tp.c
+$ gcc -c -I. hello-tp.c
----
--
--
[role="term"]
----
-gcc -c hello.c
+$ gcc -c hello.c
----
--
--
[role="term"]
----
-gcc -o hello hello.o hello-tp.o -llttng-ust -ldl
+$ gcc -o hello hello.o hello-tp.o -llttng-ust -ldl
----
--
--
[role="term"]
----
-./hello world and beyond
+$ ./hello world and beyond
----
--
+
--
[role="term"]
----
-lttng-sessiond --daemonize
+$ lttng-sessiond --daemonize
----
--
+
--
[role="term"]
----
-lttng list --userspace
+$ lttng list --userspace
----
--
+
--
[role="term"]
----
-lttng create my-user-space-session
+$ lttng create my-user-space-session
----
--
--
[role="term"]
----
-lttng enable-event --userspace hello_world:my_first_tracepoint
+$ lttng enable-event --userspace hello_world:my_first_tracepoint
----
--
--
[role="term"]
----
-lttng start
+$ lttng start
----
--
--
[role="term"]
----
-sudo lttng stop
-sudo lttng destroy
+$ lttng stop
+$ lttng destroy
----
--
+
[role="term"]
----
-babeltrace ~/lttng-traces/my-user-space-session*
+$ babeltrace ~/lttng-traces/my-user-space-session*
----
cmd:babeltrace finds all traces recursively within the given path and
[role="term"]
----
-babeltrace /tmp/my-kernel-trace | grep _switch
+$ babeltrace /tmp/my-kernel-trace | grep _switch
----
You can pipe the output of cmd:babeltrace into a tool like man:wc(1) to
[role="term"]
----
-babeltrace /tmp/my-kernel-trace | grep _open | wc --lines
+$ babeltrace /tmp/my-kernel-trace | grep _open | wc --lines
----
[role="term"]
----
-python3 top5proc.py /tmp/my-kernel-trace/kernel
+$ python3 top5proc.py /tmp/my-kernel-trace/kernel
----
Output example:
discard mode, the tracer only discards the event record that doesn't
fit.
-In discard mode, LTTng increments a count of lost event records when
-an event record is lost and saves this count to the trace. In
-overwrite mode, LTTng keeps no information when it overwrites a
-sub-buffer before consuming it.
+In discard mode, LTTng increments a count of lost event records when an
+event record is lost and saves this count to the trace. In overwrite
+mode, since LTTng 2.8, LTTng increments a count of lost sub-buffers when
+a sub-buffer is lost and saves this count to the trace. In this mode,
+the exact number of lost event records in those lost sub-buffers is not
+saved to the trace. Trace analyses can use the trace's saved discarded
+event record and sub-buffer counts to decide whether or not to perform
+the analyses even if trace data is known to be missing.
There are a few ways to decrease your probability of losing event
records.
* **High event throughput**: In general, prefer bigger sub-buffers to
lower the risk of losing event records.
+
-Having bigger sub-buffers also ensures a lower sub-buffer switching
-frequency.
+Having bigger sub-buffers also ensures a lower
+<<channel-switch-timer,sub-buffer switching frequency>>.
+
The number of sub-buffers is only meaningful if you create the channel
in overwrite mode: in this case, if a sub-buffer overwrite happens, the
An **event** is the consequence of the execution of an _instrumentation
point_, like a tracepoint that you manually place in some source code,
or a Linux kernel KProbe. An event is said to _occur_ at a specific
-time. Different actions can be taken upon the occurance of an event,
+time. Different actions can be taken upon the occurrence of an event,
like record the event's payload to a buffer.
An **event record** is the representation of an event in a sub-buffer. A
[role="term"]
----
-lttng <general options> <command> <command options>
+$ lttng <GENERAL OPTIONS> <COMMAND> <COMMAND OPTIONS>
----
The <<controlling-tracing,Tracing control>> section explores the
--
[role="term"]
----
-gcc -c app.c
+$ gcc -c app.c
----
--
--
[role="term"]
----
-gcc -o app app.o tpp.o -llttng-ust -ldl
+$ gcc -o app app.o tpp.o -llttng-ust -ldl
----
--
--
[role="term"]
----
-./app
+$ ./app
----
--
--
[role="term"]
----
-gcc -I. -c tpp.c
+$ gcc -I. -c tpp.c
----
--
--
[role="term"]
----
-ar rcs tpp.a tpp.o
+$ ar rcs tpp.a tpp.o
----
--
--
[role="term"]
----
-gcc -c app.c
+$ gcc -c app.c
----
--
--
[role="term"]
----
-gcc -o app app.o tpp.a -llttng-ust -ldl
+$ gcc -o app app.o tpp.a -llttng-ust -ldl
----
--
--
[role="term"]
----
-./app
+$ ./app
----
--
--
[role="term"]
----
-gcc -c app.c
+$ gcc -c app.c
----
--
--
[role="term"]
----
-gcc -o app app.o -ldl -L. -ltpp
+$ gcc -o app app.o -ldl -L. -ltpp
----
--
--
[role="term"]
----
-./app
+$ ./app
----
--
--
[role="term"]
----
-gcc -c app.c
+$ gcc -c app.c
----
--
--
[role="term"]
----
-gcc -o app app.o -ldl
+$ gcc -o app app.o -ldl
----
--
--
[role="term"]
----
-LD_PRELOAD=./libtpp.so ./app
+$ LD_PRELOAD=./libtpp.so ./app
----
--
--
[role="term"]
----
-./app
+$ ./app
----
--
--
[role="term"]
----
-gcc -c app.c
+$ gcc -c app.c
----
--
--
[role="term"]
----
-gcc -o app app.o -ldl
+$ gcc -o app app.o -ldl
----
--
--
[role="term"]
----
-./app
+$ ./app
----
--
--
[role="term"]
----
-gcc -I. -fpic -c emon.c
+$ gcc -I. -fpic -c emon.c
----
--
--
[role="term"]
----
-gcc -shared -o libemon.so emon.o tpp.o -llttng-ust -ldl
+$ gcc -shared -o libemon.so emon.o tpp.o -llttng-ust -ldl
----
--
--
[role="term"]
----
-gcc -c app.c
+$ gcc -c app.c
----
--
--
[role="term"]
----
-gcc -o app app.o -L. -lemon
+$ gcc -o app app.o -L. -lemon
----
--
--
[role="term"]
----
-./app
+$ ./app
----
--
--
[role="term"]
----
-gcc -I. -fpic -c emon.c
+$ gcc -I. -fpic -c emon.c
----
--
--
[role="term"]
----
-gcc -shared -o libemon.so emon.o -ldl -L. -ltpp
+$ gcc -shared -o libemon.so emon.o -ldl -L. -ltpp
----
--
--
[role="term"]
----
-gcc -c app.c
+$ gcc -c app.c
----
--
--
[role="term"]
----
-gcc -o app app.o -L. -lemon
+$ gcc -o app app.o -L. -lemon
----
--
--
[role="term"]
----
-./app
+$ ./app
----
--
--
[role="term"]
----
-gcc -I. -fpic -c emon.c
+$ gcc -I. -fpic -c emon.c
----
--
--
[role="term"]
----
-gcc -shared -o libemon.so emon.o -ldl
+$ gcc -shared -o libemon.so emon.o -ldl
----
--
--
[role="term"]
----
-gcc -c app.c
+$ gcc -c app.c
----
--
--
[role="term"]
----
-gcc -o app app.o -L. -lemon
+$ gcc -o app app.o -L. -lemon
----
--
--
[role="term"]
----
-LD_PRELOAD=./libtpp.so ./app
+$ LD_PRELOAD=./libtpp.so ./app
----
--
--
[role="term"]
----
-./app
+$ ./app
----
--
--
[role="term"]
----
-gcc -I. -fpic -c emon.c
+$ gcc -I. -fpic -c emon.c
----
--
--
[role="term"]
----
-gcc -shared -o libemon.so emon.o -ldl
+$ gcc -shared -o libemon.so emon.o -ldl
----
--
--
[role="term"]
----
-gcc -c app.c
+$ gcc -c app.c
----
--
--
[role="term"]
----
-gcc -o app app.o -L. -lemon
+$ gcc -o app app.o -L. -lemon
----
--
--
[role="term"]
----
-./app
+$ ./app
----
--
--
[role="term"]
----
-gcc -I. -fpic -c emon.c
+$ gcc -I. -fpic -c emon.c
----
--
--
[role="term"]
----
-gcc -shared -o libemon.so emon.o -ldl -L. -ltpp
+$ gcc -shared -o libemon.so emon.o -ldl -L. -ltpp
----
--
--
[role="term"]
----
-gcc -c app.c
+$ gcc -c app.c
----
--
--
[role="term"]
----
-gcc -o app app.o -ldl -L. -lemon
+$ gcc -o app app.o -ldl -L. -lemon
----
--
--
[role="term"]
----
-./app
+$ ./app
----
--
--
[role="term"]
----
-gcc -I. -fpic -c emon.c
+$ gcc -I. -fpic -c emon.c
----
--
--
[role="term"]
----
-gcc -shared -o libemon.so emon.o -ldl
+$ gcc -shared -o libemon.so emon.o -ldl
----
--
--
[role="term"]
----
-gcc -c app.c
+$ gcc -c app.c
----
--
--
[role="term"]
----
-gcc -o app app.o -ldl -L. -lemon
+$ gcc -o app app.o -ldl -L. -lemon
----
--
--
[role="term"]
----
-./app
+$ ./app
----
--
--
[role="term"]
----
-gcc -I. -fpic -c emon.c
+$ gcc -I. -fpic -c emon.c
----
--
--
[role="term"]
----
-gcc -shared -o libemon.so emon.o -ldl
+$ gcc -shared -o libemon.so emon.o -ldl
----
--
--
[role="term"]
----
-gcc -c app.c
+$ gcc -c app.c
----
--
--
[role="term"]
----
-gcc -o app app.o -L. -lemon
+$ gcc -o app app.o -L. -lemon
----
--
--
[role="term"]
----
-LD_PRELOAD=./libtpp.so ./app
+$ LD_PRELOAD=./libtpp.so ./app
----
--
--
[role="term"]
----
-./app
+$ ./app
----
--
--
[role="term"]
----
-gcc -I. -fpic -c emon.c
+$ gcc -I. -fpic -c emon.c
----
--
--
[role="term"]
----
-gcc -shared -o libemon.so emon.o
+$ gcc -shared -o libemon.so emon.o
----
--
--
[role="term"]
----
-gcc -c app.c
+$ gcc -c app.c
----
--
--
[role="term"]
----
-gcc -o app app.o tpp.o -llttng-ust -ldl -L. -lemon
+$ gcc -o app app.o tpp.o -llttng-ust -ldl -L. -lemon
----
--
--
[role="term"]
----
-./app
+$ ./app
----
--
--
[role="term"]
----
-gcc -c app.c
+$ gcc -c app.c
----
--
--
[role="term"]
----
-gcc -Wl,--export-dynamic -o app app.o tpp.o \
+$ gcc -Wl,--export-dynamic -o app app.o tpp.o \
-llttng-ust -ldl
----
--
--
[role="term"]
----
-gcc -I. -fpic -c emon.c
+$ gcc -I. -fpic -c emon.c
----
--
--
[role="term"]
----
-gcc -shared -o libemon.so emon.o
+$ gcc -shared -o libemon.so emon.o
----
--
--
[role="term"]
----
-./app
+$ ./app
----
--
|====
[role="term"]
----
-LD_PRELOAD=liblttng-ust-fork.so ./my-app
+$ LD_PRELOAD=liblttng-ust-fork.so ./my-app
----
If your tracepoint provider package is
[role="term"]
----
-LD_PRELOAD=liblttng-ust-fork.so:/path/to/tp.so ./my-app
+$ LD_PRELOAD=liblttng-ust-fork.so:/path/to/tp.so ./my-app
----
[role="term"]
----
-LD_PRELOAD=liblttng-ust-fd.so ./my-app
+$ LD_PRELOAD=liblttng-ust-fd.so ./my-app
----
Typical use cases include closing all the file descriptors after
[role="term"]
----
-gcc -o my-app my-app.o tp.o $(pkg-config --cflags --libs lttng-ust)
+$ gcc -o my-app my-app.o tp.o $(pkg-config --cflags --libs lttng-ust)
----
--
[role="term"]
----
-cd $(mktemp -d) &&
+$ cd $(mktemp -d) &&
wget http://lttng.org/files/urcu/userspace-rcu-latest-0.9.tar.bz2 &&
tar -xf userspace-rcu-latest-0.9.tar.bz2 &&
cd userspace-rcu-0.9.* &&
--
[role="term"]
----
-cd $(mktemp -d) &&
+$ cd $(mktemp -d) &&
wget http://lttng.org/files/lttng-ust/lttng-ust-latest-2.9.tar.bz2 &&
tar -xf lttng-ust-latest-2.9.tar.bz2 &&
cd lttng-ust-2.9.* &&
[role="term"]
----
-LDFLAGS='-L/usr/lib/i386-linux-gnu -L/usr/lib32'
+$ LDFLAGS='-L/usr/lib/i386-linux-gnu -L/usr/lib32'
----
====
--
[role="term"]
----
-cd $(mktemp -d) &&
+$ cd $(mktemp -d) &&
wget http://lttng.org/files/lttng-tools/lttng-tools-latest-2.9.tar.bz2 &&
tar -xf lttng-tools-latest-2.9.tar.bz2 &&
cd lttng-tools-2.9.* &&
./configure --libdir=/usr/local/lib32 CFLAGS=-m32 CXXFLAGS=-m32 \
- LDFLAGS='-L/usr/local/lib32 -L/usr/lib32' &&
+ LDFLAGS='-L/usr/local/lib32 -L/usr/lib32' \
+ --disable-bin-lttng --disable-bin-lttng-crash \
+ --disable-bin-lttng-relayd --disable-bin-lttng-sessiond &&
make &&
cd src/bin/lttng-consumerd &&
sudo make install &&
--
[role="term"]
----
-cd $(mktemp -d) &&
+$ cd $(mktemp -d) &&
wget http://lttng.org/files/lttng-tools/lttng-tools-latest-2.9.tar.bz2 &&
tar -xf lttng-tools-latest-2.9.tar.bz2 &&
cd lttng-tools-2.9.* &&
--
[role="term"]
----
-gcc -m32 -c -I. hello-tp.c
-gcc -m32 -c hello.c
-gcc -m32 -o hello hello.o hello-tp.o \
+$ gcc -m32 -c -I. hello-tp.c
+$ gcc -m32 -c hello.c
+$ gcc -m32 -o hello hello.o hello-tp.o \
-L/usr/lib32 -L/usr/local/lib32 \
-Wl,-rpath,/usr/lib32,-rpath,/usr/local/lib32 \
-llttng-ust -ldl
--
[role="term"]
----
-gcc -o app app.c -llttng-ust
+$ gcc -o app app.c -llttng-ust
----
--
--
[role="term"]
----
-lttng enable-event --userspace 'lttng_ust_tracef:*'
+$ lttng enable-event --userspace 'lttng_ust_tracef:*'
----
--
--
[role="term"]
----
-gcc -o app app.c -llttng-ust
+$ gcc -o app app.c -llttng-ust
----
--
--
[role="term"]
----
-lttng enable-event --userspace 'lttng_ust_tracelog:*'
+$ lttng enable-event --userspace 'lttng_ust_tracelog:*'
--loglevel=TRACE_WARNING
----
--
--
[role="term"]
----
-lttng enable-event --userspace 'lttng_ust_tracelog:*'
+$ lttng enable-event --userspace 'lttng_ust_tracelog:*'
--loglevel-only=TRACE_INFO
----
--
--
[role="term"]
----
-LD_PRELOAD=liblttng-ust-libc-wrapper.so my-app
+$ LD_PRELOAD=liblttng-ust-libc-wrapper.so my-app
----
--
+
--
[role="term"]
----
-LD_PRELOAD=liblttng-ust-libc-wrapper.so:liblttng-ust-dl.so my-app
+$ LD_PRELOAD=liblttng-ust-libc-wrapper.so:liblttng-ust-dl.so my-app
----
--
to the entry and exit points of functions.
man:gcc(1) and man:clang(1) have an option named
-https://gcc.gnu.org/onlinedocs/gcc/Code-Gen-Options.html[`-finstrument-functions`]
+https://gcc.gnu.org/onlinedocs/gcc/Instrumentation-Options.html[`-finstrument-functions`]
which generates instrumentation calls for entry and exit to functions.
The LTTng-UST function tracing helpers,
path:{liblttng-ust-cyg-profile.so} and
[role="term"]
----
-javac -cp /usr/share/java/jarpath/lttng-ust-agent-common.jar:/usr/share/java/jarpath/lttng-ust-agent-jul.jar Test.java
+$ javac -cp /usr/share/java/jarpath/lttng-ust-agent-common.jar:/usr/share/java/jarpath/lttng-ust-agent-jul.jar Test.java
----
<<creating-destroying-tracing-sessions,Create a tracing session>>,
[role="term"]
----
-lttng create
-lttng enable-event --jul jello
-lttng start
+$ lttng create
+$ lttng enable-event --jul jello
+$ lttng start
----
Run the compiled class:
[role="term"]
----
-java -cp /usr/share/java/jarpath/lttng-ust-agent-common.jar:/usr/share/java/jarpath/lttng-ust-agent-jul.jar:. Test
+$ java -cp /usr/share/java/jarpath/lttng-ust-agent-common.jar:/usr/share/java/jarpath/lttng-ust-agent-jul.jar:. Test
----
<<basic-tracing-session-control,Stop tracing>> and inspect the
[role="term"]
----
-lttng stop
-lttng view
+$ lttng stop
+$ lttng view
----
====
+In the resulting trace, an <<event,event record>> generated by a Java
+application using `java.util.logging` is named `lttng_jul:event` and
+has the following fields:
+
+`msg`::
+ Log record's message.
+
+`logger_name`::
+ Logger name.
+
+`class_name`::
+ Name of the class in which the log statement was executed.
+
+`method_name`::
+ Name of the method in which the log statement was executed.
+
+`long_millis`::
+ Logging time (timestamp in milliseconds).
+
+`int_loglevel`::
+ Log level integer value.
+
+`int_threadid`::
+ ID of the thread in which the log statement was executed.
+
You can use the opt:lttng-enable-event(1):--loglevel or
opt:lttng-enable-event(1):--loglevel-only option of the
man:lttng-enable-event(1) command to target a range of JUL log levels
[role="term"]
----
-javac -cp /usr/share/java/jarpath/lttng-ust-agent-common.jar:/usr/share/java/jarpath/lttng-ust-agent-log4j.jar:$LOG4JPATH Test.java
+$ javac -cp /usr/share/java/jarpath/lttng-ust-agent-common.jar:/usr/share/java/jarpath/lttng-ust-agent-log4j.jar:$LOG4JPATH Test.java
----
<<creating-destroying-tracing-sessions,Create a tracing session>>,
[role="term"]
----
-lttng create
-lttng enable-event --log4j jello
-lttng start
+$ lttng create
+$ lttng enable-event --log4j jello
+$ lttng start
----
Run the compiled class:
[role="term"]
----
-java -cp /usr/share/java/jarpath/lttng-ust-agent-common.jar:/usr/share/java/jarpath/lttng-ust-agent-log4j.jar:$LOG4JPATH:. Test
+$ java -cp /usr/share/java/jarpath/lttng-ust-agent-common.jar:/usr/share/java/jarpath/lttng-ust-agent-log4j.jar:$LOG4JPATH:. Test
----
<<basic-tracing-session-control,Stop tracing>> and inspect the
[role="term"]
----
-lttng stop
-lttng view
+$ lttng stop
+$ lttng view
----
====
+In the resulting trace, an <<event,event record>> generated by a Java
+application using log4j is named `lttng_log4j:event` and
+has the following fields:
+
+`msg`::
+ Log record's message.
+
+`logger_name`::
+ Logger name.
+
+`class_name`::
+ Name of the class in which the log statement was executed.
+
+`method_name`::
+ Name of the method in which the log statement was executed.
+
+`filename`::
+ Name of the file in which the executed log statement is located.
+
+`line_number`::
+ Line number at which the log statement was executed.
+
+`timestamp`::
+ Logging timestamp.
+
+`int_loglevel`::
+ Log level integer value.
+
+`thread_name`::
+ Name of the Java thread in which the log statement was executed.
+
You can use the opt:lttng-enable-event(1):--loglevel or
opt:lttng-enable-event(1):--loglevel-only option of the
man:lttng-enable-event(1) command to target a range of Apache log4j log levels
[role="term"]
----
-javac -cp /usr/share/java/jarpath/lttng-ust-agent-common.jar:/usr/share/java/jarpath/lttng-ust-agent-jul.jar Test.java
+$ javac -cp /usr/share/java/jarpath/lttng-ust-agent-common.jar:/usr/share/java/jarpath/lttng-ust-agent-jul.jar Test.java
----
<<creating-destroying-tracing-sessions,Create a tracing session>>
[role="term"]
----
-lttng create
-lttng enable-event --jul jello
+$ lttng create
+$ lttng enable-event --jul jello
----
<<adding-context,Add the application-specific context fields>> to the
[role="term"]
----
-lttng add-context --jul --type='$app.myRetriever:intCtx'
-lttng add-context --jul --type='$app.myRetriever:strContext'
+$ lttng add-context --jul --type='$app.myRetriever:intCtx'
+$ lttng add-context --jul --type='$app.myRetriever:strContext'
----
<<basic-tracing-session-control,Start tracing>>:
[role="term"]
----
-lttng start
+$ lttng start
----
Run the compiled class:
[role="term"]
----
-java -cp /usr/share/java/jarpath/lttng-ust-agent-common.jar:/usr/share/java/jarpath/lttng-ust-agent-jul.jar:. Test
+$ java -cp /usr/share/java/jarpath/lttng-ust-agent-common.jar:/usr/share/java/jarpath/lttng-ust-agent-jul.jar:. Test
----
<<basic-tracing-session-control,Stop tracing>> and inspect the
[role="term"]
----
-lttng stop
-lttng view
+$ lttng stop
+$ lttng view
----
====
[role="term"]
----
-lttng create
-lttng enable-event --python my-logger
-lttng start
+$ lttng create
+$ lttng enable-event --python my-logger
+$ lttng start
----
Run the Python script:
[role="term"]
----
-python test.py
+$ python test.py
----
<<basic-tracing-session-control,Stop tracing>> and inspect the recorded
[role="term"]
----
-lttng stop
-lttng view
+$ lttng stop
+$ lttng view
----
====
+In the resulting trace, an <<event,event record>> generated by a Python
+application is named `lttng_python:event` and has the following fields:
+
+`asctime`::
+ Logging time (string).
+
+`msg`::
+ Log record's message.
+
+`logger_name`::
+ Logger name.
+
+`funcName`::
+ Name of the function in which the log statement was executed.
+
+`lineno`::
+ Line number at which the log statement was executed.
+
+`int_loglevel`::
+ Log level integer value.
+
+`thread`::
+ ID of the Python thread in which the log statement was executed.
+
+`threadName`::
+ Name of the Python thread in which the log statement was executed.
+
You can use the opt:lttng-enable-event(1):--loglevel or
opt:lttng-enable-event(1):--loglevel-only option of the
man:lttng-enable-event(1) command to target a range of Python log levels
[role="term"]
----
-echo "Some message, some $variable" > /proc/lttng-logger
+$ echo "Some message, some $variable" > /proc/lttng-logger
----
Any event that the LTTng logger emits is named `lttng_logger` and
[role="term"]
----
-lttng create
-lttng enable-event --kernel lttng_logger
-lttng start
+$ lttng create
+$ lttng enable-event --kernel lttng_logger
+$ lttng start
----
Run the Bash script:
[role="term"]
----
-bash test.bash
+$ bash test.bash
----
<<basic-tracing-session-control,Stop tracing>> and inspect the recorded
[role="term"]
----
-lttng stop
-lttng view
+$ lttng stop
+$ lttng view
----
====
--
[role="term"]
----
-cd $(mktemp -d) &&
+$ cd $(mktemp -d) &&
wget http://lttng.org/files/lttng-modules/lttng-modules-latest-2.9.tar.bz2 &&
tar -xf lttng-modules-latest-2.9.tar.bz2 &&
cd lttng-modules-2.9.*
----
--
-. Edit path:{probes/Makefile} and add your new kernel module object
+. Edit path:{probes/KBuild} and add your new kernel module object
next to the existing ones:
+
--
[source,make]
-.path:{probes/Makefile}
+.path:{probes/KBuild}
----
# ...
--
[role="term"]
----
-make KERNELDIR=/path/to/linux
-sudo make modules_install
+$ make KERNELDIR=/path/to/linux
+# make modules_install && depmod -a
----
--
+
====
[role="term"]
----
-sudo lttng-sessiond --extra-kmod-probes=my_subsys,usb
+# lttng-sessiond --extra-kmod-probes=my_subsys,usb
----
====
--
====
[role="term"]
----
-sudo lttng-sessiond --kmod-probes=my_subsys,usb
+# lttng-sessiond --kmod-probes=my_subsys,usb
----
====
--
--
[role="term"]
----
-lsmod | grep lttng_probe_usb
+$ lsmod | grep lttng_probe_usb
----
--
--
[role="term"]
----
-sudo pkill lttng-sessiond
+# pkill lttng-sessiond
----
--
+
--
[role="term"]
----
-lttng-sessiond --daemonize
+$ lttng-sessiond --daemonize
----
--
--
[role="term"]
----
-sudo lttng-sessiond --daemonize
+# lttng-sessiond --daemonize
----
--
--
[role="term"]
----
-lttng create
+$ lttng create
----
--
--
[role="term"]
----
-lttng create my-session
+$ lttng create my-session
----
--
+
--
[role="term"]
----
-lttng create --output=/tmp/some-directory my-session
+$ lttng create my-session --output=/tmp/some-directory
----
--
--
[role="term"]
----
-lttng list
+$ lttng list
----
--
--
[role="term"]
----
-lttng set-session new-session
+$ lttng set-session new-session
----
--
+
--
[role="term"]
----
-lttng destroy
+$ lttng destroy
----
--
====
[role="term"]
----
-lttng list --userspace
+$ lttng list --userspace
----
====
====
[role="term"]
----
-lttng list --kernel --syscall
+$ lttng list --kernel --syscall
----
====
====
[role="term"]
----
-lttng enable-event --kernel sched_switch
+$ lttng enable-event --kernel sched_switch
----
====
====
[role="term"]
----
-lttng enable-event --kernel --syscall open,write,read,close
+$ lttng enable-event --kernel --syscall open,write,read,close
----
====
====
[role="term"]
----
-lttng enable-event --kernel sched_switch --filter='prev_comm == "bash"'
+$ lttng enable-event --kernel sched_switch --filter='prev_comm == "bash"'
----
[role="term"]
----
-lttng enable-event --kernel --all \
+$ lttng enable-event --kernel --all \
--filter='$ctx.tid == 1988 || $ctx.tid == 1534'
----
[role="term"]
----
-lttng enable-event --jul my_logger \
+$ lttng enable-event --jul my_logger \
--filter='$app.retriever:cur_msg_id > 3'
----
====
[role="term"]
----
-lttng enable-event --userspace my_app:'*' --loglevel=TRACE_INFO
+$ lttng enable-event --userspace my_app:'*' --loglevel=TRACE_INFO
----
IMPORTANT: Make sure to always quote the wildcard character when you
====
[role="term"]
----
-lttng enable-event --python my-app.'*' \
+$ lttng enable-event --python my-app.'*' \
--exclude='my-app.module,my-app.hello'
----
====
====
[role="term"]
----
-lttng enable-event --log4j --all --loglevel-only=LOG4J_WARN
+$ lttng enable-event --log4j --all --loglevel-only=LOG4J_WARN
----
====
====
[role="term"]
----
-lttng enable-event --userspace my_app:my_tracepoint --channel=my-channel
+$ lttng enable-event --userspace my_app:my_tracepoint --channel=my-channel
----
====
[role="term"]
----
-lttng enable-event --userspace my_app:my_tracepoint
-lttng enable-event --userspace my_app:my_tracepoint \
+$ lttng enable-event --userspace my_app:my_tracepoint
+$ lttng enable-event --userspace my_app:my_tracepoint \
--loglevel=TRACE_INFO
----
====
[role="term"]
----
-lttng disable-event --python my-logger
+$ lttng disable-event --python my-logger
----
====
====
[role="term"]
----
-lttng disable-event --jul '*'
+$ lttng disable-event --jul '*'
----
====
[role="term"]
----
-lttng disable-event --jul --all-events
+$ lttng disable-event --jul --all-events
----
====
--
[role="term"]
----
-lttng status
+$ lttng status
----
--
+
--
[role="term"]
----
-lttng list my-session
+$ lttng list my-session
----
--
+
--
[role="term"]
----
-lttng start
+$ lttng start
----
--
--
[role="term"]
----
-lttng stop
+$ lttng stop
----
--
+
====
[role="term"]
----
-lttng enable-channel --kernel my-channel
+$ lttng enable-channel --kernel my-channel
----
====
====
[role="term"]
----
-lttng enable-channel --userspace --num-subbuf=4 --subbuf-size=1M \
+$ lttng enable-channel --userspace --num-subbuf=4 --subbuf-size=1M \
--buffers-pid my-channel
----
====
====
[role="term"]
----
-lttng enable-channel --kernel --tracefile-count=8 \
+$ lttng enable-channel --kernel --tracefile-count=8 \
--tracefile-size=4194304 my-channel
----
====
====
[role="term"]
----
-lttng enable-channel --userspace --overwrite my-channel
+$ lttng enable-channel --userspace --overwrite my-channel
----
====
[role="term"]
----
-lttng enable-event --userspace --channel=my-channel app:tp
-lttng enable-event --userspace --channel=other-channel app:tp
+$ lttng enable-event --userspace --channel=my-channel app:tp
+$ lttng enable-event --userspace --channel=other-channel app:tp
----
If both channels are enabled, when a tracepoint named `app:tp` is
====
[role="term"]
----
-lttng disable-channel --kernel my-channel
+$ lttng disable-channel --kernel my-channel
----
====
[role="term"]
----
-lttng add-context --userspace --type=vpid --type=perf:thread:cpu-cycles
+$ lttng add-context --userspace --type=vpid --type=perf:thread:cpu-cycles
----
====
[role="term"]
----
-lttng add-context --userspace --type=perf:thread:raw:r0110:test
-lttng add-context --kernel --type=perf:cpu:raw:r0013c:x86unhalted
+$ lttng add-context --userspace --type=perf:thread:raw:r0110:test
+$ lttng add-context --kernel --type=perf:cpu:raw:r0013c:x86unhalted
----
====
[role="term"]
----
-lttng add-context --kernel --channel=my-channel --type=tid
+$ lttng add-context --kernel --channel=my-channel --type=tid
----
====
[role="term"]
----
-lttng add-context --kernel --channel=my-channel \
+$ lttng add-context --kernel --channel=my-channel \
--type='$app:retriever:cur_msg_id'
----
[role="term"]
----
-lttng track --pid=3,4,7,10,13
+$ lttng track --pid=3,4,7,10,13
----
the whitelist is:
[role="term"]
----
-lttng track --pid=1,15,16
+$ lttng track --pid=1,15,16
----
The result is:
[role="term"]
----
-lttng untrack --pid=3,7,10,13
+$ lttng untrack --pid=3,7,10,13
----
leads to this whitelist:
[role="term"]
----
-lttng track --pid --all
+$ lttng track --pid --all
----
The result is, again:
[role="term"]
----
-lttng untrack --pid --all
+$ lttng untrack --pid --all
----
gives:
[role="term"]
----
-lttng track --pid=6,11
+$ lttng track --pid=6,11
----
Result:
--
[role="term"]
----
-lttng save my-session
+$ lttng save my-session
----
--
+
--
[role="term"]
----
-lttng load my-session
+$ lttng load my-session
----
--
+
--
[role="term"]
----
-lttng-relayd
+$ lttng-relayd
----
--
--
[role="term"]
----
-lttng create my-session --set-url=net://remote-system
+$ lttng create my-session --set-url=net://remote-system
----
--
+
. On the target system, use the man:lttng(1) command-line tool as usual.
When tracing is active, the target's consumer daemon sends sub-buffers
- to the relay daemon running on the remote system intead of flushing
+ to the relay daemon running on the remote system instead of flushing
them to the local file system. The relay daemon writes the received
packets to the local file system.
--
[role="term"]
----
-lttng create --live my-session
+$ lttng create my-session --live
----
--
+
--
[role="term"]
----
-babeltrace --input-format=lttng-live net://localhost/host/hostname/my-session
+$ babeltrace --input-format=lttng-live \
+ net://localhost/host/hostname/my-session
----
--
+
[role="term"]
----
-babeltrace --input-format=lttng-live net://localhost
+$ babeltrace --input-format=lttng-live net://localhost
----
You can start the relay daemon on another system. In this case, you need
--
[role="term"]
----
-lttng create --snapshot my-session
+$ lttng create my-session --snapshot
----
--
+
--
[role="term"]
----
-lttng snapshot record --name=my-first-snapshot
+$ lttng snapshot record --name=my-first-snapshot
----
--
+
[role="term"]
----
-lttng --mi=xml enable-event --kernel --syscall open
+$ lttng --mi=xml enable-event --kernel --syscall open
----
A schema definition (XSD) is
--
[role="term"]
----
-lttng regenerate metadata
+$ lttng regenerate metadata
----
--
--
[role="term"]
----
-lttng regenerate statedump
+$ lttng regenerate statedump
----
--
--
[role="term"]
----
-lttng stop
+$ lttng stop
----
--
--
[role="term"]
----
-lttng snapshot record --name=my-snapshot
+$ lttng snapshot record --name=my-snapshot
----
--
--
[role="term"]
----
-lttng create --shm-path=/path/to/shm
+$ lttng create my-session --shm-path=/path/to/shm
----
--
--
[role="term"]
----
-lttng-crash /path/to/shm
+$ lttng-crash /path/to/shm
----
--
--
[role="term"]
----
-lttng-crash --extract=/path/to/trace /path/to/shm
+$ lttng-crash --extract=/path/to/trace /path/to/shm
----
--
or a Linux kernel KProbe.
+
An event is said to _occur_ at a specific time. Different actions can
-be taken upon the occurance of an event, like record the event's payload
+be taken upon the occurrence of an event, like record the event's payload
to a sub-buffer.
<<channel-overwrite-mode-vs-discard-mode,event loss mode>>::