2.10: update installation
[lttng-docs.git] / 2.10 / lttng-docs-2.10.txt
CommitLineData
85c29972
PP
1The LTTng Documentation
2=======================
3Philippe Proulx <pproulx@efficios.com>
347b52e8 4v2.10, 18 October 2019
85c29972
PP
5
6
7include::../common/copyright.txt[]
8
9
10include::../common/welcome.txt[]
11
12
13include::../common/audience.txt[]
14
15
16[[chapters]]
17=== What's in this documentation?
18
19The LTTng Documentation is divided into the following sections:
20
21* **<<nuts-and-bolts,Nuts and bolts>>** explains the
22 rudiments of software tracing and the rationale behind the
23 LTTng project.
24+
25You can skip this section if you’re familiar with software tracing and
26with the LTTng project.
27
28* **<<installing-lttng,Installation>>** describes the steps to
29 install the LTTng packages on common Linux distributions and from
30 their sources.
31+
32You can skip this section if you already properly installed LTTng on
33your target system.
34
35* **<<getting-started,Quick start>>** is a concise guide to
36 getting started quickly with LTTng kernel and user space tracing.
37+
38We recommend this section if you're new to LTTng or to software tracing
39in general.
40+
41You can skip this section if you're not new to LTTng.
42
43* **<<core-concepts,Core concepts>>** explains the concepts at
44 the heart of LTTng.
45+
46It's a good idea to become familiar with the core concepts
47before attempting to use the toolkit.
48
49* **<<plumbing,Components of LTTng>>** describes the various components
50 of the LTTng machinery, like the daemons, the libraries, and the
51 command-line interface.
52* **<<instrumenting,Instrumentation>>** shows different ways to
53 instrument user applications and the Linux kernel.
54+
55Instrumenting source code is essential to provide a meaningful
56source of events.
57+
58You can skip this section if you do not have a programming background.
59
60* **<<controlling-tracing,Tracing control>>** is divided into topics
61 which demonstrate how to use the vast array of features that
62 LTTng{nbsp}{revision} offers.
63* **<<reference,Reference>>** contains reference tables.
64* **<<glossary,Glossary>>** is a specialized dictionary of terms related
65 to LTTng or to the field of software tracing.
66
67
68include::../common/convention.txt[]
69
70
71include::../common/acknowledgements.txt[]
72
73
74[[whats-new]]
75== What's new in LTTng {revision}?
76
77LTTng{nbsp}{revision} bears the name _KeKriek_. From
78http://brasseriedunham.com/[Brasserie Dunham], the _**KeKriek**_ is a
79sour mashed golden wheat ale fermented with local sour cherries from
80Tougas orchards. Fresh sweet cherry notes with some tartness, lively
81carbonation with a dry finish.
82
83New features and changes in LTTng{nbsp}{revision}:
84
85* **Tracing control**:
86** You can put more than one wildcard special character (`*`), and not
87 only at the end, when you <<enabling-disabling-events,create an event
88 rule>>, in both the instrumentation point name and the literal
89 strings of
a2211984 90 link:/man/1/lttng-enable-event/v{revision}/#doc-filter-syntax[filter expressions]:
85c29972
PP
91+
92--
93[role="term"]
94----
95# lttng enable-event --kernel 'x86_*_local_timer_*' \
96 --filter='name == "*a*b*c*d*e" && count >= 23'
97----
98--
99+
100--
101[role="term"]
102----
103$ lttng enable-event --userspace '*_my_org:*msg*'
104----
105--
106
107** New trigger and notification API for
108 <<liblttng-ctl-lttng,`liblttng-ctl`>>. This new subsystem allows you
109 to register triggers which emit a notification when a given
110 condition is satisfied. As of LTTng{nbsp}{revision}, only
111 <<channel,channel>> buffer usage conditions are available.
112 Documentation is available in the
113 https://github.com/lttng/lttng-tools/tree/stable-{revision}/include/lttng[`liblttng-ctl`
90c4e38a
PP
114 header files] and in
115 <<notif-trigger-api,Get notified when a channel's buffer usage is too
116 high or too low>>.
85c29972
PP
117
118** You can now embed the whole textual LTTng-tools man pages into the
119 executables at build time with the `--enable-embedded-help`
120 configuration option. Thanks to this option, you don't need the
121 http://www.methods.co.nz/asciidoc/[AsciiDoc] and
122 https://directory.fsf.org/wiki/Xmlto[xmlto] tools at build time, and
123 a manual pager at run time, to get access to this documentation.
124
125* **User space tracing**:
126** New blocking mode: an LTTng-UST tracepoint can now block until
127 <<channel,sub-buffer>> space is available instead of discarding event
128 records in <<channel-overwrite-mode-vs-discard-mode,discard mode>>.
129 With this feature, you can be sure that no event records are
130 discarded during your application's execution at the expense of
131 performance.
132+
133For example, the following command lines create a user space tracing
134channel with an infinite blocking timeout and run an application
135instrumented with LTTng-UST which is explicitly allowed to block:
136+
137--
138[role="term"]
139----
140$ lttng create
000f69a6 141$ lttng enable-channel --userspace --blocking-timeout=inf blocking-channel
85c29972
PP
142$ lttng enable-event --userspace --channel=blocking-channel --all
143$ lttng start
144$ LTTNG_UST_ALLOW_BLOCKING=1 my-app
145----
146--
147+
148See the complete <<blocking-timeout-example,blocking timeout example>>.
149
150* **Linux kernel tracing**:
151** Linux 4.10, 4.11, and 4.12 support.
152** The thread state dump events recorded by LTTng-modules now contain
153 the task's CPU identifier. This improves the precision of the
154 scheduler model for analyses.
155** Extended man:socketpair(2) system call tracing data.
156
157
158[[nuts-and-bolts]]
159== Nuts and bolts
160
161What is LTTng? As its name suggests, the _Linux Trace Toolkit: next
162generation_ is a modern toolkit for tracing Linux systems and
163applications. So your first question might be:
164**what is tracing?**
165
166
167[[what-is-tracing]]
168=== What is tracing?
169
170As the history of software engineering progressed and led to what
171we now take for granted--complex, numerous and
172interdependent software applications running in parallel on
173sophisticated operating systems like Linux--the authors of such
174components, software developers, began feeling a natural
175urge to have tools that would ensure the robustness and good performance
176of their masterpieces.
177
178One major achievement in this field is, inarguably, the
179https://www.gnu.org/software/gdb/[GNU debugger (GDB)],
180an essential tool for developers to find and fix bugs. But even the best
181debugger won't help make your software run faster, and nowadays, faster
182software means either more work done by the same hardware, or cheaper
183hardware for the same work.
184
185A _profiler_ is often the tool of choice to identify performance
186bottlenecks. Profiling is suitable to identify _where_ performance is
187lost in a given software. The profiler outputs a profile, a statistical
188summary of observed events, which you may use to discover which
189functions took the most time to execute. However, a profiler won't
190report _why_ some identified functions are the bottleneck. Bottlenecks
191might only occur when specific conditions are met, conditions that are
192sometimes impossible to capture by a statistical profiler, or impossible
193to reproduce with an application altered by the overhead of an
194event-based profiler. For a thorough investigation of software
195performance issues, a history of execution is essential, with the
196recorded values of variables and context fields you choose, and
197with as little influence as possible on the instrumented software. This
198is where tracing comes in handy.
199
200_Tracing_ is a technique used to understand what goes on in a running
201software system. The software used for tracing is called a _tracer_,
202which is conceptually similar to a tape recorder. When recording,
203specific instrumentation points placed in the software source code
204generate events that are saved on a giant tape: a _trace_ file. You
205can trace user applications and the operating system at the same time,
206opening the possibility of resolving a wide range of problems that would
207otherwise be extremely challenging.
208
209Tracing is often compared to _logging_. However, tracers and loggers are
210two different tools, serving two different purposes. Tracers are
211designed to record much lower-level events that occur much more
212frequently than log messages, often in the range of thousands per
213second, with very little execution overhead. Logging is more appropriate
214for a very high-level analysis of less frequent events: user accesses,
215exceptional conditions (errors and warnings, for example), database
216transactions, instant messaging communications, and such. Simply put,
217logging is one of the many use cases that can be satisfied with tracing.
218
219The list of recorded events inside a trace file can be read manually
220like a log file for the maximum level of detail, but it is generally
221much more interesting to perform application-specific analyses to
222produce reduced statistics and graphs that are useful to resolve a
223given problem. Trace viewers and analyzers are specialized tools
224designed to do this.
225
226In the end, this is what LTTng is: a powerful, open source set of
227tools to trace the Linux kernel and user applications at the same time.
228LTTng is composed of several components actively maintained and
229developed by its link:/community/#where[community].
230
231
232[[lttng-alternatives]]
233=== Alternatives to noch:{LTTng}
234
235Excluding proprietary solutions, a few competing software tracers
236exist for Linux:
237
238* https://github.com/dtrace4linux/linux[dtrace4linux] is a port of
239 Sun Microsystems's DTrace to Linux. The cmd:dtrace tool interprets
240 user scripts and is responsible for loading code into the
241 Linux kernel for further execution and collecting the outputted data.
242* https://en.wikipedia.org/wiki/Berkeley_Packet_Filter[eBPF] is a
243 subsystem in the Linux kernel in which a virtual machine can execute
244 programs passed from the user space to the kernel. You can attach
245 such programs to tracepoints and KProbes thanks to a system call, and
246 they can output data to the user space when executed thanks to
247 different mechanisms (pipe, VM register values, and eBPF maps, to name
248 a few).
249* https://www.kernel.org/doc/Documentation/trace/ftrace.txt[ftrace]
250 is the de facto function tracer of the Linux kernel. Its user
251 interface is a set of special files in sysfs.
252* https://perf.wiki.kernel.org/[perf] is
253 a performance analyzing tool for Linux which supports hardware
254 performance counters, tracepoints, as well as other counters and
255 types of probes. perf's controlling utility is the cmd:perf command
256 line/curses tool.
257* http://linux.die.net/man/1/strace[strace]
258 is a command-line utility which records system calls made by a
259 user process, as well as signal deliveries and changes of process
260 state. strace makes use of https://en.wikipedia.org/wiki/Ptrace[ptrace]
261 to fulfill its function.
262* http://www.sysdig.org/[sysdig], like SystemTap, uses scripts to
263 analyze Linux kernel events. You write scripts, or _chisels_ in
264 sysdig's jargon, in Lua and sysdig executes them while the system is
265 being traced or afterwards. sysdig's interface is the cmd:sysdig
266 command-line tool as well as the curses-based cmd:csysdig tool.
267* https://sourceware.org/systemtap/[SystemTap] is a Linux kernel and
268 user space tracer which uses custom user scripts to produce plain text
269 traces. SystemTap converts the scripts to the C language, and then
270 compiles them as Linux kernel modules which are loaded to produce
271 trace data. SystemTap's primary user interface is the cmd:stap
272 command-line tool.
273
274The main distinctive features of LTTng is that it produces correlated
275kernel and user space traces, as well as doing so with the lowest
276overhead amongst other solutions. It produces trace files in the
277http://diamon.org/ctf[CTF] format, a file format optimized
278for the production and analyses of multi-gigabyte data.
279
280LTTng is the result of more than 10 years of active open source
281development by a community of passionate developers.
282LTTng{nbsp}{revision} is currently available on major desktop and server
283Linux distributions.
284
285The main interface for tracing control is a single command-line tool
286named cmd:lttng. The latter can create several tracing sessions, enable
287and disable events on the fly, filter events efficiently with custom
288user expressions, start and stop tracing, and much more. LTTng can
289record the traces on the file system or send them over the network, and
290keep them totally or partially. You can view the traces once tracing
291becomes inactive or in real-time.
292
293<<installing-lttng,Install LTTng now>> and
294<<getting-started,start tracing>>!
295
296
297[[installing-lttng]]
298== Installation
299
300**LTTng** is a set of software <<plumbing,components>> which interact to
301<<instrumenting,instrument>> the Linux kernel and user applications, and
302to <<controlling-tracing,control tracing>> (start and stop
303tracing, enable and disable event rules, and the rest). Those
304components are bundled into the following packages:
305
306* **LTTng-tools**: Libraries and command-line interface to
307 control tracing.
308* **LTTng-modules**: Linux kernel modules to instrument and
309 trace the kernel.
310* **LTTng-UST**: Libraries and Java/Python packages to instrument and
311 trace user applications.
312
313Most distributions mark the LTTng-modules and LTTng-UST packages as
314optional when installing LTTng-tools (which is always required). In the
315following sections, we always provide the steps to install all three,
316but note that:
317
318* You only need to install LTTng-modules if you intend to trace the
319 Linux kernel.
320* You only need to install LTTng-UST if you intend to trace user
321 applications.
322
323[role="growable"]
347b52e8 324.Availability of LTTng{nbsp}{revision} for major Linux distributions as of 18 October 2019.
85c29972 325|====
347b52e8 326|Distribution |Available in releases
85c29972
PP
327
328|https://www.ubuntu.com/[Ubuntu]
347b52e8
PP
329|xref:ubuntu[Ubuntu{nbsp}18.04 _Bionic Beaver_,
330Ubuntu{nbsp}19.04 _Disco Dingo_, and
331Ubuntu{nbsp}19.10 _Eoan Ermine_].
85c29972 332
347b52e8 333Ubuntu{nbsp}16.04 _Xenial Xerus_:
34d65fe0 334<<ubuntu-ppa,use the LTTng Stable{nbsp}{revision} PPA>>.
85c29972
PP
335
336|https://getfedora.org/[Fedora]
347b52e8
PP
337|xref:fedora[Fedora{nbsp}29, Fedora{nbsp}30, Fedora{nbsp}31,
338and Fedora{nbsp}32].
85c29972
PP
339
340|https://www.debian.org/[Debian]
347b52e8 341|<<debian,Debian "buster" (stable) and Debian "bullseye" (testing)>>.
85c29972
PP
342
343|https://alpinelinux.org/[Alpine Linux]
347b52e8
PP
344|xref:alpine-linux[Alpine Linux{nbsp}3.7, Alpine Linux{nbsp}3.8,
345Alpine Linux{nbsp}3.9, and Alpine Linux{nbsp}3.10].
34d65fe0
PP
346
347|https://www.opensuse.org/[openSUSE]
347b52e8 348|<<opensuse,openSUSE Leap{nbsp}15.1>>.
85c29972
PP
349
350|https://www.redhat.com/[RHEL] and https://www.suse.com/[SLES]
351|See http://packages.efficios.com/[EfficiOS Enterprise Packages].
85c29972
PP
352
353|https://buildroot.org/[Buildroot]
3575940d 354|xref:buildroot[Buildroot{nbsp}2018.02, Buildroot{nbsp}2018.05,
347b52e8
PP
355Buildroot{nbsp}2018.08, Buildroot{nbsp}2018.11, Buildroot{nbsp}2019.02,
356Buildroot{nbsp}2018.05, Buildroot{nbsp}2018.08, and
357Buildroot{nbsp}2018.11].
85c29972
PP
358
359|http://www.openembedded.org/wiki/Main_Page[OpenEmbedded] and
360https://www.yoctoproject.org/[Yocto]
347b52e8
PP
361|<<oe-yocto,Yocto Project{nbsp}2.7 _Warrior_ and
362Yocto Project{nbsp}3.0 _Zeus_>>.
85c29972
PP
363|====
364
365
366[[ubuntu]]
367=== [[ubuntu-official-repositories]]Ubuntu
368
347b52e8
PP
369LTTng{nbsp}{revision} is available on:
370
371* Ubuntu{nbsp}18.04 _Bionic Beaver_
372* Ubuntu{nbsp}19.04 _Disco Dingo_
373* Ubuntu{nbsp}19.10 _Eoan Ermine_
34d65fe0 374
347b52e8
PP
375For other releases of Ubuntu, <<ubuntu-ppa,use the LTTng
376Stable{nbsp}{revision} PPA>>.
377
378To install LTTng{nbsp}{revision} on Ubuntu{nbsp}18.04 _Bionic Beaver_,
379Ubuntu{nbsp}19.04 _Disco Dingo_, or
380Ubuntu{nbsp}19.10 _Eoan Ermine_:
34d65fe0
PP
381
382. Install the main LTTng{nbsp}{revision} packages:
383+
384--
385[role="term"]
386----
387# apt-get install lttng-tools
388# apt-get install lttng-modules-dkms
389# apt-get install liblttng-ust-dev
390----
391--
392
393. **If you need to instrument and trace
394 <<java-application,Java applications>>**, install the LTTng-UST
395 Java agent:
396+
397--
398[role="term"]
399----
400# apt-get install liblttng-ust-agent-java
401----
402--
403
404. **If you need to instrument and trace
405 <<python-application,Python{nbsp}3 applications>>**, install the
406 LTTng-UST Python agent:
407+
408--
409[role="term"]
410----
411# apt-get install python3-lttngust
412----
413--
414
415
85c29972
PP
416[[ubuntu-ppa]]
417==== noch:{LTTng} Stable {revision} PPA
418
419The https://launchpad.net/~lttng/+archive/ubuntu/stable-{revision}[LTTng
420Stable{nbsp}{revision} PPA] offers the latest stable
347b52e8 421LTTng{nbsp}{revision} packages for Ubuntu{nbsp}18.04 _Bionic Beaver_.
85c29972
PP
422
423To install LTTng{nbsp}{revision} from the LTTng Stable{nbsp}{revision} PPA:
424
425. Add the LTTng Stable{nbsp}{revision} PPA repository and update the
426 list of packages:
427+
428--
429[role="term"]
430----
431# apt-add-repository ppa:lttng/stable-2.10
432# apt-get update
433----
434--
435
436. Install the main LTTng{nbsp}{revision} packages:
437+
438--
439[role="term"]
440----
441# apt-get install lttng-tools
442# apt-get install lttng-modules-dkms
443# apt-get install liblttng-ust-dev
444----
445--
446
447. **If you need to instrument and trace
448 <<java-application,Java applications>>**, install the LTTng-UST
449 Java agent:
450+
451--
452[role="term"]
453----
454# apt-get install liblttng-ust-agent-java
455----
456--
457
458. **If you need to instrument and trace
459 <<python-application,Python{nbsp}3 applications>>**, install the
460 LTTng-UST Python agent:
461+
462--
463[role="term"]
464----
465# apt-get install python3-lttngust
466----
467--
468
469
cdb68973
PP
470[[fedora]]
471=== Fedora
472
347b52e8
PP
473To install LTTng{nbsp}{revision} on Fedora{nbsp}29, Fedora{nbsp}30,
474Fedora{nbsp}31, or Fedora{nbsp}32:
cdb68973
PP
475
476. Install the LTTng-tools{nbsp}{revision} and LTTng-UST{nbsp}{revision}
477 packages:
478+
479--
480[role="term"]
481----
482# yum install lttng-tools
483# yum install lttng-ust
484----
485--
486
487. Download, build, and install the latest LTTng-modules{nbsp}{revision}:
488+
489--
490[role="term"]
491----
492$ cd $(mktemp -d) &&
493wget http://lttng.org/files/lttng-modules/lttng-modules-latest-2.10.tar.bz2 &&
494tar -xf lttng-modules-latest-2.10.tar.bz2 &&
495cd lttng-modules-2.10.* &&
496make &&
497sudo make modules_install &&
498sudo depmod -a
499----
500--
501
502[IMPORTANT]
503.Java and Python application instrumentation and tracing
504====
505If you need to instrument and trace <<java-application,Java
506applications>> on Fedora, you need to build and install
507LTTng-UST{nbsp}{revision} <<building-from-source,from source>> and pass
508the `--enable-java-agent-jul`, `--enable-java-agent-log4j`, or
509`--enable-java-agent-all` options to the `configure` script, depending
510on which Java logging framework you use.
511
512If you need to instrument and trace <<python-application,Python
513applications>> on Fedora, you need to build and install
514LTTng-UST{nbsp}{revision} from source and pass the
515`--enable-python-agent` option to the `configure` script.
516====
517
518
519[[debian]]
520=== Debian
521
347b52e8
PP
522To install LTTng{nbsp}{revision} on Debian "buster" (stable) or
523Debian "bullseye" (testing):
cdb68973
PP
524
525. Install the main LTTng{nbsp}{revision} packages:
526+
527--
528[role="term"]
529----
530# apt-get install lttng-modules-dkms
531# apt-get install liblttng-ust-dev
532# apt-get install lttng-tools
533----
534--
535
536. **If you need to instrument and trace <<java-application,Java
537 applications>>**, install the LTTng-UST Java agent:
538+
539--
540[role="term"]
541----
542# apt-get install liblttng-ust-agent-java
543----
544--
545
546. **If you need to instrument and trace <<python-application,Python
547 applications>>**, install the LTTng-UST Python agent:
548+
549--
550[role="term"]
551----
552# apt-get install python3-lttngust
553----
554--
555
556
35df6938
PP
557[[alpine-linux]]
558=== Alpine Linux
559
560To install LTTng-tools{nbsp}{revision} and LTTng-UST{nbsp}{revision} on
347b52e8
PP
561Alpine Linux{nbsp}3.7, Alpine Linux{nbsp}3.8, Alpine Linux{nbsp}3.9, or
562Alpine Linux{nbsp}3.10:
34d65fe0 563
35df6938
PP
564. Add the LTTng packages:
565+
566--
567[role="term"]
568----
569# apk add lttng-tools
570# apk add lttng-ust-dev
571----
572--
573
35df6938
PP
574. Download, build, and install the latest LTTng-modules{nbsp}{revision}:
575+
576--
577[role="term"]
578----
579$ cd $(mktemp -d) &&
580wget http://lttng.org/files/lttng-modules/lttng-modules-latest-2.10.tar.bz2 &&
581tar -xf lttng-modules-latest-2.10.tar.bz2 &&
582cd lttng-modules-2.10.* &&
583make &&
584sudo make modules_install &&
585sudo depmod -a
586----
587--
588
589
34d65fe0
PP
590[[opensuse]]
591=== noch:{openSUSE}
592
347b52e8 593To install LTTng{nbsp}{revision} on openSUSE Leap{nbsp}15.1:
34d65fe0
PP
594
595* Install the main LTTng{nbsp}{revision} packages:
596+
597--
598[role="term"]
599----
600sudo zypper install lttng-tools
601sudo zypper install lttng-modules
602sudo zypper install lttng-ust-devel
603----
604--
605
606[IMPORTANT]
607.Java and Python application instrumentation and tracing
608====
609If you need to instrument and trace <<java-application,Java
610applications>> on openSUSE, you need to build and install
611LTTng-UST{nbsp}{revision} <<building-from-source,from source>> and pass
612the `--enable-java-agent-jul`, `--enable-java-agent-log4j`, or
613`--enable-java-agent-all` options to the `configure` script, depending
614on which Java logging framework you use.
615
616If you need to instrument and trace <<python-application,Python
617applications>> on openSUSE, you need to build and install
618LTTng-UST{nbsp}{revision} from source and pass the
619`--enable-python-agent` option to the `configure` script.
620====
621
622
85c29972
PP
623[[enterprise-distributions]]
624=== RHEL, SUSE, and other enterprise distributions
625
626To install LTTng on enterprise Linux distributions, such as Red Hat
627Enterprise Linux (RHEL) and SUSE Linux Enterprise Server (SUSE), please
628see http://packages.efficios.com/[EfficiOS Enterprise Packages].
629
630
3575940d
PP
631[[buildroot]]
632=== Buildroot
633
634To install LTTng{nbsp}{revision} on Buildroot{nbsp}2018.02,
347b52e8
PP
635Buildroot{nbsp}2018.05, Buildroot{nbsp}2018.08,
636Buildroot{nbsp}2018.11, Buildroot{nbsp}2019.02,
637Buildroot{nbsp}2019.05, Buildroot{nbsp}2019.08, or
638Buildroot{nbsp}2019.11:
3575940d
PP
639
640. Launch the Buildroot configuration tool:
641+
642--
643[role="term"]
644----
645$ make menuconfig
646----
647--
648
649. In **Kernel**, check **Linux kernel**.
650. In **Toolchain**, check **Enable WCHAR support**.
651. In **Target packages**{nbsp}&#8594; **Debugging, profiling and benchmark**,
652 check **lttng-modules** and **lttng-tools**.
653. In **Target packages**{nbsp}&#8594; **Libraries**{nbsp}&#8594;
654 **Other**, check **lttng-libust**.
655
656
347b52e8
PP
657[[oe-yocto]]
658=== OpenEmbedded and Yocto
659
660LTTng{nbsp}{revision} recipes are available in the
661http://layers.openembedded.org/layerindex/branch/master/layer/openembedded-core/[`openembedded-core`]
662layer for Yocto Project{nbsp}2.7 _Warrior_ and
663Yocto Project{nbsp}3.0 _Zeus_ under the following names:
664
665* `lttng-tools`
666* `lttng-modules`
667* `lttng-ust`
668
669With BitBake, the simplest way to include LTTng recipes in your target
670image is to add them to `IMAGE_INSTALL_append` in path:{conf/local.conf}:
671
672----
673IMAGE_INSTALL_append = " lttng-tools lttng-modules lttng-ust"
674----
675
676If you use Hob:
677
678. Select a machine and an image recipe.
679. Click **Edit image recipe**.
680. Under the **All recipes** tab, search for **lttng**.
681. Check the desired LTTng recipes.
682
683[IMPORTANT]
684.Java and Python application instrumentation and tracing
685====
686If you need to instrument and trace <<java-application,Java
687applications>> on Yocto/OpenEmbedded, you need to build and install
688LTTng-UST{nbsp}{revision} <<building-from-source,from source>> and pass
689the `--enable-java-agent-jul`, `--enable-java-agent-log4j`, or
690`--enable-java-agent-all` options to the `configure` script, depending
691on which Java logging framework you use.
692
693If you need to instrument and trace <<python-application,Python
694applications>> on Yocto/OpenEmbedded, you need to build and install
695LTTng-UST{nbsp}{revision} from source and pass the
696`--enable-python-agent` option to the `configure` script.
697====
698
699
85c29972
PP
700[[building-from-source]]
701=== Build from source
702
703To build and install LTTng{nbsp}{revision} from source:
704
705. Using your distribution's package manager, or from source, install
706 the following dependencies of LTTng-tools and LTTng-UST:
707+
708--
709* https://sourceforge.net/projects/libuuid/[libuuid]
710* http://directory.fsf.org/wiki/Popt[popt]
711* http://liburcu.org/[Userspace RCU]
712* http://www.xmlsoft.org/[libxml2]
713--
714
715. Download, build, and install the latest LTTng-modules{nbsp}{revision}:
716+
717--
718[role="term"]
719----
720$ cd $(mktemp -d) &&
721wget http://lttng.org/files/lttng-modules/lttng-modules-latest-2.10.tar.bz2 &&
722tar -xf lttng-modules-latest-2.10.tar.bz2 &&
723cd lttng-modules-2.10.* &&
724make &&
725sudo make modules_install &&
726sudo depmod -a
727----
728--
729
730. Download, build, and install the latest LTTng-UST{nbsp}{revision}:
731+
732--
733[role="term"]
734----
735$ cd $(mktemp -d) &&
736wget http://lttng.org/files/lttng-ust/lttng-ust-latest-2.10.tar.bz2 &&
737tar -xf lttng-ust-latest-2.10.tar.bz2 &&
738cd lttng-ust-2.10.* &&
739./configure &&
740make &&
741sudo make install &&
742sudo ldconfig
743----
744--
745+
746--
747[IMPORTANT]
748.Java and Python application tracing
749====
750If you need to instrument and trace <<java-application,Java
751applications>>, pass the `--enable-java-agent-jul`,
752`--enable-java-agent-log4j`, or `--enable-java-agent-all` options to the
753`configure` script, depending on which Java logging framework you use.
754
755If you need to instrument and trace <<python-application,Python
756applications>>, pass the `--enable-python-agent` option to the
757`configure` script. You can set the `PYTHON` environment variable to the
758path to the Python interpreter for which to install the LTTng-UST Python
759agent package.
760====
761--
762+
763--
764[NOTE]
765====
766By default, LTTng-UST libraries are installed to
767dir:{/usr/local/lib}, which is the de facto directory in which to
768keep self-compiled and third-party libraries.
769
770When <<building-tracepoint-providers-and-user-application,linking an
771instrumented user application with `liblttng-ust`>>:
772
773* Append `/usr/local/lib` to the env:LD_LIBRARY_PATH environment
774 variable.
775* Pass the `-L/usr/local/lib` and `-Wl,-rpath,/usr/local/lib` options to
776 man:gcc(1), man:g++(1), or man:clang(1).
777====
778--
779
780. Download, build, and install the latest LTTng-tools{nbsp}{revision}:
781+
782--
783[role="term"]
784----
785$ cd $(mktemp -d) &&
786wget http://lttng.org/files/lttng-tools/lttng-tools-latest-2.10.tar.bz2 &&
787tar -xf lttng-tools-latest-2.10.tar.bz2 &&
788cd lttng-tools-2.10.* &&
789./configure &&
790make &&
791sudo make install &&
792sudo ldconfig
793----
794--
795
796TIP: The https://github.com/eepp/vlttng[vlttng tool] can do all the
797previous steps automatically for a given version of LTTng and confine
798the installed files in a specific directory. This can be useful to test
799LTTng without installing it on your system.
800
801
802[[getting-started]]
803== Quick start
804
805This is a short guide to get started quickly with LTTng kernel and user
806space tracing.
807
808Before you follow this guide, make sure to <<installing-lttng,install>>
809LTTng.
810
811This tutorial walks you through the steps to:
812
813. <<tracing-the-linux-kernel,Trace the Linux kernel>>.
814. <<tracing-your-own-user-application,Trace a user application>> written
815 in C.
816. <<viewing-and-analyzing-your-traces,View and analyze the
817 recorded events>>.
818
819
820[[tracing-the-linux-kernel]]
821=== Trace the Linux kernel
822
823The following command lines start with the `#` prompt because you need
824root privileges to trace the Linux kernel. You can also trace the kernel
825as a regular user if your Unix user is a member of the
826<<tracing-group,tracing group>>.
827
828. Create a <<tracing-session,tracing session>> which writes its traces
829 to dir:{/tmp/my-kernel-trace}:
830+
831--
832[role="term"]
833----
834# lttng create my-kernel-session --output=/tmp/my-kernel-trace
835----
836--
837
838. List the available kernel tracepoints and system calls:
839+
840--
841[role="term"]
842----
843# lttng list --kernel
844# lttng list --kernel --syscall
845----
846--
847
848. Create <<event,event rules>> which match the desired instrumentation
849 point names, for example the `sched_switch` and `sched_process_fork`
850 tracepoints, and the man:open(2) and man:close(2) system calls:
851+
852--
853[role="term"]
854----
855# lttng enable-event --kernel sched_switch,sched_process_fork
856# lttng enable-event --kernel --syscall open,close
857----
858--
859+
860You can also create an event rule which matches _all_ the Linux kernel
861tracepoints (this will generate a lot of data when tracing):
862+
863--
864[role="term"]
865----
866# lttng enable-event --kernel --all
867----
868--
869
870. <<basic-tracing-session-control,Start tracing>>:
871+
872--
873[role="term"]
874----
875# lttng start
876----
877--
878
879. Do some operation on your system for a few seconds. For example,
880 load a website, or list the files of a directory.
46adfb4b 881. <<creating-destroying-tracing-sessions,Destroy>> the current
85c29972
PP
882 tracing session:
883+
884--
885[role="term"]
886----
85c29972
PP
887# lttng destroy
888----
889--
890+
891The man:lttng-destroy(1) command does not destroy the trace data; it
892only destroys the state of the tracing session.
46adfb4b
PP
893+
894The man:lttng-destroy(1) command also runs the man:lttng-stop(1) command
895implicitly (see <<basic-tracing-session-control,Start and stop a tracing
896session>>). You need to stop tracing to make LTTng flush the remaining
897trace data and make the trace readable.
85c29972
PP
898
899. For the sake of this example, make the recorded trace accessible to
900 the non-root users:
901+
902--
903[role="term"]
904----
905# chown -R $(whoami) /tmp/my-kernel-trace
906----
907--
908
909See <<viewing-and-analyzing-your-traces,View and analyze the
910recorded events>> to view the recorded events.
911
912
913[[tracing-your-own-user-application]]
914=== Trace a user application
915
916This section steps you through a simple example to trace a
917_Hello world_ program written in C.
918
919To create the traceable user application:
920
921. Create the tracepoint provider header file, which defines the
922 tracepoints and the events they can generate:
923+
924--
925[source,c]
926.path:{hello-tp.h}
927----
928#undef TRACEPOINT_PROVIDER
929#define TRACEPOINT_PROVIDER hello_world
930
931#undef TRACEPOINT_INCLUDE
932#define TRACEPOINT_INCLUDE "./hello-tp.h"
933
934#if !defined(_HELLO_TP_H) || defined(TRACEPOINT_HEADER_MULTI_READ)
935#define _HELLO_TP_H
936
937#include <lttng/tracepoint.h>
938
939TRACEPOINT_EVENT(
940 hello_world,
941 my_first_tracepoint,
942 TP_ARGS(
943 int, my_integer_arg,
944 char*, my_string_arg
945 ),
946 TP_FIELDS(
947 ctf_string(my_string_field, my_string_arg)
948 ctf_integer(int, my_integer_field, my_integer_arg)
949 )
950)
951
952#endif /* _HELLO_TP_H */
953
954#include <lttng/tracepoint-event.h>
955----
956--
957
958. Create the tracepoint provider package source file:
959+
960--
961[source,c]
962.path:{hello-tp.c}
963----
964#define TRACEPOINT_CREATE_PROBES
965#define TRACEPOINT_DEFINE
966
967#include "hello-tp.h"
968----
969--
970
971. Build the tracepoint provider package:
972+
973--
974[role="term"]
975----
976$ gcc -c -I. hello-tp.c
977----
978--
979
980. Create the _Hello World_ application source file:
981+
982--
983[source,c]
984.path:{hello.c}
985----
986#include <stdio.h>
987#include "hello-tp.h"
988
989int main(int argc, char *argv[])
990{
991 int x;
992
993 puts("Hello, World!\nPress Enter to continue...");
994
995 /*
996 * The following getchar() call is only placed here for the purpose
997 * of this demonstration, to pause the application in order for
998 * you to have time to list its tracepoints. It is not
999 * needed otherwise.
1000 */
1001 getchar();
1002
1003 /*
1004 * A tracepoint() call.
1005 *
1006 * Arguments, as defined in hello-tp.h:
1007 *
1008 * 1. Tracepoint provider name (required)
1009 * 2. Tracepoint name (required)
1010 * 3. my_integer_arg (first user-defined argument)
1011 * 4. my_string_arg (second user-defined argument)
1012 *
1013 * Notice the tracepoint provider and tracepoint names are
1014 * NOT strings: they are in fact parts of variables that the
1015 * macros in hello-tp.h create.
1016 */
1017 tracepoint(hello_world, my_first_tracepoint, 23, "hi there!");
1018
1019 for (x = 0; x < argc; ++x) {
1020 tracepoint(hello_world, my_first_tracepoint, x, argv[x]);
1021 }
1022
1023 puts("Quitting now!");
1024 tracepoint(hello_world, my_first_tracepoint, x * x, "x^2");
1025
1026 return 0;
1027}
1028----
1029--
1030
1031. Build the application:
1032+
1033--
1034[role="term"]
1035----
1036$ gcc -c hello.c
1037----
1038--
1039
1040. Link the application with the tracepoint provider package,
1041 `liblttng-ust`, and `libdl`:
1042+
1043--
1044[role="term"]
1045----
1046$ gcc -o hello hello.o hello-tp.o -llttng-ust -ldl
1047----
1048--
1049
1050Here's the whole build process:
1051
1052[role="img-100"]
1053.User space tracing tutorial's build steps.
1054image::ust-flow.png[]
1055
1056To trace the user application:
1057
1058. Run the application with a few arguments:
1059+
1060--
1061[role="term"]
1062----
1063$ ./hello world and beyond
1064----
1065--
1066+
1067You see:
1068+
1069--
1070----
1071Hello, World!
1072Press Enter to continue...
1073----
1074--
1075
1076. Start an LTTng <<lttng-sessiond,session daemon>>:
1077+
1078--
1079[role="term"]
1080----
1081$ lttng-sessiond --daemonize
1082----
1083--
1084+
1085Note that a session daemon might already be running, for example as
1086a service that the distribution's service manager started.
1087
1088. List the available user space tracepoints:
1089+
1090--
1091[role="term"]
1092----
1093$ lttng list --userspace
1094----
1095--
1096+
1097You see the `hello_world:my_first_tracepoint` tracepoint listed
1098under the `./hello` process.
1099
1100. Create a <<tracing-session,tracing session>>:
1101+
1102--
1103[role="term"]
1104----
1105$ lttng create my-user-space-session
1106----
1107--
1108
1109. Create an <<event,event rule>> which matches the
1110 `hello_world:my_first_tracepoint` event name:
1111+
1112--
1113[role="term"]
1114----
1115$ lttng enable-event --userspace hello_world:my_first_tracepoint
1116----
1117--
1118
1119. <<basic-tracing-session-control,Start tracing>>:
1120+
1121--
1122[role="term"]
1123----
1124$ lttng start
1125----
1126--
1127
1128. Go back to the running `hello` application and press Enter. The
1129 program executes all `tracepoint()` instrumentation points and exits.
46adfb4b 1130. <<creating-destroying-tracing-sessions,Destroy>> the current
85c29972
PP
1131 tracing session:
1132+
1133--
1134[role="term"]
1135----
85c29972
PP
1136$ lttng destroy
1137----
1138--
1139+
1140The man:lttng-destroy(1) command does not destroy the trace data; it
1141only destroys the state of the tracing session.
46adfb4b
PP
1142+
1143The man:lttng-destroy(1) command also runs the man:lttng-stop(1) command
1144implicitly (see <<basic-tracing-session-control,Start and stop a tracing
1145session>>). You need to stop tracing to make LTTng flush the remaining
1146trace data and make the trace readable.
85c29972
PP
1147
1148By default, LTTng saves the traces in
1149+$LTTNG_HOME/lttng-traces/__name__-__date__-__time__+,
1150where +__name__+ is the tracing session name. The
1151env:LTTNG_HOME environment variable defaults to `$HOME` if not set.
1152
1153See <<viewing-and-analyzing-your-traces,View and analyze the
1154recorded events>> to view the recorded events.
1155
1156
1157[[viewing-and-analyzing-your-traces]]
1158=== View and analyze the recorded events
1159
1160Once you have completed the <<tracing-the-linux-kernel,Trace the Linux
1161kernel>> and <<tracing-your-own-user-application,Trace a user
1162application>> tutorials, you can inspect the recorded events.
1163
1164Many tools are available to read LTTng traces:
1165
1166* **cmd:babeltrace** is a command-line utility which converts trace
1167 formats; it supports the format that LTTng produces, CTF, as well as a
1168 basic text output which can be ++grep++ed. The cmd:babeltrace command
1169 is part of the http://diamon.org/babeltrace[Babeltrace] project.
1170* Babeltrace also includes
1171 **https://www.python.org/[Python] bindings** so
1172 that you can easily open and read an LTTng trace with your own script,
1173 benefiting from the power of Python.
1174* http://tracecompass.org/[**Trace Compass**]
1175 is a graphical user interface for viewing and analyzing any type of
1176 logs or traces, including LTTng's.
1177* https://github.com/lttng/lttng-analyses[**LTTng analyses**] is a
1178 project which includes many high-level analyses of LTTng kernel
1179 traces, like scheduling statistics, interrupt frequency distribution,
1180 top CPU usage, and more.
1181
1182NOTE: This section assumes that the traces recorded during the previous
1183tutorials were saved to their default location, in the
1184dir:{$LTTNG_HOME/lttng-traces} directory. The env:LTTNG_HOME
1185environment variable defaults to `$HOME` if not set.
1186
1187
1188[[viewing-and-analyzing-your-traces-bt]]
1189==== Use the cmd:babeltrace command-line tool
1190
1191The simplest way to list all the recorded events of a trace is to pass
1192its path to cmd:babeltrace with no options:
1193
1194[role="term"]
1195----
1196$ babeltrace ~/lttng-traces/my-user-space-session*
1197----
1198
1199cmd:babeltrace finds all traces recursively within the given path and
1200prints all their events, merging them in chronological order.
1201
1202You can pipe the output of cmd:babeltrace into a tool like man:grep(1) for
1203further filtering:
1204
1205[role="term"]
1206----
1207$ babeltrace /tmp/my-kernel-trace | grep _switch
1208----
1209
1210You can pipe the output of cmd:babeltrace into a tool like man:wc(1) to
1211count the recorded events:
1212
1213[role="term"]
1214----
1215$ babeltrace /tmp/my-kernel-trace | grep _open | wc --lines
1216----
1217
1218
1219[[viewing-and-analyzing-your-traces-bt-python]]
1220==== Use the Babeltrace Python bindings
1221
1222The <<viewing-and-analyzing-your-traces-bt,text output of cmd:babeltrace>>
1223is useful to isolate events by simple matching using man:grep(1) and
1224similar utilities. However, more elaborate filters, such as keeping only
1225event records with a field value falling within a specific range, are
1226not trivial to write using a shell. Moreover, reductions and even the
1227most basic computations involving multiple event records are virtually
1228impossible to implement.
1229
1230Fortunately, Babeltrace ships with Python 3 bindings which makes it easy
1231to read the event records of an LTTng trace sequentially and compute the
1232desired information.
1233
1234The following script accepts an LTTng Linux kernel trace path as its
1235first argument and prints the short names of the top 5 running processes
1236on CPU 0 during the whole trace:
1237
1238[source,python]
1239.path:{top5proc.py}
1240----
1241from collections import Counter
1242import babeltrace
1243import sys
1244
1245
1246def top5proc():
1247 if len(sys.argv) != 2:
1248 msg = 'Usage: python3 {} TRACEPATH'.format(sys.argv[0])
1249 print(msg, file=sys.stderr)
1250 return False
1251
1252 # A trace collection contains one or more traces
1253 col = babeltrace.TraceCollection()
1254
1255 # Add the trace provided by the user (LTTng traces always have
1256 # the 'ctf' format)
1257 if col.add_trace(sys.argv[1], 'ctf') is None:
1258 raise RuntimeError('Cannot add trace')
1259
1260 # This counter dict contains execution times:
1261 #
1262 # task command name -> total execution time (ns)
1263 exec_times = Counter()
1264
1265 # This contains the last `sched_switch` timestamp
1266 last_ts = None
1267
1268 # Iterate on events
1269 for event in col.events:
1270 # Keep only `sched_switch` events
1271 if event.name != 'sched_switch':
1272 continue
1273
1274 # Keep only events which happened on CPU 0
1275 if event['cpu_id'] != 0:
1276 continue
1277
1278 # Event timestamp
1279 cur_ts = event.timestamp
1280
1281 if last_ts is None:
1282 # We start here
1283 last_ts = cur_ts
1284
1285 # Previous task command (short) name
1286 prev_comm = event['prev_comm']
1287
1288 # Initialize entry in our dict if not yet done
1289 if prev_comm not in exec_times:
1290 exec_times[prev_comm] = 0
1291
1292 # Compute previous command execution time
1293 diff = cur_ts - last_ts
1294
1295 # Update execution time of this command
1296 exec_times[prev_comm] += diff
1297
1298 # Update last timestamp
1299 last_ts = cur_ts
1300
1301 # Display top 5
1302 for name, ns in exec_times.most_common(5):
1303 s = ns / 1000000000
1304 print('{:20}{} s'.format(name, s))
1305
1306 return True
1307
1308
1309if __name__ == '__main__':
1310 sys.exit(0 if top5proc() else 1)
1311----
1312
1313Run this script:
1314
1315[role="term"]
1316----
1317$ python3 top5proc.py /tmp/my-kernel-trace/kernel
1318----
1319
1320Output example:
1321
1322----
1323swapper/0 48.607245889 s
1324chromium 7.192738188 s
1325pavucontrol 0.709894415 s
1326Compositor 0.660867933 s
1327Xorg.bin 0.616753786 s
1328----
1329
1330Note that `swapper/0` is the "idle" process of CPU 0 on Linux; since we
1331weren't using the CPU that much when tracing, its first position in the
1332list makes sense.
1333
1334
1335[[core-concepts]]
1336== [[understanding-lttng]]Core concepts
1337
1338From a user's perspective, the LTTng system is built on a few concepts,
1339or objects, on which the <<lttng-cli,cmd:lttng command-line tool>>
1340operates by sending commands to the <<lttng-sessiond,session daemon>>.
1341Understanding how those objects relate to eachother is key in mastering
1342the toolkit.
1343
1344The core concepts are:
1345
1346* <<tracing-session,Tracing session>>
1347* <<domain,Tracing domain>>
1348* <<channel,Channel and ring buffer>>
1349* <<"event","Instrumentation point, event rule, event, and event record">>
1350
1351
1352[[tracing-session]]
1353=== Tracing session
1354
1355A _tracing session_ is a stateful dialogue between you and
1356a <<lttng-sessiond,session daemon>>. You can
1357<<creating-destroying-tracing-sessions,create a new tracing
1358session>> with the `lttng create` command.
1359
1360Anything that you do when you control LTTng tracers happens within a
1361tracing session. In particular, a tracing session:
1362
1363* Has its own name.
1364* Has its own set of trace files.
1365* Has its own state of activity (started or stopped).
1366* Has its own <<tracing-session-mode,mode>> (local, network streaming,
1367 snapshot, or live).
1368* Has its own <<channel,channels>> which have their own
1369 <<event,event rules>>.
1370
1371[role="img-100"]
1372.A _tracing session_ contains <<channel,channels>> that are members of <<domain,tracing domains>> and contain <<event,event rules>>.
1373image::concepts.png[]
1374
1375Those attributes and objects are completely isolated between different
1376tracing sessions.
1377
1378A tracing session is analogous to a cash machine session:
1379the operations you do on the banking system through the cash machine do
1380not alter the data of other users of the same system. In the case of
1381the cash machine, a session lasts as long as your bank card is inside.
1382In the case of LTTng, a tracing session lasts from the `lttng create`
1383command to the `lttng destroy` command.
1384
1385[role="img-100"]
1386.Each Unix user has its own set of tracing sessions.
1387image::many-sessions.png[]
1388
1389
1390[[tracing-session-mode]]
1391==== Tracing session mode
1392
1393LTTng can send the generated trace data to different locations. The
1394_tracing session mode_ dictates where to send it. The following modes
1395are available in LTTng{nbsp}{revision}:
1396
1397Local mode::
1398 LTTng writes the traces to the file system of the machine being traced
1399 (target system).
1400
1401Network streaming mode::
1402 LTTng sends the traces over the network to a
1403 <<lttng-relayd,relay daemon>> running on a remote system.
1404
1405Snapshot mode::
1406 LTTng does not write the traces by default. Instead, you can request
1407 LTTng to <<taking-a-snapshot,take a snapshot>>, that is, a copy of the
1408 current tracing buffers, and to write it to the target's file system
1409 or to send it over the network to a <<lttng-relayd,relay daemon>>
1410 running on a remote system.
1411
1412Live mode::
1413 This mode is similar to the network streaming mode, but a live
1414 trace viewer can connect to the distant relay daemon to
1415 <<lttng-live,view event records as LTTng generates them>> by
1416 the tracers.
1417
1418
1419[[domain]]
1420=== Tracing domain
1421
1422A _tracing domain_ is a namespace for event sources. A tracing domain
1423has its own properties and features.
1424
1425There are currently five available tracing domains:
1426
1427* Linux kernel
1428* User space
1429* `java.util.logging` (JUL)
1430* log4j
1431* Python
1432
1433You must specify a tracing domain when using some commands to avoid
1434ambiguity. For example, since all the domains support named tracepoints
1435as event sources (instrumentation points that you manually insert in the
1436source code), you need to specify a tracing domain when
1437<<enabling-disabling-events,creating an event rule>> because all the
1438tracing domains could have tracepoints with the same names.
1439
1440Some features are reserved to specific tracing domains. Dynamic function
1441entry and return instrumentation points, for example, are currently only
1442supported in the Linux kernel tracing domain, but support for other
1443tracing domains could be added in the future.
1444
1445You can create <<channel,channels>> in the Linux kernel and user space
1446tracing domains. The other tracing domains have a single default
1447channel.
1448
1449
1450[[channel]]
1451=== Channel and ring buffer
1452
1453A _channel_ is an object which is responsible for a set of ring buffers.
1454Each ring buffer is divided into multiple sub-buffers. When an LTTng
1455tracer emits an event, it can record it to one or more
1456sub-buffers. The attributes of a channel determine what to do when
1457there's no space left for a new event record because all sub-buffers
1458are full, where to send a full sub-buffer, and other behaviours.
1459
1460A channel is always associated to a <<domain,tracing domain>>. The
1461`java.util.logging` (JUL), log4j, and Python tracing domains each have
1462a default channel which you cannot configure.
1463
1464A channel also owns <<event,event rules>>. When an LTTng tracer emits
1465an event, it records it to the sub-buffers of all
1466the enabled channels with a satisfied event rule, as long as those
1467channels are part of active <<tracing-session,tracing sessions>>.
1468
1469
1470[[channel-buffering-schemes]]
1471==== Per-user vs. per-process buffering schemes
1472
1473A channel has at least one ring buffer _per CPU_. LTTng always
1474records an event to the ring buffer associated to the CPU on which it
1475occurred.
1476
1477Two _buffering schemes_ are available when you
1478<<enabling-disabling-channels,create a channel>> in the
1479user space <<domain,tracing domain>>:
1480
1481Per-user buffering::
1482 Allocate one set of ring buffers--one per CPU--shared by all the
1483 instrumented processes of each Unix user.
1484+
1485--
1486[role="img-100"]
1487.Per-user buffering scheme.
1488image::per-user-buffering.png[]
1489--
1490
1491Per-process buffering::
1492 Allocate one set of ring buffers--one per CPU--for each
1493 instrumented process.
1494+
1495--
1496[role="img-100"]
1497.Per-process buffering scheme.
1498image::per-process-buffering.png[]
1499--
1500+
1501The per-process buffering scheme tends to consume more memory than the
1502per-user option because systems generally have more instrumented
1503processes than Unix users running instrumented processes. However, the
1504per-process buffering scheme ensures that one process having a high
1505event throughput won't fill all the shared sub-buffers of the same
1506user, only its own.
1507
1508The Linux kernel tracing domain has only one available buffering scheme
1509which is to allocate a single set of ring buffers for the whole system.
1510This scheme is similar to the per-user option, but with a single, global
1511user "running" the kernel.
1512
1513
1514[[channel-overwrite-mode-vs-discard-mode]]
1515==== Overwrite vs. discard event loss modes
1516
1517When an event occurs, LTTng records it to a specific sub-buffer (yellow
1518arc in the following animation) of a specific channel's ring buffer.
1519When there's no space left in a sub-buffer, the tracer marks it as
1520consumable (red) and another, empty sub-buffer starts receiving the
1521following event records. A <<lttng-consumerd,consumer daemon>>
1522eventually consumes the marked sub-buffer (returns to white).
1523
1524[NOTE]
1525[role="docsvg-channel-subbuf-anim"]
1526====
1527{note-no-anim}
1528====
1529
1530In an ideal world, sub-buffers are consumed faster than they are filled,
1531as is the case in the previous animation. In the real world,
1532however, all sub-buffers can be full at some point, leaving no space to
1533record the following events.
1534
1535By default, LTTng-modules and LTTng-UST are _non-blocking_ tracers: when
1536no empty sub-buffer is available, it is acceptable to lose event records
1537when the alternative would be to cause substantial delays in the
1538instrumented application's execution. LTTng privileges performance over
1539integrity; it aims at perturbing the traced system as little as possible
1540in order to make tracing of subtle race conditions and rare interrupt
1541cascades possible.
1542
1543Starting from LTTng{nbsp}2.10, the LTTng user space tracer, LTTng-UST,
1544supports a _blocking mode_. See the <<blocking-timeout-example,blocking
1545timeout example>> to learn how to use the blocking mode.
1546
1547When it comes to losing event records because no empty sub-buffer is
1548available, or because the <<opt-blocking-timeout,blocking timeout>> is
1549reached, the channel's _event loss mode_ determines what to do. The
1550available event loss modes are:
1551
1552Discard mode::
eeb71496
PP
1553 Drop the newest event records until a the tracer releases a
1554 sub-buffer.
1555+
1556This is the only available mode when you specify a
1557<<opt-blocking-timeout,blocking timeout>>.
85c29972
PP
1558
1559Overwrite mode::
1560 Clear the sub-buffer containing the oldest event records and start
1561 writing the newest event records there.
1562+
1563This mode is sometimes called _flight recorder mode_ because it's
1564similar to a
1565https://en.wikipedia.org/wiki/Flight_recorder[flight recorder]:
1566always keep a fixed amount of the latest data.
1567
1568Which mechanism you should choose depends on your context: prioritize
1569the newest or the oldest event records in the ring buffer?
1570
4089946d 1571Beware that, in overwrite mode, the tracer abandons a _whole sub-buffer_
85c29972
PP
1572as soon as a there's no space left for a new event record, whereas in
1573discard mode, the tracer only discards the event record that doesn't
1574fit.
1575
4089946d
PP
1576In discard mode, LTTng increments a count of lost event records when an
1577event record is lost and saves this count to the trace. In overwrite
1578mode, since LTTng 2.8, LTTng increments a count of lost sub-buffers when
1579a sub-buffer is lost and saves this count to the trace. In this mode,
1580the exact number of lost event records in those lost sub-buffers is not
1581saved to the trace. Trace analyses can use the trace's saved discarded
1582event record and sub-buffer counts to decide whether or not to perform
1583the analyses even if trace data is known to be missing.
85c29972
PP
1584
1585There are a few ways to decrease your probability of losing event
1586records.
1587<<channel-subbuf-size-vs-subbuf-count,Sub-buffer count and size>> shows
1588how you can fine-une the sub-buffer count and size of a channel to
1589virtually stop losing event records, though at the cost of greater
1590memory usage.
1591
1592
1593[[channel-subbuf-size-vs-subbuf-count]]
1594==== Sub-buffer count and size
1595
1596When you <<enabling-disabling-channels,create a channel>>, you can
1597set its number of sub-buffers and their size.
1598
1599Note that there is noticeable CPU overhead introduced when
1600switching sub-buffers (marking a full one as consumable and switching
1601to an empty one for the following events to be recorded). Knowing this,
1602the following list presents a few practical situations along with how
1603to configure the sub-buffer count and size for them:
1604
1605* **High event throughput**: In general, prefer bigger sub-buffers to
1606 lower the risk of losing event records.
1607+
1608Having bigger sub-buffers also ensures a lower
1609<<channel-switch-timer,sub-buffer switching frequency>>.
1610+
1611The number of sub-buffers is only meaningful if you create the channel
1612in overwrite mode: in this case, if a sub-buffer overwrite happens, the
1613other sub-buffers are left unaltered.
1614
1615* **Low event throughput**: In general, prefer smaller sub-buffers
1616 since the risk of losing event records is low.
1617+
1618Because events occur less frequently, the sub-buffer switching frequency
1619should remain low and thus the tracer's overhead should not be a
1620problem.
1621
1622* **Low memory system**: If your target system has a low memory
1623 limit, prefer fewer first, then smaller sub-buffers.
1624+
1625Even if the system is limited in memory, you want to keep the
1626sub-buffers as big as possible to avoid a high sub-buffer switching
1627frequency.
1628
1629Note that LTTng uses http://diamon.org/ctf/[CTF] as its trace format,
1630which means event data is very compact. For example, the average
1631LTTng kernel event record weights about 32{nbsp}bytes. Thus, a
1632sub-buffer size of 1{nbsp}MiB is considered big.
1633
1634The previous situations highlight the major trade-off between a few big
1635sub-buffers and more, smaller sub-buffers: sub-buffer switching
1636frequency vs. how much data is lost in overwrite mode. Assuming a
1637constant event throughput and using the overwrite mode, the two
1638following configurations have the same ring buffer total size:
1639
1640[NOTE]
1641[role="docsvg-channel-subbuf-size-vs-count-anim"]
1642====
1643{note-no-anim}
1644====
1645
1646* **2 sub-buffers of 4{nbsp}MiB each**: Expect a very low sub-buffer
1647 switching frequency, but if a sub-buffer overwrite happens, half of
1648 the event records so far (4{nbsp}MiB) are definitely lost.
1649* **8 sub-buffers of 1{nbsp}MiB each**: Expect 4{nbsp}times the tracer's
1650 overhead as the previous configuration, but if a sub-buffer
1651 overwrite happens, only the eighth of event records so far are
1652 definitely lost.
1653
1654In discard mode, the sub-buffers count parameter is pointless: use two
1655sub-buffers and set their size according to the requirements of your
1656situation.
1657
1658
1659[[channel-switch-timer]]
1660==== Switch timer period
1661
1662The _switch timer period_ is an important configurable attribute of
1663a channel to ensure periodic sub-buffer flushing.
1664
1665When the _switch timer_ expires, a sub-buffer switch happens. You can
1666set the switch timer period attribute when you
1667<<enabling-disabling-channels,create a channel>> to ensure that event
1668data is consumed and committed to trace files or to a distant relay
1669daemon periodically in case of a low event throughput.
1670
1671[NOTE]
1672[role="docsvg-channel-switch-timer"]
1673====
1674{note-no-anim}
1675====
1676
1677This attribute is also convenient when you use big sub-buffers to cope
1678with a sporadic high event throughput, even if the throughput is
1679normally low.
1680
1681
1682[[channel-read-timer]]
1683==== Read timer period
1684
1685By default, the LTTng tracers use a notification mechanism to signal a
1686full sub-buffer so that a consumer daemon can consume it. When such
1687notifications must be avoided, for example in real-time applications,
1688you can use the channel's _read timer_ instead. When the read timer
1689fires, the <<lttng-consumerd,consumer daemon>> checks for full,
1690consumable sub-buffers.
1691
1692
1693[[tracefile-rotation]]
1694==== Trace file count and size
1695
1696By default, trace files can grow as large as needed. You can set the
1697maximum size of each trace file that a channel writes when you
1698<<enabling-disabling-channels,create a channel>>. When the size of
1699a trace file reaches the channel's fixed maximum size, LTTng creates
1700another file to contain the next event records. LTTng appends a file
1701count to each trace file name in this case.
1702
1703If you set the trace file size attribute when you create a channel, the
1704maximum number of trace files that LTTng creates is _unlimited_ by
1705default. To limit them, you can also set a maximum number of trace
1706files. When the number of trace files reaches the channel's fixed
1707maximum count, the oldest trace file is overwritten. This mechanism is
1708called _trace file rotation_.
1709
1710
1711[[event]]
1712=== Instrumentation point, event rule, event, and event record
1713
1714An _event rule_ is a set of conditions which must be **all** satisfied
1715for LTTng to record an occuring event.
1716
1717You set the conditions when you <<enabling-disabling-events,create
1718an event rule>>.
1719
1720You always attach an event rule to <<channel,channel>> when you create
1721it.
1722
1723When an event passes the conditions of an event rule, LTTng records it
1724in one of the attached channel's sub-buffers.
1725
1726The available conditions, as of LTTng{nbsp}{revision}, are:
1727
1728* The event rule _is enabled_.
1729* The instrumentation point's type _is{nbsp}T_.
1730* The instrumentation point's name (sometimes called _event name_)
1731 _matches{nbsp}N_, but _is not{nbsp}E_.
1732* The instrumentation point's log level _is as severe as{nbsp}L_, or
1733 _is exactly{nbsp}L_.
1734* The fields of the event's payload _satisfy_ a filter
1735 expression{nbsp}__F__.
1736
1737As you can see, all the conditions but the dynamic filter are related to
1738the event rule's status or to the instrumentation point, not to the
1739occurring events. This is why, without a filter, checking if an event
1740passes an event rule is not a dynamic task: when you create or modify an
1741event rule, all the tracers of its tracing domain enable or disable the
1742instrumentation points themselves once. This is possible because the
1743attributes of an instrumentation point (type, name, and log level) are
1744defined statically. In other words, without a dynamic filter, the tracer
1745_does not evaluate_ the arguments of an instrumentation point unless it
1746matches an enabled event rule.
1747
1748Note that, for LTTng to record an event, the <<channel,channel>> to
1749which a matching event rule is attached must also be enabled, and the
1750tracing session owning this channel must be active.
1751
1752[role="img-100"]
1753.Logical path from an instrumentation point to an event record.
1754image::event-rule.png[]
1755
1756.Event, event record, or event rule?
1757****
1758With so many similar terms, it's easy to get confused.
1759
1760An **event** is the consequence of the execution of an _instrumentation
1761point_, like a tracepoint that you manually place in some source code,
1762or a Linux kernel KProbe. An event is said to _occur_ at a specific
1763time. Different actions can be taken upon the occurrence of an event,
1764like record the event's payload to a buffer.
1765
1766An **event record** is the representation of an event in a sub-buffer. A
1767tracer is responsible for capturing the payload of an event, current
1768context variables, the event's ID, and the event's timestamp. LTTng
1769can append this sub-buffer to a trace file.
1770
1771An **event rule** is a set of conditions which must all be satisfied for
1772LTTng to record an occuring event. Events still occur without
1773satisfying event rules, but LTTng does not record them.
1774****
1775
1776
1777[[plumbing]]
1778== Components of noch:{LTTng}
1779
1780The second _T_ in _LTTng_ stands for _toolkit_: it would be wrong
1781to call LTTng a simple _tool_ since it is composed of multiple
1782interacting components. This section describes those components,
1783explains their respective roles, and shows how they connect together to
1784form the LTTng ecosystem.
1785
1786The following diagram shows how the most important components of LTTng
1787interact with user applications, the Linux kernel, and you:
1788
1789[role="img-100"]
1790.Control and trace data paths between LTTng components.
1791image::plumbing.png[]
1792
1793The LTTng project incorporates:
1794
1795* **LTTng-tools**: Libraries and command-line interface to
1796 control tracing sessions.
1797** <<lttng-sessiond,Session daemon>> (man:lttng-sessiond(8)).
a9f3997c 1798** <<lttng-consumerd,Consumer daemon>> (cmd:lttng-consumerd).
85c29972
PP
1799** <<lttng-relayd,Relay daemon>> (man:lttng-relayd(8)).
1800** <<liblttng-ctl-lttng,Tracing control library>> (`liblttng-ctl`).
1801** <<lttng-cli,Tracing control command-line tool>> (man:lttng(1)).
1802* **LTTng-UST**: Libraries and Java/Python packages to trace user
1803 applications.
1804** <<lttng-ust,User space tracing library>> (`liblttng-ust`) and its
1805 headers to instrument and trace any native user application.
1806** <<prebuilt-ust-helpers,Preloadable user space tracing helpers>>:
1807*** `liblttng-ust-libc-wrapper`
1808*** `liblttng-ust-pthread-wrapper`
1809*** `liblttng-ust-cyg-profile`
1810*** `liblttng-ust-cyg-profile-fast`
1811*** `liblttng-ust-dl`
1812** User space tracepoint provider source files generator command-line
1813 tool (man:lttng-gen-tp(1)).
1814** <<lttng-ust-agents,LTTng-UST Java agent>> to instrument and trace
1815 Java applications using `java.util.logging` or
1816 Apache log4j 1.2 logging.
1817** <<lttng-ust-agents,LTTng-UST Python agent>> to instrument
1818 Python applications using the standard `logging` package.
1819* **LTTng-modules**: <<lttng-modules,Linux kernel modules>> to trace
1820 the kernel.
1821** LTTng kernel tracer module.
1822** Tracing ring buffer kernel modules.
1823** Probe kernel modules.
1824** LTTng logger kernel module.
1825
1826
1827[[lttng-cli]]
1828=== Tracing control command-line interface
1829
1830[role="img-100"]
1831.The tracing control command-line interface.
1832image::plumbing-lttng-cli.png[]
1833
1834The _man:lttng(1) command-line tool_ is the standard user interface to
1835control LTTng <<tracing-session,tracing sessions>>. The cmd:lttng tool
1836is part of LTTng-tools.
1837
1838The cmd:lttng tool is linked with
1839<<liblttng-ctl-lttng,`liblttng-ctl`>> to communicate with
1840one or more <<lttng-sessiond,session daemons>> behind the scenes.
1841
1842The cmd:lttng tool has a Git-like interface:
1843
1844[role="term"]
1845----
1846$ lttng <GENERAL OPTIONS> <COMMAND> <COMMAND OPTIONS>
1847----
1848
1849The <<controlling-tracing,Tracing control>> section explores the
1850available features of LTTng using the cmd:lttng tool.
1851
1852
1853[[liblttng-ctl-lttng]]
1854=== Tracing control library
1855
1856[role="img-100"]
1857.The tracing control library.
1858image::plumbing-liblttng-ctl.png[]
1859
1860The _LTTng control library_, `liblttng-ctl`, is used to communicate
1861with a <<lttng-sessiond,session daemon>> using a C API that hides the
1862underlying protocol's details. `liblttng-ctl` is part of LTTng-tools.
1863
1864The <<lttng-cli,cmd:lttng command-line tool>>
1865is linked with `liblttng-ctl`.
1866
1867You can use `liblttng-ctl` in C or $$C++$$ source code by including its
1868"master" header:
1869
1870[source,c]
1871----
1872#include <lttng/lttng.h>
1873----
1874
1875Some objects are referenced by name (C string), such as tracing
1876sessions, but most of them require to create a handle first using
1877`lttng_create_handle()`.
1878
1879The best available developer documentation for `liblttng-ctl` is, as of
1880LTTng{nbsp}{revision}, its installed header files. Every function and
1881structure is thoroughly documented.
1882
1883
1884[[lttng-ust]]
1885=== User space tracing library
1886
1887[role="img-100"]
1888.The user space tracing library.
1889image::plumbing-liblttng-ust.png[]
1890
1891The _user space tracing library_, `liblttng-ust` (see man:lttng-ust(3)),
1892is the LTTng user space tracer. It receives commands from a
1893<<lttng-sessiond,session daemon>>, for example to
1894enable and disable specific instrumentation points, and writes event
1895records to ring buffers shared with a
1896<<lttng-consumerd,consumer daemon>>.
1897`liblttng-ust` is part of LTTng-UST.
1898
1899Public C header files are installed beside `liblttng-ust` to
1900instrument any <<c-application,C or $$C++$$ application>>.
1901
1902<<lttng-ust-agents,LTTng-UST agents>>, which are regular Java and Python
1903packages, use their own library providing tracepoints which is
1904linked with `liblttng-ust`.
1905
1906An application or library does not have to initialize `liblttng-ust`
1907manually: its constructor does the necessary tasks to properly register
1908to a session daemon. The initialization phase also enables the
1909instrumentation points matching the <<event,event rules>> that you
1910already created.
1911
1912
1913[[lttng-ust-agents]]
1914=== User space tracing agents
1915
1916[role="img-100"]
1917.The user space tracing agents.
1918image::plumbing-lttng-ust-agents.png[]
1919
1920The _LTTng-UST Java and Python agents_ are regular Java and Python
1921packages which add LTTng tracing capabilities to the
1922native logging frameworks. The LTTng-UST agents are part of LTTng-UST.
1923
1924In the case of Java, the
1925https://docs.oracle.com/javase/7/docs/api/java/util/logging/package-summary.html[`java.util.logging`
1926core logging facilities] and
1927https://logging.apache.org/log4j/1.2/[Apache log4j 1.2] are supported.
1928Note that Apache Log4{nbsp}2 is not supported.
1929
1930In the case of Python, the standard
1931https://docs.python.org/3/library/logging.html[`logging`] package
1932is supported. Both Python 2 and Python 3 modules can import the
1933LTTng-UST Python agent package.
1934
1935The applications using the LTTng-UST agents are in the
1936`java.util.logging` (JUL),
1937log4j, and Python <<domain,tracing domains>>.
1938
1939Both agents use the same mechanism to trace the log statements. When an
1940agent is initialized, it creates a log handler that attaches to the root
1941logger. The agent also registers to a <<lttng-sessiond,session daemon>>.
1942When the application executes a log statement, it is passed to the
1943agent's log handler by the root logger. The agent's log handler calls a
1944native function in a tracepoint provider package shared library linked
1945with <<lttng-ust,`liblttng-ust`>>, passing the formatted log message and
1946other fields, like its logger name and its log level. This native
1947function contains a user space instrumentation point, hence tracing the
1948log statement.
1949
1950The log level condition of an
1951<<event,event rule>> is considered when tracing
1952a Java or a Python application, and it's compatible with the standard
1953JUL, log4j, and Python log levels.
1954
1955
1956[[lttng-modules]]
1957=== LTTng kernel modules
1958
1959[role="img-100"]
1960.The LTTng kernel modules.
1961image::plumbing-lttng-modules.png[]
1962
1963The _LTTng kernel modules_ are a set of Linux kernel modules
1964which implement the kernel tracer of the LTTng project. The LTTng
1965kernel modules are part of LTTng-modules.
1966
1967The LTTng kernel modules include:
1968
1969* A set of _probe_ modules.
1970+
1971Each module attaches to a specific subsystem
1972of the Linux kernel using its tracepoint instrument points. There are
1973also modules to attach to the entry and return points of the Linux
1974system call functions.
1975
1976* _Ring buffer_ modules.
1977+
1978A ring buffer implementation is provided as kernel modules. The LTTng
1979kernel tracer writes to the ring buffer; a
1980<<lttng-consumerd,consumer daemon>> reads from the ring buffer.
1981
1982* The _LTTng kernel tracer_ module.
1983* The _LTTng logger_ module.
1984+
1985The LTTng logger module implements the special path:{/proc/lttng-logger}
1986file so that any executable can generate LTTng events by opening and
1987writing to this file.
1988+
1989See <<proc-lttng-logger-abi,LTTng logger>>.
1990
1991Generally, you do not have to load the LTTng kernel modules manually
1992(using man:modprobe(8), for example): a root <<lttng-sessiond,session
1993daemon>> loads the necessary modules when starting. If you have extra
1994probe modules, you can specify to load them to the session daemon on
1995the command line.
1996
1997The LTTng kernel modules are installed in
1998+/usr/lib/modules/__release__/extra+ by default, where +__release__+ is
1999the kernel release (see `uname --kernel-release`).
2000
2001
2002[[lttng-sessiond]]
2003=== Session daemon
2004
2005[role="img-100"]
2006.The session daemon.
2007image::plumbing-sessiond.png[]
2008
2009The _session daemon_, man:lttng-sessiond(8), is a daemon responsible for
2010managing tracing sessions and for controlling the various components of
2011LTTng. The session daemon is part of LTTng-tools.
2012
2013The session daemon sends control requests to and receives control
2014responses from:
2015
2016* The <<lttng-ust,user space tracing library>>.
2017+
2018Any instance of the user space tracing library first registers to
2019a session daemon. Then, the session daemon can send requests to
2020this instance, such as:
2021+
2022--
2023** Get the list of tracepoints.
2024** Share an <<event,event rule>> so that the user space tracing library
2025 can enable or disable tracepoints. Amongst the possible conditions
2026 of an event rule is a filter expression which `liblttng-ust` evalutes
2027 when an event occurs.
2028** Share <<channel,channel>> attributes and ring buffer locations.
2029--
2030+
2031The session daemon and the user space tracing library use a Unix
2032domain socket for their communication.
2033
2034* The <<lttng-ust-agents,user space tracing agents>>.
2035+
2036Any instance of a user space tracing agent first registers to
2037a session daemon. Then, the session daemon can send requests to
2038this instance, such as:
2039+
2040--
2041** Get the list of loggers.
2042** Enable or disable a specific logger.
2043--
2044+
2045The session daemon and the user space tracing agent use a TCP connection
2046for their communication.
2047
2048* The <<lttng-modules,LTTng kernel tracer>>.
2049* The <<lttng-consumerd,consumer daemon>>.
2050+
2051The session daemon sends requests to the consumer daemon to instruct
2052it where to send the trace data streams, amongst other information.
2053
2054* The <<lttng-relayd,relay daemon>>.
2055
2056The session daemon receives commands from the
2057<<liblttng-ctl-lttng,tracing control library>>.
2058
2059The root session daemon loads the appropriate
2060<<lttng-modules,LTTng kernel modules>> on startup. It also spawns
2061a <<lttng-consumerd,consumer daemon>> as soon as you create
2062an <<event,event rule>>.
2063
2064The session daemon does not send and receive trace data: this is the
2065role of the <<lttng-consumerd,consumer daemon>> and
2066<<lttng-relayd,relay daemon>>. It does, however, generate the
2067http://diamon.org/ctf/[CTF] metadata stream.
2068
2069Each Unix user can have its own session daemon instance. The
2070tracing sessions managed by different session daemons are completely
2071independent.
2072
2073The root user's session daemon is the only one which is
2074allowed to control the LTTng kernel tracer, and its spawned consumer
2075daemon is the only one which is allowed to consume trace data from the
2076LTTng kernel tracer. Note, however, that any Unix user which is a member
2077of the <<tracing-group,tracing group>> is allowed
2078to create <<channel,channels>> in the
2079Linux kernel <<domain,tracing domain>>, and thus to trace the Linux
2080kernel.
2081
2082The <<lttng-cli,cmd:lttng command-line tool>> automatically starts a
2083session daemon when using its `create` command if none is currently
2084running. You can also start the session daemon manually.
2085
2086
2087[[lttng-consumerd]]
2088=== Consumer daemon
2089
2090[role="img-100"]
2091.The consumer daemon.
2092image::plumbing-consumerd.png[]
2093
a9f3997c 2094The _consumer daemon_, cmd:lttng-consumerd, is a daemon which shares
85c29972
PP
2095ring buffers with user applications or with the LTTng kernel modules to
2096collect trace data and send it to some location (on disk or to a
2097<<lttng-relayd,relay daemon>> over the network). The consumer daemon
2098is part of LTTng-tools.
2099
2100You do not start a consumer daemon manually: a consumer daemon is always
2101spawned by a <<lttng-sessiond,session daemon>> as soon as you create an
2102<<event,event rule>>, that is, before you start tracing. When you kill
2103its owner session daemon, the consumer daemon also exits because it is
2104the session daemon's child process. Command-line options of
2105man:lttng-sessiond(8) target the consumer daemon process.
2106
2107There are up to two running consumer daemons per Unix user, whereas only
2108one session daemon can run per user. This is because each process can be
2109either 32-bit or 64-bit: if the target system runs a mixture of 32-bit
2110and 64-bit processes, it is more efficient to have separate
2111corresponding 32-bit and 64-bit consumer daemons. The root user is an
2112exception: it can have up to _three_ running consumer daemons: 32-bit
2113and 64-bit instances for its user applications, and one more
2114reserved for collecting kernel trace data.
2115
2116
2117[[lttng-relayd]]
2118=== Relay daemon
2119
2120[role="img-100"]
2121.The relay daemon.
2122image::plumbing-relayd.png[]
2123
2124The _relay daemon_, man:lttng-relayd(8), is a daemon acting as a bridge
2125between remote session and consumer daemons, local trace files, and a
2126remote live trace viewer. The relay daemon is part of LTTng-tools.
2127
2128The main purpose of the relay daemon is to implement a receiver of
2129<<sending-trace-data-over-the-network,trace data over the network>>.
2130This is useful when the target system does not have much file system
2131space to record trace files locally.
2132
2133The relay daemon is also a server to which a
2134<<lttng-live,live trace viewer>> can
2135connect. The live trace viewer sends requests to the relay daemon to
2136receive trace data as the target system emits events. The
2137communication protocol is named _LTTng live_; it is used over TCP
2138connections.
2139
2140Note that you can start the relay daemon on the target system directly.
2141This is the setup of choice when the use case is to view events as
2142the target system emits them without the need of a remote system.
2143
2144
2145[[instrumenting]]
2146== [[using-lttng]]Instrumentation
2147
2148There are many examples of tracing and monitoring in our everyday life:
2149
2150* You have access to real-time and historical weather reports and
2151 forecasts thanks to weather stations installed around the country.
2152* You know your heart is safe thanks to an electrocardiogram.
2153* You make sure not to drive your car too fast and to have enough fuel
2154 to reach your destination thanks to gauges visible on your dashboard.
2155
2156All the previous examples have something in common: they rely on
2157**instruments**. Without the electrodes attached to the surface of your
2158body's skin, cardiac monitoring is futile.
2159
2160LTTng, as a tracer, is no different from those real life examples. If
2161you're about to trace a software system or, in other words, record its
2162history of execution, you better have **instrumentation points** in the
2163subject you're tracing, that is, the actual software.
2164
2165Various ways were developed to instrument a piece of software for LTTng
2166tracing. The most straightforward one is to manually place
2167instrumentation points, called _tracepoints_, in the software's source
2168code. It is also possible to add instrumentation points dynamically in
2169the Linux kernel <<domain,tracing domain>>.
2170
2171If you're only interested in tracing the Linux kernel, your
2172instrumentation needs are probably already covered by LTTng's built-in
2173<<lttng-modules,Linux kernel tracepoints>>. You may also wish to trace a
2174user application which is already instrumented for LTTng tracing.
2175In such cases, you can skip this whole section and read the topics of
2176the <<controlling-tracing,Tracing control>> section.
2177
2178Many methods are available to instrument a piece of software for LTTng
2179tracing. They are:
2180
2181* <<c-application,User space instrumentation for C and $$C++$$
2182 applications>>.
2183* <<prebuilt-ust-helpers,Prebuilt user space tracing helpers>>.
2184* <<java-application,User space Java agent>>.
2185* <<python-application,User space Python agent>>.
2186* <<proc-lttng-logger-abi,LTTng logger>>.
2187* <<instrumenting-linux-kernel,LTTng kernel tracepoints>>.
2188
2189
2190[[c-application]]
2191=== [[cxx-application]]User space instrumentation for C and $$C++$$ applications
2192
2193The procedure to instrument a C or $$C++$$ user application with
2194the <<lttng-ust,LTTng user space tracing library>>, `liblttng-ust`, is:
2195
2196. <<tracepoint-provider,Create the source files of a tracepoint provider
2197 package>>.
2198. <<probing-the-application-source-code,Add tracepoints to
2199 the application's source code>>.
2200. <<building-tracepoint-providers-and-user-application,Build and link
2201 a tracepoint provider package and the user application>>.
2202
2203If you need quick, man:printf(3)-like instrumentation, you can skip
2204those steps and use <<tracef,`tracef()`>> or <<tracelog,`tracelog()`>>
2205instead.
2206
2207IMPORTANT: You need to <<installing-lttng,install>> LTTng-UST to
2208instrument a user application with `liblttng-ust`.
2209
2210
2211[[tracepoint-provider]]
2212==== Create the source files of a tracepoint provider package
2213
2214A _tracepoint provider_ is a set of compiled functions which provide
2215**tracepoints** to an application, the type of instrumentation point
2216supported by LTTng-UST. Those functions can emit events with
2217user-defined fields and serialize those events as event records to one
2218or more LTTng-UST <<channel,channel>> sub-buffers. The `tracepoint()`
2219macro, which you <<probing-the-application-source-code,insert in a user
2220application's source code>>, calls those functions.
2221
2222A _tracepoint provider package_ is an object file (`.o`) or a shared
2223library (`.so`) which contains one or more tracepoint providers.
2224Its source files are:
2225
2226* One or more <<tpp-header,tracepoint provider header>> (`.h`).
2227* A <<tpp-source,tracepoint provider package source>> (`.c`).
2228
2229A tracepoint provider package is dynamically linked with `liblttng-ust`,
2230the LTTng user space tracer, at run time.
2231
2232[role="img-100"]
2233.User application linked with `liblttng-ust` and containing a tracepoint provider.
2234image::ust-app.png[]
2235
2236NOTE: If you need quick, man:printf(3)-like instrumentation, you can
2237skip creating and using a tracepoint provider and use
2238<<tracef,`tracef()`>> or <<tracelog,`tracelog()`>> instead.
2239
2240
2241[[tpp-header]]
2242===== Create a tracepoint provider header file template
2243
2244A _tracepoint provider header file_ contains the tracepoint
2245definitions of a tracepoint provider.
2246
2247To create a tracepoint provider header file:
2248
2249. Start from this template:
2250+
2251--
2252[source,c]
2253.Tracepoint provider header file template (`.h` file extension).
2254----
2255#undef TRACEPOINT_PROVIDER
2256#define TRACEPOINT_PROVIDER provider_name
2257
2258#undef TRACEPOINT_INCLUDE
2259#define TRACEPOINT_INCLUDE "./tp.h"
2260
2261#if !defined(_TP_H) || defined(TRACEPOINT_HEADER_MULTI_READ)
2262#define _TP_H
2263
2264#include <lttng/tracepoint.h>
2265
2266/*
2267 * Use TRACEPOINT_EVENT(), TRACEPOINT_EVENT_CLASS(),
2268 * TRACEPOINT_EVENT_INSTANCE(), and TRACEPOINT_LOGLEVEL() here.
2269 */
2270
2271#endif /* _TP_H */
2272
2273#include <lttng/tracepoint-event.h>
2274----
2275--
2276
2277. Replace:
2278+
2279* `provider_name` with the name of your tracepoint provider.
2280* `"tp.h"` with the name of your tracepoint provider header file.
2281
2282. Below the `#include <lttng/tracepoint.h>` line, put your
2283 <<defining-tracepoints,tracepoint definitions>>.
2284
2285Your tracepoint provider name must be unique amongst all the possible
2286tracepoint provider names used on the same target system. We
2287suggest to include the name of your project or company in the name,
2288for example, `org_lttng_my_project_tpp`.
2289
2290TIP: [[lttng-gen-tp]]You can use the man:lttng-gen-tp(1) tool to create
2291this boilerplate for you. When using cmd:lttng-gen-tp, all you need to
2292write are the <<defining-tracepoints,tracepoint definitions>>.
2293
2294
2295[[defining-tracepoints]]
2296===== Create a tracepoint definition
2297
2298A _tracepoint definition_ defines, for a given tracepoint:
2299
2300* Its **input arguments**. They are the macro parameters that the
2301 `tracepoint()` macro accepts for this particular tracepoint
2302 in the user application's source code.
2303* Its **output event fields**. They are the sources of event fields
2304 that form the payload of any event that the execution of the
2305 `tracepoint()` macro emits for this particular tracepoint.
2306
2307You can create a tracepoint definition by using the
2308`TRACEPOINT_EVENT()` macro below the `#include <lttng/tracepoint.h>`
2309line in the
2310<<tpp-header,tracepoint provider header file template>>.
2311
2312The syntax of the `TRACEPOINT_EVENT()` macro is:
2313
2314[source,c]
2315.`TRACEPOINT_EVENT()` macro syntax.
2316----
2317TRACEPOINT_EVENT(
2318 /* Tracepoint provider name */
2319 provider_name,
2320
2321 /* Tracepoint name */
2322 tracepoint_name,
2323
2324 /* Input arguments */
2325 TP_ARGS(
2326 arguments
2327 ),
2328
2329 /* Output event fields */
2330 TP_FIELDS(
2331 fields
2332 )
2333)
2334----
2335
2336Replace:
2337
2338* `provider_name` with your tracepoint provider name.
2339* `tracepoint_name` with your tracepoint name.
2340* `arguments` with the <<tpp-def-input-args,input arguments>>.
2341* `fields` with the <<tpp-def-output-fields,output event field>>
2342 definitions.
2343
2344This tracepoint emits events named `provider_name:tracepoint_name`.
2345
2346[IMPORTANT]
2347.Event name's length limitation
2348====
2349The concatenation of the tracepoint provider name and the
2350tracepoint name must not exceed **254 characters**. If it does, the
2351instrumented application compiles and runs, but LTTng throws multiple
2352warnings and you could experience serious issues.
2353====
2354
2355[[tpp-def-input-args]]The syntax of the `TP_ARGS()` macro is:
2356
2357[source,c]
2358.`TP_ARGS()` macro syntax.
2359----
2360TP_ARGS(
2361 type, arg_name
2362)
2363----
2364
2365Replace:
2366
2367* `type` with the C type of the argument.
2368* `arg_name` with the argument name.
2369
2370You can repeat `type` and `arg_name` up to 10 times to have
2371more than one argument.
2372
2373.`TP_ARGS()` usage with three arguments.
2374====
2375[source,c]
2376----
2377TP_ARGS(
2378 int, count,
2379 float, ratio,
2380 const char*, query
2381)
2382----
2383====
2384
2385The `TP_ARGS()` and `TP_ARGS(void)` forms are valid to create a
2386tracepoint definition with no input arguments.
2387
2388[[tpp-def-output-fields]]The `TP_FIELDS()` macro contains a list of
2389`ctf_*()` macros. Each `ctf_*()` macro defines one event field. See
2390man:lttng-ust(3) for a complete description of the available `ctf_*()`
2391macros. A `ctf_*()` macro specifies the type, size, and byte order of
2392one event field.
2393
2394Each `ctf_*()` macro takes an _argument expression_ parameter. This is a
2395C expression that the tracer evalutes at the `tracepoint()` macro site
2396in the application's source code. This expression provides a field's
2397source of data. The argument expression can include input argument names
2398listed in the `TP_ARGS()` macro.
2399
2400Each `ctf_*()` macro also takes a _field name_ parameter. Field names
2401must be unique within a given tracepoint definition.
2402
2403Here's a complete tracepoint definition example:
2404
2405.Tracepoint definition.
2406====
2407The following tracepoint definition defines a tracepoint which takes
2408three input arguments and has four output event fields.
2409
2410[source,c]
2411----
2412#include "my-custom-structure.h"
2413
2414TRACEPOINT_EVENT(
2415 my_provider,
2416 my_tracepoint,
2417 TP_ARGS(
2418 const struct my_custom_structure*, my_custom_structure,
2419 float, ratio,
2420 const char*, query
2421 ),
2422 TP_FIELDS(
2423 ctf_string(query_field, query)
2424 ctf_float(double, ratio_field, ratio)
2425 ctf_integer(int, recv_size, my_custom_structure->recv_size)
2426 ctf_integer(int, send_size, my_custom_structure->send_size)
2427 )
2428)
2429----
2430
2431You can refer to this tracepoint definition with the `tracepoint()`
2432macro in your application's source code like this:
2433
2434[source,c]
2435----
2436tracepoint(my_provider, my_tracepoint,
2437 my_structure, some_ratio, the_query);
2438----
2439====
2440
2441NOTE: The LTTng tracer only evaluates tracepoint arguments at run time
2442if they satisfy an enabled <<event,event rule>>.
2443
2444
2445[[using-tracepoint-classes]]
2446===== Use a tracepoint class
2447
2448A _tracepoint class_ is a class of tracepoints which share the same
2449output event field definitions. A _tracepoint instance_ is one
2450instance of such a defined tracepoint class, with its own tracepoint
2451name.
2452
2453The <<defining-tracepoints,`TRACEPOINT_EVENT()` macro>> is actually a
2454shorthand which defines both a tracepoint class and a tracepoint
2455instance at the same time.
2456
2457When you build a tracepoint provider package, the C or $$C++$$ compiler
2458creates one serialization function for each **tracepoint class**. A
2459serialization function is responsible for serializing the event fields
2460of a tracepoint to a sub-buffer when tracing.
2461
2462For various performance reasons, when your situation requires multiple
2463tracepoint definitions with different names, but with the same event
2464fields, we recommend that you manually create a tracepoint class
2465and instantiate as many tracepoint instances as needed. One positive
2466effect of such a design, amongst other advantages, is that all
2467tracepoint instances of the same tracepoint class reuse the same
2468serialization function, thus reducing
2469https://en.wikipedia.org/wiki/Cache_pollution[cache pollution].
2470
2471.Use a tracepoint class and tracepoint instances.
2472====
2473Consider the following three tracepoint definitions:
2474
2475[source,c]
2476----
2477TRACEPOINT_EVENT(
2478 my_app,
2479 get_account,
2480 TP_ARGS(
2481 int, userid,
2482 size_t, len
2483 ),
2484 TP_FIELDS(
2485 ctf_integer(int, userid, userid)
2486 ctf_integer(size_t, len, len)
2487 )
2488)
2489
2490TRACEPOINT_EVENT(
2491 my_app,
2492 get_settings,
2493 TP_ARGS(
2494 int, userid,
2495 size_t, len
2496 ),
2497 TP_FIELDS(
2498 ctf_integer(int, userid, userid)
2499 ctf_integer(size_t, len, len)
2500 )
2501)
2502
2503TRACEPOINT_EVENT(
2504 my_app,
2505 get_transaction,
2506 TP_ARGS(
2507 int, userid,
2508 size_t, len
2509 ),
2510 TP_FIELDS(
2511 ctf_integer(int, userid, userid)
2512 ctf_integer(size_t, len, len)
2513 )
2514)
2515----
2516
2517In this case, we create three tracepoint classes, with one implicit
2518tracepoint instance for each of them: `get_account`, `get_settings`, and
2519`get_transaction`. However, they all share the same event field names
2520and types. Hence three identical, yet independent serialization
2521functions are created when you build the tracepoint provider package.
2522
2523A better design choice is to define a single tracepoint class and three
2524tracepoint instances:
2525
2526[source,c]
2527----
2528/* The tracepoint class */
2529TRACEPOINT_EVENT_CLASS(
2530 /* Tracepoint provider name */
2531 my_app,
2532
2533 /* Tracepoint class name */
2534 my_class,
2535
2536 /* Input arguments */
2537 TP_ARGS(
2538 int, userid,
2539 size_t, len
2540 ),
2541
2542 /* Output event fields */
2543 TP_FIELDS(
2544 ctf_integer(int, userid, userid)
2545 ctf_integer(size_t, len, len)
2546 )
2547)
2548
2549/* The tracepoint instances */
2550TRACEPOINT_EVENT_INSTANCE(
2551 /* Tracepoint provider name */
2552 my_app,
2553
2554 /* Tracepoint class name */
2555 my_class,
2556
2557 /* Tracepoint name */
2558 get_account,
2559
2560 /* Input arguments */
2561 TP_ARGS(
2562 int, userid,
2563 size_t, len
2564 )
2565)
2566TRACEPOINT_EVENT_INSTANCE(
2567 my_app,
2568 my_class,
2569 get_settings,
2570 TP_ARGS(
2571 int, userid,
2572 size_t, len
2573 )
2574)
2575TRACEPOINT_EVENT_INSTANCE(
2576 my_app,
2577 my_class,
2578 get_transaction,
2579 TP_ARGS(
2580 int, userid,
2581 size_t, len
2582 )
2583)
2584----
2585====
2586
2587
2588[[assigning-log-levels]]
2589===== Assign a log level to a tracepoint definition
2590
2591You can assign an optional _log level_ to a
2592<<defining-tracepoints,tracepoint definition>>.
2593
2594Assigning different levels of severity to tracepoint definitions can
2595be useful: when you <<enabling-disabling-events,create an event rule>>,
2596you can target tracepoints having a log level as severe as a specific
2597value.
2598
2599The concept of LTTng-UST log levels is similar to the levels found
2600in typical logging frameworks:
2601
2602* In a logging framework, the log level is given by the function
2603 or method name you use at the log statement site: `debug()`,
2604 `info()`, `warn()`, `error()`, and so on.
2605* In LTTng-UST, you statically assign the log level to a tracepoint
2606 definition; any `tracepoint()` macro invocation which refers to
2607 this definition has this log level.
2608
2609You can assign a log level to a tracepoint definition with the
2610`TRACEPOINT_LOGLEVEL()` macro. You must use this macro _after_ the
2611<<defining-tracepoints,`TRACEPOINT_EVENT()`>> or
2612<<using-tracepoint-classes,`TRACEPOINT_INSTANCE()`>> macro for a given
2613tracepoint.
2614
2615The syntax of the `TRACEPOINT_LOGLEVEL()` macro is:
2616
2617[source,c]
2618.`TRACEPOINT_LOGLEVEL()` macro syntax.
2619----
2620TRACEPOINT_LOGLEVEL(provider_name, tracepoint_name, log_level)
2621----
2622
2623Replace:
2624
2625* `provider_name` with the tracepoint provider name.
2626* `tracepoint_name` with the tracepoint name.
2627* `log_level` with the log level to assign to the tracepoint
2628 definition named `tracepoint_name` in the `provider_name`
2629 tracepoint provider.
2630+
2631See man:lttng-ust(3) for a list of available log level names.
2632
2633.Assign the `TRACE_DEBUG_UNIT` log level to a tracepoint definition.
2634====
2635[source,c]
2636----
2637/* Tracepoint definition */
2638TRACEPOINT_EVENT(
2639 my_app,
2640 get_transaction,
2641 TP_ARGS(
2642 int, userid,
2643 size_t, len
2644 ),
2645 TP_FIELDS(
2646 ctf_integer(int, userid, userid)
2647 ctf_integer(size_t, len, len)
2648 )
2649)
2650
2651/* Log level assignment */
2652TRACEPOINT_LOGLEVEL(my_app, get_transaction, TRACE_DEBUG_UNIT)
2653----
2654====
2655
2656
2657[[tpp-source]]
2658===== Create a tracepoint provider package source file
2659
2660A _tracepoint provider package source file_ is a C source file which
2661includes a <<tpp-header,tracepoint provider header file>> to expand its
2662macros into event serialization and other functions.
2663
2664You can always use the following tracepoint provider package source
2665file template:
2666
2667[source,c]
2668.Tracepoint provider package source file template.
2669----
2670#define TRACEPOINT_CREATE_PROBES
2671
2672#include "tp.h"
2673----
2674
2675Replace `tp.h` with the name of your <<tpp-header,tracepoint provider
2676header file>> name. You may also include more than one tracepoint
2677provider header file here to create a tracepoint provider package
2678holding more than one tracepoint providers.
2679
2680
2681[[probing-the-application-source-code]]
2682==== Add tracepoints to an application's source code
2683
2684Once you <<tpp-header,create a tracepoint provider header file>>, you
2685can use the `tracepoint()` macro in your application's
2686source code to insert the tracepoints that this header
2687<<defining-tracepoints,defines>>.
2688
2689The `tracepoint()` macro takes at least two parameters: the tracepoint
2690provider name and the tracepoint name. The corresponding tracepoint
2691definition defines the other parameters.
2692
2693.`tracepoint()` usage.
2694====
2695The following <<defining-tracepoints,tracepoint definition>> defines a
2696tracepoint which takes two input arguments and has two output event
2697fields.
2698
2699[source,c]
2700.Tracepoint provider header file.
2701----
2702#include "my-custom-structure.h"
2703
2704TRACEPOINT_EVENT(
2705 my_provider,
2706 my_tracepoint,
2707 TP_ARGS(
2708 int, argc,
2709 const char*, cmd_name
2710 ),
2711 TP_FIELDS(
2712 ctf_string(cmd_name, cmd_name)
2713 ctf_integer(int, number_of_args, argc)
2714 )
2715)
2716----
2717
2718You can refer to this tracepoint definition with the `tracepoint()`
2719macro in your application's source code like this:
2720
2721[source,c]
2722.Application's source file.
2723----
2724#include "tp.h"
2725
2726int main(int argc, char* argv[])
2727{
2728 tracepoint(my_provider, my_tracepoint, argc, argv[0]);
2729
2730 return 0;
2731}
2732----
2733
2734Note how the application's source code includes
2735the tracepoint provider header file containing the tracepoint
2736definitions to use, path:{tp.h}.
2737====
2738
2739.`tracepoint()` usage with a complex tracepoint definition.
2740====
2741Consider this complex tracepoint definition, where multiple event
2742fields refer to the same input arguments in their argument expression
2743parameter:
2744
2745[source,c]
2746.Tracepoint provider header file.
2747----
2748/* For `struct stat` */
2749#include <sys/types.h>
2750#include <sys/stat.h>
2751#include <unistd.h>
2752
2753TRACEPOINT_EVENT(
2754 my_provider,
2755 my_tracepoint,
2756 TP_ARGS(
2757 int, my_int_arg,
2758 char*, my_str_arg,
2759 struct stat*, st
2760 ),
2761 TP_FIELDS(
2762 ctf_integer(int, my_constant_field, 23 + 17)
2763 ctf_integer(int, my_int_arg_field, my_int_arg)
2764 ctf_integer(int, my_int_arg_field2, my_int_arg * my_int_arg)
2765 ctf_integer(int, sum4_field, my_str_arg[0] + my_str_arg[1] +
2766 my_str_arg[2] + my_str_arg[3])
2767 ctf_string(my_str_arg_field, my_str_arg)
2768 ctf_integer_hex(off_t, size_field, st->st_size)
2769 ctf_float(double, size_dbl_field, (double) st->st_size)
2770 ctf_sequence_text(char, half_my_str_arg_field, my_str_arg,
2771 size_t, strlen(my_str_arg) / 2)
2772 )
2773)
2774----
2775
2776You can refer to this tracepoint definition with the `tracepoint()`
2777macro in your application's source code like this:
2778
2779[source,c]
2780.Application's source file.
2781----
2782#define TRACEPOINT_DEFINE
2783#include "tp.h"
2784
2785int main(void)
2786{
2787 struct stat s;
2788
2789 stat("/etc/fstab", &s);
2790 tracepoint(my_provider, my_tracepoint, 23, "Hello, World!", &s);
2791
2792 return 0;
2793}
2794----
2795
2796If you look at the event record that LTTng writes when tracing this
2797program, assuming the file size of path:{/etc/fstab} is 301{nbsp}bytes,
2798it should look like this:
2799
2800.Event record fields
2801|====
2802|Field's name |Field's value
2803|`my_constant_field` |40
2804|`my_int_arg_field` |23
2805|`my_int_arg_field2` |529
2806|`sum4_field` |389
2807|`my_str_arg_field` |`Hello, World!`
2808|`size_field` |0x12d
2809|`size_dbl_field` |301.0
2810|`half_my_str_arg_field` |`Hello,`
2811|====
2812====
2813
2814Sometimes, the arguments you pass to `tracepoint()` are expensive to
2815compute--they use the call stack, for example. To avoid this
2816computation when the tracepoint is disabled, you can use the
2817`tracepoint_enabled()` and `do_tracepoint()` macros.
2818
2819The syntax of the `tracepoint_enabled()` and `do_tracepoint()` macros
2820is:
2821
2822[source,c]
2823.`tracepoint_enabled()` and `do_tracepoint()` macros syntax.
2824----
2825tracepoint_enabled(provider_name, tracepoint_name)
2826do_tracepoint(provider_name, tracepoint_name, ...)
2827----
2828
2829Replace:
2830
2831* `provider_name` with the tracepoint provider name.
2832* `tracepoint_name` with the tracepoint name.
2833
2834`tracepoint_enabled()` returns a non-zero value if the tracepoint named
2835`tracepoint_name` from the provider named `provider_name` is enabled
2836**at run time**.
2837
2838`do_tracepoint()` is like `tracepoint()`, except that it doesn't check
2839if the tracepoint is enabled. Using `tracepoint()` with
2840`tracepoint_enabled()` is dangerous since `tracepoint()` also contains
2841the `tracepoint_enabled()` check, thus a race condition is
2842possible in this situation:
2843
2844[source,c]
2845.Possible race condition when using `tracepoint_enabled()` with `tracepoint()`.
2846----
2847if (tracepoint_enabled(my_provider, my_tracepoint)) {
2848 stuff = prepare_stuff();
2849}
2850
2851tracepoint(my_provider, my_tracepoint, stuff);
2852----
2853
2854If the tracepoint is enabled after the condition, then `stuff` is not
2855prepared: the emitted event will either contain wrong data, or the whole
2856application could crash (segmentation fault, for example).
2857
2858NOTE: Neither `tracepoint_enabled()` nor `do_tracepoint()` have an
2859`STAP_PROBEV()` call. If you need it, you must emit
2860this call yourself.
2861
2862
2863[[building-tracepoint-providers-and-user-application]]
2864==== Build and link a tracepoint provider package and an application
2865
2866Once you have one or more <<tpp-header,tracepoint provider header
2867files>> and a <<tpp-source,tracepoint provider package source file>>,
2868you can create the tracepoint provider package by compiling its source
2869file. From here, multiple build and run scenarios are possible. The
2870following table shows common application and library configurations
2871along with the required command lines to achieve them.
2872
2873In the following diagrams, we use the following file names:
2874
2875`app`::
2876 Executable application.
2877
2878`app.o`::
2879 Application's object file.
2880
2881`tpp.o`::
2882 Tracepoint provider package object file.
2883
2884`tpp.a`::
2885 Tracepoint provider package archive file.
2886
2887`libtpp.so`::
2888 Tracepoint provider package shared object file.
2889
2890`emon.o`::
2891 User library object file.
2892
2893`libemon.so`::
2894 User library shared object file.
2895
2896We use the following symbols in the diagrams of table below:
2897
2898[role="img-100"]
2899.Symbols used in the build scenario diagrams.
2900image::ust-sit-symbols.png[]
2901
2902We assume that path:{.} is part of the env:LD_LIBRARY_PATH environment
2903variable in the following instructions.
2904
2905[role="growable ust-scenarios",cols="asciidoc,asciidoc"]
2906.Common tracepoint provider package scenarios.
2907|====
2908|Scenario |Instructions
2909
2910|
2911The instrumented application is statically linked with
2912the tracepoint provider package object.
2913
2914image::ust-sit+app-linked-with-tp-o+app-instrumented.png[]
2915
2916|
2917include::../common/ust-sit-step-tp-o.txt[]
2918
2919To build the instrumented application:
2920
2921. In path:{app.c}, before including path:{tpp.h}, add the following line:
2922+
2923--
2924[source,c]
2925----
2926#define TRACEPOINT_DEFINE
2927----
2928--
2929
2930. Compile the application source file:
2931+
2932--
2933[role="term"]
2934----
2935$ gcc -c app.c
2936----
2937--
2938
2939. Build the application:
2940+
2941--
2942[role="term"]
2943----
2944$ gcc -o app app.o tpp.o -llttng-ust -ldl
2945----
2946--
2947
2948To run the instrumented application:
2949
2950* Start the application:
2951+
2952--
2953[role="term"]
2954----
2955$ ./app
2956----
2957--
2958
2959|
2960The instrumented application is statically linked with the
2961tracepoint provider package archive file.
2962
2963image::ust-sit+app-linked-with-tp-a+app-instrumented.png[]
2964
2965|
2966To create the tracepoint provider package archive file:
2967
2968. Compile the <<tpp-source,tracepoint provider package source file>>:
2969+
2970--
2971[role="term"]
2972----
2973$ gcc -I. -c tpp.c
2974----
2975--
2976
2977. Create the tracepoint provider package archive file:
2978+
2979--
2980[role="term"]
2981----
2982$ ar rcs tpp.a tpp.o
2983----
2984--
2985
2986To build the instrumented application:
2987
2988. In path:{app.c}, before including path:{tpp.h}, add the following line:
2989+
2990--
2991[source,c]
2992----
2993#define TRACEPOINT_DEFINE
2994----
2995--
2996
2997. Compile the application source file:
2998+
2999--
3000[role="term"]
3001----
3002$ gcc -c app.c
3003----
3004--
3005
3006. Build the application:
3007+
3008--
3009[role="term"]
3010----
3011$ gcc -o app app.o tpp.a -llttng-ust -ldl
3012----
3013--
3014
3015To run the instrumented application:
3016
3017* Start the application:
3018+
3019--
3020[role="term"]
3021----
3022$ ./app
3023----
3024--
3025
3026|
3027The instrumented application is linked with the tracepoint provider
3028package shared object.
3029
3030image::ust-sit+app-linked-with-tp-so+app-instrumented.png[]
3031
3032|
3033include::../common/ust-sit-step-tp-so.txt[]
3034
3035To build the instrumented application:
3036
3037. In path:{app.c}, before including path:{tpp.h}, add the following line:
3038+
3039--
3040[source,c]
3041----
3042#define TRACEPOINT_DEFINE
3043----
3044--
3045
3046. Compile the application source file:
3047+
3048--
3049[role="term"]
3050----
3051$ gcc -c app.c
3052----
3053--
3054
3055. Build the application:
3056+
3057--
3058[role="term"]
3059----
3060$ gcc -o app app.o -ldl -L. -ltpp
3061----
3062--
3063
3064To run the instrumented application:
3065
3066* Start the application:
3067+
3068--
3069[role="term"]
3070----
3071$ ./app
3072----
3073--
3074
3075|
3076The tracepoint provider package shared object is preloaded before the
3077instrumented application starts.
3078
3079image::ust-sit+tp-so-preloaded+app-instrumented.png[]
3080
3081|
3082include::../common/ust-sit-step-tp-so.txt[]
3083
3084To build the instrumented application:
3085
3086. In path:{app.c}, before including path:{tpp.h}, add the
3087 following lines:
3088+
3089--
3090[source,c]
3091----
3092#define TRACEPOINT_DEFINE
3093#define TRACEPOINT_PROBE_DYNAMIC_LINKAGE
3094----
3095--
3096
3097. Compile the application source file:
3098+
3099--
3100[role="term"]
3101----
3102$ gcc -c app.c
3103----
3104--
3105
3106. Build the application:
3107+
3108--
3109[role="term"]
3110----
3111$ gcc -o app app.o -ldl
3112----
3113--
3114
3115To run the instrumented application with tracing support:
3116
3117* Preload the tracepoint provider package shared object and
3118 start the application:
3119+
3120--
3121[role="term"]
3122----
3123$ LD_PRELOAD=./libtpp.so ./app
3124----
3125--
3126
3127To run the instrumented application without tracing support:
3128
3129* Start the application:
3130+
3131--
3132[role="term"]
3133----
3134$ ./app
3135----
3136--
3137
3138|
3139The instrumented application dynamically loads the tracepoint provider
3140package shared object.
3141
3142See the <<dlclose-warning,warning about `dlclose()`>>.
3143
3144image::ust-sit+app-dlopens-tp-so+app-instrumented.png[]
3145
3146|
3147include::../common/ust-sit-step-tp-so.txt[]
3148
3149To build the instrumented application:
3150
3151. In path:{app.c}, before including path:{tpp.h}, add the
3152 following lines:
3153+
3154--
3155[source,c]
3156----
3157#define TRACEPOINT_DEFINE
3158#define TRACEPOINT_PROBE_DYNAMIC_LINKAGE
3159----
3160--
3161
3162. Compile the application source file:
3163+
3164--
3165[role="term"]
3166----
3167$ gcc -c app.c
3168----
3169--
3170
3171. Build the application:
3172+
3173--
3174[role="term"]
3175----
3176$ gcc -o app app.o -ldl
3177----
3178--
3179
3180To run the instrumented application:
3181
3182* Start the application:
3183+
3184--
3185[role="term"]
3186----
3187$ ./app
3188----
3189--
3190
3191|
3192The application is linked with the instrumented user library.
3193
3194The instrumented user library is statically linked with the tracepoint
3195provider package object file.
3196
3197image::ust-sit+app-linked-with-lib+lib-linked-with-tp-o+lib-instrumented.png[]
3198
3199|
3200include::../common/ust-sit-step-tp-o-fpic.txt[]
3201
3202To build the instrumented user library:
3203
3204. In path:{emon.c}, before including path:{tpp.h}, add the
3205 following line:
3206+
3207--
3208[source,c]
3209----
3210#define TRACEPOINT_DEFINE
3211----
3212--
3213
3214. Compile the user library source file:
3215+
3216--
3217[role="term"]
3218----
3219$ gcc -I. -fpic -c emon.c
3220----
3221--
3222
3223. Build the user library shared object:
3224+
3225--
3226[role="term"]
3227----
3228$ gcc -shared -o libemon.so emon.o tpp.o -llttng-ust -ldl
3229----
3230--
3231
3232To build the application:
3233
3234. Compile the application source file:
3235+
3236--
3237[role="term"]
3238----
3239$ gcc -c app.c
3240----
3241--
3242
3243. Build the application:
3244+
3245--
3246[role="term"]
3247----
3248$ gcc -o app app.o -L. -lemon
3249----
3250--
3251
3252To run the application:
3253
3254* Start the application:
3255+
3256--
3257[role="term"]
3258----
3259$ ./app
3260----
3261--
3262
3263|
3264The application is linked with the instrumented user library.
3265
3266The instrumented user library is linked with the tracepoint provider
3267package shared object.
3268
3269image::ust-sit+app-linked-with-lib+lib-linked-with-tp-so+lib-instrumented.png[]
3270
3271|
3272include::../common/ust-sit-step-tp-so.txt[]
3273
3274To build the instrumented user library:
3275
3276. In path:{emon.c}, before including path:{tpp.h}, add the
3277 following line:
3278+
3279--
3280[source,c]
3281----
3282#define TRACEPOINT_DEFINE
3283----
3284--
3285
3286. Compile the user library source file:
3287+
3288--
3289[role="term"]
3290----
3291$ gcc -I. -fpic -c emon.c
3292----
3293--
3294
3295. Build the user library shared object:
3296+
3297--
3298[role="term"]
3299----
3300$ gcc -shared -o libemon.so emon.o -ldl -L. -ltpp
3301----
3302--
3303
3304To build the application:
3305
3306. Compile the application source file:
3307+
3308--
3309[role="term"]
3310----
3311$ gcc -c app.c
3312----
3313--
3314
3315. Build the application:
3316+
3317--
3318[role="term"]
3319----
3320$ gcc -o app app.o -L. -lemon
3321----
3322--
3323
3324To run the application:
3325
3326* Start the application:
3327+
3328--
3329[role="term"]
3330----
3331$ ./app
3332----
3333--
3334
3335|
3336The tracepoint provider package shared object is preloaded before the
3337application starts.
3338
3339The application is linked with the instrumented user library.
3340
3341image::ust-sit+tp-so-preloaded+app-linked-with-lib+lib-instrumented.png[]
3342
3343|
3344include::../common/ust-sit-step-tp-so.txt[]
3345
3346To build the instrumented user library:
3347
3348. In path:{emon.c}, before including path:{tpp.h}, add the
3349 following lines:
3350+
3351--
3352[source,c]
3353----
3354#define TRACEPOINT_DEFINE
3355#define TRACEPOINT_PROBE_DYNAMIC_LINKAGE
3356----
3357--
3358
3359. Compile the user library source file:
3360+
3361--
3362[role="term"]
3363----
3364$ gcc -I. -fpic -c emon.c
3365----
3366--
3367
3368. Build the user library shared object:
3369+
3370--
3371[role="term"]
3372----
3373$ gcc -shared -o libemon.so emon.o -ldl
3374----
3375--
3376
3377To build the application:
3378
3379. Compile the application source file:
3380+
3381--
3382[role="term"]
3383----
3384$ gcc -c app.c
3385----
3386--
3387
3388. Build the application:
3389+
3390--
3391[role="term"]
3392----
3393$ gcc -o app app.o -L. -lemon
3394----
3395--
3396
3397To run the application with tracing support:
3398
3399* Preload the tracepoint provider package shared object and
3400 start the application:
3401+
3402--
3403[role="term"]
3404----
3405$ LD_PRELOAD=./libtpp.so ./app
3406----
3407--
3408
3409To run the application without tracing support:
3410
3411* Start the application:
3412+
3413--
3414[role="term"]
3415----
3416$ ./app
3417----
3418--
3419
3420|
3421The application is linked with the instrumented user library.
3422
3423The instrumented user library dynamically loads the tracepoint provider
3424package shared object.
3425
3426See the <<dlclose-warning,warning about `dlclose()`>>.
3427
3428image::ust-sit+app-linked-with-lib+lib-dlopens-tp-so+lib-instrumented.png[]
3429
3430|
3431include::../common/ust-sit-step-tp-so.txt[]
3432
3433To build the instrumented user library:
3434
3435. In path:{emon.c}, before including path:{tpp.h}, add the
3436 following lines:
3437+
3438--
3439[source,c]
3440----
3441#define TRACEPOINT_DEFINE
3442#define TRACEPOINT_PROBE_DYNAMIC_LINKAGE
3443----
3444--
3445
3446. Compile the user library source file:
3447+
3448--
3449[role="term"]
3450----
3451$ gcc -I. -fpic -c emon.c
3452----
3453--
3454
3455. Build the user library shared object:
3456+
3457--
3458[role="term"]
3459----
3460$ gcc -shared -o libemon.so emon.o -ldl
3461----
3462--
3463
3464To build the application:
3465
3466. Compile the application source file:
3467+
3468--
3469[role="term"]
3470----
3471$ gcc -c app.c
3472----
3473--
3474
3475. Build the application:
3476+
3477--
3478[role="term"]
3479----
3480$ gcc -o app app.o -L. -lemon
3481----
3482--
3483
3484To run the application:
3485
3486* Start the application:
3487+
3488--
3489[role="term"]
3490----
3491$ ./app
3492----
3493--
3494
3495|
3496The application dynamically loads the instrumented user library.
3497
3498The instrumented user library is linked with the tracepoint provider
3499package shared object.
3500
3501See the <<dlclose-warning,warning about `dlclose()`>>.
3502
3503image::ust-sit+app-dlopens-lib+lib-linked-with-tp-so+lib-instrumented.png[]
3504
3505|
3506include::../common/ust-sit-step-tp-so.txt[]
3507
3508To build the instrumented user library:
3509
3510. In path:{emon.c}, before including path:{tpp.h}, add the
3511 following line:
3512+
3513--
3514[source,c]
3515----
3516#define TRACEPOINT_DEFINE
3517----
3518--
3519
3520. Compile the user library source file:
3521+
3522--
3523[role="term"]
3524----
3525$ gcc -I. -fpic -c emon.c
3526----
3527--
3528
3529. Build the user library shared object:
3530+
3531--
3532[role="term"]
3533----
3534$ gcc -shared -o libemon.so emon.o -ldl -L. -ltpp
3535----
3536--
3537
3538To build the application:
3539
3540. Compile the application source file:
3541+
3542--
3543[role="term"]
3544----
3545$ gcc -c app.c
3546----
3547--
3548
3549. Build the application:
3550+
3551--
3552[role="term"]
3553----
3554$ gcc -o app app.o -ldl -L. -lemon
3555----
3556--
3557
3558To run the application:
3559
3560* Start the application:
3561+
3562--
3563[role="term"]
3564----
3565$ ./app
3566----
3567--
3568
3569|
3570The application dynamically loads the instrumented user library.
3571
3572The instrumented user library dynamically loads the tracepoint provider
3573package shared object.
3574
3575See the <<dlclose-warning,warning about `dlclose()`>>.
3576
3577image::ust-sit+app-dlopens-lib+lib-dlopens-tp-so+lib-instrumented.png[]
3578
3579|
3580include::../common/ust-sit-step-tp-so.txt[]
3581
3582To build the instrumented user library:
3583
3584. In path:{emon.c}, before including path:{tpp.h}, add the
3585 following lines:
3586+
3587--
3588[source,c]
3589----
3590#define TRACEPOINT_DEFINE
3591#define TRACEPOINT_PROBE_DYNAMIC_LINKAGE
3592----
3593--
3594
3595. Compile the user library source file:
3596+
3597--
3598[role="term"]
3599----
3600$ gcc -I. -fpic -c emon.c
3601----
3602--
3603
3604. Build the user library shared object:
3605+
3606--
3607[role="term"]
3608----
3609$ gcc -shared -o libemon.so emon.o -ldl
3610----
3611--
3612
3613To build the application:
3614
3615. Compile the application source file:
3616+
3617--
3618[role="term"]
3619----
3620$ gcc -c app.c
3621----
3622--
3623
3624. Build the application:
3625+
3626--
3627[role="term"]
3628----
3629$ gcc -o app app.o -ldl -L. -lemon
3630----
3631--
3632
3633To run the application:
3634
3635* Start the application:
3636+
3637--
3638[role="term"]
3639----
3640$ ./app
3641----
3642--
3643
3644|
3645The tracepoint provider package shared object is preloaded before the
3646application starts.
3647
3648The application dynamically loads the instrumented user library.
3649
3650image::ust-sit+tp-so-preloaded+app-dlopens-lib+lib-instrumented.png[]
3651
3652|
3653include::../common/ust-sit-step-tp-so.txt[]
3654
3655To build the instrumented user library:
3656
3657. In path:{emon.c}, before including path:{tpp.h}, add the
3658 following lines:
3659+
3660--
3661[source,c]
3662----
3663#define TRACEPOINT_DEFINE
3664#define TRACEPOINT_PROBE_DYNAMIC_LINKAGE
3665----
3666--
3667
3668. Compile the user library source file:
3669+
3670--
3671[role="term"]
3672----
3673$ gcc -I. -fpic -c emon.c
3674----
3675--
3676
3677. Build the user library shared object:
3678+
3679--
3680[role="term"]
3681----
3682$ gcc -shared -o libemon.so emon.o -ldl
3683----
3684--
3685
3686To build the application:
3687
3688. Compile the application source file:
3689+
3690--
3691[role="term"]
3692----
3693$ gcc -c app.c
3694----
3695--
3696
3697. Build the application:
3698+
3699--
3700[role="term"]
3701----
3702$ gcc -o app app.o -L. -lemon
3703----
3704--
3705
3706To run the application with tracing support:
3707
3708* Preload the tracepoint provider package shared object and
3709 start the application:
3710+
3711--
3712[role="term"]
3713----
3714$ LD_PRELOAD=./libtpp.so ./app
3715----
3716--
3717
3718To run the application without tracing support:
3719
3720* Start the application:
3721+
3722--
3723[role="term"]
3724----
3725$ ./app
3726----
3727--
3728
3729|
3730The application is statically linked with the tracepoint provider
3731package object file.
3732
3733The application is linked with the instrumented user library.
3734
3735image::ust-sit+app-linked-with-tp-o+app-linked-with-lib+lib-instrumented.png[]
3736
3737|
3738include::../common/ust-sit-step-tp-o.txt[]
3739
3740To build the instrumented user library:
3741
3742. In path:{emon.c}, before including path:{tpp.h}, add the
3743 following line:
3744+
3745--
3746[source,c]
3747----
3748#define TRACEPOINT_DEFINE
3749----
3750--
3751
3752. Compile the user library source file:
3753+
3754--
3755[role="term"]
3756----
3757$ gcc -I. -fpic -c emon.c
3758----
3759--
3760
3761. Build the user library shared object:
3762+
3763--
3764[role="term"]
3765----
3766$ gcc -shared -o libemon.so emon.o
3767----
3768--
3769
3770To build the application:
3771
3772. Compile the application source file:
3773+
3774--
3775[role="term"]
3776----
3777$ gcc -c app.c
3778----
3779--
3780
3781. Build the application:
3782+
3783--
3784[role="term"]
3785----
3786$ gcc -o app app.o tpp.o -llttng-ust -ldl -L. -lemon
3787----
3788--
3789
3790To run the instrumented application:
3791
3792* Start the application:
3793+
3794--
3795[role="term"]
3796----
3797$ ./app
3798----
3799--
3800
3801|
3802The application is statically linked with the tracepoint provider
3803package object file.
3804
3805The application dynamically loads the instrumented user library.
3806
3807image::ust-sit+app-linked-with-tp-o+app-dlopens-lib+lib-instrumented.png[]
3808
3809|
3810include::../common/ust-sit-step-tp-o.txt[]
3811
3812To build the application:
3813
3814. In path:{app.c}, before including path:{tpp.h}, add the following line:
3815+
3816--
3817[source,c]
3818----
3819#define TRACEPOINT_DEFINE
3820----
3821--
3822
3823. Compile the application source file:
3824+
3825--
3826[role="term"]
3827----
3828$ gcc -c app.c
3829----
3830--
3831
3832. Build the application:
3833+
3834--
3835[role="term"]
3836----
3837$ gcc -Wl,--export-dynamic -o app app.o tpp.o \
3838 -llttng-ust -ldl
3839----
3840--
3841+
3842The `--export-dynamic` option passed to the linker is necessary for the
3843dynamically loaded library to ``see'' the tracepoint symbols defined in
3844the application.
3845
3846To build the instrumented user library:
3847
3848. Compile the user library source file:
3849+
3850--
3851[role="term"]
3852----
3853$ gcc -I. -fpic -c emon.c
3854----
3855--
3856
3857. Build the user library shared object:
3858+
3859--
3860[role="term"]
3861----
3862$ gcc -shared -o libemon.so emon.o
3863----
3864--
3865
3866To run the application:
3867
3868* Start the application:
3869+
3870--
3871[role="term"]
3872----
3873$ ./app
3874----
3875--
3876|====
3877
3878[[dlclose-warning]]
3879[IMPORTANT]
3880.Do not use man:dlclose(3) on a tracepoint provider package
3881====
3882Never use man:dlclose(3) on any shared object which:
3883
3884* Is linked with, statically or dynamically, a tracepoint provider
3885 package.
3886* Calls man:dlopen(3) itself to dynamically open a tracepoint provider
3887 package shared object.
3888
3889This is currently considered **unsafe** due to a lack of reference
3890counting from LTTng-UST to the shared object.
3891
3892A known workaround (available since glibc 2.2) is to use the
3893`RTLD_NODELETE` flag when calling man:dlopen(3) initially. This has the
3894effect of not unloading the loaded shared object, even if man:dlclose(3)
3895is called.
3896
3897You can also preload the tracepoint provider package shared object with
3898the env:LD_PRELOAD environment variable to overcome this limitation.
3899====
3900
3901
3902[[using-lttng-ust-with-daemons]]
3903===== Use noch:{LTTng-UST} with daemons
3904
3905If your instrumented application calls man:fork(2), man:clone(2),
3906or BSD's man:rfork(2), without a following man:exec(3)-family
3907system call, you must preload the path:{liblttng-ust-fork.so} shared
3908object when you start the application.
3909
3910[role="term"]
3911----
3912$ LD_PRELOAD=liblttng-ust-fork.so ./my-app
3913----
3914
3915If your tracepoint provider package is
3916a shared library which you also preload, you must put both
3917shared objects in env:LD_PRELOAD:
3918
3919[role="term"]
3920----
3921$ LD_PRELOAD=liblttng-ust-fork.so:/path/to/tp.so ./my-app
3922----
3923
3924
3925[role="since-2.9"]
3926[[liblttng-ust-fd]]
3927===== Use noch:{LTTng-UST} with applications which close file descriptors that don't belong to them
3928
3929If your instrumented application closes one or more file descriptors
3930which it did not open itself, you must preload the
3931path:{liblttng-ust-fd.so} shared object when you start the application:
3932
3933[role="term"]
3934----
3935$ LD_PRELOAD=liblttng-ust-fd.so ./my-app
3936----
3937
3938Typical use cases include closing all the file descriptors after
3939man:fork(2) or man:rfork(2) and buggy applications doing
3940``double closes''.
3941
3942
3943[[lttng-ust-pkg-config]]
3944===== Use noch:{pkg-config}
3945
3946On some distributions, LTTng-UST ships with a
3947https://www.freedesktop.org/wiki/Software/pkg-config/[pkg-config]
3948metadata file. If this is your case, then you can use cmd:pkg-config to
3949build an application on the command line:
3950
3951[role="term"]
3952----
3953$ gcc -o my-app my-app.o tp.o $(pkg-config --cflags --libs lttng-ust)
3954----
3955
3956
3957[[instrumenting-32-bit-app-on-64-bit-system]]
3958===== [[advanced-instrumenting-techniques]]Build a 32-bit instrumented application for a 64-bit target system
3959
3960In order to trace a 32-bit application running on a 64-bit system,
3961LTTng must use a dedicated 32-bit
3962<<lttng-consumerd,consumer daemon>>.
3963
3964The following steps show how to build and install a 32-bit consumer
3965daemon, which is _not_ part of the default 64-bit LTTng build, how to
3966build and install the 32-bit LTTng-UST libraries, and how to build and
3967link an instrumented 32-bit application in that context.
3968
3969To build a 32-bit instrumented application for a 64-bit target system,
3970assuming you have a fresh target system with no installed Userspace RCU
3971or LTTng packages:
3972
3973. Download, build, and install a 32-bit version of Userspace RCU:
3974+
3975--
3976[role="term"]
3977----
3978$ cd $(mktemp -d) &&
3979wget http://lttng.org/files/urcu/userspace-rcu-latest-0.9.tar.bz2 &&
3980tar -xf userspace-rcu-latest-0.9.tar.bz2 &&
3981cd userspace-rcu-0.9.* &&
3982./configure --libdir=/usr/local/lib32 CFLAGS=-m32 &&
3983make &&
3984sudo make install &&
3985sudo ldconfig
3986----
3987--
3988
3989. Using your distribution's package manager, or from source, install
3990 the following 32-bit versions of the following dependencies of
3991 LTTng-tools and LTTng-UST:
3992+
3993--
3994* https://sourceforge.net/projects/libuuid/[libuuid]
3995* http://directory.fsf.org/wiki/Popt[popt]
3996* http://www.xmlsoft.org/[libxml2]
3997--
3998
3999. Download, build, and install a 32-bit version of the latest
4000 LTTng-UST{nbsp}{revision}:
4001+
4002--
4003[role="term"]
4004----
4005$ cd $(mktemp -d) &&
4006wget http://lttng.org/files/lttng-ust/lttng-ust-latest-2.10.tar.bz2 &&
4007tar -xf lttng-ust-latest-2.10.tar.bz2 &&
4008cd lttng-ust-2.10.* &&
4009./configure --libdir=/usr/local/lib32 \
4010 CFLAGS=-m32 CXXFLAGS=-m32 \
4011 LDFLAGS='-L/usr/local/lib32 -L/usr/lib32' &&
4012make &&
4013sudo make install &&
4014sudo ldconfig
4015----
4016--
4017+
4018[NOTE]
4019====
4020Depending on your distribution,
402132-bit libraries could be installed at a different location than
4022`/usr/lib32`. For example, Debian is known to install
4023some 32-bit libraries in `/usr/lib/i386-linux-gnu`.
4024
4025In this case, make sure to set `LDFLAGS` to all the
4026relevant 32-bit library paths, for example:
4027
4028[role="term"]
4029----
4030$ LDFLAGS='-L/usr/lib/i386-linux-gnu -L/usr/lib32'
4031----
4032====
4033
4034. Download the latest LTTng-tools{nbsp}{revision}, build, and install
4035 the 32-bit consumer daemon:
4036+
4037--
4038[role="term"]
4039----
4040$ cd $(mktemp -d) &&
4041wget http://lttng.org/files/lttng-tools/lttng-tools-latest-2.10.tar.bz2 &&
4042tar -xf lttng-tools-latest-2.10.tar.bz2 &&
4043cd lttng-tools-2.10.* &&
4044./configure --libdir=/usr/local/lib32 CFLAGS=-m32 CXXFLAGS=-m32 \
4045 LDFLAGS='-L/usr/local/lib32 -L/usr/lib32' \
4046 --disable-bin-lttng --disable-bin-lttng-crash \
4047 --disable-bin-lttng-relayd --disable-bin-lttng-sessiond &&
4048make &&
4049cd src/bin/lttng-consumerd &&
4050sudo make install &&
4051sudo ldconfig
4052----
4053--
4054
4055. From your distribution or from source,
4056 <<installing-lttng,install>> the 64-bit versions of
4057 LTTng-UST and Userspace RCU.
4058. Download, build, and install the 64-bit version of the
4059 latest LTTng-tools{nbsp}{revision}:
4060+
4061--
4062[role="term"]
4063----
4064$ cd $(mktemp -d) &&
4065wget http://lttng.org/files/lttng-tools/lttng-tools-latest-2.10.tar.bz2 &&
4066tar -xf lttng-tools-latest-2.10.tar.bz2 &&
4067cd lttng-tools-2.10.* &&
4068./configure --with-consumerd32-libdir=/usr/local/lib32 \
4069 --with-consumerd32-bin=/usr/local/lib32/lttng/libexec/lttng-consumerd &&
4070make &&
4071sudo make install &&
4072sudo ldconfig
4073----
4074--
4075
4076. Pass the following options to man:gcc(1), man:g++(1), or man:clang(1)
4077 when linking your 32-bit application:
4078+
4079----
4080-m32 -L/usr/lib32 -L/usr/local/lib32 \
4081-Wl,-rpath,/usr/lib32,-rpath,/usr/local/lib32
4082----
4083+
4084For example, let's rebuild the quick start example in
4085<<tracing-your-own-user-application,Trace a user application>> as an
4086instrumented 32-bit application:
4087+
4088--
4089[role="term"]
4090----
4091$ gcc -m32 -c -I. hello-tp.c
4092$ gcc -m32 -c hello.c
4093$ gcc -m32 -o hello hello.o hello-tp.o \
4094 -L/usr/lib32 -L/usr/local/lib32 \
4095 -Wl,-rpath,/usr/lib32,-rpath,/usr/local/lib32 \
4096 -llttng-ust -ldl
4097----
4098--
4099
4100No special action is required to execute the 32-bit application and
4101to trace it: use the command-line man:lttng(1) tool as usual.
4102
4103
4104[role="since-2.5"]
4105[[tracef]]
4106==== Use `tracef()`
4107
4108man:tracef(3) is a small LTTng-UST API designed for quick,
4109man:printf(3)-like instrumentation without the burden of
4110<<tracepoint-provider,creating>> and
4111<<building-tracepoint-providers-and-user-application,building>>
4112a tracepoint provider package.
4113
4114To use `tracef()` in your application:
4115
4116. In the C or C++ source files where you need to use `tracef()`,
4117 include `<lttng/tracef.h>`:
4118+
4119--
4120[source,c]
4121----
4122#include <lttng/tracef.h>
4123----
4124--
4125
4126. In the application's source code, use `tracef()` like you would use
4127 man:printf(3):
4128+
4129--
4130[source,c]
4131----
4132 /* ... */
4133
4134 tracef("my message: %d (%s)", my_integer, my_string);
4135
4136 /* ... */
4137----
4138--
4139
4140. Link your application with `liblttng-ust`:
4141+
4142--
4143[role="term"]
4144----
4145$ gcc -o app app.c -llttng-ust
4146----
4147--
4148
4149To trace the events that `tracef()` calls emit:
4150
4151* <<enabling-disabling-events,Create an event rule>> which matches the
4152 `lttng_ust_tracef:*` event name:
4153+
4154--
4155[role="term"]
4156----
4157$ lttng enable-event --userspace 'lttng_ust_tracef:*'
4158----
4159--
4160
4161[IMPORTANT]
4162.Limitations of `tracef()`
4163====
4164The `tracef()` utility function was developed to make user space tracing
4165super simple, albeit with notable disadvantages compared to
4166<<defining-tracepoints,user-defined tracepoints>>:
4167
4168* All the emitted events have the same tracepoint provider and
4169 tracepoint names, respectively `lttng_ust_tracef` and `event`.
4170* There is no static type checking.
4171* The only event record field you actually get, named `msg`, is a string
4172 potentially containing the values you passed to `tracef()`
4173 using your own format string. This also means that you cannot filter
4174 events with a custom expression at run time because there are no
4175 isolated fields.
4176* Since `tracef()` uses the C standard library's man:vasprintf(3)
4177 function behind the scenes to format the strings at run time, its
4178 expected performance is lower than with user-defined tracepoints,
4179 which do not require a conversion to a string.
4180
4181Taking this into consideration, `tracef()` is useful for some quick
4182prototyping and debugging, but you should not consider it for any
4183permanent and serious applicative instrumentation.
4184====
4185
4186
4187[role="since-2.7"]
4188[[tracelog]]
4189==== Use `tracelog()`
4190
4191The man:tracelog(3) API is very similar to <<tracef,`tracef()`>>, with
4192the difference that it accepts an additional log level parameter.
4193
4194The goal of `tracelog()` is to ease the migration from logging to
4195tracing.
4196
4197To use `tracelog()` in your application:
4198
4199. In the C or C++ source files where you need to use `tracelog()`,
4200 include `<lttng/tracelog.h>`:
4201+
4202--
4203[source,c]
4204----
4205#include <lttng/tracelog.h>
4206----
4207--
4208
4209. In the application's source code, use `tracelog()` like you would use
4210 man:printf(3), except for the first parameter which is the log
4211 level:
4212+
4213--
4214[source,c]
4215----
4216 /* ... */
4217
4218 tracelog(TRACE_WARNING, "my message: %d (%s)",
4219 my_integer, my_string);
4220
4221 /* ... */
4222----
4223--
4224+
4225See man:lttng-ust(3) for a list of available log level names.
4226
4227. Link your application with `liblttng-ust`:
4228+
4229--
4230[role="term"]
4231----
4232$ gcc -o app app.c -llttng-ust
4233----
4234--
4235
4236To trace the events that `tracelog()` calls emit with a log level
4237_as severe as_ a specific log level:
4238
4239* <<enabling-disabling-events,Create an event rule>> which matches the
4240 `lttng_ust_tracelog:*` event name and a minimum level
4241 of severity:
4242+
4243--
4244[role="term"]
4245----
4246$ lttng enable-event --userspace 'lttng_ust_tracelog:*'
4247 --loglevel=TRACE_WARNING
4248----
4249--
4250
4251To trace the events that `tracelog()` calls emit with a
4252_specific log level_:
4253
4254* Create an event rule which matches the `lttng_ust_tracelog:*`
4255 event name and a specific log level:
4256+
4257--
4258[role="term"]
4259----
4260$ lttng enable-event --userspace 'lttng_ust_tracelog:*'
4261 --loglevel-only=TRACE_INFO
4262----
4263--
4264
4265
4266[[prebuilt-ust-helpers]]
4267=== Prebuilt user space tracing helpers
4268
4269The LTTng-UST package provides a few helpers in the form or preloadable
4270shared objects which automatically instrument system functions and
4271calls.
4272
4273The helper shared objects are normally found in dir:{/usr/lib}. If you
4274built LTTng-UST <<building-from-source,from source>>, they are probably
4275located in dir:{/usr/local/lib}.
4276
4277The installed user space tracing helpers in LTTng-UST{nbsp}{revision}
4278are:
4279
4280path:{liblttng-ust-libc-wrapper.so}::
4281path:{liblttng-ust-pthread-wrapper.so}::
4282 <<liblttng-ust-libc-pthread-wrapper,C{nbsp}standard library
4283 memory and POSIX threads function tracing>>.
4284
4285path:{liblttng-ust-cyg-profile.so}::
4286path:{liblttng-ust-cyg-profile-fast.so}::
4287 <<liblttng-ust-cyg-profile,Function entry and exit tracing>>.
4288
4289path:{liblttng-ust-dl.so}::
4290 <<liblttng-ust-dl,Dynamic linker tracing>>.
4291
4292To use a user space tracing helper with any user application:
4293
4294* Preload the helper shared object when you start the application:
4295+
4296--
4297[role="term"]
4298----
4299$ LD_PRELOAD=liblttng-ust-libc-wrapper.so my-app
4300----
4301--
4302+
4303You can preload more than one helper:
4304+
4305--
4306[role="term"]
4307----
4308$ LD_PRELOAD=liblttng-ust-libc-wrapper.so:liblttng-ust-dl.so my-app
4309----
4310--
4311
4312
4313[role="since-2.3"]
4314[[liblttng-ust-libc-pthread-wrapper]]
4315==== Instrument C standard library memory and POSIX threads functions
4316
4317The path:{liblttng-ust-libc-wrapper.so} and
4318path:{liblttng-ust-pthread-wrapper.so} helpers
4319add instrumentation to some C standard library and POSIX
4320threads functions.
4321
4322[role="growable"]
4323.Functions instrumented by preloading path:{liblttng-ust-libc-wrapper.so}.
4324|====
4325|TP provider name |TP name |Instrumented function
4326
4327.6+|`lttng_ust_libc` |`malloc` |man:malloc(3)
4328 |`calloc` |man:calloc(3)
4329 |`realloc` |man:realloc(3)
4330 |`free` |man:free(3)
4331 |`memalign` |man:memalign(3)
4332 |`posix_memalign` |man:posix_memalign(3)
4333|====
4334
4335[role="growable"]
4336.Functions instrumented by preloading path:{liblttng-ust-pthread-wrapper.so}.
4337|====
4338|TP provider name |TP name |Instrumented function
4339
4340.4+|`lttng_ust_pthread` |`pthread_mutex_lock_req` |man:pthread_mutex_lock(3p) (request time)
4341 |`pthread_mutex_lock_acq` |man:pthread_mutex_lock(3p) (acquire time)
4342 |`pthread_mutex_trylock` |man:pthread_mutex_trylock(3p)
4343 |`pthread_mutex_unlock` |man:pthread_mutex_unlock(3p)
4344|====
4345
4346When you preload the shared object, it replaces the functions listed
4347in the previous tables by wrappers which contain tracepoints and call
4348the replaced functions.
4349
4350
4351[[liblttng-ust-cyg-profile]]
4352==== Instrument function entry and exit
4353
4354The path:{liblttng-ust-cyg-profile*.so} helpers can add instrumentation
4355to the entry and exit points of functions.
4356
4357man:gcc(1) and man:clang(1) have an option named
4358https://gcc.gnu.org/onlinedocs/gcc/Instrumentation-Options.html[`-finstrument-functions`]
4359which generates instrumentation calls for entry and exit to functions.
4360The LTTng-UST function tracing helpers,
4361path:{liblttng-ust-cyg-profile.so} and
4362path:{liblttng-ust-cyg-profile-fast.so}, take advantage of this feature
4363to add tracepoints to the two generated functions (which contain
4364`cyg_profile` in their names, hence the helper's name).
4365
4366To use the LTTng-UST function tracing helper, the source files to
4367instrument must be built using the `-finstrument-functions` compiler
4368flag.
4369
4370There are two versions of the LTTng-UST function tracing helper:
4371
4372* **path:{liblttng-ust-cyg-profile-fast.so}** is a lightweight variant
4373 that you should only use when it can be _guaranteed_ that the
4374 complete event stream is recorded without any lost event record.
4375 Any kind of duplicate information is left out.
4376+
4377Assuming no event record is lost, having only the function addresses on
4378entry is enough to create a call graph, since an event record always
4379contains the ID of the CPU that generated it.
4380+
4381You can use a tool like man:addr2line(1) to convert function addresses
4382back to source file names and line numbers.
4383
4384* **path:{liblttng-ust-cyg-profile.so}** is a more robust variant
4385which also works in use cases where event records might get discarded or
4386not recorded from application startup.
4387In these cases, the trace analyzer needs more information to be
4388able to reconstruct the program flow.
4389
4390See man:lttng-ust-cyg-profile(3) to learn more about the instrumentation
4391points of this helper.
4392
4393All the tracepoints that this helper provides have the
4394log level `TRACE_DEBUG_FUNCTION` (see man:lttng-ust(3)).
4395
4396TIP: It's sometimes a good idea to limit the number of source files that
4397you compile with the `-finstrument-functions` option to prevent LTTng
4398from writing an excessive amount of trace data at run time. When using
4399man:gcc(1), you can use the
4400`-finstrument-functions-exclude-function-list` option to avoid
4401instrument entries and exits of specific function names.
4402
4403
4404[role="since-2.4"]
4405[[liblttng-ust-dl]]
4406==== Instrument the dynamic linker
4407
4408The path:{liblttng-ust-dl.so} helper adds instrumentation to the
4409man:dlopen(3) and man:dlclose(3) function calls.
4410
4411See man:lttng-ust-dl(3) to learn more about the instrumentation points
4412of this helper.
4413
4414
4415[role="since-2.4"]
4416[[java-application]]
4417=== User space Java agent
4418
4419You can instrument any Java application which uses one of the following
4420logging frameworks:
4421
4422* The https://docs.oracle.com/javase/7/docs/api/java/util/logging/package-summary.html[**`java.util.logging`**]
4423 (JUL) core logging facilities.
4424* http://logging.apache.org/log4j/1.2/[**Apache log4j 1.2**], since
4425 LTTng 2.6. Note that Apache Log4j{nbsp}2 is not supported.
4426
4427[role="img-100"]
4428.LTTng-UST Java agent imported by a Java application.
4429image::java-app.png[]
4430
4431Note that the methods described below are new in LTTng{nbsp}{revision}.
4432Previous LTTng versions use another technique.
4433
4434NOTE: We use http://openjdk.java.net/[OpenJDK]{nbsp}8 for development
4435and https://ci.lttng.org/[continuous integration], thus this version is
4436directly supported. However, the LTTng-UST Java agent is also tested
4437with OpenJDK{nbsp}7.
4438
4439
4440[role="since-2.8"]
4441[[jul]]
4442==== Use the LTTng-UST Java agent for `java.util.logging`
4443
4444To use the LTTng-UST Java agent in a Java application which uses
4445`java.util.logging` (JUL):
4446
4447. In the Java application's source code, import the LTTng-UST
4448 log handler package for `java.util.logging`:
4449+
4450--
4451[source,java]
4452----
4453import org.lttng.ust.agent.jul.LttngLogHandler;
4454----
4455--
4456
4457. Create an LTTng-UST JUL log handler:
4458+
4459--
4460[source,java]
4461----
4462Handler lttngUstLogHandler = new LttngLogHandler();
4463----
4464--
4465
4466. Add this handler to the JUL loggers which should emit LTTng events:
4467+
4468--
4469[source,java]
4470----
4471Logger myLogger = Logger.getLogger("some-logger");
4472
4473myLogger.addHandler(lttngUstLogHandler);
4474----
4475--
4476
4477. Use `java.util.logging` log statements and configuration as usual.
4478 The loggers with an attached LTTng-UST log handler can emit
4479 LTTng events.
4480
4481. Before exiting the application, remove the LTTng-UST log handler from
4482 the loggers attached to it and call its `close()` method:
4483+
4484--
4485[source,java]
4486----
4487myLogger.removeHandler(lttngUstLogHandler);
4488lttngUstLogHandler.close();
4489----
4490--
4491+
4492This is not strictly necessary, but it is recommended for a clean
4493disposal of the handler's resources.
4494
4495. Include the LTTng-UST Java agent's common and JUL-specific JAR files,
4496 path:{lttng-ust-agent-common.jar} and path:{lttng-ust-agent-jul.jar},
4497 in the
4498 https://docs.oracle.com/javase/tutorial/essential/environment/paths.html[class
4499 path] when you build the Java application.
4500+
4501The JAR files are typically located in dir:{/usr/share/java}.
4502+
4503IMPORTANT: The LTTng-UST Java agent must be
4504<<installing-lttng,installed>> for the logging framework your
4505application uses.
4506
4507.Use the LTTng-UST Java agent for `java.util.logging`.
4508====
4509[source,java]
4510.path:{Test.java}
4511----
4512import java.io.IOException;
4513import java.util.logging.Handler;
4514import java.util.logging.Logger;
4515import org.lttng.ust.agent.jul.LttngLogHandler;
4516
4517public class Test
4518{
4519 private static final int answer = 42;
4520
4521 public static void main(String[] argv) throws Exception
4522 {
4523 // Create a logger
4524 Logger logger = Logger.getLogger("jello");
4525
4526 // Create an LTTng-UST log handler
4527 Handler lttngUstLogHandler = new LttngLogHandler();
4528
4529 // Add the LTTng-UST log handler to our logger
4530 logger.addHandler(lttngUstLogHandler);
4531
4532 // Log at will!
4533 logger.info("some info");
4534 logger.warning("some warning");
4535 Thread.sleep(500);
4536 logger.finer("finer information; the answer is " + answer);
4537 Thread.sleep(123);
4538 logger.severe("error!");
4539
4540 // Not mandatory, but cleaner
4541 logger.removeHandler(lttngUstLogHandler);
4542 lttngUstLogHandler.close();
4543 }
4544}
4545----
4546
4547Build this example:
4548
4549[role="term"]
4550----
4551$ javac -cp /usr/share/java/jarpath/lttng-ust-agent-common.jar:/usr/share/java/jarpath/lttng-ust-agent-jul.jar Test.java
4552----
4553
4554<<creating-destroying-tracing-sessions,Create a tracing session>>,
4555<<enabling-disabling-events,create an event rule>> matching the
4556`jello` JUL logger, and <<basic-tracing-session-control,start tracing>>:
4557
4558[role="term"]
4559----
4560$ lttng create
4561$ lttng enable-event --jul jello
4562$ lttng start
4563----
4564
4565Run the compiled class:
4566
4567[role="term"]
4568----
4569$ java -cp /usr/share/java/jarpath/lttng-ust-agent-common.jar:/usr/share/java/jarpath/lttng-ust-agent-jul.jar:. Test
4570----
4571
4572<<basic-tracing-session-control,Stop tracing>> and inspect the
4573recorded events:
4574
4575[role="term"]
4576----
4577$ lttng stop
4578$ lttng view
4579----
4580====
4581
4582In the resulting trace, an <<event,event record>> generated by a Java
4583application using `java.util.logging` is named `lttng_jul:event` and
4584has the following fields:
4585
4586`msg`::
4587 Log record's message.
4588
4589`logger_name`::
4590 Logger name.
4591
4592`class_name`::
4593 Name of the class in which the log statement was executed.
4594
4595`method_name`::
4596 Name of the method in which the log statement was executed.
4597
4598`long_millis`::
4599 Logging time (timestamp in milliseconds).
4600
4601`int_loglevel`::
4602 Log level integer value.
4603
4604`int_threadid`::
4605 ID of the thread in which the log statement was executed.
4606
4607You can use the opt:lttng-enable-event(1):--loglevel or
4608opt:lttng-enable-event(1):--loglevel-only option of the
4609man:lttng-enable-event(1) command to target a range of JUL log levels
4610or a specific JUL log level.
4611
4612
4613[role="since-2.8"]
4614[[log4j]]
4615==== Use the LTTng-UST Java agent for Apache log4j
4616
4617To use the LTTng-UST Java agent in a Java application which uses
4618Apache log4j 1.2:
4619
4620. In the Java application's source code, import the LTTng-UST
4621 log appender package for Apache log4j:
4622+
4623--
4624[source,java]
4625----
4626import org.lttng.ust.agent.log4j.LttngLogAppender;
4627----
4628--
4629
4630. Create an LTTng-UST log4j log appender:
4631+
4632--
4633[source,java]
4634----
4635Appender lttngUstLogAppender = new LttngLogAppender();
4636----
4637--
4638
4639. Add this appender to the log4j loggers which should emit LTTng events:
4640+
4641--
4642[source,java]
4643----
4644Logger myLogger = Logger.getLogger("some-logger");
4645
4646myLogger.addAppender(lttngUstLogAppender);
4647----
4648--
4649
4650. Use Apache log4j log statements and configuration as usual. The
4651 loggers with an attached LTTng-UST log appender can emit LTTng events.
4652
4653. Before exiting the application, remove the LTTng-UST log appender from
4654 the loggers attached to it and call its `close()` method:
4655+
4656--
4657[source,java]
4658----
4659myLogger.removeAppender(lttngUstLogAppender);
4660lttngUstLogAppender.close();
4661----
4662--
4663+
4664This is not strictly necessary, but it is recommended for a clean
4665disposal of the appender's resources.
4666
4667. Include the LTTng-UST Java agent's common and log4j-specific JAR
4668 files, path:{lttng-ust-agent-common.jar} and
4669 path:{lttng-ust-agent-log4j.jar}, in the
4670 https://docs.oracle.com/javase/tutorial/essential/environment/paths.html[class
4671 path] when you build the Java application.
4672+
4673The JAR files are typically located in dir:{/usr/share/java}.
4674+
4675IMPORTANT: The LTTng-UST Java agent must be
4676<<installing-lttng,installed>> for the logging framework your
4677application uses.
4678
4679.Use the LTTng-UST Java agent for Apache log4j.
4680====
4681[source,java]
4682.path:{Test.java}
4683----
4684import org.apache.log4j.Appender;
4685import org.apache.log4j.Logger;
4686import org.lttng.ust.agent.log4j.LttngLogAppender;
4687
4688public class Test
4689{
4690 private static final int answer = 42;
4691
4692 public static void main(String[] argv) throws Exception
4693 {
4694 // Create a logger
4695 Logger logger = Logger.getLogger("jello");
4696
4697 // Create an LTTng-UST log appender
4698 Appender lttngUstLogAppender = new LttngLogAppender();
4699
4700 // Add the LTTng-UST log appender to our logger
4701 logger.addAppender(lttngUstLogAppender);
4702
4703 // Log at will!
4704 logger.info("some info");
4705 logger.warn("some warning");
4706 Thread.sleep(500);
4707 logger.debug("debug information; the answer is " + answer);
4708 Thread.sleep(123);
4709 logger.fatal("error!");
4710
4711 // Not mandatory, but cleaner
4712 logger.removeAppender(lttngUstLogAppender);
4713 lttngUstLogAppender.close();
4714 }
4715}
4716
4717----
4718
4719Build this example (`$LOG4JPATH` is the path to the Apache log4j JAR
4720file):
4721
4722[role="term"]
4723----
4724$ javac -cp /usr/share/java/jarpath/lttng-ust-agent-common.jar:/usr/share/java/jarpath/lttng-ust-agent-log4j.jar:$LOG4JPATH Test.java
4725----
4726
4727<<creating-destroying-tracing-sessions,Create a tracing session>>,
4728<<enabling-disabling-events,create an event rule>> matching the
4729`jello` log4j logger, and <<basic-tracing-session-control,start tracing>>:
4730
4731[role="term"]
4732----
4733$ lttng create
4734$ lttng enable-event --log4j jello
4735$ lttng start
4736----
4737
4738Run the compiled class:
4739
4740[role="term"]
4741----
4742$ java -cp /usr/share/java/jarpath/lttng-ust-agent-common.jar:/usr/share/java/jarpath/lttng-ust-agent-log4j.jar:$LOG4JPATH:. Test
4743----
4744
4745<<basic-tracing-session-control,Stop tracing>> and inspect the
4746recorded events:
4747
4748[role="term"]
4749----
4750$ lttng stop
4751$ lttng view
4752----
4753====
4754
4755In the resulting trace, an <<event,event record>> generated by a Java
4756application using log4j is named `lttng_log4j:event` and
4757has the following fields:
4758
4759`msg`::
4760 Log record's message.
4761
4762`logger_name`::
4763 Logger name.
4764
4765`class_name`::
4766 Name of the class in which the log statement was executed.
4767
4768`method_name`::
4769 Name of the method in which the log statement was executed.
4770
4771`filename`::
4772 Name of the file in which the executed log statement is located.
4773
4774`line_number`::
4775 Line number at which the log statement was executed.
4776
4777`timestamp`::
4778 Logging timestamp.
4779
4780`int_loglevel`::
4781 Log level integer value.
4782
4783`thread_name`::
4784 Name of the Java thread in which the log statement was executed.
4785
4786You can use the opt:lttng-enable-event(1):--loglevel or
4787opt:lttng-enable-event(1):--loglevel-only option of the
4788man:lttng-enable-event(1) command to target a range of Apache log4j log levels
4789or a specific log4j log level.
4790
4791
4792[role="since-2.8"]
4793[[java-application-context]]
4794==== Provide application-specific context fields in a Java application
4795
4796A Java application-specific context field is a piece of state provided
4797by the application which <<adding-context,you can add>>, using the
4798man:lttng-add-context(1) command, to each <<event,event record>>
4799produced by the log statements of this application.
4800
4801For example, a given object might have a current request ID variable.
4802You can create a context information retriever for this object and
4803assign a name to this current request ID. You can then, using the
4804man:lttng-add-context(1) command, add this context field by name to
4805the JUL or log4j <<channel,channel>>.
4806
4807To provide application-specific context fields in a Java application:
4808
4809. In the Java application's source code, import the LTTng-UST
4810 Java agent context classes and interfaces:
4811+
4812--
4813[source,java]
4814----
4815import org.lttng.ust.agent.context.ContextInfoManager;
4816import org.lttng.ust.agent.context.IContextInfoRetriever;
4817----
4818--
4819
4820. Create a context information retriever class, that is, a class which
4821 implements the `IContextInfoRetriever` interface:
4822+
4823--
4824[source,java]
4825----
4826class MyContextInfoRetriever implements IContextInfoRetriever
4827{
4828 @Override
4829 public Object retrieveContextInfo(String key)
4830 {
4831 if (key.equals("intCtx")) {
4832 return (short) 17;
4833 } else if (key.equals("strContext")) {
4834 return "context value!";
4835 } else {
4836 return null;
4837 }
4838 }
4839}
4840----
4841--
4842+
4843This `retrieveContextInfo()` method is the only member of the
4844`IContextInfoRetriever` interface. Its role is to return the current
4845value of a state by name to create a context field. The names of the
4846context fields and which state variables they return depends on your
4847specific scenario.
4848+
4849All primitive types and objects are supported as context fields.
4850When `retrieveContextInfo()` returns an object, the context field
4851serializer calls its `toString()` method to add a string field to
4852event records. The method can also return `null`, which means that
4853no context field is available for the required name.
4854
4855. Register an instance of your context information retriever class to
4856 the context information manager singleton:
4857+
4858--
4859[source,java]
4860----
4861IContextInfoRetriever cir = new MyContextInfoRetriever();
4862ContextInfoManager cim = ContextInfoManager.getInstance();
4863cim.registerContextInfoRetriever("retrieverName", cir);
4864----
4865--
4866
4867. Before exiting the application, remove your context information
4868 retriever from the context information manager singleton:
4869+
4870--
4871[source,java]
4872----
4873ContextInfoManager cim = ContextInfoManager.getInstance();
4874cim.unregisterContextInfoRetriever("retrieverName");
4875----
4876--
4877+
4878This is not strictly necessary, but it is recommended for a clean
4879disposal of some manager's resources.
4880
4881. Build your Java application with LTTng-UST Java agent support as
4882 usual, following the procedure for either the <<jul,JUL>> or
4883 <<log4j,Apache log4j>> framework.
4884
4885
4886.Provide application-specific context fields in a Java application.
4887====
4888[source,java]
4889.path:{Test.java}
4890----
4891import java.util.logging.Handler;
4892import java.util.logging.Logger;
4893import org.lttng.ust.agent.jul.LttngLogHandler;
4894import org.lttng.ust.agent.context.ContextInfoManager;
4895import org.lttng.ust.agent.context.IContextInfoRetriever;
4896
4897public class Test
4898{
4899 // Our context information retriever class
4900 private static class MyContextInfoRetriever
4901 implements IContextInfoRetriever
4902 {
4903 @Override
4904 public Object retrieveContextInfo(String key) {
4905 if (key.equals("intCtx")) {
4906 return (short) 17;
4907 } else if (key.equals("strContext")) {
4908 return "context value!";
4909 } else {
4910 return null;
4911 }
4912 }
4913 }
4914
4915 private static final int answer = 42;
4916
4917 public static void main(String args[]) throws Exception
4918 {
4919 // Get the context information manager instance
4920 ContextInfoManager cim = ContextInfoManager.getInstance();
4921
4922 // Create and register our context information retriever
4923 IContextInfoRetriever cir = new MyContextInfoRetriever();
4924 cim.registerContextInfoRetriever("myRetriever", cir);
4925
4926 // Create a logger
4927 Logger logger = Logger.getLogger("jello");
4928
4929 // Create an LTTng-UST log handler
4930 Handler lttngUstLogHandler = new LttngLogHandler();
4931
4932 // Add the LTTng-UST log handler to our logger
4933 logger.addHandler(lttngUstLogHandler);
4934
4935 // Log at will!
4936 logger.info("some info");
4937 logger.warning("some warning");
4938 Thread.sleep(500);
4939 logger.finer("finer information; the answer is " + answer);
4940 Thread.sleep(123);
4941 logger.severe("error!");
4942
4943 // Not mandatory, but cleaner
4944 logger.removeHandler(lttngUstLogHandler);
4945 lttngUstLogHandler.close();
4946 cim.unregisterContextInfoRetriever("myRetriever");
4947 }
4948}
4949----
4950
4951Build this example:
4952
4953[role="term"]
4954----
4955$ javac -cp /usr/share/java/jarpath/lttng-ust-agent-common.jar:/usr/share/java/jarpath/lttng-ust-agent-jul.jar Test.java
4956----
4957
4958<<creating-destroying-tracing-sessions,Create a tracing session>>
4959and <<enabling-disabling-events,create an event rule>> matching the
4960`jello` JUL logger:
4961
4962[role="term"]
4963----
4964$ lttng create
4965$ lttng enable-event --jul jello
4966----
4967
4968<<adding-context,Add the application-specific context fields>> to the
4969JUL channel:
4970
4971[role="term"]
4972----
4973$ lttng add-context --jul --type='$app.myRetriever:intCtx'
4974$ lttng add-context --jul --type='$app.myRetriever:strContext'
4975----
4976
4977<<basic-tracing-session-control,Start tracing>>:
4978
4979[role="term"]
4980----
4981$ lttng start
4982----
4983
4984Run the compiled class:
4985
4986[role="term"]
4987----
4988$ java -cp /usr/share/java/jarpath/lttng-ust-agent-common.jar:/usr/share/java/jarpath/lttng-ust-agent-jul.jar:. Test
4989----
4990
4991<<basic-tracing-session-control,Stop tracing>> and inspect the
4992recorded events:
4993
4994[role="term"]
4995----
4996$ lttng stop
4997$ lttng view
4998----
4999====
5000
5001
5002[role="since-2.7"]
5003[[python-application]]
5004=== User space Python agent
5005
5006You can instrument a Python 2 or Python 3 application which uses the
5007standard https://docs.python.org/3/library/logging.html[`logging`]
5008package.
5009
5010Each log statement emits an LTTng event once the
5011application module imports the
5012<<lttng-ust-agents,LTTng-UST Python agent>> package.
5013
5014[role="img-100"]
5015.A Python application importing the LTTng-UST Python agent.
5016image::python-app.png[]
5017
5018To use the LTTng-UST Python agent:
5019
5020. In the Python application's source code, import the LTTng-UST Python
5021 agent:
5022+
5023--
5024[source,python]
5025----
5026import lttngust
5027----
5028--
5029+
5030The LTTng-UST Python agent automatically adds its logging handler to the
5031root logger at import time.
5032+
5033Any log statement that the application executes before this import does
5034not emit an LTTng event.
5035+
5036IMPORTANT: The LTTng-UST Python agent must be
5037<<installing-lttng,installed>>.
5038
5039. Use log statements and logging configuration as usual.
5040 Since the LTTng-UST Python agent adds a handler to the _root_
5041 logger, you can trace any log statement from any logger.
5042
5043.Use the LTTng-UST Python agent.
5044====
5045[source,python]
5046.path:{test.py}
5047----
5048import lttngust
5049import logging
5050import time
5051
5052
5053def example():
5054 logging.basicConfig()
5055 logger = logging.getLogger('my-logger')
5056
5057 while True:
5058 logger.debug('debug message')
5059 logger.info('info message')
5060 logger.warn('warn message')
5061 logger.error('error message')
5062 logger.critical('critical message')
5063 time.sleep(1)
5064
5065
5066if __name__ == '__main__':
5067 example()
5068----
5069
5070NOTE: `logging.basicConfig()`, which adds to the root logger a basic
5071logging handler which prints to the standard error stream, is not
5072strictly required for LTTng-UST tracing to work, but in versions of
5073Python preceding 3.2, you could see a warning message which indicates
5074that no handler exists for the logger `my-logger`.
5075
5076<<creating-destroying-tracing-sessions,Create a tracing session>>,
5077<<enabling-disabling-events,create an event rule>> matching the
5078`my-logger` Python logger, and <<basic-tracing-session-control,start
5079tracing>>:
5080
5081[role="term"]
5082----
5083$ lttng create
5084$ lttng enable-event --python my-logger
5085$ lttng start
5086----
5087
5088Run the Python script:
5089
5090[role="term"]
5091----
5092$ python test.py
5093----
5094
5095<<basic-tracing-session-control,Stop tracing>> and inspect the recorded
5096events:
5097
5098[role="term"]
5099----
5100$ lttng stop
5101$ lttng view
5102----
5103====
5104
5105In the resulting trace, an <<event,event record>> generated by a Python
5106application is named `lttng_python:event` and has the following fields:
5107
5108`asctime`::
5109 Logging time (string).
5110
5111`msg`::
5112 Log record's message.
5113
5114`logger_name`::
5115 Logger name.
5116
5117`funcName`::
5118 Name of the function in which the log statement was executed.
5119
5120`lineno`::
5121 Line number at which the log statement was executed.
5122
5123`int_loglevel`::
5124 Log level integer value.
5125
5126`thread`::
5127 ID of the Python thread in which the log statement was executed.
5128
5129`threadName`::
5130 Name of the Python thread in which the log statement was executed.
5131
5132You can use the opt:lttng-enable-event(1):--loglevel or
5133opt:lttng-enable-event(1):--loglevel-only option of the
5134man:lttng-enable-event(1) command to target a range of Python log levels
5135or a specific Python log level.
5136
5137When an application imports the LTTng-UST Python agent, the agent tries
5138to register to a <<lttng-sessiond,session daemon>>. Note that you must
5139<<start-sessiond,start the session daemon>> _before_ you run the Python
5140application. If a session daemon is found, the agent tries to register
5141to it during 5{nbsp}seconds, after which the application continues
5142without LTTng tracing support. You can override this timeout value with
5143the env:LTTNG_UST_PYTHON_REGISTER_TIMEOUT environment variable
5144(milliseconds).
5145
5146If the session daemon stops while a Python application with an imported
5147LTTng-UST Python agent runs, the agent retries to connect and to
5148register to a session daemon every 3{nbsp}seconds. You can override this
5149delay with the env:LTTNG_UST_PYTHON_REGISTER_RETRY_DELAY environment
5150variable.
5151
5152
5153[role="since-2.5"]
5154[[proc-lttng-logger-abi]]
5155=== LTTng logger
5156
5157The `lttng-tracer` Linux kernel module, part of
5158<<lttng-modules,LTTng-modules>>, creates the special LTTng logger file
5159path:{/proc/lttng-logger} when it's loaded. Any application can write
5160text data to this file to emit an LTTng event.
5161
5162[role="img-100"]
5163.An application writes to the LTTng logger file to emit an LTTng event.
5164image::lttng-logger.png[]
5165
5166The LTTng logger is the quickest method--not the most efficient,
5167however--to add instrumentation to an application. It is designed
5168mostly to instrument shell scripts:
5169
5170[role="term"]
5171----
5172$ echo "Some message, some $variable" > /proc/lttng-logger
5173----
5174
5175Any event that the LTTng logger emits is named `lttng_logger` and
5176belongs to the Linux kernel <<domain,tracing domain>>. However, unlike
5177other instrumentation points in the kernel tracing domain, **any Unix
5178user** can <<enabling-disabling-events,create an event rule>> which
5179matches its event name, not only the root user or users in the
5180<<tracing-group,tracing group>>.
5181
5182To use the LTTng logger:
5183
5184* From any application, write text data to the path:{/proc/lttng-logger}
5185 file.
5186
5187The `msg` field of `lttng_logger` event records contains the
5188recorded message.
5189
5190NOTE: The maximum message length of an LTTng logger event is
51911024{nbsp}bytes. Writing more than this makes the LTTng logger emit more
5192than one event to contain the remaining data.
5193
5194You should not use the LTTng logger to trace a user application which
5195can be instrumented in a more efficient way, namely:
5196
5197* <<c-application,C and $$C++$$ applications>>.
5198* <<java-application,Java applications>>.
5199* <<python-application,Python applications>>.
5200
5201.Use the LTTng logger.
5202====
5203[source,bash]
5204.path:{test.bash}
5205----
5206echo 'Hello, World!' > /proc/lttng-logger
5207sleep 2
5208df --human-readable --print-type / > /proc/lttng-logger
5209----
5210
5211<<creating-destroying-tracing-sessions,Create a tracing session>>,
5212<<enabling-disabling-events,create an event rule>> matching the
5213`lttng_logger` Linux kernel tracepoint, and
5214<<basic-tracing-session-control,start tracing>>:
5215
5216[role="term"]
5217----
5218$ lttng create
5219$ lttng enable-event --kernel lttng_logger
5220$ lttng start
5221----
5222
5223Run the Bash script:
5224
5225[role="term"]
5226----
5227$ bash test.bash
5228----
5229
5230<<basic-tracing-session-control,Stop tracing>> and inspect the recorded
5231events:
5232
5233[role="term"]
5234----
5235$ lttng stop
5236$ lttng view
5237----
5238====
5239
5240
5241[[instrumenting-linux-kernel]]
5242=== LTTng kernel tracepoints
5243
5244NOTE: This section shows how to _add_ instrumentation points to the
5245Linux kernel. The kernel's subsystems are already thoroughly
5246instrumented at strategic places for LTTng when you
5247<<installing-lttng,install>> the <<lttng-modules,LTTng-modules>>
5248package.
5249
5250////
5251There are two methods to instrument the Linux kernel:
5252
5253. <<linux-add-lttng-layer,Add an LTTng layer>> over an existing ftrace
5254 tracepoint which uses the `TRACE_EVENT()` API.
5255+
5256Choose this if you want to instrumentation a Linux kernel tree with an
5257instrumentation point compatible with ftrace, perf, and SystemTap.
5258
5259. Use an <<linux-lttng-tracepoint-event,LTTng-only approach>> to
5260 instrument an out-of-tree kernel module.
5261+
5262Choose this if you don't need ftrace, perf, or SystemTap support.
5263////
5264
5265
5266[[linux-add-lttng-layer]]
5267==== [[instrumenting-linux-kernel-itself]][[mainline-trace-event]][[lttng-adaptation-layer]]Add an LTTng layer to an existing ftrace tracepoint
5268
5269This section shows how to add an LTTng layer to existing ftrace
5270instrumentation using the `TRACE_EVENT()` API.
5271
5272This section does not document the `TRACE_EVENT()` macro. You can
5273read the following articles to learn more about this API:
5274
5275* http://lwn.net/Articles/379903/[Using the TRACE_EVENT() macro (Part 1)]
5276* http://lwn.net/Articles/381064/[Using the TRACE_EVENT() macro (Part 2)]
5277* http://lwn.net/Articles/383362/[Using the TRACE_EVENT() macro (Part 3)]
5278
5279The following procedure assumes that your ftrace tracepoints are
5280correctly defined in their own header and that they are created in
5281one source file using the `CREATE_TRACE_POINTS` definition.
5282
5283To add an LTTng layer over an existing ftrace tracepoint:
5284
5285. Make sure the following kernel configuration options are
5286 enabled:
5287+
5288--
5289* `CONFIG_MODULES`
5290* `CONFIG_KALLSYMS`
5291* `CONFIG_HIGH_RES_TIMERS`
5292* `CONFIG_TRACEPOINTS`
5293--
5294
5295. Build the Linux source tree with your custom ftrace tracepoints.
5296. Boot the resulting Linux image on your target system.
5297+
5298Confirm that the tracepoints exist by looking for their names in the
5299dir:{/sys/kernel/debug/tracing/events/subsys} directory, where `subsys`
5300is your subsystem's name.
5301
5302. Get a copy of the latest LTTng-modules{nbsp}{revision}:
5303+
5304--
5305[role="term"]
5306----
5307$ cd $(mktemp -d) &&
5308wget http://lttng.org/files/lttng-modules/lttng-modules-latest-2.10.tar.bz2 &&
5309tar -xf lttng-modules-latest-2.10.tar.bz2 &&
5310cd lttng-modules-2.10.*
5311----
5312--
5313
5314. In dir:{instrumentation/events/lttng-module}, relative to the root
5315 of the LTTng-modules source tree, create a header file named
5316 +__subsys__.h+ for your custom subsystem +__subsys__+ and write your
5317 LTTng-modules tracepoint definitions using the LTTng-modules
5318 macros in it.
5319+
5320Start with this template:
5321+
5322--
5323[source,c]
5324.path:{instrumentation/events/lttng-module/my_subsys.h}
5325----
5326#undef TRACE_SYSTEM
5327#define TRACE_SYSTEM my_subsys
5328
5329#if !defined(_LTTNG_MY_SUBSYS_H) || defined(TRACE_HEADER_MULTI_READ)
5330#define _LTTNG_MY_SUBSYS_H
5331
5332#include "../../../probes/lttng-tracepoint-event.h"
5333#include <linux/tracepoint.h>
5334
5335LTTNG_TRACEPOINT_EVENT(
5336 /*
5337 * Format is identical to TRACE_EVENT()'s version for the three
5338 * following macro parameters:
5339 */
5340 my_subsys_my_event,
5341 TP_PROTO(int my_int, const char *my_string),
5342 TP_ARGS(my_int, my_string),
5343
5344 /* LTTng-modules specific macros */
5345 TP_FIELDS(
5346 ctf_integer(int, my_int_field, my_int)
5347 ctf_string(my_bar_field, my_bar)
5348 )
5349)
5350
5351#endif /* !defined(_LTTNG_MY_SUBSYS_H) || defined(TRACE_HEADER_MULTI_READ) */
5352
5353#include "../../../probes/define_trace.h"
5354----
5355--
5356+
5357The entries in the `TP_FIELDS()` section are the list of fields for the
5358LTTng tracepoint. This is similar to the `TP_STRUCT__entry()` part of
5359ftrace's `TRACE_EVENT()` macro.
5360+
5361See <<lttng-modules-tp-fields,Tracepoint fields macros>> for a
5362complete description of the available `ctf_*()` macros.
5363
5364. Create the LTTng-modules probe's kernel module C source file,
5365 +probes/lttng-probe-__subsys__.c+, where +__subsys__+ is your
5366 subsystem name:
5367+
5368--
5369[source,c]
5370.path:{probes/lttng-probe-my-subsys.c}
5371----
5372#include <linux/module.h>
5373#include "../lttng-tracer.h"
5374
5375/*
5376 * Build-time verification of mismatch between mainline
5377 * TRACE_EVENT() arguments and the LTTng-modules adaptation
5378 * layer LTTNG_TRACEPOINT_EVENT() arguments.
5379 */
5380#include <trace/events/my_subsys.h>
5381
5382/* Create LTTng tracepoint probes */
5383#define LTTNG_PACKAGE_BUILD
5384#define CREATE_TRACE_POINTS
5385#define TRACE_INCLUDE_PATH ../instrumentation/events/lttng-module
5386
5387#include "../instrumentation/events/lttng-module/my_subsys.h"
5388
5389MODULE_LICENSE("GPL and additional rights");
5390MODULE_AUTHOR("Your name <your-email>");
5391MODULE_DESCRIPTION("LTTng my_subsys probes");
5392MODULE_VERSION(__stringify(LTTNG_MODULES_MAJOR_VERSION) "."
5393 __stringify(LTTNG_MODULES_MINOR_VERSION) "."
5394 __stringify(LTTNG_MODULES_PATCHLEVEL_VERSION)
5395 LTTNG_MODULES_EXTRAVERSION);
5396----
5397--
5398
5399. Edit path:{probes/KBuild} and add your new kernel module object
5400 next to the existing ones:
5401+
5402--
5403[source,make]
5404.path:{probes/KBuild}
5405----
5406# ...
5407
5408obj-m += lttng-probe-module.o
5409obj-m += lttng-probe-power.o
5410
5411obj-m += lttng-probe-my-subsys.o
5412
5413# ...
5414----
5415--
5416
5417. Build and install the LTTng kernel modules:
5418+
5419--
5420[role="term"]
5421----
5422$ make KERNELDIR=/path/to/linux
5423# make modules_install && depmod -a
5424----
5425--
5426+
5427Replace `/path/to/linux` with the path to the Linux source tree where
5428you defined and used tracepoints with ftrace's `TRACE_EVENT()` macro.
5429
5430Note that you can also use the
5431<<lttng-tracepoint-event-code,`LTTNG_TRACEPOINT_EVENT_CODE()` macro>>
5432instead of `LTTNG_TRACEPOINT_EVENT()` to use custom local variables and
5433C code that need to be executed before the event fields are recorded.
5434
5435The best way to learn how to use the previous LTTng-modules macros is to
5436inspect the existing LTTng-modules tracepoint definitions in the
5437dir:{instrumentation/events/lttng-module} header files. Compare them
5438with the Linux kernel mainline versions in the
5439dir:{include/trace/events} directory of the Linux source tree.
5440
5441
5442[role="since-2.7"]
5443[[lttng-tracepoint-event-code]]
5444===== Use custom C code to access the data for tracepoint fields
5445
5446Although we recommended to always use the
5447<<lttng-adaptation-layer,`LTTNG_TRACEPOINT_EVENT()`>> macro to describe
5448the arguments and fields of an LTTng-modules tracepoint when possible,
5449sometimes you need a more complex process to access the data that the
5450tracer records as event record fields. In other words, you need local
5451variables and multiple C{nbsp}statements instead of simple
5452argument-based expressions that you pass to the
5453<<lttng-modules-tp-fields,`ctf_*()` macros of `TP_FIELDS()`>>.
5454
5455You can use the `LTTNG_TRACEPOINT_EVENT_CODE()` macro instead of
5456`LTTNG_TRACEPOINT_EVENT()` to declare custom local variables and define
5457a block of C{nbsp}code to be executed before LTTng records the fields.
5458The structure of this macro is:
5459
5460[source,c]
5461.`LTTNG_TRACEPOINT_EVENT_CODE()` macro syntax.
5462----
5463LTTNG_TRACEPOINT_EVENT_CODE(
5464 /*
5465 * Format identical to the LTTNG_TRACEPOINT_EVENT()
5466 * version for the following three macro parameters:
5467 */
5468 my_subsys_my_event,
5469 TP_PROTO(int my_int, const char *my_string),
5470 TP_ARGS(my_int, my_string),
5471
5472 /* Declarations of custom local variables */
5473 TP_locvar(
5474 int a = 0;
5475 unsigned long b = 0;
5476 const char *name = "(undefined)";
5477 struct my_struct *my_struct;
5478 ),
5479
5480 /*
5481 * Custom code which uses both tracepoint arguments
5482 * (in TP_ARGS()) and local variables (in TP_locvar()).
5483 *
5484 * Local variables are actually members of a structure pointed
5485 * to by the special variable tp_locvar.
5486 */
5487 TP_code(
5488 if (my_int) {
5489 tp_locvar->a = my_int + 17;
5490 tp_locvar->my_struct = get_my_struct_at(tp_locvar->a);
5491 tp_locvar->b = my_struct_compute_b(tp_locvar->my_struct);
5492 tp_locvar->name = my_struct_get_name(tp_locvar->my_struct);
5493 put_my_struct(tp_locvar->my_struct);
5494
5495 if (tp_locvar->b) {
5496 tp_locvar->a = 1;
5497 }
5498 }
5499 ),
5500
5501 /*
5502 * Format identical to the LTTNG_TRACEPOINT_EVENT()
5503 * version for this, except that tp_locvar members can be
5504 * used in the argument expression parameters of
5505 * the ctf_*() macros.
5506 */
5507 TP_FIELDS(
5508 ctf_integer(unsigned long, my_struct_b, tp_locvar->b)
5509 ctf_integer(int, my_struct_a, tp_locvar->a)
5510 ctf_string(my_string_field, my_string)
5511 ctf_string(my_struct_name, tp_locvar->name)
5512 )
5513)
5514----
5515
5516IMPORTANT: The C code defined in `TP_code()` must not have any side
5517effects when executed. In particular, the code must not allocate
5518memory or get resources without deallocating this memory or putting
5519those resources afterwards.
5520
5521
5522[[instrumenting-linux-kernel-tracing]]
5523==== Load and unload a custom probe kernel module
5524
5525You must load a <<lttng-adaptation-layer,created LTTng-modules probe
5526kernel module>> in the kernel before it can emit LTTng events.
5527
5528To load the default probe kernel modules and a custom probe kernel
5529module:
5530
5531* Use the opt:lttng-sessiond(8):--extra-kmod-probes option to give extra
5532 probe modules to load when starting a root <<lttng-sessiond,session
5533 daemon>>:
5534+
5535--
5536.Load the `my_subsys`, `usb`, and the default probe modules.
5537====
5538[role="term"]
5539----
5540# lttng-sessiond --extra-kmod-probes=my_subsys,usb
5541----
5542====
5543--
5544+
5545You only need to pass the subsystem name, not the whole kernel module
5546name.
5547
5548To load _only_ a given custom probe kernel module:
5549
5550* Use the opt:lttng-sessiond(8):--kmod-probes option to give the probe
5551 modules to load when starting a root session daemon:
5552+
5553--
5554.Load only the `my_subsys` and `usb` probe modules.
5555====
5556[role="term"]
5557----
5558# lttng-sessiond --kmod-probes=my_subsys,usb
5559----
5560====
5561--
5562
5563To confirm that a probe module is loaded:
5564
5565* Use man:lsmod(8):
5566+
5567--
5568[role="term"]
5569----
5570$ lsmod | grep lttng_probe_usb
5571----
5572--
5573
5574To unload the loaded probe modules:
5575
5576* Kill the session daemon with `SIGTERM`:
5577+
5578--
5579[role="term"]
5580----
5581# pkill lttng-sessiond
5582----
5583--
5584+
5585You can also use man:modprobe(8)'s `--remove` option if the session
5586daemon terminates abnormally.
5587
5588
5589[[controlling-tracing]]
5590== Tracing control
5591
5592Once an application or a Linux kernel is
5593<<instrumenting,instrumented>> for LTTng tracing,
5594you can _trace_ it.
5595
5596This section is divided in topics on how to use the various
5597<<plumbing,components of LTTng>>, in particular the <<lttng-cli,cmd:lttng
5598command-line tool>>, to _control_ the LTTng daemons and tracers.
5599
5600NOTE: In the following subsections, we refer to an man:lttng(1) command
5601using its man page name. For example, instead of _Run the `create`
5602command to..._, we use _Run the man:lttng-create(1) command to..._.
5603
5604
5605[[start-sessiond]]
5606=== Start a session daemon
5607
5608In some situations, you need to run a <<lttng-sessiond,session daemon>>
5609(man:lttng-sessiond(8)) _before_ you can use the man:lttng(1)
5610command-line tool.
5611
5612You will see the following error when you run a command while no session
5613daemon is running:
5614
5615----
5616Error: No session daemon is available
5617----
5618
5619The only command that automatically runs a session daemon is
5620man:lttng-create(1), which you use to
5621<<creating-destroying-tracing-sessions,create a tracing session>>. While
5622this is most of the time the first operation that you do, sometimes it's
5623not. Some examples are:
5624
5625* <<list-instrumentation-points,List the available instrumentation points>>.
5626* <<saving-loading-tracing-session,Load a tracing session configuration>>.
5627
5628[[tracing-group]] Each Unix user must have its own running session
5629daemon to trace user applications. The session daemon that the root user
5630starts is the only one allowed to control the LTTng kernel tracer. Users
5631that are part of the _tracing group_ can control the root session
5632daemon. The default tracing group name is `tracing`; you can set it to
5633something else with the opt:lttng-sessiond(8):--group option when you
5634start the root session daemon.
5635
5636To start a user session daemon:
5637
5638* Run man:lttng-sessiond(8):
5639+
5640--
5641[role="term"]
5642----
5643$ lttng-sessiond --daemonize
5644----
5645--
5646
5647To start the root session daemon:
5648
5649* Run man:lttng-sessiond(8) as the root user:
5650+
5651--
5652[role="term"]
5653----
5654# lttng-sessiond --daemonize
5655----
5656--
5657
5658In both cases, remove the opt:lttng-sessiond(8):--daemonize option to
5659start the session daemon in foreground.
5660
5661To stop a session daemon, use man:kill(1) on its process ID (standard
5662`TERM` signal).
5663
5664Note that some Linux distributions could manage the LTTng session daemon
5665as a service. In this case, you should use the service manager to
5666start, restart, and stop session daemons.
5667
5668
5669[[creating-destroying-tracing-sessions]]
5670=== Create and destroy a tracing session
5671
5672Almost all the LTTng control operations happen in the scope of
5673a <<tracing-session,tracing session>>, which is the dialogue between the
5674<<lttng-sessiond,session daemon>> and you.
5675
5676To create a tracing session with a generated name:
5677
5678* Use the man:lttng-create(1) command:
5679+
5680--
5681[role="term"]
5682----
5683$ lttng create
5684----
5685--
5686
5687The created tracing session's name is `auto` followed by the
5688creation date.
5689
5690To create a tracing session with a specific name:
5691
5692* Use the optional argument of the man:lttng-create(1) command:
5693+
5694--
5695[role="term"]
5696----
5697$ lttng create my-session
5698----
5699--
5700+
5701Replace `my-session` with the specific tracing session name.
5702
5703LTTng appends the creation date to the created tracing session's name.
5704
5705LTTng writes the traces of a tracing session in
5706+$LTTNG_HOME/lttng-trace/__name__+ by default, where +__name__+ is the
5707name of the tracing session. Note that the env:LTTNG_HOME environment
5708variable defaults to `$HOME` if not set.
5709
5710To output LTTng traces to a non-default location:
5711
5712* Use the opt:lttng-create(1):--output option of the man:lttng-create(1) command:
5713+
5714--
5715[role="term"]
5716----
5717$ lttng create my-session --output=/tmp/some-directory
5718----
5719--
5720
5721You may create as many tracing sessions as you wish.
5722
5723To list all the existing tracing sessions for your Unix user:
5724
5725* Use the man:lttng-list(1) command:
5726+
5727--
5728[role="term"]
5729----
5730$ lttng list
5731----
5732--
5733
5734When you create a tracing session, it is set as the _current tracing
5735session_. The following man:lttng(1) commands operate on the current
5736tracing session when you don't specify one:
5737
5738[role="list-3-cols"]
5739* `add-context`
5740* `destroy`
5741* `disable-channel`
5742* `disable-event`
5743* `enable-channel`
5744* `enable-event`
5745* `load`
5746* `regenerate`
5747* `save`
5748* `snapshot`
5749* `start`
5750* `stop`
5751* `track`
5752* `untrack`
5753* `view`
5754
5755To change the current tracing session:
5756
5757* Use the man:lttng-set-session(1) command:
5758+
5759--
5760[role="term"]
5761----
5762$ lttng set-session new-session
5763----
5764--
5765+
5766Replace `new-session` by the name of the new current tracing session.
5767
5768When you are done tracing in a given tracing session, you can destroy
5769it. This operation frees the resources taken by the tracing session
5770to destroy; it does not destroy the trace data that LTTng wrote for
5771this tracing session.
5772
5773To destroy the current tracing session:
5774
5775* Use the man:lttng-destroy(1) command:
5776+
5777--
5778[role="term"]
5779----
5780$ lttng destroy
5781----
5782--
5783
46adfb4b
PP
5784The man:lttng-destroy(1) command also runs the man:lttng-stop(1)
5785command implicitly (see <<basic-tracing-session-control,Start and stop a
5786tracing session>>). You need to stop tracing to make LTTng flush the
5787remaining trace data and make the trace readable.
5788
85c29972
PP
5789
5790[[list-instrumentation-points]]
5791=== List the available instrumentation points
5792
5793The <<lttng-sessiond,session daemon>> can query the running instrumented
5794user applications and the Linux kernel to get a list of available
5795instrumentation points. For the Linux kernel <<domain,tracing domain>>,
5796they are tracepoints and system calls. For the user space tracing
5797domain, they are tracepoints. For the other tracing domains, they are
5798logger names.
5799
5800To list the available instrumentation points:
5801
5802* Use the man:lttng-list(1) command with the requested tracing domain's
5803 option amongst:
5804+
5805--
5806* opt:lttng-list(1):--kernel: Linux kernel tracepoints (your Unix user
5807 must be a root user, or it must be a member of the
5808 <<tracing-group,tracing group>>).
5809* opt:lttng-list(1):--kernel with opt:lttng-list(1):--syscall: Linux
5810 kernel system calls (your Unix user must be a root user, or it must be
5811 a member of the tracing group).
5812* opt:lttng-list(1):--userspace: user space tracepoints.
5813* opt:lttng-list(1):--jul: `java.util.logging` loggers.
5814* opt:lttng-list(1):--log4j: Apache log4j loggers.
5815* opt:lttng-list(1):--python: Python loggers.
5816--
5817
5818.List the available user space tracepoints.
5819====
5820[role="term"]
5821----
5822$ lttng list --userspace
5823----
5824====
5825
5826.List the available Linux kernel system call tracepoints.
5827====
5828[role="term"]
5829----
5830$ lttng list --kernel --syscall
5831----
5832====
5833
5834
5835[[enabling-disabling-events]]
5836=== Create and enable an event rule
5837
5838Once you <<creating-destroying-tracing-sessions,create a tracing
5839session>>, you can create <<event,event rules>> with the
5840man:lttng-enable-event(1) command.
5841
5842You specify each condition with a command-line option. The available
5843condition options are shown in the following table.
5844
5845[role="growable",cols="asciidoc,asciidoc,default"]
5846.Condition command-line options for the man:lttng-enable-event(1) command.
5847|====
5848|Option |Description |Applicable tracing domains
5849
5850|
5851One of:
5852
5853. `--syscall`
5854. +--probe=__ADDR__+
5855. +--function=__ADDR__+
5856
5857|
5858Instead of using the default _tracepoint_ instrumentation type, use:
5859
5860. A Linux system call.
5861. A Linux https://lwn.net/Articles/132196/[KProbe] (symbol or address).
5862. The entry and return points of a Linux function (symbol or address).
5863
5864|Linux kernel.
5865
5866|First positional argument.
5867
5868|
5869Tracepoint or system call name. In the case of a Linux KProbe or
5870function, this is a custom name given to the event rule. With the
5871JUL, log4j, and Python domains, this is a logger name.
5872
51a225a5
PP
5873With a tracepoint, logger, or system call name, you can use the special
5874`*` globbing character to match anything (for example, `sched_*`,
5875`my_comp*:*msg_*`).
85c29972
PP
5876
5877|All.
5878
5879|
5880One of:
5881
5882. +--loglevel=__LEVEL__+
5883. +--loglevel-only=__LEVEL__+
5884
5885|
5886. Match only tracepoints or log statements with a logging level at
5887 least as severe as +__LEVEL__+.
5888. Match only tracepoints or log statements with a logging level
5889 equal to +__LEVEL__+.
5890
5891See man:lttng-enable-event(1) for the list of available logging level
5892names.
5893
5894|User space, JUL, log4j, and Python.
5895
5896|+--exclude=__EXCLUSIONS__+
5897
5898|
5899When you use a `*` character at the end of the tracepoint or logger
5900name (first positional argument), exclude the specific names in the
5901comma-delimited list +__EXCLUSIONS__+.
5902
5903|
5904User space, JUL, log4j, and Python.
5905
5906|+--filter=__EXPR__+
5907
5908|
5909Match only events which satisfy the expression +__EXPR__+.
5910
5911See man:lttng-enable-event(1) to learn more about the syntax of a
5912filter expression.
5913
5914|All.
5915
5916|====
5917
5918You attach an event rule to a <<channel,channel>> on creation. If you do
5919not specify the channel with the opt:lttng-enable-event(1):--channel
5920option, and if the event rule to create is the first in its
5921<<domain,tracing domain>> for a given tracing session, then LTTng
5922creates a _default channel_ for you. This default channel is reused in
5923subsequent invocations of the man:lttng-enable-event(1) command for the
5924same tracing domain.
5925
5926An event rule is always enabled at creation time.
5927
5928The following examples show how you can combine the previous
5929command-line options to create simple to more complex event rules.
5930
5931.Create an event rule targetting a Linux kernel tracepoint (default channel).
5932====
5933[role="term"]
5934----
5935$ lttng enable-event --kernel sched_switch
5936----
5937====
5938
5939.Create an event rule matching four Linux kernel system calls (default channel).
5940====
5941[role="term"]
5942----
5943$ lttng enable-event --kernel --syscall open,write,read,close
5944----
5945====
5946
5947.Create event rules matching tracepoints with filter expressions (default channel).
5948====
5949[role="term"]
5950----
5951$ lttng enable-event --kernel sched_switch --filter='prev_comm == "bash"'
5952----
5953
5954[role="term"]
5955----
5956$ lttng enable-event --kernel --all \
5957 --filter='$ctx.tid == 1988 || $ctx.tid == 1534'
5958----
5959
5960[role="term"]
5961----
5962$ lttng enable-event --jul my_logger \
5963 --filter='$app.retriever:cur_msg_id > 3'
5964----
5965
5966IMPORTANT: Make sure to always quote the filter string when you
5967use man:lttng(1) from a shell.
5968====
5969
5970.Create an event rule matching any user space tracepoint of a given tracepoint provider with a log level range (default channel).
5971====
5972[role="term"]
5973----
5974$ lttng enable-event --userspace my_app:'*' --loglevel=TRACE_INFO
5975----
5976
5977IMPORTANT: Make sure to always quote the wildcard character when you
5978use man:lttng(1) from a shell.
5979====
5980
5981.Create an event rule matching multiple Python loggers with a wildcard and with exclusions (default channel).
5982====
5983[role="term"]
5984----
5985$ lttng enable-event --python my-app.'*' \
5986 --exclude='my-app.module,my-app.hello'
5987----
5988====
5989
5990.Create an event rule matching any Apache log4j logger with a specific log level (default channel).
5991====
5992[role="term"]
5993----
5994$ lttng enable-event --log4j --all --loglevel-only=LOG4J_WARN
5995----
5996====
5997
5998.Create an event rule attached to a specific channel matching a specific user space tracepoint provider and tracepoint.
5999====
6000[role="term"]
6001----
6002$ lttng enable-event --userspace my_app:my_tracepoint --channel=my-channel
6003----
6004====
6005
6006The event rules of a given channel form a whitelist: as soon as an
6007emitted event passes one of them, LTTng can record the event. For
6008example, an event named `my_app:my_tracepoint` emitted from a user space
6009tracepoint with a `TRACE_ERROR` log level passes both of the following
6010rules:
6011
6012[role="term"]
6013----
6014$ lttng enable-event --userspace my_app:my_tracepoint
6015$ lttng enable-event --userspace my_app:my_tracepoint \
6016 --loglevel=TRACE_INFO
6017----
6018
6019The second event rule is redundant: the first one includes
6020the second one.
6021
6022
6023[[disable-event-rule]]
6024=== Disable an event rule
6025
6026To disable an event rule that you <<enabling-disabling-events,created>>
6027previously, use the man:lttng-disable-event(1) command. This command
6028disables _all_ the event rules (of a given tracing domain and channel)
6029which match an instrumentation point. The other conditions are not
6030supported as of LTTng{nbsp}{revision}.
6031
6032The LTTng tracer does not record an emitted event which passes
6033a _disabled_ event rule.
6034
6035.Disable an event rule matching a Python logger (default channel).
6036====
6037[role="term"]
6038----
6039$ lttng disable-event --python my-logger
6040----
6041====
6042
6043.Disable an event rule matching all `java.util.logging` loggers (default channel).
6044====
6045[role="term"]
6046----
6047$ lttng disable-event --jul '*'
6048----
6049====
6050
6051.Disable _all_ the event rules of the default channel.
6052====
6053The opt:lttng-disable-event(1):--all-events option is not, like the
6054opt:lttng-enable-event(1):--all option of man:lttng-enable-event(1), the
6055equivalent of the event name `*` (wildcard): it disables _all_ the event
6056rules of a given channel.
6057
6058[role="term"]
6059----
6060$ lttng disable-event --jul --all-events
6061----
6062====
6063
6064NOTE: You cannot delete an event rule once you create it.
6065
6066
6067[[status]]
6068=== Get the status of a tracing session
6069
6070To get the status of the current tracing session, that is, its
6071parameters, its channels, event rules, and their attributes:
6072
6073* Use the man:lttng-status(1) command:
6074+
6075--
6076[role="term"]
6077----
6078$ lttng status
6079----
6080--
6081+
6082
6083To get the status of any tracing session:
6084
6085* Use the man:lttng-list(1) command with the tracing session's name:
6086+
6087--
6088[role="term"]
6089----
6090$ lttng list my-session
6091----
6092--
6093+
6094Replace `my-session` with the desired tracing session's name.
6095
6096
6097[[basic-tracing-session-control]]
6098=== Start and stop a tracing session
6099
6100Once you <<creating-destroying-tracing-sessions,create a tracing
6101session>> and
6102<<enabling-disabling-events,create one or more event rules>>,
6103you can start and stop the tracers for this tracing session.
6104
6105To start tracing in the current tracing session:
6106
6107* Use the man:lttng-start(1) command:
6108+
6109--
6110[role="term"]
6111----
6112$ lttng start
6113----
6114--
6115
6116LTTng is very flexible: you can launch user applications before
6117or after the you start the tracers. The tracers only record the events
6118if they pass enabled event rules and if they occur while the tracers are
6119started.
6120
6121To stop tracing in the current tracing session:
6122
6123* Use the man:lttng-stop(1) command:
6124+
6125--
6126[role="term"]
6127----
6128$ lttng stop
6129----
6130--
6131+
6132If there were <<channel-overwrite-mode-vs-discard-mode,lost event
6133records>> or lost sub-buffers since the last time you ran
6134man:lttng-start(1), warnings are printed when you run the
6135man:lttng-stop(1) command.
6136
57dea9c4 6137IMPORTANT: You need to stop tracing to make LTTng flush the remaining
46adfb4b
PP
6138trace data and make the trace readable. Note that the
6139man:lttng-destroy(1) command (see
6140<<creating-destroying-tracing-sessions,Create and destroy a tracing
6141session>>) also runs the man:lttng-stop(1) command implicitly.
57dea9c4 6142
85c29972
PP
6143
6144[[enabling-disabling-channels]]
6145=== Create a channel
6146
6147Once you create a tracing session, you can create a <<channel,channel>>
6148with the man:lttng-enable-channel(1) command.
6149
6150Note that LTTng automatically creates a default channel when, for a
6151given <<domain,tracing domain>>, no channels exist and you
6152<<enabling-disabling-events,create>> the first event rule. This default
6153channel is named `channel0` and its attributes are set to reasonable
6154values. Therefore, you only need to create a channel when you need
6155non-default attributes.
6156
6157You specify each non-default channel attribute with a command-line
6158option when you use the man:lttng-enable-channel(1) command. The
6159available command-line options are:
6160
6161[role="growable",cols="asciidoc,asciidoc"]
6162.Command-line options for the man:lttng-enable-channel(1) command.
6163|====
6164|Option |Description
6165
6166|`--overwrite`
6167
6168|
6169Use the _overwrite_
6170<<channel-overwrite-mode-vs-discard-mode,event loss mode>> instead of
6171the default _discard_ mode.
6172
6173|`--buffers-pid` (user space tracing domain only)
6174
6175|
6176Use the per-process <<channel-buffering-schemes,buffering scheme>>
6177instead of the default per-user buffering scheme.
6178
6179|+--subbuf-size=__SIZE__+
6180
6181|
6182Allocate sub-buffers of +__SIZE__+ bytes (power of two), for each CPU,
6183either for each Unix user (default), or for each instrumented process.
6184
6185See <<channel-subbuf-size-vs-subbuf-count,Sub-buffer count and size>>.
6186
6187|+--num-subbuf=__COUNT__+
6188
6189|
6190Allocate +__COUNT__+ sub-buffers (power of two), for each CPU, either
6191for each Unix user (default), or for each instrumented process.
6192
6193See <<channel-subbuf-size-vs-subbuf-count,Sub-buffer count and size>>.
6194
6195|+--tracefile-size=__SIZE__+
6196
6197|
6198Set the maximum size of each trace file that this channel writes within
6199a stream to +__SIZE__+ bytes instead of no maximum.
6200
6201See <<tracefile-rotation,Trace file count and size>>.
6202
6203|+--tracefile-count=__COUNT__+
6204
6205|
6206Limit the number of trace files that this channel creates to
6207+__COUNT__+ channels instead of no limit.
6208
6209See <<tracefile-rotation,Trace file count and size>>.
6210
6211|+--switch-timer=__PERIODUS__+
6212
6213|
6214Set the <<channel-switch-timer,switch timer period>>
6215to +__PERIODUS__+{nbsp}µs.
6216
6217|+--read-timer=__PERIODUS__+
6218
6219|
6220Set the <<channel-read-timer,read timer period>>
6221to +__PERIODUS__+{nbsp}µs.
6222
6223|[[opt-blocking-timeout]]+--blocking-timeout=__TIMEOUTUS__+
6224
6225|
6226Set the timeout of user space applications which load LTTng-UST
6227in blocking mode to +__TIMEOUTUS__+:
6228
62290 (default)::
6230 Never block (non-blocking mode).
6231
000f69a6 6232`inf`::
85c29972
PP
6233 Block forever until space is available in a sub-buffer to record
6234 the event.
6235
6236__n__, a positive value::
6237 Wait for at most __n__ µs when trying to write into a sub-buffer.
6238
6239Note that, for this option to have any effect on an instrumented
6240user space application, you need to run the application with a set
6241env:LTTNG_UST_ALLOW_BLOCKING environment variable.
6242
6243|+--output=__TYPE__+ (Linux kernel tracing domain only)
6244
6245|
6246Set the channel's output type to +__TYPE__+, either `mmap` or `splice`.
6247
6248|====
6249
6250You can only create a channel in the Linux kernel and user space
6251<<domain,tracing domains>>: other tracing domains have their own channel
6252created on the fly when <<enabling-disabling-events,creating event
6253rules>>.
6254
6255[IMPORTANT]
6256====
6257Because of a current LTTng limitation, you must create all channels
6258_before_ you <<basic-tracing-session-control,start tracing>> in a given
6259tracing session, that is, before the first time you run
6260man:lttng-start(1).
6261
6262Since LTTng automatically creates a default channel when you use the
6263man:lttng-enable-event(1) command with a specific tracing domain, you
6264cannot, for example, create a Linux kernel event rule, start tracing,
6265and then create a user space event rule, because no user space channel
6266exists yet and it's too late to create one.
6267
6268For this reason, make sure to configure your channels properly
6269before starting the tracers for the first time!
6270====
6271
6272The following examples show how you can combine the previous
6273command-line options to create simple to more complex channels.
6274
6275.Create a Linux kernel channel with default attributes.
6276====
6277[role="term"]
6278----
6279$ lttng enable-channel --kernel my-channel
6280----
6281====
6282
6283.Create a user space channel with 4 sub-buffers or 1{nbsp}MiB each, per CPU, per instrumented process.
6284====
6285[role="term"]
6286----
6287$ lttng enable-channel --userspace --num-subbuf=4 --subbuf-size=1M \
6288 --buffers-pid my-channel
6289----
6290====
6291
71b643ed 6292.[[blocking-timeout-example]]Create a default user space channel with an infinite blocking timeout.
85c29972
PP
6293====
6294<<creating-destroying-tracing-sessions,Create a tracing-session>>,
6295create the channel, <<enabling-disabling-events,create an event rule>>,
6296and <<basic-tracing-session-control,start tracing>>:
6297
6298[role="term"]
6299----
6300$ lttng create
000f69a6 6301$ lttng enable-channel --userspace --blocking-timeout=inf blocking-channel
85c29972
PP
6302$ lttng enable-event --userspace --channel=blocking-channel --all
6303$ lttng start
6304----
6305
6306Run an application instrumented with LTTng-UST and allow it to block:
6307
6308[role="term"]
6309----
6310$ LTTNG_UST_ALLOW_BLOCKING=1 my-app
6311----
6312====
6313
6314.Create a Linux kernel channel which rotates 8 trace files of 4{nbsp}MiB each for each stream
6315====
6316[role="term"]
6317----
6318$ lttng enable-channel --kernel --tracefile-count=8 \
6319 --tracefile-size=4194304 my-channel
6320----
6321====
6322
6323.Create a user space channel in overwrite (or _flight recorder_) mode.
6324====
6325[role="term"]
6326----
6327$ lttng enable-channel --userspace --overwrite my-channel
6328----
6329====
6330
6331You can <<enabling-disabling-events,create>> the same event rule in
6332two different channels:
6333
6334[role="term"]
6335----
6336$ lttng enable-event --userspace --channel=my-channel app:tp
6337$ lttng enable-event --userspace --channel=other-channel app:tp
6338----
6339
6340If both channels are enabled, when a tracepoint named `app:tp` is
6341reached, LTTng records two events, one for each channel.
6342
6343
6344[[disable-channel]]
6345=== Disable a channel
6346
6347To disable a specific channel that you <<enabling-disabling-channels,created>>
6348previously, use the man:lttng-disable-channel(1) command.
6349
6350.Disable a specific Linux kernel channel.
6351====
6352[role="term"]
6353----
6354$ lttng disable-channel --kernel my-channel
6355----
6356====
6357
6358The state of a channel precedes the individual states of event rules
6359attached to it: event rules which belong to a disabled channel, even if
6360they are enabled, are also considered disabled.
6361
6362
6363[[adding-context]]
6364=== Add context fields to a channel
6365
6366Event record fields in trace files provide important information about
6367events that occured previously, but sometimes some external context may
6368help you solve a problem faster. Examples of context fields are:
6369
6370* The **process ID**, **thread ID**, **process name**, and
6371 **process priority** of the thread in which the event occurs.
6372* The **hostname** of the system on which the event occurs.
6373* The current values of many possible **performance counters** using
6374 perf, for example:
6375** CPU cycles, stalled cycles, idle cycles, and the other cycle types.
6376** Cache misses.
6377** Branch instructions, misses, and loads.
6378** CPU faults.
6379* Any context defined at the application level (supported for the
6380 JUL and log4j <<domain,tracing domains>>).
6381
6382To get the full list of available context fields, see
6383`lttng add-context --list`. Some context fields are reserved for a
6384specific <<domain,tracing domain>> (Linux kernel or user space).
6385
6386You add context fields to <<channel,channels>>. All the events
6387that a channel with added context fields records contain those fields.
6388
6389To add context fields to one or all the channels of a given tracing
6390session:
6391
6392* Use the man:lttng-add-context(1) command.
6393
6394.Add context fields to all the channels of the current tracing session.
6395====
6396The following command line adds the virtual process identifier and
6397the per-thread CPU cycles count fields to all the user space channels
6398of the current tracing session.
6399
6400[role="term"]
6401----
6402$ lttng add-context --userspace --type=vpid --type=perf:thread:cpu-cycles
6403----
6404====
6405
6406.Add performance counter context fields by raw ID
6407====
6408See man:lttng-add-context(1) for the exact format of the context field
6409type, which is partly compatible with the format used in
6410man:perf-record(1).
6411
6412[role="term"]
6413----
6414$ lttng add-context --userspace --type=perf:thread:raw:r0110:test
6415$ lttng add-context --kernel --type=perf:cpu:raw:r0013c:x86unhalted
6416----
6417====
6418
6419.Add a context field to a specific channel.
6420====
6421The following command line adds the thread identifier context field
6422to the Linux kernel channel named `my-channel` in the current
6423tracing session.
6424
6425[role="term"]
6426----
6427$ lttng add-context --kernel --channel=my-channel --type=tid
6428----
6429====
6430
6431.Add an application-specific context field to a specific channel.
6432====
6433The following command line adds the `cur_msg_id` context field of the
6434`retriever` context retriever for all the instrumented
6435<<java-application,Java applications>> recording <<event,event records>>
6436in the channel named `my-channel`:
6437
6438[role="term"]
6439----
6440$ lttng add-context --kernel --channel=my-channel \
6441 --type='$app:retriever:cur_msg_id'
6442----
6443
6444IMPORTANT: Make sure to always quote the `$` character when you
6445use man:lttng-add-context(1) from a shell.
6446====
6447
6448NOTE: You cannot remove context fields from a channel once you add it.
6449
6450
6451[role="since-2.7"]
6452[[pid-tracking]]
6453=== Track process IDs
6454
6455It's often useful to allow only specific process IDs (PIDs) to emit
6456events. For example, you may wish to record all the system calls made by
6457a given process (Ă  la http://linux.die.net/man/1/strace[strace]).
6458
6459The man:lttng-track(1) and man:lttng-untrack(1) commands serve this
6460purpose. Both commands operate on a whitelist of process IDs. You _add_
6461entries to this whitelist with the man:lttng-track(1) command and remove
6462entries with the man:lttng-untrack(1) command. Any process which has one
6463of the PIDs in the whitelist is allowed to emit LTTng events which pass
6464an enabled <<event,event rule>>.
6465
6466NOTE: The PID tracker tracks the _numeric process IDs_. Should a
6467process with a given tracked ID exit and another process be given this
6468ID, then the latter would also be allowed to emit events.
6469
6470.Track and untrack process IDs.
6471====
6472For the sake of the following example, assume the target system has 16
6473possible PIDs.
6474
6475When you
6476<<creating-destroying-tracing-sessions,create a tracing session>>,
6477the whitelist contains all the possible PIDs:
6478
6479[role="img-100"]
6480.All PIDs are tracked.
6481image::track-all.png[]
6482
6483When the whitelist is full and you use the man:lttng-track(1) command to
6484specify some PIDs to track, LTTng first clears the whitelist, then it
6485tracks the specific PIDs. After:
6486
6487[role="term"]
6488----
6489$ lttng track --pid=3,4,7,10,13
6490----
6491
6492the whitelist is:
6493
6494[role="img-100"]
6495.PIDs 3, 4, 7, 10, and 13 are tracked.
6496image::track-3-4-7-10-13.png[]
6497
6498You can add more PIDs to the whitelist afterwards:
6499
6500[role="term"]
6501----
6502$ lttng track --pid=1,15,16
6503----
6504
6505The result is:
6506
6507[role="img-100"]
6508.PIDs 1, 15, and 16 are added to the whitelist.
6509image::track-1-3-4-7-10-13-15-16.png[]
6510
6511The man:lttng-untrack(1) command removes entries from the PID tracker's
6512whitelist. Given the previous example, the following command:
6513
6514[role="term"]
6515----
6516$ lttng untrack --pid=3,7,10,13
6517----
6518
6519leads to this whitelist:
6520
6521[role="img-100"]
6522.PIDs 3, 7, 10, and 13 are removed from the whitelist.
6523image::track-1-4-15-16.png[]
6524
a9f3997c
PP
6525LTTng can track all possible PIDs again using the
6526opt:lttng-track(1):--all option:
85c29972
PP
6527
6528[role="term"]
6529----
6530$ lttng track --pid --all
6531----
6532
6533The result is, again:
6534
6535[role="img-100"]
6536.All PIDs are tracked.
6537image::track-all.png[]
6538====
6539
6540.Track only specific PIDs
6541====
6542A very typical use case with PID tracking is to start with an empty
6543whitelist, then <<basic-tracing-session-control,start the tracers>>, and
6544then add PIDs manually while tracers are active. You can accomplish this
6545by using the opt:lttng-untrack(1):--all option of the
6546man:lttng-untrack(1) command to clear the whitelist after you
6547<<creating-destroying-tracing-sessions,create a tracing session>>:
6548
6549[role="term"]
6550----
6551$ lttng untrack --pid --all
6552----
6553
6554gives:
6555
6556[role="img-100"]
6557.No PIDs are tracked.
6558image::untrack-all.png[]
6559
6560If you trace with this whitelist configuration, the tracer records no
6561events for this <<domain,tracing domain>> because no processes are
6562tracked. You can use the man:lttng-track(1) command as usual to track
6563specific PIDs, for example:
6564
6565[role="term"]
6566----
6567$ lttng track --pid=6,11
6568----
6569
6570Result:
6571
6572[role="img-100"]
6573.PIDs 6 and 11 are tracked.
6574image::track-6-11.png[]
6575====
6576
6577
6578[role="since-2.5"]
6579[[saving-loading-tracing-session]]
6580=== Save and load tracing session configurations
6581
6582Configuring a <<tracing-session,tracing session>> can be long. Some of
6583the tasks involved are:
6584
6585* <<enabling-disabling-channels,Create channels>> with
6586 specific attributes.
6587* <<adding-context,Add context fields>> to specific channels.
6588* <<enabling-disabling-events,Create event rules>> with specific log
6589 level and filter conditions.
6590
6591If you use LTTng to solve real world problems, chances are you have to
6592record events using the same tracing session setup over and over,
6593modifying a few variables each time in your instrumented program
6594or environment. To avoid constant tracing session reconfiguration,
6595the man:lttng(1) command-line tool can save and load tracing session
6596configurations to/from XML files.
6597
6598To save a given tracing session configuration:
6599
6600* Use the man:lttng-save(1) command:
6601+
6602--
6603[role="term"]
6604----
6605$ lttng save my-session
6606----
6607--
6608+
6609Replace `my-session` with the name of the tracing session to save.
6610
6611LTTng saves tracing session configurations to
6612dir:{$LTTNG_HOME/.lttng/sessions} by default. Note that the
6613env:LTTNG_HOME environment variable defaults to `$HOME` if not set. Use
6614the opt:lttng-save(1):--output-path option to change this destination
6615directory.
6616
6617LTTng saves all configuration parameters, for example:
6618
6619* The tracing session name.
6620* The trace data output path.
6621* The channels with their state and all their attributes.
6622* The context fields you added to channels.
6623* The event rules with their state, log level and filter conditions.
6624
6625To load a tracing session:
6626
6627* Use the man:lttng-load(1) command:
6628+
6629--
6630[role="term"]
6631----
6632$ lttng load my-session
6633----
6634--
6635+
6636Replace `my-session` with the name of the tracing session to load.
6637
6638When LTTng loads a configuration, it restores your saved tracing session
6639as if you just configured it manually.
6640
6641See man:lttng(1) for the complete list of command-line options. You
6642can also save and load all many sessions at a time, and decide in which
6643directory to output the XML files.
6644
6645
6646[[sending-trace-data-over-the-network]]
6647=== Send trace data over the network
6648
6649LTTng can send the recorded trace data to a remote system over the
6650network instead of writing it to the local file system.
6651
6652To send the trace data over the network:
6653
6654. On the _remote_ system (which can also be the target system),
6655 start an LTTng <<lttng-relayd,relay daemon>> (man:lttng-relayd(8)):
6656+
6657--
6658[role="term"]
6659----
6660$ lttng-relayd
6661----
6662--
6663
6664. On the _target_ system, create a tracing session configured to
6665 send trace data over the network:
6666+
6667--
6668[role="term"]
6669----
6670$ lttng create my-session --set-url=net://remote-system
6671----
6672--
6673+
6674Replace `remote-system` by the host name or IP address of the
6675remote system. See man:lttng-create(1) for the exact URL format.
6676
6677. On the target system, use the man:lttng(1) command-line tool as usual.
6678 When tracing is active, the target's consumer daemon sends sub-buffers
6679 to the relay daemon running on the remote system instead of flushing
6680 them to the local file system. The relay daemon writes the received
6681 packets to the local file system.
6682
6683The relay daemon writes trace files to
6684+$LTTNG_HOME/lttng-traces/__hostname__/__session__+ by default, where
6685+__hostname__+ is the host name of the target system and +__session__+
6686is the tracing session name. Note that the env:LTTNG_HOME environment
6687variable defaults to `$HOME` if not set. Use the
6688opt:lttng-relayd(8):--output option of man:lttng-relayd(8) to write
6689trace files to another base directory.
6690
6691
6692[role="since-2.4"]
6693[[lttng-live]]
6694=== View events as LTTng emits them (noch:{LTTng} live)
6695
6696LTTng live is a network protocol implemented by the <<lttng-relayd,relay
6697daemon>> (man:lttng-relayd(8)) to allow compatible trace viewers to
6698display events as LTTng emits them on the target system while tracing is
6699active.
6700
6701The relay daemon creates a _tee_: it forwards the trace data to both
6702the local file system and to connected live viewers:
6703
6704[role="img-90"]
6705.The relay daemon creates a _tee_, forwarding the trace data to both trace files and a connected live viewer.
6706image::live.png[]
6707
6708To use LTTng live:
6709
6710. On the _target system_, create a <<tracing-session,tracing session>>
6711 in _live mode_:
6712+
6713--
6714[role="term"]
6715----
6716$ lttng create my-session --live
6717----
6718--
6719+
6720This spawns a local relay daemon.
6721
6722. Start the live viewer and configure it to connect to the relay
6723 daemon. For example, with http://diamon.org/babeltrace[Babeltrace]:
6724+
6725--
6726[role="term"]
6727----
6728$ babeltrace --input-format=lttng-live \
6729 net://localhost/host/hostname/my-session
6730----
6731--
6732+
6733Replace:
6734+
6735--
6736* `hostname` with the host name of the target system.
6737* `my-session` with the name of the tracing session to view.
6738--
6739
6740. Configure the tracing session as usual with the man:lttng(1)
6741 command-line tool, and <<basic-tracing-session-control,start tracing>>.
6742
6743You can list the available live tracing sessions with Babeltrace:
6744
6745[role="term"]
6746----
6747$ babeltrace --input-format=lttng-live net://localhost
6748----
6749
6750You can start the relay daemon on another system. In this case, you need
6751to specify the relay daemon's URL when you create the tracing session
6752with the opt:lttng-create(1):--set-url option. You also need to replace
6753`localhost` in the procedure above with the host name of the system on
6754which the relay daemon is running.
6755
6756See man:lttng-create(1) and man:lttng-relayd(8) for the complete list of
6757command-line options.
6758
6759
6760[role="since-2.3"]
6761[[taking-a-snapshot]]
6762=== Take a snapshot of the current sub-buffers of a tracing session
6763
6764The normal behavior of LTTng is to append full sub-buffers to growing
6765trace data files. This is ideal to keep a full history of the events
6766that occurred on the target system, but it can
6767represent too much data in some situations. For example, you may wish
6768to trace your application continuously until some critical situation
6769happens, in which case you only need the latest few recorded
6770events to perform the desired analysis, not multi-gigabyte trace files.
6771
6772With the man:lttng-snapshot(1) command, you can take a snapshot of the
6773current sub-buffers of a given <<tracing-session,tracing session>>.
6774LTTng can write the snapshot to the local file system or send it over
6775the network.
6776
6777To take a snapshot:
6778
6779. Create a tracing session in _snapshot mode_:
6780+
6781--
6782[role="term"]
6783----
6784$ lttng create my-session --snapshot
6785----
6786--
6787+
6788The <<channel-overwrite-mode-vs-discard-mode,event loss mode>> of
6789<<channel,channels>> created in this mode is automatically set to
6790_overwrite_ (flight recorder mode).
6791
6792. Configure the tracing session as usual with the man:lttng(1)
6793 command-line tool, and <<basic-tracing-session-control,start tracing>>.
6794
6795. **Optional**: When you need to take a snapshot,
6796 <<basic-tracing-session-control,stop tracing>>.
6797+
6798You can take a snapshot when the tracers are active, but if you stop
6799them first, you are sure that the data in the sub-buffers does not
6800change before you actually take the snapshot.
6801
6802. Take a snapshot:
6803+
6804--
6805[role="term"]
6806----
6807$ lttng snapshot record --name=my-first-snapshot
6808----
6809--
6810+
6811LTTng writes the current sub-buffers of all the current tracing
6812session's channels to trace files on the local file system. Those trace
6813files have `my-first-snapshot` in their name.
6814
6815There is no difference between the format of a normal trace file and the
6816format of a snapshot: viewers of LTTng traces also support LTTng
6817snapshots.
6818
6819By default, LTTng writes snapshot files to the path shown by
6820`lttng snapshot list-output`. You can change this path or decide to send
6821snapshots over the network using either:
6822
6823. An output path or URL that you specify when you create the
6824 tracing session.
6825. An snapshot output path or URL that you add using
6826 `lttng snapshot add-output`
6827. An output path or URL that you provide directly to the
6828 `lttng snapshot record` command.
6829
6830Method 3 overrides method 2, which overrides method 1. When you
6831specify a URL, a relay daemon must listen on a remote system (see
6832<<sending-trace-data-over-the-network,Send trace data over the network>>).
6833
6834
6835[role="since-2.6"]
6836[[mi]]
6837=== Use the machine interface
6838
6839With any command of the man:lttng(1) command-line tool, you can set the
6840opt:lttng(1):--mi option to `xml` (before the command name) to get an
6841XML machine interface output, for example:
6842
6843[role="term"]
6844----
6845$ lttng --mi=xml enable-event --kernel --syscall open
6846----
6847
6848A schema definition (XSD) is
6849https://github.com/lttng/lttng-tools/blob/stable-2.10/src/common/mi-lttng-3.0.xsd[available]
6850to ease the integration with external tools as much as possible.
6851
6852
6853[role="since-2.8"]
6854[[metadata-regenerate]]
6855=== Regenerate the metadata of an LTTng trace
6856
6857An LTTng trace, which is a http://diamon.org/ctf[CTF] trace, has both
6858data stream files and a metadata file. This metadata file contains,
6859amongst other things, information about the offset of the clock sources
6860used to timestamp <<event,event records>> when tracing.
6861
6862If, once a <<tracing-session,tracing session>> is
6863<<basic-tracing-session-control,started>>, a major
6864https://en.wikipedia.org/wiki/Network_Time_Protocol[NTP] correction
6865happens, the trace's clock offset also needs to be updated. You
6866can use the `metadata` item of the man:lttng-regenerate(1) command
6867to do so.
6868
6869The main use case of this command is to allow a system to boot with
6870an incorrect wall time and trace it with LTTng before its wall time
6871is corrected. Once the system is known to be in a state where its
6872wall time is correct, it can run `lttng regenerate metadata`.
6873
6874To regenerate the metadata of an LTTng trace:
6875
6876* Use the `metadata` item of the man:lttng-regenerate(1) command:
6877+
6878--
6879[role="term"]
6880----
6881$ lttng regenerate metadata
6882----
6883--
6884
6885[IMPORTANT]
6886====
6887`lttng regenerate metadata` has the following limitations:
6888
6889* Tracing session <<creating-destroying-tracing-sessions,created>>
6890 in non-live mode.
6891* User space <<channel,channels>>, if any, are using
6892 <<channel-buffering-schemes,per-user buffering>>.
6893====
6894
6895
6896[role="since-2.9"]
6897[[regenerate-statedump]]
6898=== Regenerate the state dump of a tracing session
6899
6900The LTTng kernel and user space tracers generate state dump
6901<<event,event records>> when the application starts or when you
6902<<basic-tracing-session-control,start a tracing session>>. An analysis
6903can use the state dump event records to set an initial state before it
6904builds the rest of the state from the following event records.
6905http://tracecompass.org/[Trace Compass] is a notable example of an
6906application which uses the state dump of an LTTng trace.
6907
6908When you <<taking-a-snapshot,take a snapshot>>, it's possible that the
6909state dump event records are not included in the snapshot because they
6910were recorded to a sub-buffer that has been consumed or overwritten
6911already.
6912
6913You can use the `lttng regenerate statedump` command to emit the state
6914dump event records again.
6915
6916To regenerate the state dump of the current tracing session, provided
6917create it in snapshot mode, before you take a snapshot:
6918
6919. Use the `statedump` item of the man:lttng-regenerate(1) command:
6920+
6921--
6922[role="term"]
6923----
6924$ lttng regenerate statedump
6925----
6926--
6927
6928. <<basic-tracing-session-control,Stop the tracing session>>:
6929+
6930--
6931[role="term"]
6932----
6933$ lttng stop
6934----
6935--
6936
6937. <<taking-a-snapshot,Take a snapshot>>:
6938+
6939--
6940[role="term"]
6941----
6942$ lttng snapshot record --name=my-snapshot
6943----
6944--
6945
6946Depending on the event throughput, you should run steps 1 and 2
6947as closely as possible.
6948
6949NOTE: To record the state dump events, you need to
6950<<enabling-disabling-events,create event rules>> which enable them.
6951LTTng-UST state dump tracepoints start with `lttng_ust_statedump:`.
6952LTTng-modules state dump tracepoints start with `lttng_statedump_`.
6953
6954
6955[role="since-2.7"]
6956[[persistent-memory-file-systems]]
6957=== Record trace data on persistent memory file systems
6958
6959https://en.wikipedia.org/wiki/Non-volatile_random-access_memory[Non-volatile random-access memory]
6960(NVRAM) is random-access memory that retains its information when power
6961is turned off (non-volatile). Systems with such memory can store data
6962structures in RAM and retrieve them after a reboot, without flushing
6963to typical _storage_.
6964
6965Linux supports NVRAM file systems thanks to either
6966http://pramfs.sourceforge.net/[PRAMFS] or
6967https://www.kernel.org/doc/Documentation/filesystems/dax.txt[DAX]{nbsp}+{nbsp}http://lkml.iu.edu/hypermail/linux/kernel/1504.1/03463.html[pmem]
6968(requires Linux 4.1+).
6969
6970This section does not describe how to operate such file systems;
6971we assume that you have a working persistent memory file system.
6972
6973When you create a <<tracing-session,tracing session>>, you can specify
6974the path of the shared memory holding the sub-buffers. If you specify a
6975location on an NVRAM file system, then you can retrieve the latest
6976recorded trace data when the system reboots after a crash.
6977
6978To record trace data on a persistent memory file system and retrieve the
6979trace data after a system crash:
6980
6981. Create a tracing session with a sub-buffer shared memory path located
6982 on an NVRAM file system:
6983+
6984--
6985[role="term"]
6986----
6987$ lttng create my-session --shm-path=/path/to/shm
6988----
6989--
6990
6991. Configure the tracing session as usual with the man:lttng(1)
6992 command-line tool, and <<basic-tracing-session-control,start tracing>>.
6993
6994. After a system crash, use the man:lttng-crash(1) command-line tool to
6995 view the trace data recorded on the NVRAM file system:
6996+
6997--
6998[role="term"]
6999----
7000$ lttng-crash /path/to/shm
7001----
7002--
7003
7004The binary layout of the ring buffer files is not exactly the same as
7005the trace files layout. This is why you need to use man:lttng-crash(1)
7006instead of your preferred trace viewer directly.
7007
7008To convert the ring buffer files to LTTng trace files:
7009
7010* Use the opt:lttng-crash(1):--extract option of man:lttng-crash(1):
7011+
7012--
7013[role="term"]
7014----
7015$ lttng-crash --extract=/path/to/trace /path/to/shm
7016----
7017--
7018
7019
90c4e38a
PP
7020[role="since-2.10"]
7021[[notif-trigger-api]]
7022=== Get notified when a channel's buffer usage is too high or too low
7023
7024With LTTng's $$C/C++$$ notification and trigger API, your user
7025application can get notified when the buffer usage of one or more
7026<<channel,channels>> becomes too low or too high. You can use this API
7027and enable or disable <<event,event rules>> during tracing to avoid
7028<<channel-overwrite-mode-vs-discard-mode,discarded event records>>.
7029
7030.Have a user application get notified when an LTTng channel's buffer usage is too high.
7031====
7032In this example, we create and build an application which gets notified
7033when the buffer usage of a specific LTTng channel is higher than
703475{nbsp}%. We only print that it is the case in the example, but we
7035could as well use the API of <<liblttng-ctl-lttng,`liblttng-ctl`>> to
7036disable event rules when this happens.
7037
7038. Create the application's C source file:
7039+
7040--
7041[source,c]
7042.path:{notif-app.c}
7043----
7044#include <stdio.h>
7045#include <assert.h>
7046#include <lttng/domain.h>
7047#include <lttng/action/action.h>
7048#include <lttng/action/notify.h>
7049#include <lttng/condition/condition.h>
7050#include <lttng/condition/buffer-usage.h>
7051#include <lttng/condition/evaluation.h>
7052#include <lttng/notification/channel.h>
7053#include <lttng/notification/notification.h>
7054#include <lttng/trigger/trigger.h>
7055#include <lttng/endpoint.h>
7056
7057int main(int argc, char *argv[])
7058{
d2a86fb9
PP
7059 int exit_status = 0;
7060 struct lttng_notification_channel *notification_channel;
7061 struct lttng_condition *condition;
7062 struct lttng_action *action;
7063 struct lttng_trigger *trigger;
7064 const char *tracing_session_name;
7065 const char *channel_name;
7066
7067 assert(argc >= 3);
7068 tracing_session_name = argv[1];
7069 channel_name = argv[2];
90c4e38a
PP
7070
7071 /*
d2a86fb9
PP
7072 * Create a notification channel. A notification channel
7073 * connects the user application to the LTTng session daemon.
7568806b 7074 * This notification channel can be used to listen to various
d2a86fb9
PP
7075 * types of notifications.
7076 */
7077 notification_channel = lttng_notification_channel_create(
7078 lttng_session_daemon_notification_endpoint);
7079
7080 /*
7081 * Create a "high buffer usage" condition. In this case, the
7082 * condition is reached when the buffer usage is greater than or
7568806b
PP
7083 * equal to 75 %. We create the condition for a specific tracing
7084 * session name, channel name, and for the user space tracing
7085 * domain.
90c4e38a 7086 *
d2a86fb9
PP
7087 * The "low buffer usage" condition type also exists.
7088 */
7089 condition = lttng_condition_buffer_usage_high_create();
7090 lttng_condition_buffer_usage_set_threshold_ratio(condition, .75);
7091 lttng_condition_buffer_usage_set_session_name(
7092 condition, tracing_session_name);
7093 lttng_condition_buffer_usage_set_channel_name(condition,
7094 channel_name);
7095 lttng_condition_buffer_usage_set_domain_type(condition,
7096 LTTNG_DOMAIN_UST);
7097
7098 /*
7099 * Create an action (get a notification) to take when the
7100 * condition created above is reached.
7101 */
7102 action = lttng_action_notify_create();
7103
7104 /*
7105 * Create a trigger. A trigger associates a condition to an
7106 * action: the action is executed when the condition is reached.
90c4e38a 7107 */
d2a86fb9 7108 trigger = lttng_trigger_create(condition, action);
90c4e38a 7109
d2a86fb9
PP
7110 /* Register the trigger to LTTng. */
7111 lttng_register_trigger(trigger);
90c4e38a
PP
7112
7113 /*
d2a86fb9
PP
7114 * Now that we have registered a trigger, a notification will be
7115 * emitted everytime its condition is met. To receive this
7116 * notification, we must subscribe to notifications that match
7117 * the same condition.
90c4e38a 7118 */
7568806b
PP
7119 lttng_notification_channel_subscribe(notification_channel,
7120 condition);
90c4e38a
PP
7121
7122 /*
7568806b
PP
7123 * Notification loop. You can put this in a dedicated thread to
7124 * avoid blocking the main thread.
90c4e38a 7125 */
d2a86fb9
PP
7126 for (;;) {
7127 struct lttng_notification *notification;
7128 enum lttng_notification_channel_status status;
7129 const struct lttng_evaluation *notification_evaluation;
7130 const struct lttng_condition *notification_condition;
7131 double buffer_usage;
7132
7133 /* Receive the next notification. */
7134 status = lttng_notification_channel_get_next_notification(
7568806b 7135 notification_channel, &notification);
d2a86fb9
PP
7136
7137 switch (status) {
7138 case LTTNG_NOTIFICATION_CHANNEL_STATUS_OK:
7139 break;
7140 case LTTNG_NOTIFICATION_CHANNEL_STATUS_NOTIFICATIONS_DROPPED:
7141 /*
7142 * The session daemon can drop notifications if
7143 * a monitoring application is not consuming the
7144 * notifications fast enough.
7145 */
7146 continue;
7147 case LTTNG_NOTIFICATION_CHANNEL_STATUS_CLOSED:
7148 /*
7149 * The notification channel has been closed by the
7150 * session daemon. This is typically caused by a session
7151 * daemon shutting down.
7152 */
7153 goto end;
7154 default:
7155 /* Unhandled conditions or errors. */
7156 exit_status = 1;
7157 goto end;
7158 }
7159
7160 /*
7161 * A notification provides, amongst other things:
7162 *
7163 * * The condition that caused this notification to be
7164 * emitted.
7165 * * The condition evaluation, which provides more
7166 * specific information on the evaluation of the
7167 * condition.
7168 *
7169 * The condition evaluation provides the buffer usage
7568806b 7170 * value at the moment the condition was reached.
d2a86fb9
PP
7171 */
7172 notification_condition = lttng_notification_get_condition(
7173 notification);
7174 notification_evaluation = lttng_notification_get_evaluation(
7175 notification);
7176
7177 /* We're subscribed to only one condition. */
7178 assert(lttng_condition_get_type(notification_condition) ==
7179 LTTNG_CONDITION_TYPE_BUFFER_USAGE_HIGH);
7180
7181 /*
7182 * Get the exact sampled buffer usage from the
7183 * condition evaluation.
7184 */
7185 lttng_evaluation_buffer_usage_get_usage_ratio(
7186 notification_evaluation, &buffer_usage);
7187
7188 /*
7189 * At this point, instead of printing a message, we
7190 * could do something to reduce the channel's buffer
7191 * usage, like disable specific events.
7192 */
7193 printf("Buffer usage is %f %% in tracing session \"%s\", "
7568806b
PP
7194 "user space channel \"%s\".\n", buffer_usage * 100,
7195 tracing_session_name, channel_name);
d2a86fb9
PP
7196 lttng_notification_destroy(notification);
7197 }
90c4e38a
PP
7198
7199end:
d2a86fb9
PP
7200 lttng_action_destroy(action);
7201 lttng_condition_destroy(condition);
7202 lttng_trigger_destroy(trigger);
7203 lttng_notification_channel_destroy(notification_channel);
7204 return exit_status;
90c4e38a
PP
7205}
7206----
7207--
7208
7209. Build the `notif-app` application, linking it to `liblttng-ctl`:
7210+
7211--
7212[role="term"]
7213----
7214$ gcc -o notif-app notif-app.c -llttng-ctl
7215----
7216--
7217
7218. <<creating-destroying-tracing-sessions,Create a tracing session>>,
7219 <<enabling-disabling-events,create an event rule>> matching all the
7220 user space tracepoints, and
7221 <<basic-tracing-session-control,start tracing>>:
7222+
7223--
7224[role="term"]
7225----
7226$ lttng create my-session
7227$ lttng enable-event --userspace --all
7228$ lttng start
7229----
7230--
7231+
7232If you create the channel manually with the man:lttng-enable-channel(1)
7233command, you can control how frequently are the current values of the
7234channel's properties sampled to evaluate user conditions with the
7235opt:lttng-enable-channel(1):--monitor-timer option.
7236
7237. Run the `notif-app` application. This program accepts the
7238 <<tracing-session,tracing session>> name and the user space channel
7239 name as its two first arguments. The channel which LTTng automatically
7240 creates with the man:lttng-enable-event(1) command above is named
7241 `channel0`:
7242+
7243--
7244[role="term"]
7245----
7246$ ./notif-app my-session channel0
7247----
7248--
7249
7250. In another terminal, run an application with a very high event
7251 throughput so that the 75{nbsp}% buffer usage condition is reached.
7252+
7253In the first terminal, the application should print lines like this:
7254+
7255----
7256Buffer usage is 81.45197 % in tracing session "my-session", user space
7257channel "channel0".
7258----
7259+
7260If you don't see anything, try modifying the condition in
7261path:{notif-app.c} to a lower value (0.1, for example), rebuilding it
7262(step 2) and running it again (step 4).
7263====
7264
7265
85c29972
PP
7266[[reference]]
7267== Reference
7268
7269[[lttng-modules-ref]]
7270=== noch:{LTTng-modules}
7271
7272
7273[role="since-2.9"]
7274[[lttng-tracepoint-enum]]
7275==== `LTTNG_TRACEPOINT_ENUM()` usage
7276
7277Use the `LTTNG_TRACEPOINT_ENUM()` macro to define an enumeration:
7278
7279[source,c]
7280----
7281LTTNG_TRACEPOINT_ENUM(name, TP_ENUM_VALUES(entries))
7282----
7283
7284Replace:
7285
7286* `name` with the name of the enumeration (C identifier, unique
7287 amongst all the defined enumerations).
7288* `entries` with a list of enumeration entries.
7289
7290The available enumeration entry macros are:
7291
7292+ctf_enum_value(__name__, __value__)+::
7293 Entry named +__name__+ mapped to the integral value +__value__+.
7294
7295+ctf_enum_range(__name__, __begin__, __end__)+::
7296 Entry named +__name__+ mapped to the range of integral values between
7297 +__begin__+ (included) and +__end__+ (included).
7298
7299+ctf_enum_auto(__name__)+::
7300 Entry named +__name__+ mapped to the integral value following the
7301 last mapping's value.
7302+
7303The last value of a `ctf_enum_value()` entry is its +__value__+
7304parameter.
7305+
7306The last value of a `ctf_enum_range()` entry is its +__end__+ parameter.
7307+
7308If `ctf_enum_auto()` is the first entry in the list, its integral
7309value is 0.
7310
7311Use the `ctf_enum()` <<lttng-modules-tp-fields,field definition macro>>
7312to use a defined enumeration as a tracepoint field.
7313
7314.Define an enumeration with `LTTNG_TRACEPOINT_ENUM()`.
7315====
7316[source,c]
7317----
7318LTTNG_TRACEPOINT_ENUM(
7319 my_enum,
7320 TP_ENUM_VALUES(
7321 ctf_enum_auto("AUTO: EXPECT 0")
7322 ctf_enum_value("VALUE: 23", 23)
7323 ctf_enum_value("VALUE: 27", 27)
7324 ctf_enum_auto("AUTO: EXPECT 28")
7325 ctf_enum_range("RANGE: 101 TO 303", 101, 303)
7326 ctf_enum_auto("AUTO: EXPECT 304")
7327 )
7328)
7329----
7330====
7331
7332
7333[role="since-2.7"]
7334[[lttng-modules-tp-fields]]
7335==== Tracepoint fields macros (for `TP_FIELDS()`)
7336
7337[[tp-fast-assign]][[tp-struct-entry]]The available macros to define
7338tracepoint fields, which must be listed within `TP_FIELDS()` in
7339`LTTNG_TRACEPOINT_EVENT()`, are:
7340
7341[role="func-desc growable",cols="asciidoc,asciidoc"]
7342.Available macros to define LTTng-modules tracepoint fields
7343|====
7344|Macro |Description and parameters
7345
7346|
7347+ctf_integer(__t__, __n__, __e__)+
7348
7349+ctf_integer_nowrite(__t__, __n__, __e__)+
7350
7351+ctf_user_integer(__t__, __n__, __e__)+
7352
7353+ctf_user_integer_nowrite(__t__, __n__, __e__)+
7354|
7355Standard integer, displayed in base 10.
7356
7357+__t__+::
7358 Integer C type (`int`, `long`, `size_t`, ...).
7359
7360+__n__+::
7361 Field name.
7362
7363+__e__+::
7364 Argument expression.
7365
7366|
7367+ctf_integer_hex(__t__, __n__, __e__)+
7368
7369+ctf_user_integer_hex(__t__, __n__, __e__)+
7370|
7371Standard integer, displayed in base 16.
7372
7373+__t__+::
7374 Integer C type.
7375
7376+__n__+::
7377 Field name.
7378
7379+__e__+::
7380 Argument expression.
7381
7382|+ctf_integer_oct(__t__, __n__, __e__)+
7383|
7384Standard integer, displayed in base 8.
7385
7386+__t__+::
7387 Integer C type.
7388
7389+__n__+::
7390 Field name.
7391
7392+__e__+::
7393 Argument expression.
7394
7395|
7396+ctf_integer_network(__t__, __n__, __e__)+
7397
7398+ctf_user_integer_network(__t__, __n__, __e__)+
7399|
7400Integer in network byte order (big-endian), displayed in base 10.
7401
7402+__t__+::
7403 Integer C type.
7404
7405+__n__+::
7406 Field name.
7407
7408+__e__+::
7409 Argument expression.
7410
7411|
7412+ctf_integer_network_hex(__t__, __n__, __e__)+
7413
7414+ctf_user_integer_network_hex(__t__, __n__, __e__)+
7415|
7416Integer in network byte order, displayed in base 16.
7417
7418+__t__+::
7419 Integer C type.
7420
7421+__n__+::
7422 Field name.
7423
7424+__e__+::
7425 Argument expression.
7426
7427|
7428+ctf_enum(__N__, __t__, __n__, __e__)+
7429
7430+ctf_enum_nowrite(__N__, __t__, __n__, __e__)+
7431
7432+ctf_user_enum(__N__, __t__, __n__, __e__)+
7433
7434+ctf_user_enum_nowrite(__N__, __t__, __n__, __e__)+
7435|
7436Enumeration.
7437
7438+__N__+::
7439 Name of a <<lttng-tracepoint-enum,previously defined enumeration>>.
7440
7441+__t__+::
7442 Integer C type (`int`, `long`, `size_t`, ...).
7443
7444+__n__+::
7445 Field name.
7446
7447+__e__+::
7448 Argument expression.
7449
7450|
7451+ctf_string(__n__, __e__)+
7452
7453+ctf_string_nowrite(__n__, __e__)+
7454
7455+ctf_user_string(__n__, __e__)+
7456
7457+ctf_user_string_nowrite(__n__, __e__)+
7458|
7459Null-terminated string; undefined behavior if +__e__+ is `NULL`.
7460
7461+__n__+::
7462 Field name.
7463
7464+__e__+::
7465 Argument expression.
7466
7467|
7468+ctf_array(__t__, __n__, __e__, __s__)+
7469
7470+ctf_array_nowrite(__t__, __n__, __e__, __s__)+
7471
7472+ctf_user_array(__t__, __n__, __e__, __s__)+
7473
7474+ctf_user_array_nowrite(__t__, __n__, __e__, __s__)+
7475|
7476Statically-sized array of integers.
7477
7478+__t__+::
7479 Array element C type.
7480
7481+__n__+::
7482 Field name.
7483
7484+__e__+::
7485 Argument expression.
7486
7487+__s__+::
7488 Number of elements.
7489
7490|
7491+ctf_array_bitfield(__t__, __n__, __e__, __s__)+
7492
7493+ctf_array_bitfield_nowrite(__t__, __n__, __e__, __s__)+
7494
7495+ctf_user_array_bitfield(__t__, __n__, __e__, __s__)+
7496
7497+ctf_user_array_bitfield_nowrite(__t__, __n__, __e__, __s__)+
7498|
7499Statically-sized array of bits.
7500
7501The type of +__e__+ must be an integer type. +__s__+ is the number
7502of elements of such type in +__e__+, not the number of bits.
7503
7504+__t__+::
7505 Array element C type.
7506
7507+__n__+::
7508 Field name.
7509
7510+__e__+::
7511 Argument expression.
7512
7513+__s__+::
7514 Number of elements.
7515
7516|
7517+ctf_array_text(__t__, __n__, __e__, __s__)+
7518
7519+ctf_array_text_nowrite(__t__, __n__, __e__, __s__)+
7520
7521+ctf_user_array_text(__t__, __n__, __e__, __s__)+
7522
7523+ctf_user_array_text_nowrite(__t__, __n__, __e__, __s__)+
7524|
7525Statically-sized array, printed as text.
7526
7527The string does not need to be null-terminated.
7528
7529+__t__+::
7530 Array element C type (always `char`).
7531
7532+__n__+::
7533 Field name.
7534
7535+__e__+::
7536 Argument expression.
7537
7538+__s__+::
7539 Number of elements.
7540
7541|
7542+ctf_sequence(__t__, __n__, __e__, __T__, __E__)+
7543
7544+ctf_sequence_nowrite(__t__, __n__, __e__, __T__, __E__)+
7545
7546+ctf_user_sequence(__t__, __n__, __e__, __T__, __E__)+
7547
7548+ctf_user_sequence_nowrite(__t__, __n__, __e__, __T__, __E__)+
7549|
7550Dynamically-sized array of integers.
7551
7552The type of +__E__+ must be unsigned.
7553
7554+__t__+::
7555 Array element C type.
7556
7557+__n__+::
7558 Field name.
7559
7560+__e__+::
7561 Argument expression.
7562
7563+__T__+::
7564 Length expression C type.
7565
7566+__E__+::
7567 Length expression.
7568
7569|
7570+ctf_sequence_hex(__t__, __n__, __e__, __T__, __E__)+
7571
7572+ctf_user_sequence_hex(__t__, __n__, __e__, __T__, __E__)+
7573|
7574Dynamically-sized array of integers, displayed in base 16.
7575
7576The type of +__E__+ must be unsigned.
7577
7578+__t__+::
7579 Array element C type.
7580
7581+__n__+::
7582 Field name.
7583
7584+__e__+::
7585 Argument expression.
7586
7587+__T__+::
7588 Length expression C type.
7589
7590+__E__+::
7591 Length expression.
7592
7593|+ctf_sequence_network(__t__, __n__, __e__, __T__, __E__)+
7594|
7595Dynamically-sized array of integers in network byte order (big-endian),
7596displayed in base 10.
7597
7598The type of +__E__+ must be unsigned.
7599
7600+__t__+::
7601 Array element C type.
7602
7603+__n__+::
7604 Field name.
7605
7606+__e__+::
7607 Argument expression.
7608
7609+__T__+::
7610 Length expression C type.
7611
7612+__E__+::
7613 Length expression.
7614
7615|
7616+ctf_sequence_bitfield(__t__, __n__, __e__, __T__, __E__)+
7617
7618+ctf_sequence_bitfield_nowrite(__t__, __n__, __e__, __T__, __E__)+
7619
7620+ctf_user_sequence_bitfield(__t__, __n__, __e__, __T__, __E__)+
7621
7622+ctf_user_sequence_bitfield_nowrite(__t__, __n__, __e__, __T__, __E__)+
7623|
7624Dynamically-sized array of bits.
7625
7626The type of +__e__+ must be an integer type. +__s__+ is the number
7627of elements of such type in +__e__+, not the number of bits.
7628
7629The type of +__E__+ must be unsigned.
7630
7631+__t__+::
7632 Array element C type.
7633
7634+__n__+::
7635 Field name.
7636
7637+__e__+::
7638 Argument expression.
7639
7640+__T__+::
7641 Length expression C type.
7642
7643+__E__+::
7644 Length expression.
7645
7646|
7647+ctf_sequence_text(__t__, __n__, __e__, __T__, __E__)+
7648
7649+ctf_sequence_text_nowrite(__t__, __n__, __e__, __T__, __E__)+
7650
7651+ctf_user_sequence_text(__t__, __n__, __e__, __T__, __E__)+
7652
7653+ctf_user_sequence_text_nowrite(__t__, __n__, __e__, __T__, __E__)+
7654|
7655Dynamically-sized array, displayed as text.
7656
7657The string does not need to be null-terminated.
7658
7659The type of +__E__+ must be unsigned.
7660
7661The behaviour is undefined if +__e__+ is `NULL`.
7662
7663+__t__+::
7664 Sequence element C type (always `char`).
7665
7666+__n__+::
7667 Field name.
7668
7669+__e__+::
7670 Argument expression.
7671
7672+__T__+::
7673 Length expression C type.
7674
7675+__E__+::
7676 Length expression.
7677|====
7678
7679Use the `_user` versions when the argument expression, `e`, is
7680a user space address. In the cases of `ctf_user_integer*()` and
7681`ctf_user_float*()`, `&e` must be a user space address, thus `e` must
7682be addressable.
7683
7684The `_nowrite` versions omit themselves from the session trace, but are
7685otherwise identical. This means the `_nowrite` fields won't be written
7686in the recorded trace. Their primary purpose is to make some
7687of the event context available to the
7688<<enabling-disabling-events,event filters>> without having to
7689commit the data to sub-buffers.
7690
7691
7692[[glossary]]
7693== Glossary
7694
7695Terms related to LTTng and to tracing in general:
7696
7697Babeltrace::
7698 The http://diamon.org/babeltrace[Babeltrace] project, which includes
7699 the cmd:babeltrace command, some libraries, and Python bindings.
7700
7701<<channel-buffering-schemes,buffering scheme>>::
7702 A layout of sub-buffers applied to a given channel.
7703
7704<<channel,channel>>::
7705 An entity which is responsible for a set of ring buffers.
7706+
7707<<event,Event rules>> are always attached to a specific channel.
7708
7709clock::
7710 A reference of time for a tracer.
7711
7712<<lttng-consumerd,consumer daemon>>::
7713 A process which is responsible for consuming the full sub-buffers
7714 and write them to a file system or send them over the network.
7715
7716<<channel-overwrite-mode-vs-discard-mode,discard mode>>:: The event loss
7717 mode in which the tracer _discards_ new event records when there's no
7718 sub-buffer space left to store them.
7719
7720event::
7721 The consequence of the execution of an instrumentation
7722 point, like a tracepoint that you manually place in some source code,
7723 or a Linux kernel KProbe.
7724+
7725An event is said to _occur_ at a specific time. Different actions can
7726be taken upon the occurrence of an event, like record the event's payload
7727to a sub-buffer.
7728
7729<<channel-overwrite-mode-vs-discard-mode,event loss mode>>::
7730 The mechanism by which event records of a given channel are lost
7731 (not recorded) when there is no sub-buffer space left to store them.
7732
7733[[def-event-name]]event name::
7734 The name of an event, which is also the name of the event record.
7735 This is also called the _instrumentation point name_.
7736
7737event record::
7738 A record, in a trace, of the payload of an event which occured.
7739
7740<<event,event rule>>::
7741 Set of conditions which must be satisfied for one or more occuring
7742 events to be recorded.
7743
7744`java.util.logging`::
7745 Java platform's
7746 https://docs.oracle.com/javase/7/docs/api/java/util/logging/package-summary.html[core logging facilities].
7747
7748<<instrumenting,instrumentation>>::
7749 The use of LTTng probes to make a piece of software traceable.
7750
7751instrumentation point::
7752 A point in the execution path of a piece of software that, when
7753 reached by this execution, can emit an event.
7754
7755instrumentation point name::
7756 See _<<def-event-name,event name>>_.
7757
7758log4j::
7759 A http://logging.apache.org/log4j/1.2/[logging library] for Java
7760 developed by the Apache Software Foundation.
7761
7762log level::
7763 Level of severity of a log statement or user space
7764 instrumentation point.
7765
7766LTTng::
7767 The _Linux Trace Toolkit: next generation_ project.
7768
7769<<lttng-cli,cmd:lttng>>::
7770 A command-line tool provided by the LTTng-tools project which you
7771 can use to send and receive control messages to and from a
7772 session daemon.
7773
7774LTTng analyses::
7775 The https://github.com/lttng/lttng-analyses[LTTng analyses] project,
7776 which is a set of analyzing programs that are used to obtain a
7777 higher level view of an LTTng trace.
7778
7779cmd:lttng-consumerd::
7780 The name of the consumer daemon program.
7781
7782cmd:lttng-crash::
7783 A utility provided by the LTTng-tools project which can convert
7784 ring buffer files (usually
7785 <<persistent-memory-file-systems,saved on a persistent memory file system>>)
7786 to trace files.
7787
7788LTTng Documentation::
7789 This document.
7790
7791<<lttng-live,LTTng live>>::
7792 A communication protocol between the relay daemon and live viewers
7793 which makes it possible to see events "live", as they are received by
7794 the relay daemon.
7795
7796<<lttng-modules,LTTng-modules>>::
7797 The https://github.com/lttng/lttng-modules[LTTng-modules] project,
7798 which contains the Linux kernel modules to make the Linux kernel
7799 instrumentation points available for LTTng tracing.
7800
7801cmd:lttng-relayd::
7802 The name of the relay daemon program.
7803
7804cmd:lttng-sessiond::
7805 The name of the session daemon program.
7806
7807LTTng-tools::
7808 The https://github.com/lttng/lttng-tools[LTTng-tools] project, which
7809 contains the various programs and libraries used to
7810 <<controlling-tracing,control tracing>>.
7811
7812<<lttng-ust,LTTng-UST>>::
7813 The https://github.com/lttng/lttng-ust[LTTng-UST] project, which
7814 contains libraries to instrument user applications.
7815
7816<<lttng-ust-agents,LTTng-UST Java agent>>::
7817 A Java package provided by the LTTng-UST project to allow the
7818 LTTng instrumentation of `java.util.logging` and Apache log4j 1.2
7819 logging statements.
7820
7821<<lttng-ust-agents,LTTng-UST Python agent>>::
7822 A Python package provided by the LTTng-UST project to allow the
7823 LTTng instrumentation of Python logging statements.
7824
7825<<channel-overwrite-mode-vs-discard-mode,overwrite mode>>::
7826 The event loss mode in which new event records overwrite older
7827 event records when there's no sub-buffer space left to store them.
7828
7829<<channel-buffering-schemes,per-process buffering>>::
7830 A buffering scheme in which each instrumented process has its own
7831 sub-buffers for a given user space channel.
7832
7833<<channel-buffering-schemes,per-user buffering>>::
7834 A buffering scheme in which all the processes of a Unix user share the
7835 same sub-buffer for a given user space channel.
7836
7837<<lttng-relayd,relay daemon>>::
7838 A process which is responsible for receiving the trace data sent by
7839 a distant consumer daemon.
7840
7841ring buffer::
7842 A set of sub-buffers.
7843
7844<<lttng-sessiond,session daemon>>::
7845 A process which receives control commands from you and orchestrates
7846 the tracers and various LTTng daemons.
7847
7848<<taking-a-snapshot,snapshot>>::
7849 A copy of the current data of all the sub-buffers of a given tracing
7850 session, saved as trace files.
7851
7852sub-buffer::
7853 One part of an LTTng ring buffer which contains event records.
7854
7855timestamp::
7856 The time information attached to an event when it is emitted.
7857
7858trace (_noun_)::
7859 A set of files which are the concatenations of one or more
7860 flushed sub-buffers.
7861
7862trace (_verb_)::
7863 The action of recording the events emitted by an application
7864 or by a system, or to initiate such recording by controlling
7865 a tracer.
7866
7867Trace Compass::
7868 The http://tracecompass.org[Trace Compass] project and application.
7869
7870tracepoint::
7871 An instrumentation point using the tracepoint mechanism of the Linux
7872 kernel or of LTTng-UST.
7873
7874tracepoint definition::
7875 The definition of a single tracepoint.
7876
7877tracepoint name::
7878 The name of a tracepoint.
7879
7880tracepoint provider::
7881 A set of functions providing tracepoints to an instrumented user
7882 application.
7883+
7884Not to be confused with a _tracepoint provider package_: many tracepoint
7885providers can exist within a tracepoint provider package.
7886
7887tracepoint provider package::
7888 One or more tracepoint providers compiled as an object file or as
7889 a shared library.
7890
7891tracer::
7892 A software which records emitted events.
7893
7894<<domain,tracing domain>>::
7895 A namespace for event sources.
7896
7897<<tracing-group,tracing group>>::
7898 The Unix group in which a Unix user can be to be allowed to trace the
7899 Linux kernel.
7900
7901<<tracing-session,tracing session>>::
7902 A stateful dialogue between you and a <<lttng-sessiond,session
7903 daemon>>.
7904
7905user application::
7906 An application running in user space, as opposed to a Linux kernel
7907 module, for example.
This page took 0.309936 seconds and 4 git commands to generate.