Francis Deslauriers [Fri, 9 Feb 2018 20:15:23 +0000 (15:15 -0500)]
dlopen() liblttng-ust.so from constructor to prevent unloading
The support of probe provider dlclose() allows for the following
problematic scenario:
- Application is not linked against the liblttng-ust.so
- Application dlopen() a probe provider library that is linked against
liblttng-ust.so
- Application dlclose() the probe provider
In this scenario, the probe provider has a dependency on
liblttng-ust.so, so when it's loaded by the application, liblttng-ust.so
is loaded too. The probe provider library now has the only reference to
the liblttng-ust.so library. When the application calls dlclose() on
it, all its references are dropped, thus triggering the unloading of
both the probe provider library and liblttng-ust.so.
This scenario is problematic because lttng ust_listener_threads are in
DETACHED state. We cannot join them and therefore we cannot unload the
library containing the code they run. Only the operating system can free
those resources.
The reason why those threads are in DETACHED state is to quickly
teardown applications on process exit.
A possible solution to investigate: if we can determine whether
liblttng-ust.so is being dlopen (directly or undirectly) or it's linked
against the application, we could set the detached state accordingly.
To prevent that unloading, we pin it in memory by grabbing an extra
reference on the library, with a RTLD_NODELETE flag. This will prevent
the dynamic loader from ever removing the liblttng-ust.so library from
the process' address space.
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Francis Deslauriers [Fri, 9 Feb 2018 20:15:22 +0000 (15:15 -0500)]
Add probe provider unregister function
Also, ensure that enumerations and events are removed from their
respective hashtables when _lttng_{event, enum}_destroy functions are
called.
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Francis Deslauriers [Fri, 9 Feb 2018 20:15:21 +0000 (15:15 -0500)]
Cleanup: Move version numbers in separate variables in configure script
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Mathieu Desnoyers [Fri, 9 Feb 2018 20:15:20 +0000 (15:15 -0500)]
Remove duplicate provider name checks
It's now possible to register a probe provider with a name that has
already been registered. This is useful when wanting to load a new
version of a shared library on a already running process.
Changes are necessary in the lttng-session daemon to support cases where
the newly register event has a different probe payload.
Taking a simple case where a probe provider is registered twice, the
tracepoint call site will have two probes registered to it and thus will
generate two events in the trace.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Mathieu Desnoyers [Tue, 6 Feb 2018 13:58:59 +0000 (08:58 -0500)]
Fix: remove for() loop initial declarations
Breaks sles12sp2 builds:
08:51:35 lttng-ust-fd-tracker.c: In function 'dup_std_fd':
08:51:35 lttng-ust-fd-tracker.c:174:2: error: 'for' loop initial
declarations are only allowed in C99 mode
08:51:35 for (int i = 0; i < STDERR_FILENO + 1; i++) {
08:51:35 ^
08:51:35 lttng-ust-fd-tracker.c:174:2: note: use option -std=c99 or
-std=gnu99 to compile your code
08:51:35 lttng-ust-fd-tracker.c:195:11: error: redefinition of 'i'
08:51:35 for (int i = 0; i < fd_to_close_count; i++) {
08:51:35 ^
08:51:35 lttng-ust-fd-tracker.c:174:11: note: previous definition of 'i'
was here
08:51:35 for (int i = 0; i < STDERR_FILENO + 1; i++) {
08:51:35 ^
08:51:35 lttng-ust-fd-tracker.c:195:2: error: 'for' loop initial
declarations are only allowed in C99 mode
08:51:35 for (int i = 0; i < fd_to_close_count; i++) {
08:51:35 ^
08:51:35 Makefile:412: recipe for target 'lttng-ust-fd-tracker.lo'
failed
08:51:35 make[2]: *** [lttng-ust-fd-tracker.lo] Error 1
08:51:35 make[2]: *** Waiting for unfinished jobs....
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Jonathan Rajotte [Mon, 5 Feb 2018 22:58:19 +0000 (17:58 -0500)]
Use ust_lock during sock_info operations for atomicity against fork
Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Jonathan Rajotte [Mon, 5 Feb 2018 22:58:18 +0000 (17:58 -0500)]
Force tracked fd to be bigger than STDERR_FILENO
This allow ust to be proactive regarding std* fd manipulation done by
external source.
A good example of this is the "daemon" function that can dup2 statically
the std* fd and close them silently if the were already used.
Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Mathieu Desnoyers [Tue, 5 Dec 2017 16:55:07 +0000 (11:55 -0500)]
Fix: numa: dynamically check that numa is available
Kernels with CONFIG_NUMA=n cause libnuma to print those messages:
"set_mempolicy: Function not implemented".
Check for numa availability before using it.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Michael Jeanson [Tue, 21 Nov 2017 16:11:15 +0000 (11:11 -0500)]
Fix: specify SONAME in python-lttngust LoadLibrary
When loading the python agent library with ctypes in the python
bindings, specify the SONAME. This will make sure we load the proper
library in the event of a SONAME bump and the bindings will work without
having to install the "dev" package which in most distros contains the
non-versionned ".so".
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Mathieu Desnoyers [Sun, 12 Nov 2017 19:51:24 +0000 (14:51 -0500)]
Fix: ensure fd tracker is initialized when called from constructors
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Jonathan Rajotte [Fri, 10 Nov 2017 16:06:41 +0000 (11:06 -0500)]
Fix: fd of an elf object must be registered to the fd tracker
The open call take place inside ust, it must be tracked to prevent external
closing.
The bug can be hit during tracing of an application for which the probe
provider is loaded using LD_PRELOAD in combination with the fd utility
shared object. The application is responsible for closing all possible fd.
Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Jonathan Rajotte [Thu, 9 Nov 2017 22:37:14 +0000 (17:37 -0500)]
Cleanup: clock description for metadata was moved to lttng-sessiond
Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Mathieu Desnoyers [Tue, 7 Nov 2017 20:33:40 +0000 (15:33 -0500)]
Fix: liblttng-ust-fd.so: override fclose symbol
fclose() invoked from application or libraries end up calling close()
from within the libc, which bypasses our file descriptor tracking.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Mathieu Desnoyers [Tue, 7 Nov 2017 14:38:18 +0000 (09:38 -0500)]
Revert "Use initial-exec TLS model"
This reverts commit
553bbf7f38652084ed7966c7817b8ccb372b14e1.
The initial-exec model seems to behave differently than global-dynamic
with respect to lazy initialization, causing locks to be taken then
first time each thread touch the TLS. This introduces deadlocks with
library constructors waiting on other threads.
This will require further investigation.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Mathieu Desnoyers [Tue, 7 Nov 2017 14:37:46 +0000 (09:37 -0500)]
Revert "Fix: typo: DECLARE_URCU_TLS_IE -> DEFINE_URCU_TLS_IE"
This reverts commit
364979886a80f7c993b3210af48a93f7d65f4530.
The initial-exec model seems to behave differently than global-dynamic
with respect to lazy initialization, causing locks to be taken then
first time each thread touch the TLS. This introduces deadlocks with
library constructors waiting on other threads.
This will require further investigation.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Philippe Proulx [Mon, 6 Nov 2017 20:46:03 +0000 (15:46 -0500)]
configure.ac: add --disable-examples option to not build/install examples
Some environments and distributions do not need the LTTng-UST examples
to be built because they remove them anyway. Continue to build them by
default, but add --disable-examples to explicitly disable them.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Michael Jeanson [Mon, 6 Nov 2017 19:09:30 +0000 (14:09 -0500)]
Disable NUMA by default on 32bit arm
There is currently no NUMA support on 32bit arm, disable the dependency
on libnuma by default on this architecture. It can still be force with
--enable-numa.
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Mathieu Desnoyers [Sun, 5 Nov 2017 14:08:03 +0000 (09:08 -0500)]
Update Makefile.am output about libnuma not found
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Mathieu Desnoyers [Sun, 5 Nov 2017 14:02:12 +0000 (09:02 -0500)]
Update readme to document opt dep on numa
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Mathieu Desnoyers [Sun, 5 Nov 2017 14:01:04 +0000 (09:01 -0500)]
numa support: allow disabling numa support
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Mathieu Desnoyers [Sun, 5 Nov 2017 13:30:08 +0000 (08:30 -0500)]
Update README to document dependency on libnuma
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Mathieu Desnoyers [Sun, 5 Nov 2017 12:45:02 +0000 (07:45 -0500)]
Take NUMA configuration into account for UST buffer allocation
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Mathieu Desnoyers [Sat, 21 Oct 2017 10:45:42 +0000 (12:45 +0200)]
Fix: typo: DECLARE_URCU_TLS_IE -> DEFINE_URCU_TLS_IE
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Mathieu Desnoyers [Thu, 19 Oct 2017 13:27:27 +0000 (09:27 -0400)]
Use initial-exec TLS model
Use initial-exec TLS model for TLS variables. Those are faster, and
signal-handler-safe.
Link: https://docs.oracle.com/cd/E23824_01/html/819-0690/chapter8-20.html
Link: https://gcc.gnu.org/onlinedocs/gcc/Common-Variable-Attributes.html#Common-Variable-Attributes
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Jonathan Rajotte [Tue, 19 Sep 2017 16:08:34 +0000 (12:08 -0400)]
lttng-gen-tp: formatting
Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Jonathan Rajotte [Tue, 19 Sep 2017 16:08:33 +0000 (12:08 -0400)]
Fix: lttng-gen-tp: only replace file extension
Previous replace was done on the complete path. A path containing
.c or .o would result in a corrupted file path.
Reported-by: Gunnar Strand <Gunnar.Strand@ericsson.com>
Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Mathieu Desnoyers [Wed, 23 Aug 2017 15:27:16 +0000 (08:27 -0700)]
Fix: move fsync after ftruncate
Move fsync after ftruncate to ensure we sync up all metadata after the
entire initialization of the buffer.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Mathieu Desnoyers [Wed, 23 Aug 2017 15:17:48 +0000 (08:17 -0700)]
Fix: sync buffer file metadata on buffer allocation
Synchronizing the file metadata on disk after zeroing the whole file (on
buffer allocation) will make the crash extraction feature (--shm-path
create option) more robust. It ensures the content of the file metadata
backing the buffers does not have to be updated while tracing into the
memory map. Therefore, the on-disk metadata will never be out of sync at
the point where a system crash occurs.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Philippe Proulx [Fri, 28 Jul 2017 19:30:09 +0000 (15:30 -0400)]
lttng-ust(3): reword and fix style of `LTTNG_UST_ALLOW_BLOCKING` variable
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Philippe Proulx [Fri, 28 Jul 2017 19:25:07 +0000 (15:25 -0400)]
lttng-ust(3): specify "If set" instead of "if set to 1" for some variables
The actual library's behaviour is to check if the environment variable
is set, not only if it's set to the string `1`.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Philippe Proulx [Thu, 27 Jul 2017 23:28:40 +0000 (19:28 -0400)]
Fix: doc/man: use a single XSL file and match local names
Matching the local name instead of the full name, that is:
*[local-name() = 'co']
instead of just `co` matches both the non-namespaced element and the
DocBook-namespaced element whether we're using the DocBook 4.5 or
DocBook 5.0 stylesheets.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Liguang Li [Thu, 22 Jun 2017 08:56:29 +0000 (16:56 +0800)]
Fix: Expand the shmobj size for the sequence number
This patch fixes a bug which was introduced by:
1ff31389b ("Generate and export the sequence number")
Signed-off-by: Liguang Li <liguang.li@windriver.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Mathieu Desnoyers [Fri, 26 May 2017 15:59:54 +0000 (17:59 +0200)]
Introduce LTTNG_UST_ALLOW_BLOCKING env. var.
Introduce the LTTNG_UST_ALLOW_BLOCKING env. var. to control whether
applications are allowed to block when a buffer is full. If set, it
allows the tracer to block the application when buffers are full.
The blocking is now controlled by a per-channel configuration option in
the LTTng control interface for channels with the "--blocking-timeout"
parameter, which is specified in usec (or -1 to block forever).
This replaces the LTTNG_UST_BLOCKING_RETRY_TIMEOUT env. var., which
actually never made it into a stable release (we therefore remove this
env. var).
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Philippe Proulx [Sat, 29 Oct 2016 17:32:57 +0000 (13:32 -0400)]
Fix: lttng-ust-elf.c: define NT_GNU_BUILD_ID if not defined
On uClibc, NT_GNU_BUILD_ID is not defined, so we define it
manually in this case.
This definition is the number given to the .note.gnu.build-id section
(of type SHT_NOTE) of a linked ELF object.
The ELF note headers give name "GNU" and type 3 (NT_GNU_BUILD_ID)
for a build ID note, of which there can be only one in a linked object
[...]
See http://fedoraproject.org/wiki/RolandMcGrath/BuildID
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Mathieu Desnoyers [Sat, 27 May 2017 13:14:41 +0000 (15:14 +0200)]
Calculate context length outside of retry loop
Allow context length calculation to have side-effects which trigger
event tracing by moving the calculation outside of the buffer space
reservation retry loop.
This also paves the way to have dynamically sized contexts in lttng-ust,
which would expect to put their size of the internal stack. Note that
the context length calculation is performed *after* the event payload
field length calculation, so the stack needs to be used accordingly.
Currently, the only dynamically sized contexts we have are provided by
Java integration, which keeps its own stack.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Michael Jeanson [Tue, 9 May 2017 20:55:44 +0000 (16:55 -0400)]
Fix: Quote CMAKE variable assignment in Makefile
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Michael Jeanson [Tue, 9 May 2017 20:10:46 +0000 (16:10 -0400)]
Rework configure script
* Harmonize with our other projects
* Add missing checks
* Fix help strings indentation
* Fix SED tool detection
* Add mini-report
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Michael Jeanson [Tue, 9 May 2017 19:24:40 +0000 (15:24 -0400)]
Fix: Typo in doc/examples/Makefile.am
$(CPPLAGS) -> $(CPPFLAGS)
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Michael Jeanson [Tue, 9 May 2017 18:25:01 +0000 (14:25 -0400)]
Fix: Don't override user variables within the build system
Instead use the appropriatly prefixed AM_* variables as to not interfere
when a user variable is passed to a make command. The proper use of flag
variables is documented at :
https://www.gnu.org/software/automake/manual/automake.html#Flag-Variables-Ordering
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Michael Jeanson [Tue, 9 May 2017 18:46:38 +0000 (14:46 -0400)]
Move m4 scripts to m4 dir
Harmonize build system configuration across our projects.
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Mathieu Desnoyers [Sat, 6 May 2017 02:26:31 +0000 (22:26 -0400)]
doc: remove duplicate LTTNG_UST_BLOCKING_RETRY_TIMEOUT man page entry
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Mathieu Desnoyers [Sat, 6 May 2017 01:04:54 +0000 (21:04 -0400)]
Version 2.10.0-rc1
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Jérémie Galarneau [Fri, 5 May 2017 15:52:07 +0000 (11:52 -0400)]
Clean-up: remove unused variables to silence gcc warning
GCC 6.3.1 warns that two variables (write_offset and consumed_cur)
in lib_ring_buffer_snapshot_sample_positions() are unused.
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Jérémie Galarneau [Fri, 5 May 2017 14:33:06 +0000 (10:33 -0400)]
Add ustctl_snapshot_sample_positions ustctl command
The protocol's minor version is bumped since a new API entry
point is introduced. The so name's "current" and "age" fields are
bumped in accordance with the libtool guidelines[1].
[1] https://www.gnu.org/software/libtool/manual/html_node/Updating-version-info.html
Signed-off-by: Jérémie Galarneau <jeremie.galarneau@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Francis Deslauriers [Fri, 17 Mar 2017 20:55:21 +0000 (16:55 -0400)]
Cleanup: formatting in strutils_star_glob_match explanation
Replace tabs for spaces in example scenario.
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Philippe Proulx [Wed, 15 Mar 2017 00:48:18 +0000 (20:48 -0400)]
doc/man: add typical `$` and `#` prompts to command lines
It is more instinctive for the typical reader to immediately recognize
command lines when they start with the classic prompts.
On the online version of the man pages, those prompts are treated
specially to make them non-selectable. This makes it possible to copy
multiple command lines at once (without copying the prompts) and to
paste them to your shell.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Mathieu Desnoyers [Sun, 12 Mar 2017 02:18:27 +0000 (21:18 -0500)]
Fix: add missing getenv.h include to ustctl.c
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Mathieu Desnoyers [Fri, 10 Mar 2017 23:08:25 +0000 (18:08 -0500)]
Fix: race between lttng-ust getenv() and application setenv()
The LTTng-UST listener threads invoke getenv(), which can cause issues
if the application issues setenv() concurrently. This is a legitimate
use by the application because it may have a single thread and not be
aware that it runs with liblttng-ust.
Fix this by keeping our own environment variable table for the variables
we care about. Initialize this table within the lttng-ust library
constructor, when we don't race with the application.
As this thread shows:
https://sourceware.org/bugzilla/show_bug.cgi?id=5069#c10
getenv() does _not_ appear to be thread-safe if an application uses
setenv() or putenv().
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Mathieu Desnoyers [Sat, 11 Mar 2017 13:43:09 +0000 (08:43 -0500)]
Use SIZE_MAX instead of -1ULL for size_t parameter
strutils_star_glob_match() receives a size_t. Passing -1ULL truncates
the value implicitly on systems where size_t is 32-bit. It is cleaner to
use SIZE_T.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Mathieu Desnoyers [Sat, 11 Mar 2017 13:38:49 +0000 (08:38 -0500)]
filter: use SIZE_MAX rather than UINT_MAX and tuncating -1ULL
The backing type is a size_t, so use SIZE_MAX to represent infinity.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Mathieu Desnoyers [Fri, 10 Mar 2017 21:49:42 +0000 (16:49 -0500)]
Fix: out of bound array access in filter code
Found by Coverity:
*** CID
1372124: Memory - illegal accesses (OVERRUN)
/liblttng-ust/lttng-filter.c: 139 in print_op()
133
134 const char *print_op(enum filter_op op)
135 {
136 if (op >= NR_FILTER_OPS)
137 return "UNKNOWN";
138 else
>>> CID
1372124: Memory - illegal accesses (OVERRUN)
>>> Overrunning array "opnames" of 74 8-byte elements at element index 78 (byte offset 624) using index "op" (which evaluates to 78).
139 return opnames[op];
140 }
141
142 static
143 int apply_field_reloc(struct lttng_event *event,
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Alexandre Montplaisir [Fri, 10 Mar 2017 18:51:39 +0000 (13:51 -0500)]
Correctly clean all generated JAR files
Compatibility symlinks should also be cleaned correctly by "make clean".
Also cleanup the generated .stamp file that was added to the -common
package.
Signed-off-by: Alexandre Montplaisir <alexmonthy@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Alexandre Montplaisir [Fri, 10 Mar 2017 18:35:27 +0000 (13:35 -0500)]
Fix: List missing file in Java agent's Makefile
EventNamePattern.java was added with
c0e418b, it should be listed in
the corresponding Makefile.
Signed-off-by: Alexandre Montplaisir <alexmonthy@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Alexandre Montplaisir [Fri, 17 Feb 2017 20:22:39 +0000 (15:22 -0500)]
Support generic globbing patterns in the Java agent
Replace the separate eventNames and eventNamePrefixes maps by
one map tracking generic Patterns instead. This will allow
matching against patterns containing more than one wildcard
character, which is now supported by UST.
Signed-off-by: Alexandre Montplaisir <alexmonthy@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Alexandre Montplaisir [Fri, 17 Feb 2017 21:13:20 +0000 (16:13 -0500)]
Log more information in the Java TCP client
Instead of just logging "event enabled" and "event disabled",
also print what event rule was enabled or disabled.
Also print what type of response is sent (success or failure).
Signed-off-by: Alexandre Montplaisir <alexmonthy@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Alexandre Montplaisir [Fri, 17 Feb 2017 19:02:40 +0000 (14:02 -0500)]
Cleanup: Remove unused import in JUL Java agent
Signed-off-by: Alexandre Montplaisir <alexmonthy@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Philippe Proulx [Fri, 17 Feb 2017 09:26:59 +0000 (04:26 -0500)]
Add support for star globbing patterns in event names
This patch adds support for full star-only globbing patterns used in
the event names (enabler names).
strutils_star_glob_match() is always used to perform the match when
the enabler is LTTNG_ENABLER_STAR_GLOB. This enabler is set when it is
detected that its name contains at least one non-escaped star with
strutils_is_star_glob_pattern().
While exclusions could be checked before the enabler name match to this
date, they must now be checked after we know there's a match because the
intersection of exclusion names and event event name is not always
checked on the LTTng-tools side (too much complexity for too little
gain).
The match itself is performed by strutils_star_glob_match(), the same
function that the filter interpreter uses.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Philippe Proulx [Fri, 17 Feb 2017 09:14:40 +0000 (04:14 -0500)]
Filtering: add support for star-only globbing patterns
This patch adds the support for "full" star-only globbing patterns to be
used in filter literal strings. A star-only globbing pattern is a
globbing pattern with the star (`*`) being the only special character.
This means `?` and character sets (`[abc-k]`) are not supported here. We
cannot support them without a strategy to differentiate the globbing
pattern because `?` and `[` are not special characters in filter literal
strings right now. The eventual strategy to support them would probably
look like this:
filename =* "?sys*.[ch]"
The filter bytecode generator in LTTng-tools's session daemon creates
the new FILTER_OP_LOAD_STAR_GLOB_STRING operation when the interpreter
should load a star globbing pattern literal string. Even if both
"plain", or legacy strings and star globbing pattern strings are literal
strings, they do not represent the same thing, that is, the == and !=
operators act differently.
The validation process checks that:
1. There's no binary operator between two
FILTER_OP_LOAD_STAR_GLOB_STRING operations. It is illegal to compare
two star globbing patterns, as this is not trivial to implement, and
completely useless as far as I know.
2. Only the == and != binary operators are allowed between a
star globbing pattern and a string.
For the special case of star globbing patterns with a star at the end
only, the current behaviour is not changed to preserve a maximum of
backward compatibility. This is also why the UST ABI version is changed
from 7.1 to 7.2, not to 8.0.
== or != operations between REG_STRING and REG_STAR_GLOB_STRING
registers is specialized to FILTER_OP_EQ_STAR_GLOB_STRING and
FILTER_OP_NE_STAR_GLOB_STRING. Which side is the actual globbing pattern
(the one with the REG_STAR_GLOB_STRING type) is checked at execution
time. The strutils_star_glob_match() function is used to perform the
match operation. See the implementation for more details.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Philippe Proulx [Fri, 17 Feb 2017 09:04:35 +0000 (04:04 -0500)]
Add string utilities
The new string-utils.c file has a few utility functions to manipulate
and check strings. See string-utils.c for more details.
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Francis Deslauriers [Thu, 2 Mar 2017 16:56:05 +0000 (11:56 -0500)]
Fix: (un)install targets of Python agent
This Makefile was using Distutils' setup.py to install the Python agent
but was using the Autoconf's $pkgpythondir variable for the uninstall
process. The two folders can be different on some distributions which
made the uninstall attempting to delete a non-existant folder and
effectively not uninstalling.
We now run a phony installation of the bindings in a temporary directory
and use the tree structure of the install folder to infere the location
of the files on the system to delete them.
Also, we print a warning if the install directory is not included in the
PYTHONPATH variable.
Signed-off-by: Francis Deslauriers <francis.deslauriers@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Jonathan Rajotte [Tue, 21 Feb 2017 23:10:11 +0000 (18:10 -0500)]
Fix: include config.h to resolve HAVE_DLMOPEN
Fixes commit
42330adcefcd1830dad89e2a960c93d8dd1da125
Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Jonathan Rajotte [Tue, 21 Feb 2017 21:00:27 +0000 (16:00 -0500)]
Validate the presence of dlmopen at configure time
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Michael Jeanson [Mon, 20 Feb 2017 21:48:56 +0000 (16:48 -0500)]
Fix: CMake examples integration
Fix multiple issues with the CMake examples integration with our build
system.
Fixes #1089
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Vicente Olivert Riera [Wed, 30 Nov 2016 16:52:51 +0000 (16:52 +0000)]
doc/examples/Makefile.am: define C and C++ compilers for CMake
This prevents build failures when cross-compiling.
If we don't define the compiler, it will use the one from the host
machine. For instance "/usr/bin/c++", which is incorrect.
The failure looks like this:
.............................................................
[ 10%] Building CXX object CMakeFiles/tracepoint-provider.dir/tracepoint-provider.cpp.o
/usr/bin/c++ -Dtracepoint_provider_EXPORTS -I/br/output/build/lttng-libust-2.9.0/doc/examples/cmake-multiple-shared-libraries/. -isystem /br/output/build/lttng-libust-2.9.0/include -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -Os -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -fPIC -o CMakeFiles/tracepoint-provider.dir/tracepoint-provider.cpp.o -c /br/output/build/lttng-libust-2.9.0/doc/examples/cmake-multiple-shared-libraries/tracepoint-provider.cpp
In file included from /br/output/build/lttng-libust-2.9.0/include/lttng/tracepoint.h:29:0,
from /br/output/build/lttng-libust-2.9.0/doc/examples/cmake-multiple-shared-libraries/tracepoint-provider.h:32,
from /br/output/build/lttng-libust-2.9.0/doc/examples/cmake-multiple-shared-libraries/tracepoint-provider.cpp:26:
/br/output/build/lttng-libust-2.9.0/include/lttng/tracepoint-rcu.h:26:27: fatal error: urcu/compiler.h: No such file or directory
#include <urcu/compiler.h>
^
compilation terminated.
.............................................................
Acked-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Signed-off-by: Vicente Olivert Riera <Vincent.Riera@imgtec.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Philippe Proulx [Tue, 29 Nov 2016 17:10:57 +0000 (12:10 -0500)]
lttng-ust(3): document `lttng_ust_loaded` symbol
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Philippe Proulx [Tue, 29 Nov 2016 17:04:22 +0000 (12:04 -0500)]
lttng-ust(3): document `perf:thread:raw:rN:NAME` context
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Philippe Proulx [Tue, 29 Nov 2016 03:36:13 +0000 (22:36 -0500)]
lttng-ust-dl(3): update documentation
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Philippe Proulx [Tue, 29 Nov 2016 03:16:34 +0000 (22:16 -0500)]
lttng-ust(3): document liblttng-ust-fd
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Philippe Proulx [Tue, 29 Nov 2016 03:12:51 +0000 (22:12 -0500)]
doc/man: document the lttng_ust_lib events
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Philippe Proulx [Tue, 29 Nov 2016 02:35:34 +0000 (21:35 -0500)]
doc/man: add full stop to descriptive table cells
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Philippe Proulx [Tue, 29 Nov 2016 02:34:26 +0000 (21:34 -0500)]
doc/man: reorganize source for tables
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Philippe Proulx [Tue, 29 Nov 2016 02:28:58 +0000 (21:28 -0500)]
lttng-ust(3): sort ctf_*() macro parameter definitions
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Philippe Proulx [Tue, 29 Nov 2016 02:27:19 +0000 (21:27 -0500)]
lttng-ust(3): fix alignment and indentation
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Philippe Proulx [Tue, 29 Nov 2016 02:22:19 +0000 (21:22 -0500)]
lttng-ust(3): document new ctf_*() array/sequence network/hex macros
Signed-off-by: Philippe Proulx <eeppeliteloop@gmail.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Mathieu Desnoyers [Mon, 28 Nov 2016 17:40:13 +0000 (12:40 -0500)]
Filter code relicensing to MIT license
Relicense the filtering code to MIT license.
I am the only author of this code.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Mathieu Desnoyers [Mon, 21 Nov 2016 20:58:29 +0000 (15:58 -0500)]
Performance: add missing "caa_unlikely" on fast-path
There is a missing branch prediction hint on the return value of
lib_ring_buffer_reserve().
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Mathieu Desnoyers [Wed, 9 Nov 2016 13:31:13 +0000 (08:31 -0500)]
Fix: blocking mode: add missing stdbool.h include
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Mathieu Desnoyers [Fri, 4 Nov 2016 13:42:03 +0000 (07:42 -0600)]
Implement LTTNG_UST_BLOCKING_RETRY_TIMEOUT
Add LTTNG_UST_BLOCKING_RETRY_TIMEOUT environment variable:
LTTNG_UST_BLOCKING_RETRY_TIMEOUT
Maximum duration (milliseconds) to retry event tracing when
there’s no space left for the event record in the
sub-buffer.
0 (default)
Never block the application.
Positive value
Block the application for the specified number of
milliseconds. If there’s no space left after this
duration, discard the event record.
Negative value
Block the application until there’s space left for the
event record.
This option can be useful in workloads generating very
large trace data throughput, where blocking the application
is an acceptable trade-off to prevent discarding event
records.
Warning
Setting this environment variable to a non-zero value
may significantly affect application timings.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
CC: "Carlos O'Donell" <carlos@redhat.com>
Mathieu Desnoyers [Tue, 18 Oct 2016 22:43:04 +0000 (18:43 -0400)]
Fix: loglevel and model_emf_uri build fix
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Mathieu Desnoyers [Tue, 18 Oct 2016 21:56:42 +0000 (17:56 -0400)]
Fix: loglevel and model_emf_uri with g++ compiled probes
Fix the loglevel and model_emf_uri features for probe providers compiled
with g++. They were previously effectless because of C++ symbol name
mangling. The weakref was refering to the non-mangled symbol, but C++
emits a mangled symbol for the static variable.
Fix this by emitting an extern "C" symbol with hidden visibility on C++.
With a C compiled, this simply turns a static variable into a variable
with hidden visibility.
Fixes: #1069
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Michael Jeanson [Fri, 14 Oct 2016 20:45:20 +0000 (16:45 -0400)]
Fix: Out of tree build of liblttng-ust-java
Signed-off-by: Michael Jeanson <mjeanson@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Mathieu Desnoyers [Fri, 7 Oct 2016 19:25:43 +0000 (15:25 -0400)]
Version 2.9.0-rc1
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Jonathan Rajotte [Tue, 4 Oct 2016 19:17:05 +0000 (15:17 -0400)]
Fix: cmake example with configure based oot build
Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Mathieu Desnoyers [Mon, 3 Oct 2016 17:18:53 +0000 (13:18 -0400)]
Fix: perform statedump before replying to sessiond
If a stop command immediately follows a start command, the consumer
daemon will stop event recording in the ring buffers shared memory
control structures before the sessiond sends further commands to the
application. Therefore, a stop-after-start may be performed concurrently
with the statedump, leading to have parts of the statedump being
missing. This case may always happen if an application exits during
statedump, but it is not expected to have incomplete statedump in the
stop-after-start use case.
The session daemon statedump regeneration tests expect that the
statedump is completed when the regeneration command returns. This also
requires that we perform the statedump in lttng-ust before replying to
the session daemon command.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Jonathan Rajotte [Mon, 3 Oct 2016 15:15:21 +0000 (11:15 -0400)]
build: check for CXX_WORKS for cmake example
Tracepoints and tracepoint provider compilation need a functional
c++ compiler (g++ > 4.7).
Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Mathieu Desnoyers [Mon, 12 Sep 2016 16:04:13 +0000 (12:04 -0400)]
Implement liblttng-ust-fd
This library overrides close() and closeall() libc functions, and uses
lttng_ust_safe_close_fd() to check whether the application can
interact with the file descriptor or if it should be left to lttng-ust.
This takes care of bugs caused by applications doing bulk close() or
closefrom() of file descriptors soon after forking.
Fixes: #253
Fixes: #626
CC: Aravind HT <aravind.ht@gmail.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Mathieu Desnoyers [Tue, 13 Sep 2016 20:17:40 +0000 (16:17 -0400)]
Introduce file descriptor tracker
Introduce a tracker for file descriptors used by lttng-ust. It exposes
a new API in an internal header lttng_ust_safe_close_fd(), which is
meant to be used by a LD_PRELOADed library overriding close() and
closefrom() (BSD).
This takes care of bugs caused by applications doing bulk close() or
closefrom() of file descriptors soon after forking.
We need to hold the ust_lock() to protect the fd tracker lock against
fork. Since the fd tracker is needed across connect() (which allocates a
file descriptor), we need to hold the ust_lock across connect().
Fixes: #253
Fixes: #626
Suggested-by: Aravind HT <aravind.ht@gmail.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Mathieu Desnoyers [Tue, 13 Sep 2016 20:17:04 +0000 (16:17 -0400)]
Fix: honor send timeout on unix socket connect
Needed if we want to hold the ust_lock() while we connect to the session
daemon without blocking the application forever if the session daemon is
hung on SIGSTOP.
This only triggers if we launchs _many_ applications with a session
daemon SIGSTOP'd (e.g. 1000 in parallel), so we fill the socket queue,
and applications hang there until the session daemon is SIGCONT'd.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Mathieu Desnoyers [Sat, 1 Oct 2016 22:59:17 +0000 (18:59 -0400)]
Fix: perform TLS fixup in all UST entry points from each thread
Each entry point into lttng-ust that end up taking the ust lock need to
perform a TLS fixup for each thread. Add a TLS fixup in both listener
threads, in fork and base address dump helper libs, and in app context
and tracepoint probe registration/unregistration functions, which can be
called from application threads.
Those ensure we don't take the libc dl lock within the ust lock when
performing the TLS lazy fixup.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Jonathan Rajotte [Mon, 3 Oct 2016 14:42:45 +0000 (10:42 -0400)]
Fix: build: pass configure flags to cmake example
Signed-off-by: Jonathan Rajotte <jonathan.rajotte-julien@efficios.com>
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Mathieu Desnoyers [Sun, 25 Sep 2016 16:31:11 +0000 (12:31 -0400)]
Performance: implement lttng_inline_memcpy
Because all length parameters received for serializing data coming from
applications go through a callback, they are never constant, and it
hurts performance to perform a call to memcpy each time.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Mathieu Desnoyers [Sun, 25 Sep 2016 14:50:22 +0000 (10:50 -0400)]
Performance: mark ring buffer do_copy callers always inline
The underlying copy operation is more efficient if the size is a
constant, which only happens if this function is inlined in the caller.
Otherwise, we end up calling memcpy for each field.
Force inlining for performance reasons for:
- lib_ring_buffer_write,
- lib_ring_buffer_do_strcpy,
- lib_ring_buffer_strcpy.
Note that in lttng-ust, the probe provider serialization functions need
to call the lttng_event_write() client callback, which will fallback to
the memcpy operation.
Inlining those functions helps for the event header code, which can
inline them.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Mathieu Desnoyers [Mon, 19 Sep 2016 02:52:39 +0000 (22:52 -0400)]
Performance: cache the backend pages pointer in context
Getting the backend pages pointer requires walking through the ring
buffer backend tables through multiple shmp operations. Cache the
current value so it can be re-used for all backend write operations
writing fields for the same event.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Mathieu Desnoyers [Sat, 24 Sep 2016 14:25:10 +0000 (10:25 -0400)]
Add ctx_len field to ring buffer context
Allow extending the structure beyond its original size.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Mathieu Desnoyers [Mon, 19 Sep 2016 02:10:53 +0000 (22:10 -0400)]
ring buffer: handle shmp errors
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Mathieu Desnoyers [Mon, 19 Sep 2016 01:45:32 +0000 (21:45 -0400)]
Cleanup: libringbuffer: remove duplicate shmp() invocations in slow paths
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Mathieu Desnoyers [Mon, 19 Sep 2016 01:38:38 +0000 (21:38 -0400)]
Performance: Only dereference shmp once
Dereferencing shared-memory pointers is expensive: many bound checks are
performed. Ensure the shmp is only dereferenced once in the commit
fast-path.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Mathieu Desnoyers [Mon, 19 Sep 2016 01:31:02 +0000 (21:31 -0400)]
Performance: Mark channel and buffer event disabled check unlikely
Channel and buffer are typically enabled when tracing, else the probes
would not be connected. Ensure the compiler optimizes for the right
usual case.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Mathieu Desnoyers [Sun, 18 Sep 2016 04:25:58 +0000 (00:25 -0400)]
Performance: Relax atomicity constraints for crash handling
Use a store rather than a cmpxchg() for the update of the
sequential commit counter. This speeds up commit. The downside
is that short race windows between the if() check to see if the
counter is larger than the new value and the update could result
in the counter going backwards, in unlikely preemption or signal
delivery scenarios.
Accept that we may lose a few events in a crash dump for the
benefit of tracing speed.
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Mathieu Desnoyers [Mon, 12 Sep 2016 22:58:50 +0000 (18:58 -0400)]
Fix: compat: -ust suffix sprintf error handling
Signed-off-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
This page took 0.045525 seconds and 4 git commands to generate.