1 #LyX 1.5.5 created this file. For more info see http://www.lyx.org/
10 \font_typewriter default
11 \font_default_family default
17 \paperfontsize default
25 \paperorientation portrait
28 \paragraph_separation skip
30 \quotes_language english
33 \paperpagestyle default
34 \tracking_changes false
43 A Set of APIs for a Third-Generation Trace Viewer
48 Pierre-Marc Fournier, Michel Dagenais, Mathieu Desnoyers
53 <pierre-marc.fournier _at_ polymtl.ca>
62 École Polytechnique de Montréal
69 \begin_layout Standard
70 This document proposes some APIs for a trace viewing and analysis infrastructure.
73 \begin_layout Standard
74 Design choices were made based on the experience acquired by the design,
75 development and use of two previous generations of trace viewers, respectively
76 the Trace Visualizer from the Linux Trace Toolkit and its successor, the
77 LTTV from the LTTng tracing toolkit.
85 Provide an infrastructure for fast, interactive visualization and analysis
86 of huge traces (>10 GB) on standard workstations
90 \begin_layout Standard
91 Efficiency is of great importance.
92 It is tempting to think that as computers get faster, it will be possible
93 to run more complex analyses on traces in a shorter time.
94 In fact, the size of the traces grows as computers get more powerful, because
95 they do more operations than before in the same amount of time and therefore
97 The current tendency to parallelize systems is another factor that results
106 \begin_layout Itemize
107 Allow efficient manipulation of traces that are larger than the system memory
110 \begin_layout Itemize
111 Support processing of live traces
114 \begin_layout Itemize
115 Allow merging of traces of different formats
118 \begin_layout Itemize
119 Design with both command line utilities (C language) and graphical interfaces
120 (Java and others) in mind, sharing code and standardizing interfaces as
125 \begin_layout Standard
126 Both command-line interfaces and graphical interfaces have an important
127 role to play in trace analysis.
128 While graphical interfaces allow for complex views, command-line utilities
129 can be used quickly in more environments; they can also be more convenient
130 in some circumstances.
131 A good design should be such that they can share code and APIs.
135 \begin_layout Section
139 \begin_layout Itemize
140 All data structures must be accessed through API methods, in order to allow
141 for internal change without breaking compatibility.
144 \begin_layout Itemize
145 The APIs in this document are described as C functions and datatypes.
146 When an operation applies to an abstraction that could be an object in
147 an object-oriented language, it is specified as the first argument of the
151 \begin_layout Subsection
155 \begin_layout Standard
156 The framework needs to represent points in the time line and time ranges.
157 This is done by two data structures, respectively:
160 \begin_layout Itemize
164 \begin_layout Itemize
165 struct trace_time_range
168 \begin_layout Standard
169 The preferred representation for times is seconds.
172 \begin_layout Standard
173 Ranges are a span between two points in time.
174 They are represented as these two points.
175 One or both of them can have the special value
182 \begin_layout Standard
183 Each time is accompanied by an uncertainty value.
186 \begin_layout Subsection
190 \begin_layout Standard
191 The following modules should be part of the framework.
192 This list does not include modules related to the GUI, the controlling
193 of traces and the transfer of traces.
196 \begin_layout Description
197 Trace Has one instance per open trace.
198 Allows the access to a trace by exporting the Low-level Trace Reading API.
199 The underlying implementation may change from one trace to another to adapt
200 to the specific format of each trace.
203 \begin_layout Description
204 Traceset Receives data from several Traces using their Low-Level Trace Reading
205 API and merge them in a single stream.
206 May also receive data from other instances of Traceset.
207 Exports the High-level Trace Reading API and the Low-level Trace Reading
209 Handles time offsetting if necessary to align traces.
213 \begin_layout Description
215 providers They receive the stream of events from the Traceset, using
216 the High-level Trace Reading API.
217 They detect state changes.
218 Each one is designed to detect states of a particular level (kernel, applicatio
219 n-level, UML states).
220 It pushes the detected states to the State module using the State Setting
224 \begin_layout Description
225 State Records changes in the system state received from State providers
226 by the State Setting API.
227 It implements an efficient state searching engine.
228 Exports the State Accessing API to allow plugins to access the information.
231 \begin_layout Description
233 description Receives the stream of events from the Traceset using the
234 High-level Trace Reading API.
235 It reads the events that describe other events and stores the information
236 they contain, possibly using the State module.
237 Other components can access this information through an exported API.
240 \begin_layout Description
242 These are the graphical views, analyses and other components that need
243 access to the events.
244 They receive data from a traceset using the High-level Trace Reading API.
245 They use the services they need (State, Event Description, and/or others).
246 They may export a special API if other components need to access their
250 \begin_layout Section
251 Low level reading of traces
254 \begin_layout Subsection
258 \begin_layout Standard
259 This API presents a uniform way of accessing traces at a low level, that
260 is to access randomly the events of the trace in an physical-encoding-independa
264 \begin_layout Standard
265 \begin_inset Note Note
268 \begin_layout Subsubsection
272 \begin_layout Standard
273 Timestamps, at the framework level, should be in an architecture-independent
275 One possibility would be to have them stored in seconds (with decimals).
276 Offsetting, if necessary, could be handled by the traceset (trace merging)
278 An uncertainty value should accompany the timestamp.
279 Timestamps in cycles should be available if they are provided by the tracer.
287 \begin_layout Standard
288 \begin_inset Note Note
291 \begin_layout Subsubsection
295 \begin_layout Itemize
296 void get_supported_formats(void)
300 \begin_layout Standard
304 \begin_layout Standard
308 \begin_layout Itemize
309 a list of supported formats
313 \begin_layout Itemize
318 \begin_layout Standard
322 \begin_layout Itemize
327 \begin_layout Itemize
331 \begin_layout Itemize
332 other special url to establish a network connection
335 \begin_layout Itemize
340 \begin_layout Itemize
341 trace format (optional, for cases where auto detection would not work or
345 \begin_layout Standard
349 \begin_layout Itemize
358 \begin_layout Subsection
359 Methods of the Low Level Trace Reading API
362 \begin_layout Itemize
363 void close(struct trace *tr)
367 \begin_layout Standard
368 Close the trace and unallocate all resources associated to this struct trace
369 including the handle.
370 After this call, tr is invalid and its memory is freed.
373 \begin_layout Standard
377 \begin_layout Itemize
381 \begin_layout Standard
386 \begin_layout Itemize
387 struct event *current_event(struct trace *tr, struct cursor *cur)
391 \begin_layout Standard
392 Return the event pointed by the cursor cur.
395 \begin_layout Standard
399 \begin_layout Itemize
403 \begin_layout Itemize
404 cur: the cursor indicating the position of the wanted event
407 \begin_layout Standard
411 \begin_layout Itemize
412 the event currently pointed by the position pointer
415 \begin_layout Itemize
416 NULL, if current_event is called on a new cursor that is positioned before
417 the beginning of the trace
421 \begin_layout Itemize
422 int advance(struct trace *tr, struct trace_cursor *cur)
426 \begin_layout Standard
427 Advance the cursor to the next event in the trace.
430 \begin_layout Standard
434 \begin_layout Itemize
438 \begin_layout Itemize
439 cur: the cursor that should be advanced
442 \begin_layout Standard
446 \begin_layout Itemize
451 \begin_layout Itemize
452 Success (TRACE_ADVANCE_OK)
455 \begin_layout Itemize
456 The cursor was not advanced because the end of trace is reached (end of
457 a non-live trace or of a completed live trace) (TRACE_ADVANCE_END)
460 \begin_layout Itemize
461 The cursor was not advanced because no new events are available at this
462 time (with a live trace that is still running) (TRACE_ADVANCE_TRY_AGAIN)
467 \begin_layout Itemize
468 int seek_time(struct trace *tr, struct trace_cursor *cur, struct trace_time
473 \begin_layout Standard
474 Place a cursor at a particular time index in a trace.
477 \begin_layout Standard
481 \begin_layout Itemize
482 tr: the trace that should be seeked
485 \begin_layout Itemize
486 cur: the cursor to seek
489 \begin_layout Itemize
490 time: the time to seek to
493 \begin_layout Standard
497 \begin_layout Itemize
501 \begin_layout Itemize
502 != 0: time out of range
506 \begin_layout Itemize
507 struct trace_cursor *trace_new_cursor(struct trace *tr)
511 \begin_layout Standard
512 Return a new cursor associated to the trace.
513 The position of this new cursor is just before the first event, therefore
514 advance() must be called before reading the first event.
517 \begin_layout Standard
521 \begin_layout Itemize
525 \begin_layout Standard
529 \begin_layout Itemize
534 \begin_layout Itemize
535 struct trace_cursor *trace_cursor_copy(struct trace_cursor *cur)
539 \begin_layout Standard
543 \begin_layout Standard
547 \begin_layout Itemize
548 cur: the cursor to copy
551 \begin_layout Standard
555 \begin_layout Itemize
556 a new cursor that is at the same location as cur
560 \begin_layout Itemize
561 void trace_cursor_destroy(struct trace_cursor *cur)
565 \begin_layout Standard
566 Free all resources associated to a cursor.
567 After this call, cur is invalid.
570 \begin_layout Standard
574 \begin_layout Itemize
575 cur: the cursor to destroy
578 \begin_layout Standard
583 \begin_layout Itemize
584 struct trace_time get_time_start(struct trace *tr)
588 \begin_layout Standard
592 \begin_layout Itemize
596 \begin_layout Standard
600 \begin_layout Itemize
601 the timestamp of the first event in the trace
605 \begin_layout Itemize
606 struct trace_time get_time_end(struct trace *tr)
610 \begin_layout Standard
614 \begin_layout Standard
618 \begin_layout Itemize
619 the timestamp of the last event in the trace
623 \begin_layout Itemize
624 register_callback_new_event(struct trace *tr, struct trace_time after, void
625 (*cb)(struct trace *, void *priv), void *private)
629 \begin_layout Standard
630 Register a callback that is called when a new event becomes available in
632 It is also called when the live trace ends.
635 \begin_layout Standard
639 \begin_layout Itemize
640 after (timestamp) : call only if the timestamp of the event is later than
646 \begin_layout Itemize
647 cb: the callback function
648 \begin_inset Note Note
651 \begin_layout Standard
652 specify its args and return val
660 \begin_layout Standard
664 \begin_layout Itemize
665 Result (success or failure)
669 \begin_layout Section
670 High level reading of traces
673 \begin_layout Subsection
677 \begin_layout Standard
678 When reading/analyzing/viewing several traces of heterogenous types, these
679 traces are read by translator modules, which export the Low Level Trace
681 The traceset service then uses this API to read each of these traces individual
682 ly, merging them along the way.
683 It may apply timestamp offsetting or other synchronization techniques.
684 To allow views and analyses to access events, it in turn exports the High
685 Level Trace Reading API.
688 \begin_layout Standard
689 The goal of this API is to provide a uniform way for analyses and views
690 to obtain large sets of trace events from a traceset (merge of many traces
694 \begin_layout Subsection
695 Methods of the high-level trace reading API
698 \begin_layout Itemize
699 struct request_handle *traceset_new_event_request(struct traceset *tr, struct
700 trace_time t1, struct trace_time t2, struct event_filter *filter, void
701 (*cb)(void *priv, ), void *private)
705 \begin_layout Standard
706 Request a range of events from a traceset
709 \begin_layout Standard
713 \begin_layout Itemize
717 \begin_layout Itemize
721 \begin_layout Itemize
722 t2: stop timestamp (special value for infinity, for live traces)
725 \begin_layout Itemize
726 filter: filter with complex expressions
729 \begin_layout Itemize
730 private: private pointer to be passed to the callback
733 \begin_layout Standard
737 \begin_layout Itemize
738 handle to the request for cancelling it
742 \begin_layout Itemize
743 void event_request_cancel(struct request_handle *req)
747 \begin_layout Standard
751 \begin_layout Standard
755 \begin_layout Itemize
756 req: the handle to the request
760 \begin_layout Itemize
761 void traceset_process(struct traceset *tr, struct timeval *tv)
765 \begin_layout Standard
766 Process requests for the time specified by tv, then return to the caller.
767 The time is a wall time and has no relationship with the timestamps of
769 It is used to process a part of the trace while allowing a GUI to remain
773 \begin_layout Standard
777 \begin_layout Itemize
781 \begin_layout Itemize
782 tv: the duration for which to process the trace
786 \begin_layout Section
790 \begin_layout Standard
791 States are key/value pairs associated with a time range.
792 Keys can be (and generally are) duplicated as long as they do not apply
793 to overlapping ranges.
796 \begin_layout Standard
797 Keys are character strings.
798 They are organized in a filesystem-like hierarchy.
801 \begin_layout Standard
802 Each key/value pair is associated either to a specific trace or to the traceset.
805 \begin_layout Standard
806 State can persist between runs of the program.
807 This is useful both to reduce pre-calculation times when re-opening a trace,
808 and also to keep general parameters like bookmarks.
811 \begin_layout Standard
812 It is possible to assign a state to the time range -infinity..infinity to
813 indicate that it is global to the trace.
816 \begin_layout Standard
817 Values may be of various types:
820 \begin_layout Itemize
824 \begin_layout Itemize
828 \begin_layout Itemize
832 \begin_layout Itemize
836 \begin_layout Itemize
840 \begin_layout Itemize
841 blob (binary block of arbitrary length)
844 \begin_layout Itemize
848 \begin_layout Itemize
852 \begin_layout Itemize
856 \begin_layout Subsection
857 Methods of the State Accessing API
860 \begin_layout Itemize
861 struct state_value *state_get_value_at_time(char *key, struct trace_time
866 \begin_layout Standard
867 Request the value of a given key at a point in time
870 \begin_layout Standard
874 \begin_layout Itemize
875 var: the state variables (string)
878 \begin_layout Itemize
882 \begin_layout Standard
886 \begin_layout Itemize
888 A struct state_value contains the value and the time interval that applies
893 \begin_layout Itemize
894 struct state_value_range **state_get_values_in_range(char *key, struct state_val
895 ue *val, struct trace_time_range range)
899 \begin_layout Standard
900 Request all the state changes of a given set of state variables between
904 \begin_layout Standard
908 \begin_layout Itemize
912 \begin_layout Itemize
913 range: the time range
916 \begin_layout Standard
920 \begin_layout Itemize
925 \begin_layout Itemize
926 Other functions for getting values for a set of keys at once?
929 \begin_layout Subsection
930 Methods of the State Setting API
933 \begin_layout Itemize
934 set a particular state
937 \begin_layout Itemize
941 \begin_layout Section
942 Describing event types
945 \begin_layout Subsection
949 \begin_layout Standard
950 Because tracepoints may be created dynamically, information about the descriptio
951 n of events is just as dynamic.
952 In this context, one simple way to communicate the event description informatio
953 n to upper layers would be to send them as events, as it is done in recent
955 The core events used to describe other events are the only ones whose descripti
956 on is hardcoded in the framework.
959 \begin_layout Standard
960 These event-type-describing events could then be received and interpreted
961 by the Event Description Service, which would be a client to the high-level
962 tracing API at the same level as normal views and analyses.
963 It would store the information and allow the other views and analyses to
964 access it via this API.
967 \begin_layout Standard
968 Each event has a timestamp, a name and arguments of various types.
969 The framework should support the following types:
972 \begin_layout Itemize
976 \begin_layout Itemize
980 \begin_layout Itemize
984 \begin_layout Itemize
988 \begin_layout Itemize
992 \begin_layout Subsection
993 Events-describing events
996 \begin_layout Itemize
997 Event type declaration event
1001 \begin_layout Standard
1002 Announce the existence of an event type
1005 \begin_layout Itemize
1010 \begin_layout Itemize
1011 Argument declaration event
1015 \begin_layout Standard
1016 Announce the existence of an event argument
1019 \begin_layout Itemize
1023 \begin_layout Itemize
1027 \begin_layout Itemize
1032 \begin_layout Itemize
1037 \begin_layout Standard
1038 Announce that an event type ceased to exist
1041 \begin_layout Itemize
1046 \begin_layout Subsection
1047 Methods of the Event Type Description API
1050 \begin_layout Standard
1051 The event type description service provides the following functions.
1054 \begin_layout Itemize
1055 GArray<struct event_type *> *traceset_get_all_event_types(struct traceset
1060 \begin_layout Standard
1061 Get the list of all the event types
1064 \begin_layout Standard
1068 \begin_layout Itemize
1069 ts: the traceset of which we want the event types
1072 \begin_layout Standard
1076 \begin_layout Itemize
1077 A GArray of of struct event_type.
1078 The GArray must be gfree()'d by the caller when it is done reading it.
1082 \begin_layout Itemize
1083 struct event_type *traceset_get_event_type_by_name(struct traceset *ts,
1088 \begin_layout Standard
1089 Find an event type by name
1092 \begin_layout Standard
1096 \begin_layout Itemize
1097 ts: the traceset of which we want the event type
1100 \begin_layout Itemize
1101 name: the name of the of the event type we are looking for
1104 \begin_layout Standard
1108 \begin_layout Itemize
1109 A pointer to the event type (must not be free'd) or NULL if not found
1113 \begin_layout Itemize
1114 GArray<struct event arg *> *event_type_get_all_args(struct event_type *evtype)
1118 \begin_layout Standard
1119 Get the list of arguments of an event
1122 \begin_layout Standard
1126 \begin_layout Itemize
1127 eventype: the event type of which we want the arguments
1130 \begin_layout Standard
1134 \begin_layout Itemize
1135 A GArray of struct event_args.
1136 The GArray must be gfree()'d by the caller when it is done reading it.
1140 \begin_layout Itemize
1141 struct event_arg *event_type_get_arg_by_name(struct event_type *evtype)
1145 \begin_layout Standard
1146 Find an argument by name
1150 \begin_layout Itemize
1151 Functions for accessing struct event_arg fields
1154 \begin_layout Section
1158 \begin_layout Subsection
1162 \begin_layout Standard
1163 Events contain the following information.
1166 \begin_layout Itemize
1170 \begin_layout Itemize
1171 Event type identifier - an event id (integer) - hidden to the API users,
1172 manipulated as pointers/references to struct event_type
1175 \begin_layout Itemize
1176 A reference to the trace it was in
1179 \begin_layout Subsection
1180 Methods of the Event inspecting API
1183 \begin_layout Itemize
1184 struct event_type *event_get_type(struct traceset *ts, struct event *ev)
1188 \begin_layout Standard
1189 get the event type corresponding to an event
1192 \begin_layout Standard
1196 \begin_layout Itemize
1200 \begin_layout Itemize
1204 \begin_layout Standard
1208 \begin_layout Itemize
1209 The event type or NULL if no information
1213 \begin_layout Itemize
1214 struct trace_time event_get_time(struct event *ev)
1218 \begin_layout Standard
1223 \begin_layout Itemize
1224 struct trace *event_get_trace(struct event *ev)
1227 \begin_layout Itemize
1228 get the name of the machine on which the event occured or other location
1232 \begin_layout Itemize
1233 get information on the type of tracing technology that was used
1236 \begin_layout Itemize
1237 get the corresponding tracepoint (machine/tracing technology/name/location
1238 in code(if available))
1241 \begin_layout Itemize
1242 uint32 event_read_arg_uint32(struct event *ev, struct event_arg *arg)
1245 \begin_layout Itemize
1246 int32 event_read_arg_int32(struct event *ev, struct event_arg *arg)
1249 \begin_layout Itemize
1250 uint64 event_read_arg_uint64(struct event *ev, struct event_arg *arg)
1253 \begin_layout Itemize
1254 int64 event_read_arg_int64(struct event *ev, struct event_arg *arg)
1257 \begin_layout Itemize
1258 float32 event_read_arg_float32(struct event *ev, struct event_arg *arg)
1261 \begin_layout Itemize
1262 float64 event_read_arg_float64(struct event *ev, struct event_arg *arg)
1265 \begin_layout Section
1269 \begin_layout Standard
1270 A filtering API is proposed.
1273 \begin_layout Section
1274 Controlling the tracing of a system
1277 \begin_layout Subsection
1281 \begin_layout Standard
1282 The goal of this API is to provide a portable way of controlling tracing
1283 of heterogenous systems.
1286 \begin_layout Standard
1287 It should enable to do the following actions.
1290 \begin_layout Itemize
1291 Set the parameters of a trace (channels, buffering, destination of data
1292 (file, network, process)...)
1295 \begin_layout Itemize
1296 Control the recording of events (start, stop, pause the trace)
1299 \begin_layout Itemize
1300 Create tracepoints (on dynamic tracers) and control which tracepoints are
1301 activated (some may be at trace level, others might only permit system
1305 \begin_layout Subsection
1306 Methods of the Trace Control API
1309 \begin_layout Itemize
1310 List the static tracepoints available on a system
1314 \begin_layout Standard
1315 These may be static tracepoints (active or inactive) or dynamic tracepoints
1316 (active or proposed).
1320 \begin_layout Itemize
1321 Add a new dynamic tracepoint
1324 \begin_layout Itemize
1325 Activate a tracepoint
1328 \begin_layout Itemize
1329 Deactivate a tracepoint
1332 \begin_layout Itemize
1333 List available probes
1336 \begin_layout Itemize
1337 Connect a probe to a tracepoint
1340 \begin_layout Itemize
1344 \begin_layout Itemize
1348 \begin_layout Itemize
1349 \begin_inset Note Greyedout
1352 \begin_layout Standard
1361 \begin_layout Standard
1362 \begin_inset Note Note
1365 \begin_layout Section*
1369 \begin_layout Itemize
1370 extend APIs for monitoring (case where the flow of events is constant and
1371 we cannot keep the whole trace forever)
1375 \begin_layout Itemize
1376 views must be able to register the fact that they need the whole trace if
1380 \begin_layout Itemize
1381 there must be an elegant way to destroy old events
1385 \begin_layout Itemize
1386 add API or infos about interaction with gui? ex: how traceset_process should