1. 15 Mar, 2013 14 commits
    • Steven Rostedt (Red Hat)'s avatar
      tracing: Clear all trace buffers when unloaded module event was used · 873c642f
      Steven Rostedt (Red Hat) authored
      
      
      Currently we do not know what buffer a module event was enabled in.
      On unload, it is safest to clear all buffer instances, not just the
      top level buffer.
      
      Todo: Clear only the buffer that the event was used in. The
      infrastructure is there to do this, but it makes the code a bit
      more complex. Lets get the current code vetted before we add that.
      Signed-off-by: default avatarSteven Rostedt <rostedt@goodmis.org>
      873c642f
    • Steven Rostedt (Red Hat)'s avatar
      tracing/ring-buffer: Move poll wake ups into ring buffer code · 15693458
      Steven Rostedt (Red Hat) authored
      
      
      Move the logic to wake up on ring buffer data into the ring buffer
      code itself. This simplifies the tracing code a lot and also has the
      added benefit that waiters on one of the instance buffers can be woken
      only when data is added to that instance instead of data added to
      any instance.
      Signed-off-by: default avatarSteven Rostedt <rostedt@goodmis.org>
      15693458
    • Steven Rostedt's avatar
      tracing: Fix read blocking on trace_pipe_raw · b627344f
      Steven Rostedt authored
      
      
      If the ring buffer is empty, a read to trace_pipe_raw wont block.
      The tracing code has the infrastructure to wake up waiting readers,
      but the trace_pipe_raw doesn't take advantage of that.
      
      When a read is done to trace_pipe_raw without the O_NONBLOCK flag
      set, have the read block until there's data in the requested buffer.
      Signed-off-by: default avatarSteven Rostedt <rostedt@goodmis.org>
      b627344f
    • Steven Rostedt's avatar
      tracing: Fix polling on trace_pipe_raw · cc60cdc9
      Steven Rostedt authored
      
      
      The trace_pipe_raw never implemented polling and this was casing
      issues for several utilities. This is now implemented.
      
      Blocked reads still are on the TODO list.
      Reported-by: default avatarMauro Carvalho Chehab <mchehab@redhat.com>
      Tested-by: default avatarMauro Carvalho Chehab <mchehab@redhat.com>
      Signed-off-by: default avatarSteven Rostedt <rostedt@goodmis.org>
      cc60cdc9
    • Steven Rostedt (Red Hat)'s avatar
      tracing: Do not block on splice if either file or splice NONBLOCK flag is set · 189e5784
      Steven Rostedt (Red Hat) authored
      
      
      Currently only the splice NONBLOCK flag is checked to determine if
      the splice read should block or not. But the file descriptor NONBLOCK
      flag also needs to be checked.
      Signed-off-by: default avatarSteven Rostedt <rostedt@goodmis.org>
      189e5784
    • Steven Rostedt's avatar
      tracing: Add rmdir to remove multibuffer instances · 0c8916c3
      Steven Rostedt authored
      
      
      Add a method to the hijacked dentry descriptor of the
      "instances" directory to allow for rmdir to remove an
      instance of a multibuffer.
      
      Example:
      
        cd /debug/tracing/instances
        mkdir hello
        ls
      hello/
        rmdir hello
        ls
      
      Like the mkdir method, the i_mutex is dropped for the instances
      directory. The instances directory is created at boot up and can
      not be renamed or removed. The trace_types_lock mutex is used to
      synchronize adding and removing of instances.
      
      I've run several stress tests with different threads trying to
      create and delete directories of the same name, and it has stood
      up fine.
      
      Cc: Al Viro <viro@zeniv.linux.org.uk>
      Signed-off-by: default avatarSteven Rostedt <rostedt@goodmis.org>
      0c8916c3
    • Steven Rostedt's avatar
      tracing: Add interface to allow multiple trace buffers · 277ba044
      Steven Rostedt authored
      
      
      Add the interface ("instances" directory) to add multiple buffers
      to ftrace. To create a new instance, simply do a mkdir in the
      instances directory:
      
      This will create a directory with the following:
      
       # cd instances
       # mkdir foo
       # ls foo
      buffer_size_kb        free_buffer  trace_clock    trace_pipe
      buffer_total_size_kb  set_event    trace_marker   tracing_enabled
      events/               trace        trace_options  tracing_on
      
      Currently only events are able to be set, and there isn't a way
      to delete a buffer when one is created (yet).
      
      Note, the i_mutex lock is dropped from the parent "instances"
      directory during the mkdir operation. As the "instances" directory
      can not be renamed or deleted (created on boot), I do not see
      any harm in dropping the lock. The creation of the sub directories
      is protected by trace_types_lock mutex, which only lets one
      instance get into the code path at a time. If two tasks try to
      create or delete directories of the same name, only one will occur
      and the other will fail with -EEXIST.
      
      Cc: Al Viro <viro@zeniv.linux.org.uk>
      Signed-off-by: default avatarSteven Rostedt <rostedt@goodmis.org>
      277ba044
    • Steven Rostedt's avatar
      tracing: Replace the static global per_cpu arrays with allocated per_cpu · a7603ff4
      Steven Rostedt authored
      
      
      The global and max-tr currently use static per_cpu arrays for the CPU data
      descriptors. But in order to get new allocated trace_arrays, they need to
      be allocated per_cpu arrays. Instead of using the static arrays, switch
      the global and max-tr to use allocated data.
      Signed-off-by: default avatarSteven Rostedt <rostedt@goodmis.org>
      a7603ff4
    • Steven Rostedt's avatar
      tracing: Pass the ftrace_file to the buffer lock reserve code · ccb469a1
      Steven Rostedt authored
      
      
      Pass the struct ftrace_event_file *ftrace_file to the
      trace_event_buffer_lock_reserve() (new function that replaces the
      trace_current_buffer_lock_reserver()).
      
      The ftrace_file holds a pointer to the trace_array that is in use.
      In the case of multiple buffers with different trace_arrays, this
      allows different events to be recorded into different buffers.
      
      Also fixed some of the stale comments in include/trace/ftrace.h
      Signed-off-by: default avatarSteven Rostedt <rostedt@goodmis.org>
      ccb469a1
    • Steven Rostedt's avatar
      tracing: Encapsulate global_trace and remove dependencies on global vars · 2b6080f2
      Steven Rostedt authored
      
      
      The global_trace variable in kernel/trace/trace.c has been kept 'static' and
      local to that file so that it would not be used too much outside of that
      file. This has paid off, even though there were lots of changes to make
      the trace_array structure more generic (not depending on global_trace).
      
      Removal of a lot of direct usages of global_trace is needed to be able to
      create more trace_arrays such that we can add multiple buffers.
      Signed-off-by: default avatarSteven Rostedt <rostedt@goodmis.org>
      2b6080f2
    • Steven Rostedt's avatar
      tracing: Use RING_BUFFER_ALL_CPUS for TRACE_PIPE_ALL_CPU · ae3b5093
      Steven Rostedt authored
      
      
      Both RING_BUFFER_ALL_CPUS and TRACE_PIPE_ALL_CPU are defined as
      -1 and used to say that all the ring buffers are to be modified
      or read (instead of just a single cpu, which would be >= 0).
      
      There's no reason to keep TRACE_PIPE_ALL_CPU as it is also started
      to be used for more than what it was created for, and now that
      the ring buffer code added a generic RING_BUFFER_ALL_CPUS define,
      we can clean up the trace code to use that instead and remove
      the TRACE_PIPE_ALL_CPU macro.
      Signed-off-by: default avatarSteven Rostedt <rostedt@goodmis.org>
      ae3b5093
    • Steven Rostedt's avatar
      tracing: Separate out trace events from global variables · ae63b31e
      Steven Rostedt authored
      
      
      The trace events for ftrace are all defined via global variables.
      The arrays of events and event systems are linked to a global list.
      This prevents multiple users of the event system (what to enable and
      what not to).
      
      By adding descriptors to represent the event/file relation, as well
      as to which trace_array descriptor they are associated with, allows
      for more than one set of events to be defined. Once the trace events
      files have a link between the trace event and the trace_array they
      are associated with, we can create multiple trace_arrays that can
      record separate events in separate buffers.
      Signed-off-by: default avatarSteven Rostedt <rostedt@goodmis.org>
      ae63b31e
    • Steven Rostedt (Red Hat)'s avatar
      tracing: Prevent buffer overwrite disabled for latency tracers · 613f04a0
      Steven Rostedt (Red Hat) authored
      
      
      The latency tracers require the buffers to be in overwrite mode,
      otherwise they get screwed up. Force the buffers to stay in overwrite
      mode when latency tracers are enabled.
      
      Added a flag_changed() method to the tracer structure to allow
      the tracers to see what flags are being changed, and also be able
      to prevent the change from happing.
      
      Cc: stable@vger.kernel.org
      Signed-off-by: default avatarSteven Rostedt <rostedt@goodmis.org>
      613f04a0
    • Steven Rostedt (Red Hat)'s avatar
      tracing: Keep overwrite in sync between regular and snapshot buffers · 80902822
      Steven Rostedt (Red Hat) authored
      
      
      Changing the overwrite mode for the ring buffer via the trace
      option only sets the normal buffer. But the snapshot buffer could
      swap with it, and then the snapshot would be in non overwrite mode
      and the normal buffer would be in overwrite mode, even though the
      option flag states otherwise.
      
      Keep the two buffers overwrite modes in sync.
      
      Cc: stable@vger.kernel.org
      Signed-off-by: default avatarSteven Rostedt <rostedt@goodmis.org>
      80902822
  2. 14 Mar, 2013 1 commit
  3. 12 Mar, 2013 1 commit
    • Steven Rostedt (Red Hat)'s avatar
      tracing: Fix race in snapshot swapping · 2721e72d
      Steven Rostedt (Red Hat) authored
      
      
      Although the swap is wrapped with a spin_lock, the assignment
      of the temp buffer used to swap is not within that lock.
      It needs to be moved into that lock, otherwise two swaps
      happening on two different CPUs, can end up using the wrong
      temp buffer to assign in the swap.
      
      Luckily, all current callers of the swap function appear to have
      their own locks. But in case something is added that allows two
      different callers to call the swap, then there's a chance that
      this race can trigger and corrupt the buffers.
      
      New code is coming soon that will allow for this race to trigger.
      
      I've Cc'd stable, so this bug will not show up if someone backports
      one of the changes that can trigger this bug.
      
      Cc: stable@vger.kernel.org
      Signed-off-by: default avatarSteven Rostedt <rostedt@goodmis.org>
      2721e72d
  4. 07 Mar, 2013 2 commits
    • Steven Rostedt (Red Hat)'s avatar
      tracing: Do not return EINVAL in snapshot when not allocated · c9960e48
      Steven Rostedt (Red Hat) authored
      
      
      To use the tracing snapshot feature, writing a '1' into the snapshot
      file causes the snapshot buffer to be allocated if it has not already
      been allocated and dose a 'swap' with the main buffer, so that the
      snapshot now contains what was in the main buffer, and the main buffer
      now writes to what was the snapshot buffer.
      
      To free the snapshot buffer, a '0' is written into the snapshot file.
      
      To clear the snapshot buffer, any number but a '0' or '1' is written
      into the snapshot file. But if the file is not allocated it returns
      -EINVAL error code. This is rather pointless. It is better just to
      do nothing and return success.
      Acked-by: default avatarHiraku Toyooka <hiraku.toyooka.gu@hitachi.com>
      Signed-off-by: default avatarSteven Rostedt <rostedt@goodmis.org>
      c9960e48
    • Steven Rostedt (Red Hat)'s avatar
      tracing: Add help of snapshot feature when snapshot is empty · d8741e2e
      Steven Rostedt (Red Hat) authored
      
      
      When cat'ing the snapshot file, instead of showing an empty trace
      header like the trace file does, show how to use the snapshot
      feature.
      
      Also, this is a good place to show if the snapshot has been allocated
      or not. Users may want to "pre allocate" the snapshot to have a fast
      "swap" of the current buffer. Otherwise, a swap would be slow and might
      fail as it would need to allocate the snapshot buffer, and that might
      fail under tight memory constraints.
      
      Here's what it looked like before:
      
       # tracer: nop
       #
       # entries-in-buffer/entries-written: 0/0   #P:4
       #
       #                              _-----=> irqs-off
       #                             / _----=> need-resched
       #                            | / _---=> hardirq/softirq
       #                            || / _--=> preempt-depth
       #                            ||| /     delay
       #           TASK-PID   CPU#  ||||    TIMESTAMP  FUNCTION
       #              | |       |   ||||       |         |
      
      Here's what it looks like now:
      
       # tracer: nop
       #
       #
       # * Snapshot is freed *
       #
       # Snapshot commands:
       # echo 0 > snapshot : Clears and frees snapshot buffer
       # echo 1 > snapshot : Allocates snapshot buffer, if not already allocated.
       #                      Takes a snapshot of the main buffer.
       # echo 2 > snapshot : Clears snapshot buffer (but does not allocate)
       #                      (Doesn't have to be '2' works with any number that
       #                       is not a '0' or '1')
      Acked-by: default avatarHiraku Toyooka <hiraku.toyooka.gu@hitachi.com>
      Signed-off-by: default avatarSteven Rostedt <rostedt@goodmis.org>
      d8741e2e
  5. 07 Feb, 2013 1 commit
  6. 01 Feb, 2013 1 commit
    • Steven Rostedt (Red Hat)'s avatar
      tracing: Init current_trace to nop_trace and remove NULL checks · d840f718
      Steven Rostedt (Red Hat) authored
      
      
      On early boot up, when the ftrace ring buffer is initialized, the
      static variable current_trace is initialized to &nop_trace.
      Before this initialization, current_trace is NULL and will never
      become NULL again. It is always reassigned to a ftrace tracer.
      
      Several places check if current_trace is NULL before it uses
      it, and this check is frivolous, because at the point in time
      when the checks are made the only way current_trace could be
      NULL is if ftrace failed its allocations at boot up, and the
      paths to these locations would probably not be possible.
      
      By initializing current_trace to &nop_trace where it is declared,
      current_trace will never be NULL, and we can remove all these
      checks of current_trace being NULL which never needed to be
      checked in the first place.
      
      Cc: Dan Carpenter <dan.carpenter@oracle.com>
      Cc: Hiraku Toyooka <hiraku.toyooka.gu@hitachi.com>
      Signed-off-by: default avatarSteven Rostedt <rostedt@goodmis.org>
      d840f718
  7. 30 Jan, 2013 3 commits
  8. 29 Jan, 2013 1 commit
  9. 26 Jan, 2013 1 commit
  10. 25 Jan, 2013 1 commit
  11. 23 Jan, 2013 1 commit
    • Steven Rostedt's avatar
      tracing: Fix race with max_tr and changing tracers · 34600f0e
      Steven Rostedt authored
      
      
      There's a race condition between the setting of a new tracer and
      the update of the max trace buffers (the swap). When a new tracer
      is added, it sets current_trace to nop_trace before disabling
      the old tracer. At this moment, if the old tracer uses update_max_tr(),
      the update may trigger the warning against !current_trace->use_max-tr,
      as nop_trace doesn't have that set.
      
      As update_max_tr() requires that interrupts be disabled, we can
      add a check to see if current_trace == nop_trace and bail if it
      does. Then when disabling the current_trace, set it to nop_trace
      and run synchronize_sched(). This will make sure all calls to
      update_max_tr() have completed (it was called with interrupts disabled).
      
      As a clean up, this commit also removes shrinking and recreating
      the max_tr buffer if the old and new tracers both have use_max_tr set.
      The old way use to always shrink the buffer, and then expand it
      for the next tracer. This is a waste of time.
      Signed-off-by: default avatarSteven Rostedt <rostedt@goodmis.org>
      34600f0e
  12. 22 Jan, 2013 1 commit
    • Steven Rostedt's avatar
      tracing: Remove the extra 4 bytes of padding in events · b000c806
      Steven Rostedt authored
      Due to a userspace issue with PowerTop v2beta, which hardcoded
      the offset of event fields that it was using, it broke when
      we removed the Big Kernel Lock counter from the event header.
      
       (commit e6e1e259 "tracing: Remove lock_depth from event entry")
      
      Because this broke userspace, it was determined that we must
      keep those 4 bytes around.
      
       (commit a3a4a5ac
      
       "Regression: partial revert "tracing: Remove lock_depth from event entry"")
      
      This unfortunately wastes space in the ring buffer. 4 bytes per
      event, where a lot of events are just 24 bytes. That's 16% of the
      buffer wasted. A million events will add 4 megs of white space
      into the buffer.
      
      It was later noticed that PowerTop v2beta could not work on systems
      where the kernel was 64 bit but the userspace was 32 bits.
      The reason was because the offsets are different between the
      two and the hard coded offset of one would not work with the other.
      
      With PowerTop v2 final, it implemented the same interface that both
      perf and trace-cmd use. That is, it reads the format file of
      the event to find the offsets of the fields it needs. This fixes
      the problem with running powertop on a 32 bit userspace running
      on a 64 bit kernel. It also no longer requires the 4 byte padding.
      
      As PowerTop v2 has been out for a while, and is included in all
      major distributions, it is time that we can safely remove the
      4 bytes of padding. Users of PowerTop v2beta should upgrade to
      PowerTop v2 final.
      
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Acked-by: default avatarArjan van de Ven <arjan@linux.intel.com>
      Signed-off-by: default avatarSteven Rostedt <rostedt@goodmis.org>
      b000c806
  13. 21 Jan, 2013 3 commits
  14. 14 Jan, 2013 1 commit
  15. 11 Jan, 2013 1 commit
    • Steven Rostedt's avatar
      tracing: Fix regression with irqsoff tracer and tracing_on file · 2df8f8a6
      Steven Rostedt authored
      Commit 02404baf
      
       "tracing: Remove deprecated tracing_enabled file"
      removed the tracing_enabled file as it never worked properly and
      the tracing_on file should be used instead. But the tracing_on file
      didn't call into the tracers start/stop routines like the
      tracing_enabled file did. This caused trace-cmd to break when it
      enabled the irqsoff tracer.
      
      If you just did "echo irqsoff > current_tracer" then it would work
      properly. But the tool trace-cmd disables tracing first by writing
      "0" into the tracing_on file. Then it writes "irqsoff" into
      current_tracer and then writes "1" into tracing_on. Unfortunately,
      the above commit changed the irqsoff tracer to check the tracing_on
      status instead of the tracing_enabled status. If it's disabled then
      it does not start the tracer internals.
      
      The problem is that writing "1" into tracing_on does not call the
      tracers "start" routine like writing "1" into tracing_enabled did.
      This makes the irqsoff tracer not start when using the trace-cmd
      tool, and is a regression for userspace.
      
      Simple fix is to have the tracing_on file call the tracers start()
      method when being enabled (and the stop() method when disabled).
      Signed-off-by: default avatarSteven Rostedt <rostedt@goodmis.org>
      2df8f8a6
  16. 10 Jan, 2013 1 commit
    • Steven Rostedt's avatar
      tracing: Fix regression of trace_options file setting · a8dd2176
      Steven Rostedt authored
      
      
      The latest change to allow trace options to be set on the command
      line also broke the trace_options file.
      
      The zeroing of the last byte of the option name that is echoed into
      the trace_option file was removed with the consolidation of some
      of the code. The compare between the option and what was written to
      the trace_options file fails because the string holding the data
      written doesn't terminate with a null character.
      
      A zero needs to be added to the end of the string copied from
      user space.
      Signed-off-by: default avatarSteven Rostedt <rostedt@goodmis.org>
      a8dd2176
  17. 06 Dec, 2012 1 commit
  18. 19 Nov, 2012 1 commit
  19. 15 Nov, 2012 1 commit
  20. 13 Nov, 2012 3 commits
    • Yoshihiro YUNOMAE's avatar
      tracing: Show raw time stamp on stats per cpu using counter or tsc mode for trace_clock · 11043d8b
      Yoshihiro YUNOMAE authored
      Show raw time stamp values for stats per cpu if you choose counter or tsc mode
      for trace_clock. Although a unit of tracing time stamp is nsec in local or global mode,
      the units in counter and TSC mode are tracing counter and cycles respectively.
      Link: http://lkml.kernel.org/r/1352837903-32191-3-git-send-email-dhsharp@google.com
      
      
      
      Cc: Frederic Weisbecker <fweisbec@gmail.com>
      Cc: Ingo Molnar <mingo@redhat.com>
      Signed-off-by: default avatarYoshihiro YUNOMAE <yoshihiro.yunomae.ez@hitachi.com>
      Signed-off-by: default avatarDavid Sharp <dhsharp@google.com>
      Signed-off-by: default avatarSteven Rostedt <rostedt@goodmis.org>
      11043d8b
    • David Sharp's avatar
      tracing: Format non-nanosec times from tsc clock without a decimal point. · 8be0709f
      David Sharp authored
      With the addition of the "tsc" clock, formatting timestamps to look like
      fractional seconds is misleading. Mark clocks as either in nanoseconds or
      not, and format non-nanosecond timestamps as decimal integers.
      
      Tested:
      $ cd /sys/kernel/debug/tracing/
      $ cat trace_clock
      [local] global tsc
      $ echo sched_switch > set_event
      $ echo 1 > tracing_on ; sleep 0.0005 ; echo 0 > tracing_on
      $ cat trace
                <idle>-0     [000]  6330.555552: sched_switch: prev_comm=swapper prev_pid=0 prev_prio=120 prev_state=R ==> next_comm=bash next_pid=29964 next_prio=120
                 sleep-29964 [000]  6330.555628: sched_switch: prev_comm=bash prev_pid=29964 prev_prio=120 prev_state=S ==> next_comm=swapper next_pid=0 next_prio=120
        ...
      $ echo 1 > options/latency-format
      $ cat trace
        <idle>-0       0 4104553247us+: sched_switch: prev_comm=swapper prev_pid=0 prev_prio=120 prev_state=R ==> next_comm=bash next_pid=29964 next_prio=120
         sleep-29964   0 4104553322us+: sched_switch: prev_comm=bash prev_pid=29964 prev_prio=120 prev_state=S ==> next_comm=swapper next_pid=0 next_prio=120
        ...
      $ echo tsc > trace_clock
      $ cat trace
      $ echo 1 > tracing_on ; sleep 0.0005 ; echo 0 > tracing_on
      $ echo 0 > options/latency-format
      $ cat trace
                <idle>-0     [000] 16490053398357: sched_switch: prev_comm=swapper prev_pid=0 prev_prio=120 prev_state=R ==> next_comm=bash next_pid=31128 next_prio=120
                 sleep-31128 [000] 16490053588518: sched_switch: prev_comm=bash prev_pid=31128 prev_prio=120 prev_state=S ==> next_comm=swapper next_pid=0 next_prio=120
        ...
      echo 1 > options/latency-format
      $ cat trace
        <idle>-0       0 91557653238+: sched_switch: prev_comm=swapper prev_pid=0 prev_prio=120 prev_state=R ==> next_comm=bash next_pid=31128 next_prio=120
         sleep-31128   0 91557843399+: sched_switch: prev_comm=bash prev_pid=31128 prev_prio=120 prev_state=S ==> next_comm=swapper next_pid=0 next_prio=120
        ...
      
      v2:
      Move arch-specific bits out of generic code.
      v4:
      Fix x86_32 build due to 64-bit division.
      
      Google-Bug-Id: 6980623
      Link: http://lkml.kernel.org/r/1352837903-32191-2-git-send-email-dhsharp@google.com
      
      
      
      Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
      Signed-off-by: default avatarDavid Sharp <dhsharp@google.com>
      Signed-off-by: default avatarSteven Rostedt <rostedt@goodmis.org>
      8be0709f
    • David Sharp's avatar
      tracing,x86: Add a TSC trace_clock · 8cbd9cc6
      David Sharp authored
      In order to promote interoperability between userspace tracers and ftrace,
      add a trace_clock that reports raw TSC values which will then be recorded
      in the ring buffer. Userspace tracers that also record TSCs are then on
      exactly the same time base as the kernel and events can be unambiguously
      interlaced.
      
      Tested: Enabled a tracepoint and the "tsc" trace_clock and saw very large
      timestamp values.
      
      v2:
      Move arch-specific bits out of generic code.
      v3:
      Rename "x86-tsc", cleanups
      v7:
      Generic arch bits in Kbuild.
      
      Google-Bug-Id: 6980623
      Link: http://lkml.kernel.org/r/1352837903-32191-1-git-send-email-dhsharp@google.com
      
      Acked-by: default avatarIngo Molnar <mingo@kernel.org>
      Cc: Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>
      Cc: Ingo Molnar <mingo@kernel.org>
      Cc: Thomas Gleixner <tglx@linutronix.de>
      Cc: "H. Peter Anvin" <hpa@linux.intel.com>
      Signed-off-by: default avatarDavid Sharp <dhsharp@google.com>
      Signed-off-by: default avatarSteven Rostedt <rostedt@goodmis.org>
      8cbd9cc6