Lalit Maganti

Lalit is a founding engineer on the Perfetto project and the creator of its trace processing library. During his eight years at Google, he has worn many hats, from building on-device probes to collect trace data to creating distributed systems that analyse millions of Android traces at scale. Over the last year, he has been focused on bridging the gap between Perfetto's powerful analysis and visualization capabilities and the tools used by the wider open-source Linux community.


Session

09-03
10:15
30min
Perfetto: The Swiss Army Knife of Linux Client/Embedded Tracing
Lalit Maganti

The Linux kernel and userspace provide numerous sources for performance data, including perf, ftrace, eBPF, and /proc interfaces. While effective individually, these sources are often siloed, requiring developers to write custom scripts to build a comprehensive view of system behaviour. This challenge is compounded by tools that prioritize the recording experience, often leaving visualization as a secondary concern.

Perfetto is an open-source suite of tools for performance analysis, featuring the Perfetto UI—an offline, web-based, timeline-oriented visualizer. A key design principle of the UI is its ability to analyze and visualize not only Perfetto's native format but also a range of widely used tracing formats, including Linux perf output, Firefox profiler data, and ftrace text logs.

This talk demonstrates how the Perfetto UI can be used to investigate complex performance problems, independent of the original data collection method. We will walk through several practical analysis scenarios:

  • Analyze Linux perf CPU profiles: diving into individual samples and using the UI's time-based selection and dynamic flamegraph viewer to pinpoint where CPU time is spent.
  • Investigate Linux scheduler traces (sched_switch, sched_waking): to diagnose why an application may be latent even when its CPU usage is low.
  • Visualize custom user space trace data: using Mesa (the Linux 3D graphics library) as a case study to solve performance problems during rendering.
  • Examine community tools built on Perfetto: including pthread_trace(for lock contention), systing (eBPF-based tracing), magic-trace (Intel Processor Trace), and viztracer (Python profiling).
  • Demonstrate the UI’s new superpower: merging multiple, disparate trace files onto a single timeline to create a holistic, multi-data source, system-wide view of performance.
Bonjour 50