Skip to content

Commit

Permalink
Rewrite based on FlameGraphs (#128)
Browse files Browse the repository at this point in the history
  • Loading branch information
timholy authored Jan 22, 2020
1 parent e2d13d5 commit 3b4fdcd
Show file tree
Hide file tree
Showing 18 changed files with 280 additions and 1,533 deletions.
1 change: 0 additions & 1 deletion .travis.yml
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@ language: julia
os:
- linux
julia:
- 0.7
- 1.0
- 1
- nightly
Expand Down
15 changes: 8 additions & 7 deletions Project.toml
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
name = "ProfileView"
uuid = "c46f51b8-102a-5cf2-8d2c-8597cb0e0da7"
author = ["Tim Holy <[email protected]>"]
version = "0.5.3"
version = "0.6.0"

[deps]
Cairo = "159f3aea-2a34-519c-b102-8c37f9878175"
Colors = "5ae59095-9a9b-59fe-a467-6f913c188581"
FileIO = "5789e2e9-d7fb-5bc7-8068-2c6fae9b9549"
FlameGraphs = "08572546-2f56-4bcf-ba4e-bab62c3a3f89"
Graphics = "a2bd30eb-e257-5431-a919-1863eab51364"
Gtk = "4c0ca9eb-093a-5379-98c5-f87ac0bbbf44"
GtkReactive = "27996c0f-39cd-5cc1-a27a-05f136f946b6"
Expand All @@ -16,15 +17,15 @@ Profile = "9abbd945-dff8-562f-b5e8-e1ebf5ef1b79"
UUIDs = "cf7118a7-6976-5b1a-9a39-7adc72f591a4"

[compat]
Cairo = "0.5, 0.6, 0.7, 0.8, 1"
Cairo = "0.6, 0.8, 1"
Colors = "0.9, 0.10, 0.11"
FileIO = "1"
Graphics = "0.2, 0.3, 0.4, 1"
Gtk = "0.16, 0.17, 0.18, 1"
GtkReactive = "0.5.1, 0.6, 0.7, 1"
FlameGraphs = "0.1"
Graphics = "0.4, 1"
Gtk = "0.18, 1"
GtkReactive = "0.7, 1"
IntervalSets = "0.2, 0.3"
julia = "0.7, 1"

julia = "1"

[extras]
Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"
Expand Down
106 changes: 25 additions & 81 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
# ProfileView.jl

[![Build Status](https://travis-ci.org/timholy/ProfileView.jl.svg)](https://travis-ci.org/timholy/ProfileView.jl)
[![PkgEval][pkgeval-img]][pkgeval-url]

Note this README is for users of Julia 0.7 and higher; users of earlier versions
should see [this page](https://github.com/timholy/ProfileView.jl/tree/julia0.6).
**NOTE**: Jupyter/IJulia and SVG support has migrated to the [ProfileSVG](https://github.com/timholy/ProfileSVG.jl) package.

# Introduction

Expand All @@ -16,6 +16,7 @@ garbage collection as potential candidates for optimization.

This type of plot is known as a [flame
graph](https://github.com/brendangregg/FlameGraph).
The main logic is handled by the [FlameGraphs](https://github.com/timholy/FlameGraphs.jl) package; this package is just a visualization front-end.

## Installation

Expand Down Expand Up @@ -51,22 +52,21 @@ using ProfileView

If you're following along, you should see something like this:

![ProfileView](readme_images/pv1.jpg)
![ProfileView](readme_images/pv1.png)

(Note that collected profiles can vary from run-to-run, so don't be alarmed
if you get something different.)
This plot is a visual representation of the *call graph* of the code that you just profiled.
The "root" of the tree is at the bottom; if you move your mouse the long horizontal
bars near the bottom, you should fine one for `eval_user_input` in REPL.jl.
As is explained [elsewhere][Profiling],
these are what run your code in the REPL.
bars near the bottom, you'll find functions in Base or REPL.jl that are responsible for taking the commands you type and executing them.
If you move your mouse upwards, you'll eventually get to the function(s) you ran with `@profile`.

While the vertical axis therefore represents nesting depth, the
horizontal axis represents the amount of time (more precisely, the
number of backtraces) spent at each line. One sees on the 4th line
from the bottom, there are several differently-colored bars, each
corresponding to a different line of `profile_test`. The fact that
number of backtraces) spent at each line. The row at which the single
long bar breaks up into multiple different-colored bars corresponds
to the execution of different lines from `profile_test`.
The fact that
they are all positioned on top of the lower peach-colored bar means that all
of these lines are called by the same "parent" function. Within a
block of code, they are sorted in order of increasing line number, to
Expand All @@ -75,20 +75,18 @@ make it easier for you to compare to the source code.
From this visual representation, we can very quickly learn several
things about this function:

- The most deeply-nested call corresponds to the `mapslices(sort, B; dims=1)` call.
(If you hover over the top-most bars you will see they correspond to lines in `sort.jl`.)
In contrast, the call to `maximum` (the lowest blue bar) resolves to just two (non-inlined) calls.
- On the right side, you see a stack of calls to functions in `sort.jl`.
This is because sorting is recursive.

- `mapslices(sum, A; dims=2)` is considerably more expensive than
`mapslices(sort, B; dims=1)`. (This is because it has to process more
data.)
- `mapslices(sum, A; dims=2)` is considerably more expensive (the corresponding bar is horizontally wider) than
`mapslices(sort, B; dims=1)`. This is because it has to process more
data.

It is also worth noting that red is a special color: it is reserved for function
calls that have to be resolved at run-time (by virtue of their
execution of the C functions `jl_invoke` or
`jl_apply_generic`). Because run-time dispatch (aka, run-time method lookup or
calls that have to be resolved at run-time.
Because run-time dispatch (aka, dynamic dispatch, run-time method lookup, or
a virtual call) often has a significant
impact on performance, we highlight the problematic call in red. It's
impact on performance, ProfileView highlights the problematic call in red. It's
worth noting that some red is unavoidable; for example, the REPL can't
predict in advance the return types from what users type at the
prompt, and so `eval_user_input` is red.
Expand Down Expand Up @@ -120,32 +118,18 @@ contribution to its total run time.
with CTRL-scroll. You can also use your keyboard (arrow keys, plus
SHIFT and CTRL modifiers). Double-click to restore the full view.

- To use the Gtk interface in Juno or IJulia, set `PROFILEVIEW_USEGTK = true` in
the `Main` module before `using ProfileView`.

- The toolbar at the top contains two icons to load and save profile
data, respectively. Clicking the save icon will prompt you for a
filename. Launching `ProfileView.view(nothing)` opens a blank
window; you can populate it with saved data by clicking on the
"open" icon.

### IJulia (SVG) Interface

- Double-clicking on a bar will zoom in the graph around the bar.

- Double-clicking on the background will zoom out to show the entire graph.

- Click-drag anywhere on the graph will allow for panning.

## Command-line options

The `view` command has the following syntax:
```
function view(data = Profile.fetch(); lidict = nothing, C = false, colorgc = true, fontsize = 12, combine = true, pruned = [])
function view([fcolor,] data = Profile.fetch(); lidict = nothing, C = false, fontsize = 14, kwargs...)
```
Here is the meaning of the different arguments:

- The first is the vector containing backtraces. You can use `data1 =
- `fcolor` optionally allows you to control the scheme used to select
bar color. This can be quite extensively customized; see [FlameGraphs](https://timholy.github.io/FlameGraphs.jl/stable/) for details.

- `data` is the vector containing backtraces. You can use `data1 =
copy(Profile.fetch()); Profile.clear()` to store and examine results
from multiple profile runs simultaneously.

Expand All @@ -157,14 +141,9 @@ Here is the meaning of the different arguments:
from C backtraces to learn about garbage-collection and to
disambiguate the call graph).

- `colorgc`, when `true`, causes lines triggering garbage-collection
to be displayed in red.

- `fontsize` controls the size of the font displayed as a tooltip.

- `combine` is explained [elsewhere][Profiling].

- `pruned` is a list of functions (see example) whose call tree will not be displayed. This is useful to control the output of very deep (or recursive) functions. Example: `pruned = [("sort!", "sort.jl"), ("some_function_name", "some_filename.jl")]`
These are the main options, but there are others; see FlameGraphs for more details.

## Source locations & Revise (new in ProfileView 0.5.3)

Expand All @@ -174,43 +153,8 @@ If you use Revise and are tracking the source files (either as a package or with
the source locations (file and line number) reported by ProfileView
will match the current code at the time the window is created.

### Saving profile data manually

If you're using the Gtk backend, the easiest approach is to click on
the "Save as" icon.

From the REPL, you can save profile data for later viewing and analysis using the JLD file format.
The main trick is that the backtrace data, on its own, is only valid within a particular
julia session. To become portable, you have to save "line information" that looks
up the particular line number in the source code corresponding to a particular
machine instruction. Here's an example:

```julia
li, lidict = Profile.retrieve()
using JLD
@save "/tmp/foo.jlprof" li lidict
```
Now open a new julia session, and try the following:
```
using HDF5, JLD, ProfileView
@load "/tmp/foo.jlprof"
ProfileView.view(li, lidict=lidict)
```

### Saving ProfileView visualizations

You can share your profiling results with others either as an SVG file or
as an IJulia notebook. Simply use
```
ProfileView.svgwrite("profile_results.svg")
```
or
```
ProfileView.svgwrite("profile_results.svg", bt, lidict)
```
if you've `retrieve`d stored data.

Alternatively, run ProfileView inside an IJulia notebook and then save the notebook.

[Julia]: http://julialang.org "Julia"
[Profiling]: https://docs.julialang.org/en/latest/manual/profile/#Profiling-1
[pkgeval-img]: https://juliaci.github.io/NanosoldierReports/pkgeval_badges/P/ProfileView.svg
[pkgeval-url]: https://juliaci.github.io/NanosoldierReports/pkgeval_badges/report.html
32 changes: 32 additions & 0 deletions README_save.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# This functionality is currently broken (looking for volunteers to fix it)

(Once fixed it can move back to the README)

- The toolbar at the top contains two icons to load and save profile
data, respectively. Clicking the save icon will prompt you for a
filename. Launching `ProfileView.view(nothing)` opens a blank
window; you can populate it with saved data by clicking on the
"open" icon.

### Saving profile data manually

If you're using the Gtk backend, the easiest approach is to click on
the "Save as" icon.

From the REPL, you can save profile data for later viewing and analysis using the JLD file format.
The main trick is that the backtrace data, on its own, is only valid within a particular
julia session. To become portable, you have to save "line information" that looks
up the particular line number in the source code corresponding to a particular
machine instruction. Here's an example:

```julia
li, lidict = Profile.retrieve()
using JLD
@save "/tmp/foo.jlprof" li lidict
```
Now open a new julia session, and try the following:
```
using HDF5, JLD, ProfileView
@load "/tmp/foo.jlprof"
ProfileView.view(li, lidict=lidict)
```
Binary file removed readme_images/pv1.jpg
Binary file not shown.
Binary file added readme_images/pv1.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading

2 comments on commit 3b4fdcd

@timholy
Copy link
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@JuliaRegistrator
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Registration pull request created: JuliaRegistries/General/8265

After the above pull request is merged, it is recommended that a tag is created on this repository for the registered package version.

This will be done automatically if Julia TagBot is installed, or can be done manually through the github interface, or via:

git tag -a v0.6.0 -m "<description of version>" 3b4fdcddd692d8693d2feadf9e18af94a29b50da
git push origin v0.6.0

Please sign in to comment.