Benchmark
For sure, that is an important question: how does Toothpick compare to other DI libraries, and especially the Daggers ?
In order to clearly compare the performances of Toothpick, we have setup an experiment, derived from the experiment of NimbleDroid. This experiment emphasizes startup times, with a large number of injections.
To benchmark, we have used:
-
100 different classes to inject,
-
each time an injection is requested, a new instance of these classes is created, they are not singletons (not scoped in TP language).
-
all benchmarks have been performed locally on a stock Nexus 5. It would have been too long to compare them on a cloud of devices like NimbleDroid.
-
each entry in the table is the average of 50 launches
-
we compared the DI libraries using a variable number of injections. NimbleDroid only compares the DI frameworks using 55 000 injections, which is not realistic. We provide more data and the injection counts used are 51 120 / 2^(i in 9..0): 100, 200, 400, 800, 1 600, 3 200, 6 400, 12 800, 25 600, 51 200. Note that the injection counts grow exponentially.
-
we have compared the following libraries:
-
RoboGuice 4
-
Dagger 1
-
Dagger 2
-
Toothpick
We provide a fork of the Nimble Droid experiment that we have used for benchmarking the DI libraries: https://github.com/stephanenicolas/DIDemoApps.
All durations below are in milliseconds, they are the average of 50 launches on a Nexus 5 device.
| Injections | RG 4 | Dagger1 | Dagger2 | Tootpick |
|---|---|---|---|---|
| 100 | 242.93 | 29.34 | 30.02 | 28.30 |
| 200 | 274.68 | 30.14 | 30.36 | 28.07 |
| 400 | 277.75 | 30.89 | 31.66 | 31.30 |
| 800 | 291.05 | 33.36 | 32.82 | 35.44 |
| 1600 | 338.95 | 34.32 | 34.86 | 47.56 |
| 3200 | 413.59 | 38.66 | 38.00 | 53.33 |
| 6400 | 526.93 | 45.05 | 42.45 | 65.93 |
| 12800 | 694.30 | 61.23 | 54.48 | 104.49 |
| 25600 | 1013.59 | 98.59 | 76.50 | 205.37 |
| 51200 | 1691.72 | 136.89 | 120.98 | 335.40 |
On an average of 50 runs, the cost of setting up the injection system, i.e creating an injector (RG), a component (Daggers), and a scope (Toothpick) are:
- RoboGuice 4: 105 ms
- Dagger 1: 20 ms
- Dagger 2: 22 ms
- Toothpick: 1 ms
As we can see on this chart, RoboGuice is clearly slower than all other libraries. Although, we consider this graph to be interesting to get an idea of performances with a large number of injections (from 100 to 55 000 injections), we think the highest end is not very realistic and prefer the following graph were the number of injections fit into a more realistic range [100-6 400]:
Already, 6 400 injections is huge. According to our experience, an application with 1 000 injections is already a very large app. Most applications should use between 100 and 1 000 injections at startup. For 100 injections, RoboGuice is already much slower than other DI libraries. So we can discard it to see clearly how the other libs perform. Indeed, the simple cost of creating the first injector in RoboGuice makes it slow compared to other libraries.
Toothpick, Dagger 1 & Dagger 2 are all in a very tight bracket between 100 and 1 000 injections. For a small number of injections, the cost of creating the ObjectGraph in Dagger 1, and the DaggerD2EComponent in Dagger 2 is relatively high (around 20 ms for both libs on a Nexus 5), which makes Toothpick faster as the cost of initialization of a scope is very low (1 ms).
For 1 000 injections, which seems to us a very reasonable number of injections, we can observe the following:
- Dagger 1: 33 ms
- Dagger 2: 31 ms
- Toothpick: 35 ms
For 6 400 injections, which is probably too high a number to be realistic, the performances are:
- Dagger 1: 45 ms
- Dagger 2: 42 ms
- Toothpick: 66 ms
thus, the actual cost of using Toothpick compared to Dagger 2 for 6 400 injections is 24 ms.
If your application needs incredibly fast performances, and you think it's worth sacrificing testing and add some extra boiler plate code, then you should use Dagger 2.
Otherwise, we think that Toothpick is quite a reasonable option. Not only does it perform as well as Daggers for startup scenarios, in a reasonable number of injections, but it can even be better than Dagger 2 for all the parts of your app that use only a small number of injections (like the view holders of a recycler view, secondary activities outside of startup, etc..).