Benchmark

How fast is Toothpick ?

For sure, that is an important question: how does Toothpick compare to other DI libraries, and especially the Daggers ?

In order to clearly compare the performances of Toothpick, we have setup an experiment, derived from the experiment of NimbleDroid. This experiment emphasis startup times, with a large number of injections.

Benchmark setup

To benchmark, we have used:

• 100 different classes to inject,
• each time an injection is requested, a new instance of these classes is created, they are not singletons (not scoped in TP language).
• all benchmarks have been performed locally on a stock Nexus 5. It would have been too long to compare them on a cloud of devices like NimbleDroid.
• each entry in the table is the average of 50 launches
• we compared the DI libraries using a variable number of injections. NimbleDroid only compares the DI frameworks using 55 000 injections, which is not realistic. We provide more data and the injection count used is 51 120 / 2^(i in 9..0).
• we have compared the following librairies:
• RoboGuice 4
• Dagger 1
• Dagger 2
• Toothpick

Benchmark raw data

All durations below are in milliseconds, they are the average of 50 launches on a Nexus 5 device.

Injections	RG 4	Dagger1	Dagger2	Tootpick
100	242.93	29.34	30.02	28.30
200	274.68	30.14	30.36	28.07
400	277.75	30.89	31.66	31.30
800	291.05	33.36	32.82	35.44
1600	338.95	34.32	34.86	47.56
3200	413.59	38.66	38.00	53.33
6400	526.93	45.05	42.45	65.93
12800	694.30	61.23	54.48	104.49
25600	1013.59	98.59	76.50	205.37
51200	1691.72	136.89	120.98	335.40

On an average of 50 runs, the cost of setting up the injection system, i.e creating an injector (RG), a component (Daggers), and a scope (Toothpick) are:

• RoboGuice 4: 105 ms
• Dagger 1: 20 ms
• Dagger 2: 22 ms
• Toothpick: 1 ms

Charts and interpretation

As we can see on this chart, RoboGuice is clearly slower than all other librairies. But though we consider this graph to be interesting to get an idea of performances with a large number of injections (from 100 to 55 000 injections), we think the highest end is not very realistic and prefer the following graph were the number of injections fits into a more realistic range:

Already, 6 400 injections is huge. According to our experience, an application with 1 000 injections is already a very large app. Most applications should use between 100 and 1 000 injections at startup. For 1 000 injections, RoboGuice is already much slower than other DI librairies. So we can discard it to see clearly how the other libs perform. Indeed, the simple cost of creating the first injector in RoboGuice makes it slow compared to other librairies.

Toothpick, Dagger 1 & Dagger 2 are all in a very tight bracket between 100 and 1 000 injections. For a small number of injections, the cost of creating the ObjectGraph in Dagger 1, and the DaggerD2EComponent in Dagger 2 is relatively high (around 20 ms for both libs on a Nexus 5), which makes Toothpick faster as the cost of initialization of a scope is very low (1 ms).

For 1 000 injections, which seems to us a very reasonable number of injections, we can observe the following:

• Dagger 1: 33 ms
• Dagger 2: 31 ms
• Toothpick: 35 ms

For 6 400 injections, which is probably too high a number to be realistic, the performances are:

• Dagger 1: 45 ms
• Dagger 2: 42 ms
• Toothpick: 66 ms

thus, the actual cost of using Toothpick compared to Dagger 2 for 6 400 injections is 24 ms.

If your application needs incredibly fast performances, and you think it's worth sacrificing testing and add some extra boiler plate code, then you should use Dagger 2. Otherwise, we think that Toothpick is quite a reasonable option. And it can even be better than Dagger 2 for all the parts of your app that use only a small number of injections (like the view holders of a recycler view, secondary activities outside of startup, etc..).