[wip] Add s3 instrumentation #6590

nhulston · 2025-01-23T20:45:09Z

Summary of changes

Reason for change

Implementation details

Test coverage

Other details

datadog-ddstaging · 2025-01-23T21:43:30Z

Datadog Report

Branch report: nicholas.hulston/add-s3-instrumentation
Commit report: 10a4362
Test service: dd-trace-dotnet

❌ 368 Failed (0 Known Flaky), 577124 Passed, 4821 Skipped, 88h 33m 10.88s Total Time
❄️ 55 New Flaky
⌛ 2 Performance Regressions

❌ Failed Tests (368)

This report shows up to 5 failed tests.

IntegrationDisabled - Datadog.Trace.ClrProfiler.IntegrationTests.AWS.AwsS3Tests - Details
Expand for error
```
 Expected exit code: 0, actual exit code: 134.
```
IntegrationDisabled - Datadog.Trace.ClrProfiler.IntegrationTests.AWS.AwsS3Tests - Details
Expand for error
```
 Expected exit code: 0, actual exit code: 134.
```
IntegrationDisabled - Datadog.Trace.ClrProfiler.IntegrationTests.AWS.AwsS3Tests - Details
Expand for error
```
 Expected exit code: 0, actual exit code: 134.
```
IntegrationDisabled - Datadog.Trace.ClrProfiler.IntegrationTests.AWS.AwsS3Tests - Details
Expand for error
```
 Expected exit code: 0, actual exit code: 134.
```
IntegrationDisabled - Datadog.Trace.ClrProfiler.IntegrationTests.AWS.AwsS3Tests - Details
Expand for error
```
 Expected exit code: 0, actual exit code: 134.
```

New Flaky Tests (55)

TrimmerTest - Datadog.Trace.ClrProfiler.IntegrationTests.AppTrimmingTests - Last Failure

Expand for error

 application not found: /project/artifacts/publish/Samples.Trimming/release_net9.0_linux-x64/Samples.Trimming

TrimmerTest - Datadog.Trace.ClrProfiler.IntegrationTests.AppTrimmingTests - Last Failure

Expand for error

 application not found: /project/artifacts/publish/Samples.Trimming/release_net8.0_linux-x64/Samples.Trimming

TrimmerTest - Datadog.Trace.ClrProfiler.IntegrationTests.AppTrimmingTests - Last Failure

Expand for error

 application not found: /project/artifacts/publish/Samples.Trimming/release_net8.0_linux-musl-x64/Samples.Trimming

TrimmerTest - Datadog.Trace.ClrProfiler.IntegrationTests.AppTrimmingTests - Last Failure

Expand for error

 application not found: /project/artifacts/publish/Samples.Trimming/release_net9.0_linux-musl-x64/Samples.Trimming

TrimmerTest - Datadog.Trace.ClrProfiler.IntegrationTests.AppTrimmingTests - Last Failure

Expand for error

 application not found: /project/artifacts/publish/Samples.Trimming/release_net7.0_linux-musl-x64/Samples.Trimming

⌛ Performance Regressions vs Default Branch (2)

Baseline - Samples.FakeDbCommand.windows.net462.json.scenarios 72.24ms (+3.11ms, +5%) - Details
Baseline - Samples.FakeDbCommand.windows.netcoreapp31.json.scenarios 110.16ms (+2.43ms, +2%) - Details

andrewlock · 2025-01-23T22:17:10Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6590) - mean (69ms)  : 66, 72
     .   : milestone, 69,
    master - mean (69ms)  : 67, 71
     .   : milestone, 69,

    section CallTarget+Inlining+NGEN
    This PR (6590) - mean (982ms)  : 964, 1001
     .   : milestone, 982,
    master - mean (983ms)  : 958, 1007
     .   : milestone, 983,

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6590) - mean (108ms)  : 106, 111
     .   : milestone, 108,
    master - mean (108ms)  : 106, 110
     .   : milestone, 108,

    section CallTarget+Inlining+NGEN
    This PR (6590) - mean (683ms)  : 671, 695
     .   : milestone, 683,
    master - mean (683ms)  : 665, 700
     .   : milestone, 683,

gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6590) - mean (92ms)  : 90, 93
     .   : milestone, 92,
    master - mean (91ms)  : 89, 93
     .   : milestone, 91,

    section CallTarget+Inlining+NGEN
    This PR (6590) - mean (631ms)  : 612, 649
     .   : milestone, 631,
    master - mean (637ms)  : 619, 656
     .   : milestone, 637,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.6.2) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6590) - mean (190ms)  : 186, 194
     .   : milestone, 190,
    master - mean (190ms)  : 185, 195
     .   : milestone, 190,

    section CallTarget+Inlining+NGEN
    This PR (6590) - mean (1,090ms)  : 1059, 1121
     .   : milestone, 1090,
    master - mean (1,089ms)  : 1052, 1126
     .   : milestone, 1089,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6590) - mean (275ms)  : 271, 278
     .   : milestone, 275,
    master - mean (276ms)  : 271, 282
     .   : milestone, 276,

    section CallTarget+Inlining+NGEN
    This PR (6590) - mean (870ms)  : 839, 900
     .   : milestone, 870,
    master - mean (873ms)  : 843, 903
     .   : milestone, 873,

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Baseline
    This PR (6590) - mean (263ms)  : 260, 267
     .   : milestone, 263,
    master - mean (264ms)  : 259, 269
     .   : milestone, 264,

    section CallTarget+Inlining+NGEN
    This PR (6590) - mean (847ms)  : 809, 885
     .   : milestone, 847,
    master - mean (849ms)  : 821, 876
     .   : milestone, 849,

andrewlock · 2025-01-23T22:36:04Z

Benchmarks Report for tracer 🐌

Benchmarks for #6590 compared to master:

2 benchmarks are faster, with geometric mean 1.143
All benchmarks have the same allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`StartStopWithChild`	net6.0	7.88μs	43.9ns	297ns	0.015	0.0075	0	5.61 KB
master	`StartStopWithChild`	netcoreapp3.1	10.2μs	51.6ns	242ns	0.0193	0.00963	0	5.8 KB
master	`StartStopWithChild`	net472	16.1μs	53.7ns	208ns	1.04	0.303	0.0957	6.2 KB
#6590	`StartStopWithChild`	net6.0	7.99μs	44.6ns	289ns	0.0136	0.00453	0	5.61 KB
#6590	`StartStopWithChild`	netcoreapp3.1	10.3μs	57.9ns	389ns	0.0213	0.0107	0	5.8 KB
#6590	`StartStopWithChild`	net472	16.3μs	48.8ns	189ns	1.05	0.316	0.0973	6.21 KB

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	485μs	157ns	545ns	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	647μs	452ns	1.69μs	0	2.7 KB
master	`WriteAndFlushEnrichedTraces`	net472	864μs	505ns	1.95μs	0.425	3.3 KB
#6590	`WriteAndFlushEnrichedTraces`	net6.0	499μs	299ns	1.12μs	0	2.7 KB
#6590	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	659μs	349ns	1.35μs	0	2.7 KB
#6590	`WriteAndFlushEnrichedTraces`	net472	878μs	711ns	2.75μs	0.437	3.3 KB

Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendRequest`	net6.0	128μs	533ns	2.06μs	0.187	14.47 KB
master	`SendRequest`	netcoreapp3.1	147μs	131ns	492ns	0.216	17.27 KB
master	`SendRequest`	net472	0.01ns	0.00339ns	0.0131ns	0	0 b
#6590	`SendRequest`	net6.0	133μs	375ns	1.45μs	0.197	14.47 KB
#6590	`SendRequest`	netcoreapp3.1	148μs	273ns	1.06μs	0.225	17.27 KB
#6590	`SendRequest`	net472	0.00136ns	0.000755ns	0.00292ns	0	0 b

Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	551μs	2.26μs	8.77μs	0.548	0	0	41.77 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	663μs	3.53μs	18.3μs	0.332	0	0	41.85 KB
master	`WriteAndFlushEnrichedTraces`	net472	824μs	3.66μs	14.2μs	8.12	2.44	0.406	53.26 KB
#6590	`WriteAndFlushEnrichedTraces`	net6.0	580μs	2.98μs	14μs	0.536	0	0	41.76 KB
#6590	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	650μs	3.1μs	14.9μs	0.322	0	0	41.71 KB
#6590	`WriteAndFlushEnrichedTraces`	net472	851μs	2.73μs	10.2μs	8.22	2.47	0.411	53.3 KB

Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`ExecuteNonQuery`	net6.0	1.38μs	0.823ns	3.19ns	0.0141	0	1.02 KB
master	`ExecuteNonQuery`	netcoreapp3.1	1.78μs	2.92ns	10.9ns	0.0133	0	1.02 KB
master	`ExecuteNonQuery`	net472	2.06μs	1.36ns	4.89ns	0.156	0.00103	987 B
#6590	`ExecuteNonQuery`	net6.0	1.26μs	0.729ns	2.63ns	0.014	0	1.02 KB
#6590	`ExecuteNonQuery`	netcoreapp3.1	1.8μs	1.77ns	6.86ns	0.0134	0	1.02 KB
#6590	`ExecuteNonQuery`	net472	1.97μs	2.58ns	9.99ns	0.157	0.000979	987 B

Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`CallElasticsearch`	net6.0	1.25μs	0.675ns	2.53ns	0.0138	976 B
master	`CallElasticsearch`	netcoreapp3.1	1.64μs	0.841ns	2.92ns	0.0132	976 B
master	`CallElasticsearch`	net472	2.56μs	1.11ns	4.16ns	0.157	995 B
master	`CallElasticsearchAsync`	net6.0	1.31μs	0.601ns	2.25ns	0.0131	952 B
master	`CallElasticsearchAsync`	netcoreapp3.1	1.58μs	1.15ns	4.44ns	0.0133	1.02 KB
master	`CallElasticsearchAsync`	net472	2.62μs	1.08ns	4.05ns	0.167	1.05 KB
#6590	`CallElasticsearch`	net6.0	1.17μs	0.619ns	2.32ns	0.0135	976 B
#6590	`CallElasticsearch`	netcoreapp3.1	1.57μs	1.73ns	6.49ns	0.0132	976 B
#6590	`CallElasticsearch`	net472	2.63μs	3.47ns	13.4ns	0.158	995 B
#6590	`CallElasticsearchAsync`	net6.0	1.32μs	0.702ns	2.72ns	0.0131	952 B
#6590	`CallElasticsearchAsync`	netcoreapp3.1	1.61μs	0.811ns	3.14ns	0.0137	1.02 KB
#6590	`CallElasticsearchAsync`	net472	2.7μs	2.38ns	9.21ns	0.167	1.05 KB

Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteAsync`	net6.0	1.2μs	0.741ns	2.87ns	0.0134	952 B
master	`ExecuteAsync`	netcoreapp3.1	1.71μs	1.01ns	3.78ns	0.0126	952 B
master	`ExecuteAsync`	net472	1.78μs	0.305ns	1.1ns	0.145	915 B
#6590	`ExecuteAsync`	net6.0	1.27μs	0.705ns	2.64ns	0.0135	952 B
#6590	`ExecuteAsync`	netcoreapp3.1	1.62μs	0.668ns	2.59ns	0.013	952 B
#6590	`ExecuteAsync`	net472	1.86μs	0.64ns	2.39ns	0.145	915 B

Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendAsync`	net6.0	4.18μs	1.36ns	5.1ns	0.0314	2.31 KB
master	`SendAsync`	netcoreapp3.1	5.24μs	2.64ns	9.53ns	0.0364	2.85 KB
master	`SendAsync`	net472	7.38μs	1.68ns	6.52ns	0.494	3.12 KB
#6590	`SendAsync`	net6.0	4.51μs	1.31ns	4.89ns	0.0316	2.31 KB
#6590	`SendAsync`	netcoreapp3.1	5.22μs	1.77ns	6.86ns	0.039	2.85 KB
#6590	`SendAsync`	net472	7.37μs	1.29ns	4.83ns	0.496	3.12 KB

Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	1.45μs	0.621ns	2.24ns	0.0233	1.64 KB
master	`EnrichedLog`	netcoreapp3.1	2.41μs	1.28ns	4.96ns	0.0216	1.64 KB
master	`EnrichedLog`	net472	2.5μs	1.26ns	4.88ns	0.25	1.57 KB
#6590	`EnrichedLog`	net6.0	1.45μs	0.511ns	1.91ns	0.0234	1.64 KB
#6590	`EnrichedLog`	netcoreapp3.1	2.45μs	1.09ns	4.1ns	0.0221	1.64 KB
#6590	`EnrichedLog`	net472	2.57μs	1.25ns	4.69ns	0.249	1.57 KB

Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`EnrichedLog`	net6.0	116μs	129ns	499ns	0	0	4.28 KB
master	`EnrichedLog`	netcoreapp3.1	120μs	123ns	461ns	0.0601	0	4.28 KB
master	`EnrichedLog`	net472	152μs	189ns	733ns	0.682	0.227	4.46 KB
#6590	`EnrichedLog`	net6.0	117μs	135ns	525ns	0	0	4.28 KB
#6590	`EnrichedLog`	netcoreapp3.1	120μs	119ns	445ns	0	0	4.28 KB
#6590	`EnrichedLog`	net472	151μs	128ns	495ns	0.676	0.225	4.46 KB

Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	3.08μs	0.764ns	2.86ns	0.0308	2.2 KB
master	`EnrichedLog`	netcoreapp3.1	4.26μs	2.26ns	8.76ns	0.0297	2.2 KB
master	`EnrichedLog`	net472	4.79μs	1.86ns	7.22ns	0.32	2.02 KB
#6590	`EnrichedLog`	net6.0	2.92μs	1.32ns	5.13ns	0.0302	2.2 KB
#6590	`EnrichedLog`	netcoreapp3.1	4.08μs	0.977ns	3.52ns	0.0285	2.2 KB
#6590	`EnrichedLog`	net472	4.88μs	1.16ns	4.5ns	0.32	2.02 KB

Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendReceive`	net6.0	1.36μs	0.575ns	2.23ns	0.0163	1.14 KB
master	`SendReceive`	netcoreapp3.1	1.81μs	3.43ns	12.8ns	0.015	1.14 KB
master	`SendReceive`	net472	2.06μs	0.783ns	2.82ns	0.183	1.16 KB
#6590	`SendReceive`	net6.0	1.37μs	0.757ns	2.93ns	0.0158	1.14 KB
#6590	`SendReceive`	netcoreapp3.1	1.77μs	0.55ns	2.13ns	0.0151	1.14 KB
#6590	`SendReceive`	net472	2.13μs	2.79ns	10.8ns	0.183	1.16 KB

Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.78μs	3.01ns	11.3ns	0.0223	1.6 KB
master	`EnrichedLog`	netcoreapp3.1	3.86μs	6.13ns	23.7ns	0.0211	1.65 KB
master	`EnrichedLog`	net472	4.35μs	2.39ns	9.24ns	0.323	2.04 KB
#6590	`EnrichedLog`	net6.0	2.84μs	0.895ns	3.47ns	0.0226	1.6 KB
#6590	`EnrichedLog`	netcoreapp3.1	3.91μs	1.61ns	6.23ns	0.0215	1.65 KB
#6590	`EnrichedLog`	net472	4.64μs	3.79ns	14.7ns	0.323	2.04 KB

Benchmarks.Trace.SpanBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #6590

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net6.0	1.147	466.99	407.15
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net472	1.139	868.45	762.24

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`StartFinishSpan`	net6.0	467ns	0.582ns	2.25ns	0.00817	576 B
master	`StartFinishSpan`	netcoreapp3.1	613ns	1.03ns	4.01ns	0.0077	576 B
master	`StartFinishSpan`	net472	647ns	1.09ns	4.22ns	0.0916	578 B
master	`StartFinishScope`	net6.0	489ns	0.485ns	1.88ns	0.00966	696 B
master	`StartFinishScope`	netcoreapp3.1	697ns	1.31ns	5.06ns	0.00946	696 B
master	`StartFinishScope`	net472	868ns	1.64ns	6.34ns	0.104	658 B
#6590	`StartFinishSpan`	net6.0	407ns	0.254ns	0.986ns	0.00817	576 B
#6590	`StartFinishSpan`	netcoreapp3.1	565ns	0.847ns	3.17ns	0.00779	576 B
#6590	`StartFinishSpan`	net472	615ns	0.336ns	1.3ns	0.0915	578 B
#6590	`StartFinishScope`	net6.0	542ns	0.147ns	0.548ns	0.0098	696 B
#6590	`StartFinishScope`	netcoreapp3.1	743ns	0.832ns	3.22ns	0.00924	696 B
#6590	`StartFinishScope`	net472	762ns	0.596ns	2.31ns	0.104	658 B

Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunOnMethodBegin`	net6.0	693ns	1.63ns	6.31ns	0.00976	696 B
master	`RunOnMethodBegin`	netcoreapp3.1	910ns	1.74ns	6.73ns	0.00905	696 B
master	`RunOnMethodBegin`	net472	1.04μs	3.03ns	11.7ns	0.104	658 B
#6590	`RunOnMethodBegin`	net6.0	665ns	0.49ns	1.9ns	0.00969	696 B
#6590	`RunOnMethodBegin`	netcoreapp3.1	952ns	0.659ns	2.55ns	0.00944	696 B
#6590	`RunOnMethodBegin`	net472	1.03μs	0.519ns	2.01ns	0.104	658 B

nhulston added 5 commits January 23, 2025 15:13

add s3

fd847c2

auto generate on windows

0b0e72b

update InstrumentMethod header with generator header

324c3ec

update sample app s3 version

0ad273c

update sample app

10a4362

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[wip] Add s3 instrumentation #6590

[wip] Add s3 instrumentation #6590

nhulston commented Jan 23, 2025

datadog-ddstaging bot commented Jan 23, 2025 •

edited

Loading

andrewlock commented Jan 23, 2025

andrewlock commented Jan 23, 2025

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Faster 🎉 in #6590

Raw results

Raw results

[wip] Add s3 instrumentation #6590

Are you sure you want to change the base?

[wip] Add s3 instrumentation #6590

Conversation

nhulston commented Jan 23, 2025

Summary of changes

Reason for change

Implementation details

Test coverage

Other details

datadog-ddstaging bot commented Jan 23, 2025 • edited Loading

Datadog Report

❌ Failed Tests (368)

New Flaky Tests (55)

⌛ Performance Regressions vs Default Branch (2)

andrewlock commented Jan 23, 2025

Execution-Time Benchmarks Report ⏱️

andrewlock commented Jan 23, 2025

Benchmarks Report for tracer 🐌

Benchmark details

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Raw results

Faster 🎉 in #6590

Raw results

Raw results

datadog-ddstaging bot commented Jan 23, 2025 •

edited

Loading