Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf(udp): multi-message receive on apple #2302

Merged
merged 5 commits into from
Jan 9, 2025

Conversation

mxinden
Copy link
Collaborator

@mxinden mxinden commented Dec 23, 2024

Do multi-message receive via quinn-rs/quinn#1993 on apple platforms.


Localhost benchmark (Download) with PMTUD disabled shows ~11% performance improvement.

$ critcmp main mmsg                                              
group                                         main                                   mmsg
-----                                         ----                                   ----
1-conn/1-100mb-resp (aka. Download)/client    1.11   790.5±12.51ms   126.5 MB/sec    1.00   711.5±31.65ms   140.5 MB/sec

Fixes #2279.

Copy link

github-actions bot commented Dec 23, 2024

Failed Interop Tests

QUIC Interop Runner, client vs. server, differences relative to ef9e4dc.

neqo-latest as client

neqo-latest as server

All results

Succeeded Interop Tests

QUIC Interop Runner, client vs. server

neqo-latest as client

neqo-latest as server

Unsupported Interop Tests

QUIC Interop Runner, client vs. server

neqo-latest as client

neqo-latest as server

@mxinden mxinden changed the title feat(udp): multi-message receive on apple perf(udp): multi-message receive on apple Dec 23, 2024
neqo-udp/src/lib.rs Outdated Show resolved Hide resolved
@mxinden mxinden force-pushed the apple-mmsg branch 7 times, most recently from aeb6fff to 6f98357 Compare December 27, 2024 19:03
Copy link

codecov bot commented Dec 27, 2024

Codecov Report

Attention: Patch coverage is 87.03704% with 7 lines in your changes missing coverage. Please review.

Project coverage is 93.33%. Comparing base (db0a332) to head (4aac744).
Report is 3 commits behind head on main.

Files with missing lines Patch % Lines
neqo-udp/src/lib.rs 87.03% 4 Missing and 3 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2302      +/-   ##
==========================================
- Coverage   93.34%   93.33%   -0.02%     
==========================================
  Files         114      114              
  Lines       36908    36889      -19     
  Branches    36908    36889      -19     
==========================================
- Hits        34451    34429      -22     
- Misses       1675     1679       +4     
+ Partials      782      781       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@mxinden mxinden marked this pull request as ready for review December 27, 2024 19:49
neqo-udp/src/lib.rs Outdated Show resolved Hide resolved
neqo-udp/src/lib.rs Outdated Show resolved Hide resolved
@larseggert larseggert enabled auto-merge January 8, 2025 17:27
Copy link

github-actions bot commented Jan 9, 2025

Benchmark results

Performance differences relative to ef9e4dc.

decode 4096 bytes, mask ff: No change in performance detected.
       time:   [11.168 µs 11.205 µs 11.248 µs]
       change: [-0.3788% +0.1613% +0.7455%] (p = 0.58 > 0.05)

Found 18 outliers among 100 measurements (18.00%)
1 (1.00%) low severe
5 (5.00%) low mild
1 (1.00%) high mild
11 (11.00%) high severe

decode 1048576 bytes, mask ff: 💔 Performance has regressed.
       time:   [3.0172 ms 3.0265 ms 3.0375 ms]
       change: [+3.9724% +4.5153% +5.0587%] (p = 0.00 < 0.05)

Found 10 outliers among 100 measurements (10.00%)
1 (1.00%) high mild
9 (9.00%) high severe

decode 4096 bytes, mask 7f: No change in performance detected.
       time:   [19.533 µs 19.582 µs 19.638 µs]
       change: [-0.2929% +0.0864% +0.4468%] (p = 0.66 > 0.05)

Found 27 outliers among 100 measurements (27.00%)
4 (4.00%) low severe
6 (6.00%) low mild
2 (2.00%) high mild
15 (15.00%) high severe

decode 1048576 bytes, mask 7f: 💔 Performance has regressed.
       time:   [5.1594 ms 5.1706 ms 5.1834 ms]
       change: [+1.2562% +1.6462% +2.0193%] (p = 0.00 < 0.05)

Found 14 outliers among 100 measurements (14.00%)
1 (1.00%) low mild
13 (13.00%) high severe

decode 4096 bytes, mask 3f: No change in performance detected.
       time:   [5.5262 µs 5.5440 µs 5.5706 µs]
       change: [-1.0699% -0.4711% +0.0274%] (p = 0.10 > 0.05)

Found 7 outliers among 100 measurements (7.00%)
4 (4.00%) high mild
3 (3.00%) high severe

decode 1048576 bytes, mask 3f: 💔 Performance has regressed.
       time:   [1.7625 ms 1.7694 ms 1.7776 ms]
       change: [+23.916% +24.654% +25.381%] (p = 0.00 < 0.05)

Found 14 outliers among 100 measurements (14.00%)
3 (3.00%) high mild
11 (11.00%) high severe

coalesce_acked_from_zero 1+1 entries: Change within noise threshold.
       time:   [99.199 ns 99.471 ns 99.750 ns]
       change: [+0.0355% +0.4283% +0.7920%] (p = 0.02 < 0.05)

Found 6 outliers among 100 measurements (6.00%)
2 (2.00%) high mild
4 (4.00%) high severe

coalesce_acked_from_zero 3+1 entries: Change within noise threshold.
       time:   [117.56 ns 117.92 ns 118.31 ns]
       change: [+0.5061% +0.8668% +1.2566%] (p = 0.00 < 0.05)

Found 15 outliers among 100 measurements (15.00%)
1 (1.00%) low mild
14 (14.00%) high severe

coalesce_acked_from_zero 10+1 entries: Change within noise threshold.
       time:   [117.16 ns 117.69 ns 118.31 ns]
       change: [-0.0775% +0.6481% +1.2617%] (p = 0.04 < 0.05)

Found 20 outliers among 100 measurements (20.00%)
5 (5.00%) low severe
1 (1.00%) low mild
5 (5.00%) high mild
9 (9.00%) high severe

coalesce_acked_from_zero 1000+1 entries: Change within noise threshold.
       time:   [98.535 ns 98.674 ns 98.837 ns]
       change: [+0.8541% +1.8132% +2.7847%] (p = 0.00 < 0.05)

Found 11 outliers among 100 measurements (11.00%)
4 (4.00%) high mild
7 (7.00%) high severe

RxStreamOrderer::inbound_frame(): Change within noise threshold.
       time:   [111.19 ms 111.23 ms 111.26 ms]
       change: [-0.2096% -0.1446% -0.0814%] (p = 0.00 < 0.05)

Found 7 outliers among 100 measurements (7.00%)
3 (3.00%) low mild
4 (4.00%) high mild

SentPackets::take_ranges: No change in performance detected.
       time:   [5.4984 µs 5.6614 µs 5.8311 µs]
       change: [-15.570% -4.1665% +4.2182%] (p = 0.64 > 0.05)

Found 8 outliers among 100 measurements (8.00%)
5 (5.00%) high mild
3 (3.00%) high severe

transfer/pacing-false/varying-seeds: Change within noise threshold.
       time:   [41.883 ms 41.958 ms 42.037 ms]
       change: [-2.7910% -2.5481% -2.2906%] (p = 0.00 < 0.05)

Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) high mild

transfer/pacing-true/varying-seeds: Change within noise threshold.
       time:   [42.297 ms 42.372 ms 42.446 ms]
       change: [-2.3559% -2.1151% -1.8833%] (p = 0.00 < 0.05)

Found 1 outliers among 100 measurements (1.00%)
1 (1.00%) low mild

transfer/pacing-false/same-seed: Change within noise threshold.
       time:   [41.615 ms 41.674 ms 41.732 ms]
       change: [-2.8392% -2.6525% -2.4468%] (p = 0.00 < 0.05)
transfer/pacing-true/same-seed: Change within noise threshold.
       time:   [42.313 ms 42.372 ms 42.430 ms]
       change: [-2.6838% -2.4963% -2.3095%] (p = 0.00 < 0.05)

Found 2 outliers among 100 measurements (2.00%)
2 (2.00%) low mild

1-conn/1-100mb-resp/mtu-1504 (aka. Download)/client: No change in performance detected.
       time:   [883.30 ms 893.00 ms 902.85 ms]
       thrpt:  [110.76 MiB/s 111.98 MiB/s 113.21 MiB/s]
change:
       time:   [-1.0177% +0.5464% +2.0802%] (p = 0.49 > 0.05)
       thrpt:  [-2.0379% -0.5434% +1.0282%]
1-conn/10_000-parallel-1b-resp/mtu-1504 (aka. RPS)/client: No change in performance detected.
       time:   [302.24 ms 304.42 ms 306.68 ms]
       thrpt:  [32.607 Kelem/s 32.849 Kelem/s 33.086 Kelem/s]
change:
       time:   [-0.3781% +0.5948% +1.6414%] (p = 0.25 > 0.05)
       thrpt:  [-1.6149% -0.5913% +0.3795%]

Found 4 outliers among 100 measurements (4.00%)
4 (4.00%) high mild

1-conn/1-1b-resp/mtu-1504 (aka. HPS)/client: Change within noise threshold.
       time:   [34.065 ms 34.258 ms 34.469 ms]
       thrpt:  [29.012  elem/s 29.191  elem/s 29.355  elem/s]
change:
       time:   [-2.2643% -1.3232% -0.4249%] (p = 0.00 < 0.05)
       thrpt:  [+0.4267% +1.3410% +2.3168%]

Found 5 outliers among 100 measurements (5.00%)
3 (3.00%) high mild
2 (2.00%) high severe

1-conn/1-100mb-resp/mtu-1504 (aka. Upload)/client: 💚 Performance has improved.
       time:   [1.5954 s 1.6102 s 1.6251 s]
       thrpt:  [61.534 MiB/s 62.105 MiB/s 62.682 MiB/s]
change:
       time:   [-6.1738% -4.9823% -3.8219%] (p = 0.00 < 0.05)
       thrpt:  [+3.9738% +5.2435% +6.5801%]

Client/server transfer results

Transfer of 33554432 bytes over loopback.

Client Server CC Pacing MTU Mean [ms] Min [ms] Max [ms]
gquiche gquiche 1504 547.4 ± 58.3 511.1 677.7
neqo gquiche reno on 1504 778.8 ± 74.0 715.6 953.2
neqo gquiche reno 1504 775.2 ± 53.1 736.7 913.8
neqo gquiche cubic on 1504 778.8 ± 76.1 736.8 992.0
neqo gquiche cubic 1504 779.0 ± 25.6 757.9 847.7
msquic msquic 1504 226.9 ± 100.2 100.6 404.6
neqo msquic reno on 1504 280.3 ± 142.4 197.4 693.1
neqo msquic reno 1504 267.3 ± 100.8 208.9 518.4
neqo msquic cubic on 1504 272.5 ± 72.4 210.8 425.2
neqo msquic cubic 1504 250.8 ± 66.5 212.7 423.5
gquiche neqo reno on 1504 734.9 ± 146.7 578.7 1028.4
gquiche neqo reno 1504 724.5 ± 125.6 555.7 928.6
gquiche neqo cubic on 1504 706.5 ± 101.7 540.6 897.3
gquiche neqo cubic 1504 709.0 ± 125.9 562.3 918.3
msquic neqo reno on 1504 497.5 ± 40.5 469.6 603.7
msquic neqo reno 1504 517.3 ± 47.6 481.3 615.8
msquic neqo cubic on 1504 498.0 ± 58.5 467.6 662.8
msquic neqo cubic 1504 494.9 ± 72.0 456.2 698.2
neqo neqo reno on 1504 522.9 ± 65.3 456.9 685.8
neqo neqo reno 1504 564.3 ± 75.3 481.7 759.5
neqo neqo cubic on 1504 521.1 ± 34.3 478.0 584.0
neqo neqo cubic 1504 607.2 ± 153.9 503.2 1028.2

⬇️ Download logs

@larseggert larseggert disabled auto-merge January 9, 2025 17:56
@larseggert larseggert merged commit 3c48567 into mozilla:main Jan 9, 2025
62 of 63 checks passed
mxinden added a commit to mxinden/neqo that referenced this pull request Jan 13, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

perf(udp): enable multi-message IO on apple
3 participants