Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

4.0.0: pytest is failing in tests/test_utils.py::test_commit_info_error unit #236

Open
kloczek opened this issue Mar 5, 2023 · 0 comments
Open

4.0.0: pytest is failing in tests/test_utils.py::test_commit_info_error unit #236

kloczek opened this issue Mar 5, 2023 · 0 comments

Comments

@kloczek
Copy link

kloczek commented Mar 5, 2023 •
edited
Loading

Looks like this unit should be skipped when there is no git

+ PYTHONPATH=/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-benchmark-4.0.0-3.fc35.x86_64/usr/lib64/python3.8/site-packages:/home/tkloczko/rpmbuild/BUILDROOT/python-pytest-benchmark-4.0.0-3.fc35.x86_64/usr/lib/python3.8/site-packages
+ /usr/bin/pytest -ra -m 'not network' --ignore tests/test_elasticsearch_storage.py
============================= test session starts ==============================
platform linux -- Python 3.8.16, pytest-7.2.1, pluggy-1.0.0
benchmark: 4.0.0 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /home/tkloczko/rpmbuild/BUILD/pytest-benchmark-4.0.0, configfile: pytest.ini, testpaths: tests
plugins: benchmark-4.0.0, xdist-3.2.0, aspectlib-1.5.2
collected 225 items

tests/test_benchmark.py ................................................ [ 21%]
.....                                                                    [ 23%]
tests/test_calibration.py ..................................             [ 38%]
tests/test_cli.py ...........                                            [ 43%]
tests/test_doctest.rst .                                                 [ 44%]
tests/test_normal.py ..........                                          [ 48%]
tests/test_pedantic.py ...................                               [ 56%]
tests/test_sample.py ....                                                [ 58%]
tests/test_skip.py s                                                     [ 59%]
tests/test_stats.py ..............                                       [ 65%]
tests/test_storage.py ........................................           [ 83%]
tests/test_utils.py ...ssssssssss.F...ss.ss.ss.ss.....                   [ 98%]
tests/test_with_testcase.py ..                                           [ 99%]
tests/test_with_weaver.py ..                                             [100%]

=================================== FAILURES ===================================
____________________________ test_commit_info_error ____________________________
/home/tkloczko/rpmbuild/BUILD/pytest-benchmark-4.0.0/tests/test_utils.py:127: in test_commit_info_error
    assert info['error'].lower().startswith("calledprocesserror(128, 'fatal: not a git repository")
E   assert False
E    +  where False = <built-in method startswith of str object at 0x7fea33260dc0>("calledprocesserror(128, 'fatal: not a git repository")
E    +    where <built-in method startswith of str object at 0x7fea33260dc0> = "filenotfounderror(2, 'no such file or directory')".startswith
E    +      where "filenotfounderror(2, 'no such file or directory')" = <built-in method lower of str object at 0x7fea33260ea0>()
E    +        where <built-in method lower of str object at 0x7fea33260ea0> = "FileNotFoundError(2, 'No such file or directory')".lower

----------------------------------------------------------------------------------------------------------------------------------- benchmark: 61 tests ------------------------------------------------------------------------------------------------------------------------------------
Name (time in ns)                                                     Min                             Max                            Mean                 StdDev                          Median                    IQR            Outliers               OPS             Rounds  Iterations
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
test_calibrate_stuck[False-0-1e-10]                                5.0500 (1.0)                    5.0500 (1.0)                    5.0500 (1.0)           0.0000 (1.0)                    5.0500 (1.0)           0.0000 (1.0)           0;0  198,019,801.9802 (1.0)            1           2
test_calibrate_stuck[False-1-1e-10]                                5.0500 (1.0)                    5.0500 (1.0)                    5.0500 (1.0)           0.0000 (1.0)                    5.0500 (1.0)           0.0000 (1.0)           0;0  198,019,801.9802 (1.0)            1           2
test_calibrate_stuck[False--1-1e-10]                               5.0500 (1.00)                   5.0500 (1.00)                   5.0500 (1.00)          0.0000 (1.0)                    5.0500 (1.00)          0.0000 (1.0)           0;0  198,019,801.9796 (1.00)           1           2
test_calibrate_stuck[True--1-1e-10]                               10.0000 (1.98)                  10.0000 (1.98)                  10.0000 (1.98)          0.0000 (1.0)                   10.0000 (1.98)          0.0000 (1.0)           0;0  100,000,000.0001 (0.51)           1           1
test_calibrate_stuck[True-0-1e-10]                                10.0000 (1.98)                  10.0000 (1.98)                  10.0000 (1.98)          0.0000 (1.0)                   10.0000 (1.98)          0.0000 (1.0)           0;0  100,000,000.0001 (0.51)           1           1
test_calibrate_stuck[True-1-1e-10]                                10.0000 (1.98)                  10.0000 (1.98)                  10.0000 (1.98)          0.0000 (1.0)                   10.0000 (1.98)          0.0000 (1.0)           0;0  100,000,000.0001 (0.51)           1           1
test_calibrate_stuck[False--1-1e-09]                              50.5000 (10.00)                 50.5000 (10.00)                 50.5000 (10.00)         0.0000 (1.0)                   50.5000 (10.00)         0.0000 (1.0)           0;0   19,801,980.1980 (0.10)           1           2
test_calibrate_stuck[False-0-1e-09]                               50.5000 (10.00)                 50.5000 (10.00)                 50.5000 (10.00)         0.0000 (1.0)                   50.5000 (10.00)         0.0000 (1.0)           0;0   19,801,980.1980 (0.10)           1           2
test_calibrate_stuck[False-1-1e-09]                               50.5000 (10.00)                 50.5000 (10.00)                 50.5000 (10.00)         0.0000 (1.0)                   50.5000 (10.00)         0.0000 (1.0)           0;0   19,801,980.1980 (0.10)           1           2
test_xfast                                                        66.7572 (13.22)                123.9777 (24.55)                 71.5150 (14.16)         3.5820 (inf)                   71.5256 (14.16)         0.0000 (1.0)        22;375   13,983,078.7807 (0.07)        1354         100
test_calibrate_stuck[True--1-1e-09]                              100.0000 (19.80)                100.0000 (19.80)                100.0000 (19.80)         0.0000 (1.0)                  100.0000 (19.80)         0.0000 (1.0)           0;0   10,000,000.0000 (0.05)           1           1
test_calibrate_stuck[True-0-1e-09]                               100.0000 (19.80)                100.0000 (19.80)                100.0000 (19.80)         0.0000 (1.0)                  100.0000 (19.80)         0.0000 (1.0)           0;0   10,000,000.0000 (0.05)           1           1
test_calibrate_stuck[True-1-1e-09]                               100.0000 (19.80)                100.0000 (19.80)                100.0000 (19.80)         0.0000 (1.0)                  100.0000 (19.80)         0.0000 (1.0)           0;0   10,000,000.0000 (0.05)           1           1
test_calibrate_xfast                                             109.3117 (21.65)              6,494.5500 (>1000.0)              111.7397 (22.13)        18.7764 (inf)                  110.8716 (21.95)         0.6286 (inf)    11454;118971    8,949,373.6255 (0.05)      941693         100
test_rounds_iterations                                           334.0188 (66.14)                560.8890 (111.07)               385.4496 (76.33)        62.9072 (inf)                  362.0051 (71.68)        51.9562 (inf)           2;2    2,594,373.0546 (0.01)          15          10
test_proto[LocalsSimpleProxy]                                    402.7970 (79.76)            114,749.8027 (>1000.0)              456.2419 (90.34)       415.6832 (inf)                  419.2465 (83.02)        33.2948 (inf)     236;12179    2,191,819.5905 (0.01)       78156          20
test_calibrate_fast                                              444.0080 (87.92)             53,213.6764 (>1000.0)              463.9979 (91.88)        96.5740 (inf)                  460.5778 (91.20)         9.2356 (inf)    12260;66573    2,155,182.1205 (0.01)     1902533          12
test_rounds                                                      447.9662 (88.71)              1,393.0257 (275.85)               638.6079 (126.46)      260.5217 (inf)                  532.0180 (105.35)      276.4282 (inf)           2;1    1,565,906.1164 (0.01)          15           1
test_proto[LocalsCachedPropertyProxy]                            448.8975 (88.89)             16,906.9972 (>1000.0)              511.4434 (101.28)      211.2767 (inf)                  490.8070 (97.19)        28.1725 (inf)     1889;5077    1,955,250.6569 (0.01)      175105           1
test_warmup_rounds                                               523.8689 (103.74)               896.1651 (177.46)               634.4169 (125.63)      153.4711 (inf)                  579.9811 (114.85)      174.0991 (inf)           1;0    1,576,250.4756 (0.01)           5           1
test_iterations                                                  558.0019 (110.50)               558.0019 (110.50)               558.0019 (110.50)        0.0000 (1.0)                  558.0019 (110.50)        0.0000 (1.0)           0;0    1,792,108.5271 (0.01)           1          10
test_proto[CachedPropertyProxy]                                  631.9024 (125.13)             9,700.8888 (>1000.0)              688.6240 (136.36)      126.2901 (inf)                  679.1670 (134.49)       27.0084 (inf)       94;1542    1,452,171.2937 (0.01)       80127           1
test_proto[SimpleProxy]                                          738.0731 (146.15)            54,013.2169 (>1000.0)              800.9582 (158.61)      201.2293 (inf)                  786.0363 (155.65)       21.1876 (inf)    2018;12366    1,248,504.6308 (0.01)      177456           1
test_setup_many_rounds                                           905.0127 (179.21)             2,499.9026 (495.03)             1,212.3259 (240.06)      479.6699 (inf)                1,027.9473 (203.55)      152.9697 (inf)           1;2      824,860.7225 (0.00)          10           1
test_single                                                    1,452.8632 (287.70)             1,452.8632 (287.70)             1,452.8632 (287.70)        0.0000 (1.0)                1,452.8632 (287.70)        0.0000 (1.0)           0;0      688,296.0410 (0.00)           1           1
test_can_use_both_args_and_setup_without_return                1,922.0170 (380.60)             1,922.0170 (380.60)             1,922.0170 (380.60)        0.0000 (1.0)                1,922.0170 (380.60)        0.0000 (1.0)           0;0      520,286.7712 (0.00)           1           1
test_args_kwargs                                               3,504.1012 (693.88)             3,504.1012 (693.88)             3,504.1012 (693.88)        0.0000 (1.0)                3,504.1012 (693.88)        0.0000 (1.0)           0;0      285,379.8868 (0.00)           1           1
test_setup                                                     3,807.0139 (753.86)             3,807.0139 (753.86)             3,807.0139 (753.86)        0.0000 (1.0)                3,807.0139 (753.86)        0.0000 (1.0)           0;0      262,673.0656 (0.00)           1           1
test_foo                                                       6,034.9703 (>1000.0)          152,045.8609 (>1000.0)           64,790.4150 (>1000.0)   8,395.6336 (inf)               67,616.1144 (>1000.0)     147.8475 (inf)     2351;3620       15,434.3818 (0.00)       16167           1
test_fast                                                      8,741.1609 (>1000.0)           97,583.9794 (>1000.0)           62,108.6773 (>1000.0)   8,485.5536 (inf)               60,020.9460 (>1000.0)   7,230.3228 (inf)     1462;1017       16,100.8098 (0.00)       14085           1
test_parametrized[0]                                          14,543.9990 (>1000.0)          115,676.0845 (>1000.0)           68,715.2769 (>1000.0)   9,043.4774 (inf)               64,658.0011 (>1000.0)  11,797.0631 (inf)       905;352       14,552.8046 (0.00)       14271           1
test_parametrized[1]                                          15,130.9650 (>1000.0)          148,372.9575 (>1000.0)           75,416.8173 (>1000.0)   6,040.9576 (inf)               76,527.0088 (>1000.0)     147.1490 (inf)     2827;4163       13,259.6420 (0.00)       14358           1
test_foo2                                                     15,847.1521 (>1000.0)           78,969.1694 (>1000.0)           65,483.4025 (>1000.0)   8,188.2788 (inf)               67,656.8598 (>1000.0)     128.0569 (inf)      846;1687       15,271.0452 (0.00)       11936           1
test_setup_cprofile                                           18,751.0159 (>1000.0)           18,751.0159 (>1000.0)           18,751.0159 (>1000.0)       0.0000 (1.0)               18,751.0159 (>1000.0)       0.0000 (1.0)           0;0       53,330.4439 (0.00)           1           1
test_calibrate_slow                                           26,900.0884 (>1000.0)        3,155,903.8907 (>1000.0)           77,413.4750 (>1000.0)   9,333.6087 (inf)               76,854.1358 (>1000.0)     157.8592 (inf)    3542;106115       12,917.6477 (0.00)      683390           1
test_calibrate                                                44,321.8742 (>1000.0)        4,201,730.9461 (>1000.0)           46,529.1507 (>1000.0)  38,899.3567 (inf)               45,694.1780 (>1000.0)     604.8940 (inf)     155;23645       21,491.9031 (0.00)      225556           1
test_parametrized[4]                                          47,705.1362 (>1000.0)          138,234.8128 (>1000.0)           77,325.6275 (>1000.0)   2,465.4677 (inf)               76,652.0388 (>1000.0)     128.0569 (inf)      880;1719       12,932.3231 (0.00)       10735           1
test_parametrized[3]                                          54,944.0738 (>1000.0)        3,035,787.0273 (>1000.0)           77,622.7805 (>1000.0)  30,397.4406 (inf)               76,571.9451 (>1000.0)     117.8123 (inf)        3;1585       12,882.8160 (0.00)        9539           1
test_parametrized[2]                                          57,581.8121 (>1000.0)          122,495.9269 (>1000.0)           77,343.8228 (>1000.0)   2,520.5551 (inf)               76,618.9769 (>1000.0)     115.9497 (inf)      890;1732       12,929.2808 (0.00)       10802           1
test_slow                                                  1,048,668.0549 (>1000.0)        1,193,783.9445 (>1000.0)        1,121,650.8574 (>1000.0)  43,953.8094 (inf)            1,095,492.8584 (>1000.0)  88,272.7909 (inf)         493;0          891.5430 (0.00)         929           1
test_slower                                               10,149,935.0276 (>1000.0)       10,202,737.0594 (>1000.0)       10,174,600.1088 (>1000.0)  10,092.0557 (inf)           10,174,617.1713 (>1000.0)  12,523.4947 (inf)          28;3           98.2840 (0.00)          99           1
test_weave_fixture                                        10,153,203.9698 (>1000.0)       10,199,252.9817 (>1000.0)       10,176,762.1934 (>1000.0)  18,238.2979 (inf)           10,170,449.9684 (>1000.0)  26,611.0292 (inf)           2;0           98.2631 (0.00)           5           1
test_weave_method                                         10,166,110.0052 (>1000.0)       10,181,545.0471 (>1000.0)       10,174,346.4824 (>1000.0)   7,295.0436 (inf)           10,177,953.1687 (>1000.0)  13,160.2283 (inf)           2;0           98.2864 (0.00)           5           1
test_calibrate_stuck[False--1-0.01]                      504,999,999.9992 (>1000.0)      504,999,999.9992 (>1000.0)      504,999,999.9992 (>1000.0)       0.0000 (1.0)          504,999,999.9992 (>1000.0)       0.0000 (1.0)           0;0            1.9802 (0.00)           1           2
test_calibrate_stuck[False-0-0.01]                       504,999,999.9992 (>1000.0)      504,999,999.9992 (>1000.0)      504,999,999.9992 (>1000.0)       0.0000 (1.0)          504,999,999.9992 (>1000.0)       0.0000 (1.0)           0;0            1.9802 (0.00)           1           2
test_calibrate_stuck[False-1-0.01]                       504,999,999.9992 (>1000.0)      504,999,999.9992 (>1000.0)      504,999,999.9992 (>1000.0)       0.0000 (1.0)          504,999,999.9992 (>1000.0)       0.0000 (1.0)           0;0            1.9802 (0.00)           1           2
test_calibrate_stuck[True--1-0.01]                     1,000,000,000.0000 (>1000.0)    1,000,000,000.0000 (>1000.0)    1,000,000,000.0000 (>1000.0)       0.0000 (1.0)        1,000,000,000.0000 (>1000.0)       0.0000 (1.0)           0;0            1.0000 (0.00)           1           1
test_calibrate_stuck[True-0-0.01]                      1,000,000,000.0000 (>1000.0)    1,000,000,000.0000 (>1000.0)    1,000,000,000.0000 (>1000.0)       0.0000 (1.0)        1,000,000,000.0000 (>1000.0)       0.0000 (1.0)           0;0            1.0000 (0.00)           1           1
test_calibrate_stuck[True-1-0.01]                      1,000,000,000.0000 (>1000.0)    1,000,000,000.0000 (>1000.0)    1,000,000,000.0000 (>1000.0)       0.0000 (1.0)        1,000,000,000.0000 (>1000.0)       0.0000 (1.0)           0;0            1.0000 (0.00)           1           1
test_calibrate_stuck[False--1-1.000000000000001]      50,500,000,000.0000 (>1000.0)   50,500,000,000.0000 (>1000.0)   50,500,000,000.0000 (>1000.0)       0.0000 (1.0)       50,500,000,000.0000 (>1000.0)       0.0000 (1.0)           0;0            0.0198 (0.00)           1           2
test_calibrate_stuck[False--1-1]                      50,500,000,000.0000 (>1000.0)   50,500,000,000.0000 (>1000.0)   50,500,000,000.0000 (>1000.0)       0.0000 (1.0)       50,500,000,000.0000 (>1000.0)       0.0000 (1.0)           0;0            0.0198 (0.00)           1           2
test_calibrate_stuck[False-0-1.000000000000001]       50,500,000,000.0000 (>1000.0)   50,500,000,000.0000 (>1000.0)   50,500,000,000.0000 (>1000.0)       0.0000 (1.0)       50,500,000,000.0000 (>1000.0)       0.0000 (1.0)           0;0            0.0198 (0.00)           1           2
test_calibrate_stuck[False-0-1]                       50,500,000,000.0000 (>1000.0)   50,500,000,000.0000 (>1000.0)   50,500,000,000.0000 (>1000.0)       0.0000 (1.0)       50,500,000,000.0000 (>1000.0)       0.0000 (1.0)           0;0            0.0198 (0.00)           1           2
test_calibrate_stuck[False-1-1.000000000000001]       50,500,000,000.0000 (>1000.0)   50,500,000,000.0000 (>1000.0)   50,500,000,000.0000 (>1000.0)       0.0000 (1.0)       50,500,000,000.0000 (>1000.0)       0.0000 (1.0)           0;0            0.0198 (0.00)           1           2
test_calibrate_stuck[False-1-1]                       50,500,000,000.0000 (>1000.0)   50,500,000,000.0000 (>1000.0)   50,500,000,000.0000 (>1000.0)       0.0000 (1.0)       50,500,000,000.0000 (>1000.0)       0.0000 (1.0)           0;0            0.0198 (0.00)           1           2
test_calibrate_stuck[True--1-1.000000000000001]      100,000,000,000.0000 (>1000.0)  100,000,000,000.0000 (>1000.0)  100,000,000,000.0000 (>1000.0)       0.0000 (1.0)      100,000,000,000.0000 (>1000.0)       0.0000 (1.0)           0;0            0.0100 (0.00)           1           1
test_calibrate_stuck[True--1-1]                      100,000,000,000.0000 (>1000.0)  100,000,000,000.0000 (>1000.0)  100,000,000,000.0000 (>1000.0)       0.0000 (1.0)      100,000,000,000.0000 (>1000.0)       0.0000 (1.0)           0;0            0.0100 (0.00)           1           1
test_calibrate_stuck[True-0-1.000000000000001]       100,000,000,000.0000 (>1000.0)  100,000,000,000.0000 (>1000.0)  100,000,000,000.0000 (>1000.0)       0.0000 (1.0)      100,000,000,000.0000 (>1000.0)       0.0000 (1.0)           0;0            0.0100 (0.00)           1           1
test_calibrate_stuck[True-0-1]                       100,000,000,000.0000 (>1000.0)  100,000,000,000.0000 (>1000.0)  100,000,000,000.0000 (>1000.0)       0.0000 (1.0)      100,000,000,000.0000 (>1000.0)       0.0000 (1.0)           0;0            0.0100 (0.00)           1           1
test_calibrate_stuck[True-1-1.000000000000001]       100,000,000,000.0000 (>1000.0)  100,000,000,000.0000 (>1000.0)  100,000,000,000.0000 (>1000.0)       0.0000 (1.0)      100,000,000,000.0000 (>1000.0)       0.0000 (1.0)           0;0            0.0100 (0.00)           1           1
test_calibrate_stuck[True-1-1]                       100,000,000,000.0000 (>1000.0)  100,000,000,000.0000 (>1000.0)  100,000,000,000.0000 (>1000.0)       0.0000 (1.0)      100,000,000,000.0000 (>1000.0)       0.0000 (1.0)           0;0            0.0100 (0.00)           1           1
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

Legend:
  Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  OPS: Operations Per Second, computed as 1 / Mean
----------------------------- cProfile (time in s) -----------------------------
tests/test_pedantic.py::test_setup_cprofile
ncalls  tottime percall cumtime percall filename:lineno(function)
1       0.0000  0.0000  0.0000  0.0000  pytest-benchmark-4.0.0/tests/test_pedantic.py:29(stuff)
1       0.0000  0.0000  0.0000  0.0000  ~:0(<method 'append' of 'list' objects>)
1       0.0000  0.0000  0.0000  0.0000  ~:0(<method 'disable' of '_lsprof.Profiler' objects>)

=========================== short test summary info ============================
SKIPPED [1] tests/test_skip.py:5: bla
SKIPPED [2] tests/test_utils.py:60: 'git' not available on $PATH
SKIPPED [2] tests/test_utils.py:60: 'hg' not available on $PATH
SKIPPED [2] tests/test_utils.py:80: 'git' not available on $PATH
SKIPPED [2] tests/test_utils.py:80: 'hg' not available on $PATH
SKIPPED [1] tests/test_utils.py:94: 'git' not available on $PATH
SKIPPED [1] tests/test_utils.py:94: 'hg' not available on $PATH
SKIPPED [4] tests/test_utils.py:160: 'git' not available on $PATH
SKIPPED [4] tests/test_utils.py:160: 'hg' not available on $PATH
FAILED tests/test_utils.py::test_commit_info_error - assert False
============ 1 failed, 205 passed, 19 skipped in 478.31s (0:07:58) =============
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@kloczek