ixgbe: Add 1000BASE-BX support

[toreamun]

Add support for 1000BASE-BX (BiDi) single-mode SFP modules, commonly used for last mile fiber connections to homes and businesses.

BASE-BX10 is bit number 6 in the Ethernet Compliance Codes defined in SFF-8472, section 5.4.
In addition to this bit, the SFP module’s nominal signaling rate must be checked to make sure it
is 1000BASE-BX and not another BASE-BX variant, like 100BASE-BX. The hardware officially
only supports 1000BASE-BX according to the datasheet.

A new media type is also added to /sys/net/if_media.h, and the driver registers for this type when the 1000BASE-BX physical layer is detected.

The change has been tested with Intel X520 82599ES (SFI/SFP+):

$ ifconfig -vvv ix0
ix0: flags=1008843<UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST,LOWER_UP> metric 0 mtu 1500
        options=4803828<VLAN_MTU,JUMBO_MTU,WOL_UCAST,WOL_MCAST,WOL_MAGIC,HWSTATS,MEXTPG>
        ether a8:b8:e0:05:97:b1
        media: Ethernet autoselect (1000baseBX-10 <full-duplex>)
        status: active
        nd6 options=29<PERFORMNUD,IFDISABLED,AUTO_LINKLOCAL>
        drivername: ix0
        plugged: SFP/SFP+/SFP28 BASE-BX10 (SC)
        vendor: Tsuhan PN: THMPRS-3511-10A SN: F21061204830 DATE: 2021-06-11
        module temperature: 46.00 C voltage: 3.46 Volts
        lane 1: RX power: 0.15 mW (-8.10 dBm) TX bias: 17.97 mA

        SFF8472 DUMP (0xA0 0..127 range):
         03 04 01 00 00 00 40 12 00 01 00 01 0d 00 0a 64
         00 00 00 00 54 73 75 68 61 6e 20 20 20 20 20 20
         20 20 20 20 00 00 00 00 54 48 4d 50 52 53 2d 33
         35 31 31 2d 31 30 41 20 41 20 20 20 05 1e 00 12
         00 1a 00 00 46 32 31 30 36 31 32 30 34 38 33 30
         20 20 20 20 32 31 30 36 31 31 20 20 68 f0 01 cf
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
         00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

[emaste]

CC @kev009

[kev009]

I have something similar pending for igb(4). So far been testing 1000Base-BX10 modules so I can test your patch on ix(4). I am planning on picking up 100m, 10g modules.

@toreamun I found information on xxBASE-BX10 a little confusing but am under the impression it is available in 100m,1g,10g variants (and maybe theoretically 10mbit). In theory the lower rates might work fine if SGMII is available and I would be surprised if the 10g rate doesn't work natively on ix(4).

[toreamun]

I have something similar pending for igb(4). So far been testing 1000Base-BX10 modules so I can test your patch on ix(4). I am planning on picking up 100m, 10g modules.

Thanx @kev009. It would be helpful if you could share an SFF-8472 dump (ifconfig -vvv) of tested cards, especially for cards other than 1000BASE-BX10.

I found information on xxBASE-BX10 a little confusing but am under the impression it is available in 100m,1g,10g variants (and maybe theoretically 10mbit).

I agree that the information is a bit confusing at this point. It is crystal clear from the datasheet that 1000BASE-BX is supported, so I chose the conservative approach of checking the nominal rate. Support for other rates is not explicit in the datasheet. I will try to review the datasheet once more later today.

In theory the lower rates might work fine if SGMII is available and I would be surprised if the 10g rate doesn't work natively on ix(4).

It would be great to have this tested before enabling it. It may also require additional media types in /sys/net/if_media.h. I believe that distances other than 10 km could also work. I named the added media type IFM_1000_BX10 to align with OpenBSD, but perhaps a better name would be simply IFM_1000_BX, as users cannot change the built-in maximum range of their transceiver anyway?

[toreamun]

The Intel datasheet only talks about 1000BASE-BX, never 1000BASE-BX10. This may suggest that the hardware provides support for 1000BASE-BX in general, without limiting it specifically to 10 km (the BX10 variant). It is very likely that the Intel 82599 could support other 1000BASE-BX modules (like 20 km) since range is determined by the optical characteristics of the transceiver, not the controller. I have pushed and update where IFM_1000_BX10 now is changed to IFM_1000_BX. The update also includes a related commit I forgot in the initial push

[toreamun]

I will get my hands on both 100BASE-BX and 10GBASE-BX tranceivers and an extra card in a few weeks, and can then see if I can add support for them. I think that is best done in one or two separate PR and focus on 1000BASE-BX in this PR.

[kev009]

Are you ok dropping all this machinery around the bitrate check? If you have the right comp_code for 1g I think it is safe to proceed and SFPs tend too often to be miscoded. We can always add it back if there is unforeseen issue.

[kev009]

I agree with the change to BX, for instance LX can come in 2km or extended distances as well. By convention it seems we use the trailing numbers for lanes.

[kev009]

I went ahead and committed the if_media changes. Please rebase and squash on main once you have reviewed.

[toreamun]

I went ahead and committed the if_media changes. Please rebase and squash on main once you have reviewed.

I have now done a rebase to get the media commit from main. Do you want me to squash the other commits into fewer or single commit?

[kev009]

Yeah everything left here should be a single commit

[toreamun]

Yeah everything left here should be a single commit

Done

[toreamun]

@kev009 Automatic kernel cross build and style checker is happy, but the smoke test fails: "Failed to start an instance: FAILED_PRECONDITION: Monthly compute limit exceeded!"

[kev009]

@toreamun I haven't been in a rush to push this because I want to understand how it might interact with at least 10Gbase-BX. I have some 10GBase-BX optics, and some 100Mbit-BX optics to check.. not sure the later are the SGMII capable so only expecting the later to be relevant. I might have some time this weekend to finish this up.

[toreamun]

Hi @kev009 , I just wanted to kindly follow up on the status of this pull request. Please let me know if there’s anything I can do to help move it forward.

Additionally, I’ve realized that supporting 100BASE-BX will require further changes beyond this PR, so it might be best to set that aside for now and focus on the current improvements. I have tested using two 100BaseBX modules, and changes to the link setup and use of AUTOC.LMS flags is probably required if this is even possible.