gosmi state and roadmap?

[toni-moreno]

Hi @sleepinggenius2 , I've been planning to add direct MIB support to my snmpcollector https://github.com/toni-moreno/snmpcollector project, and I would like to know the state and the
roadmap for your great work.

• is this library tested with lots of different MIB files?
• are you planning to improve it?
• what is the current state for this project

Thank you very much!!

[sleepinggenius2]

I will do my best to answer each of your questions below.

Is this library tested with lots of different MIB files?

Yes. I've tested this against all the MIB files that I currently have available to me, which is 4400+ files across 240+ vendors, including the standard IETF, IANA, IEEE, UC Davis, and CableLabs MIBs. The parser should support all RFC-compliant MIBs, but it does provide some leniency for the most common errors that I have seen across different vendors. That is not something that can currently be configured, so it would not be directly suited for use in a linter.

Are you planning to improve it?

The biggest outstanding todo list item is adding full test coverage. That was not really possible as a wrapper around libsmi, but would be now that it is pure Go. Of course, I also want to see if there are any optimizations that can be done. Most of the improvements currently come from things that are uncovered using the library in various other projects.

What is the current state for this project?

This project was originally written because I needed it for use in another project I was working on and I couldn't find anything else out there at that time. After proving it could meet my needs, I felt that it was something that the community could benefit from as well. I've been using it on a regular basis in a number of projects, mostly in conjunction with the projects in my other repos, from the very beginning. The cgo version was a quick and dirty solution at the beginning, but brought about numerous issues over the years, which final necessitated either patching libsmi or rewriting everything natively in Go, which is the route I ended up taking. The project should be fairly stable at this time, but I am still working toward a 1.0 release, so there may be small API changes. I know it is relatively fast, but I have not personally used it in real-time applications, only through generated code from my mib2go project, so I cannot vouch for the performance in that situation.

I'm also quite intrigued by your project, as that's along the same lines as some of the other (currently non-public) projects that I've been working on. I'm not sure of your background, but from one who deals with MIBs and SNMP on nearly a daily basis, I would definitely suggest testing your project against as many different devices as possible and not necessarily having faith that the MIBs will give you the whole story. For example, one of the most common things that I've seen is devices that have not properly implemented scalars and completely ignore the implicit .0 index at the end of the OID. I would also suggest looking into my fork of gosnmp if you start running into scaling issues. There are probably still numerous optimizations that can be made, but I've implemented quite a few that help significantly with both CPU and memory consumption when you start scaling out to polling large numbers of devices in parallel, especially as you get into the 10s to 100s of thousands. The largest I've tested with the current version is 1 minute polling of ~78k devices, with less than 4 cores and 8G of RAM. Hope this helps!

[sleepinggenius2]

I just ran a parser test sequentially for 4454 MIB files on a 3.6GHz i7 with no errors encountered during parsing. The significant outlier at the end is a single 25MB 651k line MIB file. I have included graphs both with and without that data point as well as the test code that I used.

With:

Without:

package main

import (
    "bufio"
    "fmt"
    "os"
    "time"

    "github.com/sleepinggenius2/gosmi"
)

func main() {
    var module, moduleName string
    var err error
    var start time.Time

    gosmi.Init()

    scanner := bufio.NewScanner(os.Stdin)
    for scanner.Scan() {
        module = scanner.Text()
        start = time.Now()
        moduleName, err = gosmi.LoadModule(module)
        if err == nil {
            fmt.Println(moduleName, int64(time.Since(start)))
        } else {
            fmt.Println("ERROR", err)
        }
    }

    gosmi.Exit()
}

[toni-moreno]

Hi @sleepinggenius2 Thank you a lot for your extensive answer. We will play with it and If enough fast to translate Names to OID's we will include in the next release of our snmpcollector.

Thank you very much!

[Sugaroverdose]

@sleepinggenius2 why did you decided to maintain your own gosnmp fork btw? isn't it better to merge your changes into soniah/gosnmp

[sleepinggenius2]

The fork was originally created out of necessity to be able to set the SO_REUSEPORT socket option to support more than 64k simultaneous clients. Unfortunately, I then ran into significant CPU and memory constraints at that scale as well. I ended up making a number of changes to address those issues as well as some specific changes to better integrate with my other libraries.

It has always been my intention to merge my changes back in, but the codebase has diverged in significant enough ways at this point that I have not yet had the time to do so. I'm hoping to be able to create all the appropriate PRs, tests, etc. at some point in the not too distant future.