Handle data path changes between versions of Lustre

[joehandzik]

@roclark has identified some pieces of data that have changed their path home between versions of Lustre. An example is some set of ldlm data that is now in sysfs instead of procfs. We need to develop a way to allow the user to specify which version of Lustre they're running, and change our expectations of data locations depending on version.

My first guess at a design would involve making structures to hold the different path options for metric search locations, and take a Lustre version argument at the command line (no argument would default to the most recent version of Lustre that we've developed for). We could then dynamically load a different path for any metric that needs it.

The unfortunate part here is that it makes our design confusing. Now, we have metrics that were in proc for some versions, and will be in sys for others. It's...not something I'm sure how to deal with, but we can thank the Lustre community later for that. :)

[roclark]

For what it's worth, this ticket is the best I could find that is tracking the progress on the transition. Though there are some items that are already in /sys/fs/lustre as you mention, not everything is there yet and it looks like they are still working on it. Reading the comments it appears it is targeted for a point release of Lustre 2.10 (which just GA'ed yesterday), so we should keep monitoring this to make sure we can fully transition alongside Lustre.

Also note that this ticket has been mentioned a lot about tracking the transition to sysfs.

Finally, the changes first started with Lustre 2.8, so anyone using versions prior to 2.8 will be able to use the lustre_exporter as-is with all metrics collected (save ones that were added/removed/changed in old versions). With 2.8, there was a small transition to the other location.

[ghost]

The canonical way would be to gather data through lctl get_param that hides the path changes from the users, rather than directly scraping the procfs/sysfs paths. Multiple params can be queried at once to minimize the overhead of calling an external executable.

[joehandzik]

@sckobras Yeah, that's definitely an option. We wanted to at least enable procfs scraping first, keeping in spirit with how the Node Exporter works. We'll need to take a look at how wildly these paths change between releases, possibly just supporting major releases with procfs and minor release changes would require using lctl via a switch.

Of course, we may just realize that's a lot of effort to go to for not a lot of gain.