Announcing Rust crate tracing-rfc-5424
Introduction
I've just published a new crate to crates.io that provides syslog support for the tracing ecosystem. You can see it here and the project home page is here. The project got started when I needed to log to a local syslog daemon from an app that uses tracing. I started down the same path recommended by David Barsky in this rust-lang thread: "Hey, there's this syslog crate, why not just call it from a Layer you write?"
I started down that path and found that that crate's Logger abstraction takes a &mut self on all it's methods, which makes it difficult to use in a Layer implementation (which only gets &self). I opened an issue here. "No problem," I thought, "the transport piece is easy. I can still make make use of the underlying components. Hmmmm…. what the heck are RFCs 5424 & 3164? The crate gives no guidance on which to use, so I need to understand them. Oh… Formatter5424 represents the hostname as a String, well having read the RFC I know that's not right… oh the heck with it."
That's how I wound up writing tracing-rfc-5424 – a tracing-subscriber Layer implementation that sends tracing Events (and, soon, Spans) to a syslog daemon.
Background
syslog
The term "syslog" collectively refers to implementations and standards for system-wide logging on Unix-like operating systems, including Linux. Originally developed in the eighties as part of the sendmail project, it has spread far & wide due to its simplicity & effectiveness. It became a de facto standard; RFC 3164 (2001) documented the observed behavior of the syslog communications protocol, while RFC 5424 (2009) extended & formalized it.
If you're running Linux, you can likely see /var/log/syslog (or /var/log/messages) right now. The two primary daemon implementations of which I am aware are syslog-ng and rsyslog.
tracing
The tracing crate is a "scoped, structured logging and diagnostics system". It provides a superset of the features offered by logging crates such as fern and log4rs particularly suited to asynchronous programming. Of particular interest is that it makes a very clear distinction between producers of events & their consumers (Subscribers, in tracing parlance); so much so that the tracing crate provides no support for consuming events, other than the definition of the Subscriber trait.
tracing-subscriber
The tracing-subscriber crate (also part of the Tokio project) provides a few basic implementations along with "utilities for implementing and composing tracing subscribers." A key notion introduced by this crate is the idea of a Layer. "Unlike Subscribers, which implement a complete strategy for how trace data is collected, Layers provide modular implementations of specific behaviors." The concern here is that, in general, a consumer of tracing event data may want to do various things at the same time with that data: write it to a log file, just write it to stdout, shuttle it off to a log collection daemon, produce metrics based on it, and so on.
This could easily give rise to a geometric explosion in types: LogFile, LogFileWithStdout, LogFileWithLogStash, LogFileWithLogStashAndStdout, and so forth. The idea behind Layers is to decompose each facet of event handling into its own type, and then "stack them up" in a Subscriber as the application developer desires. In a sense, this design pattern is reminiscent of Alexandrescu's concept of traits classes in C++– a way of decomposing functionality into orthogonal facets and composing them linearly rather than geometrically.
Rationale For a New Crate
There was some prior art. There was the aforementioned rust-syslog, but I found it's enforcement of the syslog standards to be more lax than I preferred. To take one example, RFC 5424 defines the "hostname" field as an ASCII string less than 256 characters in length, but rust-syslog represents it as a String: i.e. UTF-8 encoded text of unlimited length.
There's a relatively new crate named syslog-tracing that works by using FFI to directly call the libc syslog API. This approach admittedly has some advantages, such as reusing a battle-tested implementation of the client-side syslog protocol, but on the other hand leaves you effectively coding to a C API.
Naming & Crate Squatting
The name tracing-syslog seemed most natural to me, but has been already claimed (by an empty project) on crates.io. I've reached-out to the author, but haven't heard anything back (I believe he's in Russia, so he likely has bigger problems on his mind than name-squatting). I moved on to syslog-tracing, but before I published, that was claimed as well. While preparing to publish my crate, I wound-up re-factoring the repo into a library package & a test package and I've taken the opportunity to rename the library crate to tracing-rfc-5424.
As always, bug reports, pull requests & suggestions are welcome.
10/09/22 10:25