Re: Encoding unknown values in the extended_switchstructure

Hi Peter,

I appreciate the hierarchical nature of the data_format tag. Perhaps
the example of taking two bytes was highly unfortunate given that the
field is logically split in 20+12 bits.

Of course I share your view on handling unknowns in a future sFlow
version and the recommendation to developers of structures. Indeed
the '0xFFFFFFFF' approach can result the least impacting - except
imagine somebody does sanity checks on those fields, which i guess
it's why you dropped an email in the first place.

As said, I was focusing only on the case in which enterprise value
is 0 leaving in the hands of specific developers, enterprise value
!= 0, to find a way to meaningfully handle unknowns - given that
likely they will be in control of both the agent and the collector.

While thinking at an acceptable trick on the data_format tag, i
stumbled upon the 32 bit length field just besides that. Do you see
a chance of recommending carving part of that field (despite its
name) for the bitmap - when enterprise value equals to zero? At this
propo, I couldn't find any explicit reference to what is thought to
be the maximum size of an sFlow datagram; did i miss such indication
or should that be implicitely inferred?

Cheers,
Paolo

On Sat, May 23, 2009 at 09:13:30AM -0700, Peter Phaal wrote:
> It would be nice to have a general purpose "unknown" mechanism and the use
> of a bitmap to indicate which elements in a structure fits well with XDR
> encoding.
>
> However, your proposal to take the upper two bytes from the data_format
> field has some problems. The sFlow version 5 data_format field is fully
> defined. Your proposed change would not be backward compatible for sFlow
> collectors since they would need to be altered to ignore the bits (or use
> them to mask the structure).
>
> The data_format uniquely identifies the format of an opaque structure in
> the sFlow specification. A data_format is contructed as follows:
> - The most significant 20 bits correspond to the SMI Private Enterprise
> Code of the entity responsible for the structure definition. A value
> of zero is used to denote standard structures defined by sflow.org.
> - The least significant 12 bits are a structure format number assigned
> by the enterprise that should uniquely identify the the format of the
> structure.
>
> I just checked IANA and the largest currently assigned enterprise number is
> now 33572. Using the upper two bytes of the data_format would overlap with
> enterprise codes, breaking vendor extensibility - an important feature of
> sFlow version 5.
>
> I think you may have misunderstood the role of the structure number in the
> data_format tag. The entire data_format element (32 bits) is treated as a
> unique structure id. The allocation scheme is hierarchical, providing each
> enterprise publishing structures with a full 12 bits that they can assign as
> they choose (i.e. the structure number by itself does not uniquely
> distinguish a structure, it must be combined with the assigning entity to
> provide a unique id).
>
> Currently, sFlow, version 5 defines unknown fields as follows:
> The following values should be used for fields that are
> unknown (unless otherwise indicated in the structure
> definitions).
> - Unknown integer value. Use a value of 0 to indicate that
> a value is unknown.
> - Unknown counter. Use the maximum counter value to indicate
> that the counter is not available. Within any given sFlow
> session a particular counter must be always available, or
> always unavailable. An available counter may temporarily
> have the max value just before it rolls to zero. This is
> permitted.
> - Unknown string. Use the zero length empty string. */
>
> Of course there is also the option of omitting a structure all together to
> indicate that none of its fields are known to the agent.
>
> These rules work reasonably well, except for the case of the extended_switch
> structure. By "otherwise indicating" the unknown values as 0xFFFFFFFF the
> proposal is the smallest possible change that should be backward compatible
> for existing agents and collectors.
>
> If an sFlow version 6 is ever defined then your suggestion of using a bit
> array to provide a general unknown value mechanism would be an attractive
> extension.
>
> As new structures are defined for sFlow version 5, adopting a convention in
> which the first member of the structure is a bit array indicating any
> unknown elements in the structure would be something that authors of
> structures should consider if there is any possibility that elements of the
> structure might not be known.
>
> Peter
>
> > -----Original Message-----
> > From: Paolo Lucente [mailto:pl+list@pmacct.net]
> > Sent: Saturday, May 23, 2009 4:59 AM
> > To: Peter Phaal
> > Cc: sflow@sflow.org
> > Subject: Re: [sFlow] Encoding unknown values in the
> > extended_switchstructure
> >
> > Hi Peter,
> >
> > Not really an objection - rather a proposal for your consideration.
> >
> > What about bitmapping fields contained in standard elements within,
> > say, the upper two bytes of the element tag?
> >
> > typedef struct _SFLFlow_sample_element {
> > struct _SFLFlow_sample_element *nxt;
> > u_int32_t tag; /* SFLFlow_type_tag */ <-----
> > u_int32_t length;
> > SFLFlow_type flowType;
> > } SFLFlow_sample_element;
> >
> > ie.
> >
> > #define EX_SWITCH_SRC_VLAN 0x00010000
> > #define EX_SWITCH_SRC_PRIORITY 0x00020000
> > #define EX_SWITCH_DST_VLAN 0x00040000
> > #define EX_SWITCH_DST_PRIORITY 0x00080000
> >
> > It would have the beauty of being backward compatible, meaning the
> > collector can just ignore it and not take benefit from it, and will
> > maintain intact the 32 bit field - which is not a requirement today,
> > for this specific case, but you might be less lucky tomorrow trying
> > to do the same trick elsewhere; so a more coherent approach.
> >
> > Companies developing closed extensions and thus making use of the
> > upper bytes of the tag field, ie.
> >
> > /* enterprise = 4300 (inmon)...*/
> > SFLFLOW_EX_PROCESS = (4300 << 12) + 3
> >
> > might (or will need to) use a different way to signal some fields
> > are not populated within their elements.
> >
> > The only con of this approach is that it would put an upper bound
> > to the number of fields that can be packed within an element, ie.
> > 15 fields if using the upper 2 bytes (keeping out the '0x00000000'
> > value), but i see you have never been a big fan of flat structures
> > ( :-) ) so i wouldn't see this as a true biggie. Quickly scanning
> > through the existing elements, all seem to fit this approach.
> >
> > Cheers,
> > Paolo
> >
> >
> > On Fri, May 22, 2009 at 11:07:55AM -0700, Peter Phaal wrote:
> > > There currently isn't a satisfactory way to indicate that one or more of
> > the
> > > fields in the extended_switch structure is unknown to the sFlow
> > exporter.
> > >
> > > /* Extended Switch Data */
> > > /* opaque = flow_data; enterprise = 0; format = 1001 */
> > > /* Note: For untagged ingress ports, use the assigned vlan and priority
> > > of the port for the src_vlan and src_priority values.
> > > For untagged egress ports, use the values for dst_vlan and
> > > dst_priority that would have been placed in the 802.Q tag
> > > had the egress port been a tagged member of the VLAN instead
> > > of an untagged member. */
> > >
> > > struct extended_switch {
> > > unsigned int src_vlan; /* The 802.1Q VLAN id of incoming frame */
> > > unsigned int src_priority; /* The 802.1p priority of incoming frame
> > */
> > > unsigned int dst_vlan; /* The 802.1Q VLAN id of outgoing frame */
> > > unsigned int dst_priority; /* The 802.1p priority of outgoing frame
> > */
> > > }
> > >
> > > Since 0 is a valid value for both 802.1Q VLAN 802.1p priority it should
> > not
> > > be used to encode the unknown value. Rather than omit the whole
> > structure
> > > because one field isn't known, it makes sense to define an unambiguous
> > value
> > > to indicate that a field is unknown.
> > >
> > > Fortunately these fields have relatively small maximum values, vlan <=
> > 4096
> > > and priority <= 7, so an unambiguous choice for an unknown value would
> > be
> > > 0xffffffff (Note: this same value that sFlow currently uses to indicate
> > that
> > > a 32 bit counter value is unknown). If there are no objections, I think
> > it
> > > would make sense to include a note to this effect on Specifications page
> > on
> > > sFlow.org.
> > >
> > > Peter
Received on Sat May 23 14:57:01 2009