Re: CEE Field List

I think you are going in a slightly wrong direction.

I also am confused.  I thought that this mailing list was the working group but I continually see working group products coming from off-list; I feel excluded.

Here is a radical proposal.  Instead of name value pairs, use triplets.

It occurs to me that there are two interesting pieces of "name/type" metadata, the representational or syntactic type and the semantic type, for lack of better terms.

The representational type defines the syntax of the information that follows, e.g. ipv4 address, string, integer, timestamp, float, whatever.

The semantic type defines the usage of the data in the context of the event record, e.g. source address, user name, source port, detection time, event duration (ms), etc.

Taken together these map to a more complete story.

Separately they are also useful.  One might choose to correlate all ipv4 addresses regardless of whether they are source or destination addresses.  One might choose to store them in a manner that is most efficient based on the representational type (timestamps as 64-bit numerics vs. variable-length strings, etc.).  It also allows for semantic types that might have different representational types, e.g. an ipv6 packet teredo tunneled over an ipv4 network would cause events on intervening devices that would differ in representational type (ipv6addr vs. ipv4addr) but would be the same in semantic type (srcaddr).  Clever manipulation might even provide for correlation in such a case.

Also: an event record processing device might choose to only operate on representational types if it performs storage and/or forwarding functions but does not perform analysis functions.

This does not invalidate your principle of transport/log file format independence although you might require some syntactic sugar for non-delimited formats, e.g. ipv4addr_srcaddr=192.168.0.1

Best regards,
Eric

Eric Fitzgerald
Microsoft Corporation

-----Original Message-----
From: Raffael Marty [mailto:rmarty@...]
Sent: Thursday, March 06, 2008 11:12 AM
To: CEE-DISCUSSION-LIST@...
Subject: [CEE-DISCUSSION-LIST] CEE Field List

We have been busy working on the Common Event Syntax. As part of the syntax, we came up with a list of field names that should be used in log messages. A common name for fields helps cross-correlate log records between different products and log files. The list of field names is independent of the exact syntax that is used to write the log messages or the transport/format. Whether the data is written in an XML file, a flat text file, a CSV file, or using a binary encoding, a common set of field names helps cross-correlating these log messages.

A sample message that uses these field names could look as follows:

Feb 22 08:57:21 ram sudo[11033]: src_ip=10.2.2.1 dest_host=ram name=what an event dvc_location=home

Here are some specific questions we would like to pose to the
community:

- is the list more or less complete?
- are the descriptions meaningful? where do we need to tighten them up?
- do the data types make sense?
- how should we handle lists of values? For example, an event might talk about multiple ports.
- any other comments?