Goal

The goal of Common Event Expression (CEE) is to define a standardized event definition (CEE Profile). In order to encourage use and improve compatibility with existing standards, a CEE Event can be encoded using one or more Common Log Syntax (CLS) Encodings. Each CLS Encoding defines how a CEE Event is encoded into an exchangeable record format, compliant with a specific syntax encoding.

The combination of the CEE Profile and CLS specifications enable the efficient and lossless storage, exchange, and consumption of event records. The event record format was designed for maximum compatibility and can be encoded to interoperate with various standard syntaxes. CLS Encodings are defined for recording of CEE Events using XML or JSON. Additional encodings may be defined to enable CEE Event representation in other syntaxes.

BACK TO TOP

Process

CEE makes a distinction between the event and the event encoding process. Once an event occurs that should be recorded, the relevant data from that event is captured and later encoded as an event record. Then, the event record can be easily logged or shared with any device that can understand the event record encoding.

In order for event consumers to consume CLS-encoded event records, the consumer must simply decode the CLS-encoded record to obtain the original CEE Event. The process encoding and decoding of event records using a CLS Encoding MUST be performed without the loss of any CEE Event data.

+---------+             +---------+  encode  +---------+
|         |  describe   |         | =======> |         |
| Profile | ==========> |  Event  |          | Record  |
|         |             |         | <======= |         |
+---------+             +---------+  decode  +---------+

BACK TO TOP

CEE Event & Profile

The CEE Event structure consists of a common header, an extensible body profile, and zero or more module elements. The header is a collection of fields that are required by all events, such as the time and event producer information. The profile is the core of an event. A profile is defined by the user or developer, and provides the specific fields and data unique to that type of event.

Additionally, CEE places the following constraints on a CEE Event:

More detailed information on CEE Events and Event Profiles can be found in the CEE Profile specification.

BACK TO TOP

Field Types

The CEE Profile supports the following field types:

1. xs:string
2. xs:byte
3. xs:unsignedByte
4. xs:short
5. xs:unsignedShort
6. xs:int
7. xs:unsignedInt
8. xs:long
9. xs:unsignedLong
10. xs:float
11. xs:double
12. xs:dateTime
13. xs:hexBinary
14. xs:anyURI
15. xs:boolean
16. tag
17. ipAddress
18. macAddress
19. hostname
20. anyCEEType

Each value type is a derivative of the string type and MUST be representable as a sequence of UTF-8 encoded Unicode characters. The NUL (ABNF %00) character MAY exist in a CEE Event, but MUST NOT appear in any encoded event record.

More information for each of these types may be found in the CEE Profile: Field Types section.

BACK TO TOP

Encoding Specification Requirements

Encoding Identification

Each syntax encoding SHOULD define a MIME type to allow consumers to correctly identify the CLS syntax used to encode CEE event record.

The MIME type SHOULD be an application type with a subtype of cee+ENCODING, where ENCODING is the common short name for the underlying syntax encoding. For example, application/cee+json would identify a JSON encoded CEE event record, while application/cee+xml is an XML encoded record.

Event Transport

This specification does not define or require the use of any specific transport or protocol for transferring CEE event records. A syntax encoding MAY define or suggest compatible protocols, but SHOULD NOT mandate the use of any specific protocol.

BACK TO TOP

JSON Encoding

The JavaScript Object Notation (JSON) format is a text-based data exchange format and is used by a larger number of devices and web services. The simplicity, readability, and efficiency of JSON makes it ideal for encoding event record data. This document describes a method for encoding a CEE Event into a JSON event record.

The use of the CEE JSON Encoding should be indicated by the use of the MIME or Internet media type: application/cee+json

Requirements

A JSON-encoded event record MUST be represented as a sequence of valid UTF-8 encoded Unicode characters. NUL characters (ABNF %x00) MUST NOT appear, even in an escaped format (i.e., \u0000).

Reserved Characters

There are several Unicode characters that MUST only appear in an escaped form and MUST NOT appear in their native form within a JSON-encoded value. This set of disallowed string characters include the quote, backslash ('', ABNF %x5C), and ASCII control (ABNF %x00-%x1F) characters. In CEE, the line feed (ABNF %x0A) and carriage return (ABNF %x0D) SHOULD also be escaped.

Other JSON reserved characters, such as the slash ('/', ABNF %x5C) character, MAY be escaped. JSON uses a leading slash character for escaping. Other characters MAY appear in an escaped form. A list of JSON encoding escape sequences is provided in the following table:

JSON Event Encoding

A CEE Event is encoded into a JSON Object according to the following ABNF definition:

CLS-JSON    = obj-s %x22 "Event" %x22 n-sep ( EVENT / EVENTLIST ) obj-e
EVENTLIST   = arr-s EVENT *( nv-sep EVENT ) arr-e
EVENT       = obj-s EVENT-HDR [EVENT-TYPE] [EVENT-PROFILE] [MODULES] obj-e

EVENT-HDR   = EVENT-FIELDS
EVENT-TYPE  = nv-sep %x22 "Type" %x22 n-sep obj-s EVENT-FIELDS obj-e
EVENT-PROF  = nv-sep %x22 "Profile" %x22 n-sep obj-s PROFILE obj-e
PROFILE     = %x22 PROFILE-NAME %x22 n-sep obj-s EVENT-FIELDS obj-e
PROFILE-NAME= CEE-NAME

EVENT-FIELDS= EVENT-FIELD *( nv-sep EVENT-FIELD )
EVENT-FIELD = %x22 FIELD-NAME %x22 n-sep FIELD-VALUE
FIELD-NAME  = CEE-NAME
FIELD-VALUE = JSON-DATATYPE / arr-s [JSON-DATA-ARRAY] arr-e

; CEE Event Modules
MODULES     = %x22 "Module" %x22 n-sep arr-s MODULE *( nv-sep MODULE ) arr-e
MODULE      = obj-s %x22 EXT-TYPE %x22 n-sep EXT-DATA obj-e
EXT-TYPE    = "Augment"
EXT-DATA    = EVENT-DATA

; JSON structures
nv-sep = ws "," ws  ; JSON name-value separator
n-sep  = ws ":" ws  ; JSON name separator
arr-s  = ws "[" ws  ; JSON array begin
arr-e  = ws "]" ws  ; JSON array end
obj-s  = ws "{" ws  ; JSON object begin
obj-e  = ws "}" ws  ; JSON object end
ws     = *( %x20 / %x09 / %x0A / %x0D )  ; Space, HTab, CR, or LF

JSON-FIELD       = %x22 FIELD-NAME %x22 n-sep JSON-FIELD-VALUE

JSON-DATA-ARRAY  = JSON-DATATYPE *254( nv-sep JSON-DATATYPE )
JSON-DATATYPE    = JSON-TYPE / JSON-NIL

; string values cannot contain unescaped " \
; or ASCII control characters (%x00-1F)
JSON-TYPE        = json-boolean / json-integer / json-string
JSON-NIL         = arr-s arr-e  ; []

json-string      = %x22 string  %x22
json-boolean     =      boolean
json-number      =      integer / float

; From the CEE Infoset and CLS Encoding Specification:
CEE-NAME   = CEE-SCHAR *31CEE-CHAR
CEE-SCHAR  = ALPHA / %x5F
CEE-CHAR   = CEE-SCHAR / DIGIT

NONZERO-DIGIT = %x31-39
DIGIT = %x30 / NONZERO-DIGIT

In order to reduce the encoding size and overhead, a JSON event record SHOULD be encoded compactly by removing all optional whitespace characters: space, tab, carriage return, or line feeds, as permitted in RFC 4648.

Mapping JSON to CEE Field Types

JSON offers native support for four value types: string, integer, float, and boolean.

The JSON number and boolean types map directly to CEE Field Types, giving the following conditions:

• JSON null literal is not supported
• JSON number values with a fraction or exponential component map to CEE float values; all other number values map to CEE integer
• Integer values MUST be representable within a 64-bit integer, in the range [-(2^63), 2^63-1] signed or [0, 2^64-1] unsigned
• Float value MUST be representable with an IEEE 754 64-bit floating point format.

For maximal compatibility, integer, float, and boolean CEE Field Types SHOULD be encoded using the equivalent, built-in JSON types. Values corresponding to these native types SHOULD NOT appear within quote characters.

All JSON string values may be mapped to the base string CEE Field Type or one of its derivations.

Examples

Example 1 - Valid

The JSON Event Record corresponding to the most minimal CEE-compliant event.

{"Event":{"p_proc":"auth","p_sys":"system.example.com","time":"2011-12-20T12:38:05.123456-05:00"}}

Example 2 - Valid

A CEE Event generated using only the base CEE Event structure defined in cee.xsd.

{"Event":{"crit":123,"id":"abc","p_app":"application","p_proc":"auth","p_proc_id":123,"p_sys":"system.example.com","pri":10,"time":"2011-12-20T12:38:05.123456-05:00","Type":{"action":"login","domain":"app","object":"account","service":"web","status":"success"}},"Module":[{"Augment":{"p_proc":"my_process","p_proc_id":123,"p_sys":"syslog-relay.example.com","time":"2011-12-20T17:41:12Z"}},{"Augment":{"p_proc":"my_process","p_proc_id":123,"p_sys":"syslog-relay.example.com","time":"2011-12-20T17:41:12Z"}}]}

Example 3 - Valid

A CEE JSON Record that contains a Type block as well as a custom profile.

{"Event":{"p_proc":"proc1","p_sys":"host.vendor.com","time":"2012-01-18T05:55:12.4321-05:00","Type":{"action":"login","status":"ongoing","tags":":hipaa"},"Profile":{"CustomProfile":{"schema":"http://vendor.com/events/cee-profile.xsd","new_field":"a string value","new_val":1234,"product_host":"source.example.com"}}}}

Example 4 - Valid

An example of a custom profile being used within the Augment module of a CEE JSON Event Record.

{"Event":{"p_proc":"proc1","p_sys":"host.domain.com","time":"2012-01-21T13:00:01Z"},"Module":[{"Augment":{"p_proc":"aug_proc1","p_sys":"relay.domain.com","time":"2012-01-21T13:02:17.0125Z","Profile":{"VendorDictProfile":{"schema":"http://vendor-domain.com/cee-profiles/default.xsd","schema_ver":1.01,"dst_ip":"10.10.0.2","src_ip":"127.0.0.1"}}}}]}

BACK TO TOP

XML Encoding

The W3C Extensible Markup Language (XML) 1.0 format is a text-based data exchange format and is used by a larger number of devices and web services. The simplicity, readability, and efficiency of XML makes it ideal for encoding event record data. This document describes a method for encoding a CEE Event into an XML event record.

The use of the CEE XML Encoding should be indicated by the use of the MIME or Internet media type: application/cee+xml

Requirements

Each XML-encoded CEE event record MUST be encoded in a format that conforms to the XML specification and SHOULD validate against the official CEE XML Schema. NUL characters (ABNF %x00) MUST NOT appear, even in an escaped format (i.e., �).

Each CEE XML event record MUST be encoded using UTF-8. The CEE XML MAY include an XML header. If the XML header is present, the encoding attribute MUST be set to UTF-8. All insignificant whitespace occurring in and around XML elements and other structures MUST be ignored.

Each CEE XML record MUST NOT include any XML namespace declarations. As all of the CEE Event definitions MUST be part of the same namespace (http://cee.mitre.org), the specification of this namespace in the XML event record is unnecessary and redundant.

CEE XML event records are encoded using "Simple XML". That is the XML records should contain only elements. The XML records MUST NOT include any XML attributes, Processing Instructions (PI), or Document Type Definitions (DTD). CEE field values MAY be enclosed within an XML CDATA section (e.g., <![CDATA[field <element/> value]]>).

Reserved Characters

There are several characters that MUST only appear in an escaped form and MUST NOT appear in their native form within an XML-encoded field value. This set of disallowed string characters include the less-than sign (<, ABNF %x3C), ampersand (&, ABNF %x26), and ASCII control (ABNF %x01-%x1F) characters. The greater-than sign (>, ABNF %x3E) MAY be encoded, but is not mandatory.

The W3C XML specification defines two valid escape sequences: a character entity and a numeric character reference. Each escape sequence MUST begin when an ampersand and end with a semicolon (;, ABNF %x3B). Valid character entity references are defined in the XML specification, but only for a few of the more commonly escaped characters; not every XML reserved character has a character entity reference.

However, the more generic, numeric reference character reference can be used to represent any character except illegal characters. The numeric reference escape sequences MUST begin with an ampersand; followed by a pound sign (#, ABNF %x23); the Unicode character value as a decimal, or an x (ABNF %x78) and a hexadecimal value; and a final semicolon. A list of valid XML escape sequences for reserved XML characters is provided below. Additional valid XML escape sequences are defined in the XML specification.

XML Infoset Encoding

A CEE Event is encoded into an XML event record according to the following ABNF definition:

CLS-XML     = ele-s "CEE" ele-e *EVENT ele-se "CEE" ele-e
EVENT       = ele-s "Event" ele-e EVENT-DATA ele-se "Event" ele-e

EVENT-DATA  = EVENT-HDR [EVENT-TYPE] [EVENT-PROFILE] [MODULES]
EVENT-HDR   = EVENT-FIELDS
EVENT-TYPE  = ele-s "Type" ele-e EVENT-FIELDS ele-se "Type" ele-e
EVENT-PROF  = ele-s "Profile" ele-e PROFILE ele-se "Profile" ele-e
PROFILE     = ele-s PROFILE-NAME ele-e EVENT-FIELDS ele-se PROFILE-NAME ele-e
PROFILE-NAME= CEE-NAME

EVENT-FIELDS= EVENT-FIELD *( EVENT-FIELD )
EVENT-FIELD = ele-s FIELD-NAME ele-e FIELD-VALUE ele-se FIELD-NAME ele-e
FIELD-NAME  = CEE-NAME
FIELD-VALUE = string

; CEE Event Modules
MODULES     = MODULE *( MODULE )
PLUGIN      = ele-s "Module" ele-e  EXT-TYPE ele-se "Module" ele-e
EXT-TYPE    = EXT-AUGMENT
EXT-AUGMENT = ele-s "Augment" ele-e EXT-DATA ele-se "Augment" ele-e
EXT-DATA    = EVENT-DATA

; XML structures
ele-s  = ws "<" ws  ; XML element start
ele-se = ws "</" ws  ; XML element start
ele-e  = ws ">" ws  ; XML element end

; From the CEE Infoset and CLS Encoding Specification:
CEE-NAME   = CEE-SCHAR *31CEE-CHAR
CEE-SCHAR  = ALPHA / %x5F
CEE-CHAR   = CEE-SCHAR / DIGIT

NONZERO-DIGIT = %x31-39
DIGIT = %x30 / NONZERO-DIGIT

Additionally, CEE field values SHOULD be encoded with no leading or trailing whitespace. Whitespace characters in XML encodings are defined as a space ( , ABNF %x20), horizontal tab (ABNF %x09), line feed (ABNF %x0A), or carriage return (ABNF %x0D).

When processing CEE XML field values, leading and trailing whitespace SHOULD be ignored. Any whitespace characters appearing within the middle of a value MUST NOT be collapsed and MUST be handled exactly as provided by the value.

Any XML record SHOULD successfully validate against the CEE Event Schema (cee.xsd) when combined with the Event Profile schema specified in the Profile's schema field. If specified, the schema version, schema-ver, may need to be taken into account.

Mapping XML to CEE Value Types

All XML values are contained within XML elements as character sequences.

Based on the content of the XML value, the CEE XML Event Schema, and any corresponding Event Profile schemas, it is up to the application to choose the best mapping for their needs. All CEE applications MUST at least map an XML-encoded field value to the xs:string field type.

Examples

Example 1 - Valid

The XML Event Record corresponding to the most minimal CEE-compliant event.

<CEE>
  <Event>
    <p_proc>auth</p_proc>
    <p_sys>system.example.com</p_sys>
    <time>2011-12-20T12:38:05.123456-05:00</time>
  </Event>
</CEE>

Example 2 - Valid

A CEE Event generated using only the base CEE Event structure defined in cee.xsd.

<?xml version="1.0" encoding="UTF-8"?>
<CEE>
  <Event>
    <crit>123</crit>
    <id>abc</id>
    <p_app>application</p_app>
    <p_proc>auth</p_proc>
    <p_proc_id>123</p_proc_id>
    <p_sys>system.example.com</p_sys>
    <pri>10</pri>
    <time>2011-12-20T12:38:05.123456-05:00</time>
    <Type>
      <action>login</action>
      <domain>app</domain>
      <object>account</object>
      <service>web</service>
      <status>success</status>
    </Type>
  </Event>
  <Module>
    <Augment>
      <p_proc>my_proc</p_proc>
      <p_proc_id>123</p_proc_id>
      <p_sys>syslog-relay.example.com</p_sys>
      <time>2011-12-20T17:41:12Z</time>
    </Augment>
  </Module>
  <Module>
    <Augment>
      <p_proc>my_process</p_proc>
      <p_proc_id>123</p_proc_id>
      <p_sys>syslog-relay.example.com</p_sys>
      <time>2011-12-20T17:41:12Z</time>
    </Augment>
  </Module>
</CEE>

Example 3 - Valid

A CEE XML Record that contains a Type block as well as a custom profile.

<CEE>
  <Event>
    <p_proc>proc1</p_proc>
    <p_sys>host.vendor.com</p_sys>
    <time>2012-01-18T05:55:12.4321-05:00</time>
    <Type>
      <action>login</action>
      <status>ongoing</status>
      <tags>:hipaa</tags>
    </Type>
    <Profile>
      <CustomProfile>
        <schema>http://vendor.com/events/cee-profile.xsd</schema>
        <new_field>a string value</new_field>
        <new_val>1234</new_val>
        <product_host>source.example.com</product_host>
      </CustomProfile>
    </Profile>
  </Event>
</CEE>

Example 4 - Valid

An example of a custom profile being used within the Augment module of a CEE XML Event Record.

<CEE>
  <Event>
    <p_proc>proc1</p_proc>
    <p_sys>host.domain.com</p_sys>
    <time>2012-01-21T13:00:01Z</time>
  </Event>
  <Module>
    <Augment>
      <p_proc>aug_proc1</p_proc>
      <p_sys>relay.domain.com</p_sys>
      <time>2012-01-21T13:02:17.0125Z</time>
      <Profile>
        <VendorDictProfile>
          <schema>http://vendor-domain.com/cee-profiles/default.xsd</schema>
          <schema_ver>1.01</schema_ver>
          <dst_ip>10.10.0.2</dst_ip>
          <src_ip>127.0.0.1</src_ip>
        </VendorDictProfile>
      </Profile>
    </Augment>
  </Module>
</CEE>

BACK TO TOP

Page Last Updated: August 10, 2012