Introduction

The CEE Log Transport (CLT) provides the technical support necessary for a secure, interoperable, and reliable log infrastructure. A log infrastructure requires more than just standardized event records, support is needed for international string encodings, secure logging services, standardized event interfaces, and secure, verifiable log trails.

The CLT defines a listing of requirements that a CLT Protocol must meet. For example, a CLT Protocol must be able to transmit a Common Log Syntax (CLS) encoded CEE Event. More advanced CLT Protocols may provide things like encryption and full acknowledgments. A CLT Protocol may be able to specify or transmit CEE Profiles and additional event-related information, such as packet captures or file data.

CLT also defines transport mappings. A CLT Mapping defines a standardized way for CEE Events to be transmitted over a certain CLT Protocol. One use for a CLT Mapping is to define how to send CEE Events over the RFC5425 TLS Syslog protocol. This Mapping would define that the CEE Event must be encoded using an RFC5424 Syslog-compatible CLS Encoding and placed at a certain point in the Syslog message. The CLT Mapping may need to define additional indicators, such as flags to indicate that the data is an encoded CEE Event and the character encoding used (e.g., UTF-8).

The CLT provides the features necessary to support the end-to-end audit process by extending the event record representation to include the essential confidentiality, integrity, and availability of audit services.

BACK TO TOP

CLT Protocol

The CLT Protocol consists of a session of at least one event channel. Each channel consists of zero or more protocol packets. Each packet has a packet header and body. The body contains the event data within zero or more CLS-encoded CEE Event Records.

The CLT Protocol session is a full-duplex pipe where the computer at either end is capable of initiating a message exchange. A CLT Protocol session carries one or more channels simultaneously. A channel is a stream of "typed" messages.

The actual CEE events are sent from the Event Sender and are received by the Event Receiver. Each session communication may be sent directly from the Sender to the Receiver (Direct Transmission) or transmitted by way of one or more session-forwarding Relays (Multi-Hop Transmission).

BACK TO TOP

CLT Protocol Requirements

Many uses for events records require them to be transported from the originating system. CLT Protocols must meet some basic requirements in order to do this reliably and efficiently. This requirements document categorizes requirements in four groups based on conformance levels. Conformance Level 0 is the mandatory conformance level and includes basic capability. Further conformance levels describe optional capabilities that more advanced CLT protocols should support. Conformance Level 1 is core capability providing the minimum set of requirements for robustness. Conformance Level 2 contains a set of additional requirements that address logging in the presence of attackers. Conformance Level 3 contains an additional set of requirements that address local administrative attack scenarios. Conformance Level 3 is the most robust requirements set.

Conformance Level 0 - Core Requirements

0.1 Publish

The CLT Protocol shall be a published protocol specification with no licensing barriers to interoperability, no royalties and no approval process. The CLT Protocol shall utilize only those protocols and standards which are openly available.

0.2 Transport

The CLT Protocol shall be able to transport at least one form of CEE encoded event records within the body of the protocol packet.

0.3 Self-Identification

The CLT Protocol shall make evident by the protocol as to which data belongs to CEE event records and the CEE defined encoding used when a transmission is made consisting of CEE and non-CEE data.

0.3.1 Identification of CEE Events

The CLT Protocol shall have a mechanism that identifies the CEE Events within the protocol packet body.

0.3.2 Encoding Identifier

A CLT Protocol capable of handling more than one (1) data encoding method shall provide evidence as to the encoding being used.

0.4 Timestamp

Each protocol packet shall have timestamp that indicates the date and time the packet was transmitted. A valid timestamp must indicate year, month, day, hour, minute, and second. Including sub-seconds is encouraged.

Conformance Level 1 - Basic Capabilities

One of the core capabilities is for the log transport to perform in limited bandwidth environments. Additional core capabilities include log transport tamper detection.

1.1 Event Record Delivery

The CLT Protocol shall preserve the integrity of the logical order of a channel's packets such that the Event Receiver will be able to reconstitute the original logical order.

1.2 Compression of Records

The CLT Protocol shall provide a method that allows the Event Sender to compress the packet body prior to transmission.

1.3 Missing Record Detection

The CLT Protocol shall be able to accurately and reliably detect missing transport packets.

1.4 Message Identification

The CLT Protocol shall support message identifiers.

1.4.1 Packet Duplication

The CLT Protocol shall be able to identify when duplicate packets have been received.

1.4.2 Packet Acknowledgement

The CLT Protocol shall support at least one method of allowing the Event Receiver to acknowledge (ACK) that a packet has been received.

1.4.3 Packet Retransmission

The CLT Protocol shall be able to retransmit individual packets on request at least until the Event Receiver has acknowledged reception or until a nominal time has elapsed.

1.5 Packet Traversal Traceability

The CLT Protocol shall be capable of tracing and recording the path a packet traverses. The intent of this requirement is to allow packets to be traced through Session Relay devices such as NATs.

1.6 Tamper Detection

The CLT Protocol shall accurately and reliably detect any evidence of tampering or data corruption through the use of digital signatures or other anti-tamper mechanisms.

Conformance Level 2 - Securely Log in the Presence of Attackers

The core theme for the Conformance Level 2 transport requirements is the capability for CLT Protocol to securely log in the presence of attackers. Additional requirements include Confidentiality, Authenticity, and transmission Encryption.

2.1 Full Integrity Acknowledgements

The CLT Protocol shall be capable of producing packet level acknowledgements that contains data through which the sender can verify the integrity of a received packet. This requirement is an extension of that of Requirement 1.4.2: Packet Acknowledgement.

2.2 Message Replay Protection

The CLT Protocol shall protect against message replay.

2.3 Event Integrity

The CLT Protocol shall accurately and reliably detect any repeated or unexpected message identifiers.

2.3.1 Chain of Modification

The CLT Protocol shall maintain the chain of modification of CEE Event data that is modified while in transit.

2.3.2 Reproduction of Original Event

The CLT Protocol shall be able to reproduce by request, the original CEE Event that is modified while in transit.

2.4 Transmission Encryption

The CLT Protocol shall support transmission encryption using best practice encryption algorithms. One way to achieve this requirement is by using Transport Layer Security (TLS).

2.5 Confidentiality

The CLT Protocol shall maintain confidentiality of data, minimally within the packet body. The CLT Protocol cryptography modules shall be capable of performing data encryption using data encryption best practices.

2.6 Authenticity

The CLT Protocol shall maintain authenticity of the data in transit.

2.6.1 Use of SASL, GSS-API, and Kerberos

The CLT Protocol shall support authenticity using Simple Authentication and Security Layer (SASL), Generic Security Services Application Program Interface (GSS-API), and Kerberos.

Conformance Level 3 - Secure Against Local Administration Attacks

The core theme for the Conformance Level 3 transport requirements is the capability for CLT Protocol to securely log in the presence of local administration attacks.

3.1 Tamper Resistant

The CLT Protocol shall maintain integrity mechanisms resistant to tampering by local administrator (e.g. perfect forward secrecy).

3.2 Record Channels

The CLT Protocol shall be able to provide support for multiple channels within a session.

3.3 Profile Channels

A CLT Protocol channel shall be able to have CEE-specific metadata bound to it, such as a CEE Event Profile. This metadata can be used by the Event Sender/Receiver to exchange record format data or reduce duplicative data from being sent.

BACK TO TOP

CTL Implementation Requirements

Conformance Level 0 - Core Requirements

0.1 Support CLT Protocol Level 0

The implementation must support at least a Conformance Level 0 of the CLT Protocol.

Conformance Level 1 - Basic Requirements

1.1 Support CLT Protocol Level 1

The implementation must support at least a Conformance Level 1 of the CLT Protocol.

1.2 Sender-side Buffering

The CLT Protocol shall be capable of sender-side buffering, e.g. event record is retained in a recoverable fashion on sender until server indicates that event has been received.

1.2.1 Single Log Record Buffering

The client shall retain each log record until the server has acknowledged (ACKed) the reception of the message; ideally this ACK should include a hash or signature whereby the sender can validate the message was correctly received.

1.2.2 Batch Log Record Buffering

The client shall retain each batch of log records until the server has ACKed the reception of the batch message; ideally this ACK should include a hash or signature whereby the sender can validate the message was correctly received.

1.2.3 Enable and Disable Switch

The CLT Protocol shall have a switch that enables and disables send-side buffering.

1.3 Log in Limited Network Environments

The CLT Protocol shall be capable of reordering the event transmission queue so that the most important messages receive priority. Many environments utilize Network Address Translation (NAT). The CLT Protocol shall be capable of functioning correctly in that environment.

1.3.1 Retransmission Priority

The Priority of the event retransmission queue shall be settable by the application.

1.3.2 Network Address Translation (NAT)

The CLT Protocol shall be capable of communicating in a Network Address Translation (NAT) environment.

Conformance Level 2 - Log In The Presence of Attackers

2.1 Support CLT Protocol Level 2

The implementation must support at least a Conformance Level 2 of the CLT Protocol.

Conformance Level 3 - Secure Against Local Administration Attacks

3.1 Support CLT Protocol Level 3

The implementation must support at least a Conformance Level 3 of the CLT Protocol.

3.2 Secure against local administration attacks

The CLT Protocol shall, on average, transmit event records within 1 second of event record creation.

3.3 Event Source Channel Binding

Applications sending events over the CLT Protocol shall be able to bind the event records to a CLT Protocol channel.

3.4 Event Destination Channel Binding

Applications receiving events over a CLT Protocol channel shall be able to reconstruct the full event record based on the channel contents and previously exchanged channel metadata.

3.5 Channel Profiles

Applications shall be able to bind one or more event profiles to a registered channel.

3.6 Continuous Operation

The CLT Protocol shall be able to support load balancing and gracefully failover to backup servers when the primary is lost.

BACK TO TOP

Transport Mappings

Transport mappings specify standard interoperability mapping between CEE and popular log transmission protocols.

Mappings have been defined between CEE and the following protocol:

• Syslog

CEE over Syslog Mapping

Since Syslog is the de facto standard in log transport protocols and it is supported by numerous products, CEE provides a way to send CEE Event data over Syslog. This document defines a standard process to encode CEE Events using a CEE Log Syntax (CLS) Encoding and place the encoded event into a Syslog message.

Syslog

There exist several different formats for Syslog messages. Within this document, they will be separated into two groups: legacy and RFC5424. While the basic process for transmitting CEE Events is the same for both Syslog groups, there are some nuances surrounding the use of legacy Syslog of which implementers must be cognizant.

Legacy Syslog

Most people are familiar with legacy Syslog. Legacy Syslog is characterized by its timestamp format missing the year and the event message consisting only of an unstructured line of text. The various formats of legacy Syslog messages are identified in RFC3164.

Besides the variance in message formats, not all legacy Syslog implementations can handle 8-bit character sets. That is, some implementations use only the lower 7 bits of each byte. This causes problems when trying to send binary data or extended character sets (e.g., extended ASCII, UTF-8) that rely on "8-bit clean" processing.

RFC5424 Syslog

The Syslog specification was updated in March 2009 with the release of RFC5424. This update brought many needed enhancements, including a standardized timestamp that includes the year, Unicode support (UTF-8), and a way of providing more structured content within Syslog messages.

One of the lesser recognized improvements was to make the entire Syslog protocol and RFC5424 implementations 8-bit clean. This step was necessary to enable full Unicode support.

While RFC5424 Syslog is preferred over legacy versions, there are still many environments and platforms that have been built on top of legacy Syslog implementations. All Syslog implementations supporting CEE SHOULD be RFC5424-compatible.

CLT Syslog Mapping

It is possible to use Syslog to transport CEE Events. To do this, the CEE Event must be encoded using a CLS Encoding compatible with the Syslog implementation and placed into the body of a new Syslog message.

Each Syslog message MUST conform to the CEE CLS Specification as well as the appropriate Syslog specification, such as RFC3164 or RFC5424. This includes the complete Syslog header and content.

This document defines a CLT mapping that is conformant to both CEE and Syslog specifications via the inclusion of a CLS JSON (Javascript Object Notation) encoded event within a Syslog message. The resulting CLT Syslog message has the following ABNF (Augmented Backus-Naur Form) definition:

SYSLOG        = HEADER BODY
HEADER        = PRI [VERSION SP] TIMESTAMP SP HOSTNAME SP
BODY          = MSG [ CEE-EVENT ]

PRI           = "<" PRIVAL ">"
PRIVAL        = 1*3DIGIT ; range 0 .. 191
VERSION       = NONZERO-DIGIT 0*2DIGIT
MSG           = MSG-7BIT / MSG-8BIT
MSG-7BIT      = *( SP / PRINTUSASCII )
MSG-8BIT      = [BOM] UTF-8-STRING

CEE-EVENT     = CEE-FLAG JSON-RECORD
              ; CLS-JSON grammar defined in the
              ;  CLS JSON Encoding Specification
CEE-FLAG      = "cee:" [SP]  ; %x63.65.65.3A

UTF-8-STRING  = *OCTET  ; UTF-8 string, RFC 3629
; MUST NOT contain the NUL %x00 character
OCTET         = %x00-FF
PRINTUSASCII  = %x21-7E
ALPHANUM      = ALPHA / DIGIT
SP            = %x20
NONZERO-DIGIT = %x31-39
DIGIT         = %x30 / NONZERO-DIGIT
BOM           = %xEF.BB.BF

Syslog Header

The standard Syslog header MUST be used. The actual formatting of the Syslog header is dependent on the Syslog protocol version and may vary based on the implementation. Regardless, these header values are for the Syslog protocol and are independent of CEE. The Syslog header values SHOULD NOT be used to add or modify any values within an enclosed CEE Event.

Syslog Body

The CEE Event MUST be represented using a CLS Encoding. For compliance with this specification, the CEE Event MUST be represented using the CLS JSON Encoding.

In contrast to Section 2 of RFC4627, the JSON encoding suitable for Syslog transport MUST NOT contain insignificant whitespace before or after any of the six JSON structural characters.

CEE Event Flag

The beginning of the encoded CEE Event MUST be identified by the CEE Event Flag. Within Syslog, the CEE Event Flag is cee: (ABNF %x63.63.65.3A).

The CEE Event Flag MAY be followed by one space (' ', ABNF %x20) character.

A CLS JSON Encoded CEE Event MUST appear immediately following a CEE Event Flag. Other, non-CEE, non-JSON content MUST NOT appear in the Syslog body after a CEE Event Flag. A Syslog message MUST NOT contain more than one CEE Event Flag.

Character Encoding

All CLS Encodings, including the CLS JSON Encoding, assume an 8-bit clean environment. Therefore, it is important that the event producer understand a Syslog format that will be used, especially whether implementation is a 7-bit or 8-bit clean implementation.

If the Syslog implementation is 8-bit clean, then the implementation supports UTF-8 and no additional encodings are necessary beyond those required by the JSON CLS Specification. However, if the implementation is only 7-bit, then all characters not in the ASCII character set (ABNF %x00-7F) MUST be additionally escaped.

When using a 7-bit Syslog implementation or have requirements for 7-bit Syslog compatibility, then all UTF-8 encoded characters that cannot be represented using the lower 7 bits of an 8-bit byte MUST be escaped. These characters SHOULD be escaped using the JSON escape sequences RFC4627, especially the Unicode escape: \u followed by four (4) hexadecimal digits (ABNF %x5C.75 4HEXDIG).

Transmission

A Syslog message containing a CEE Event should be able to be transmitted using any Syslog-based transport mechanism. Due to the potential priority or sensitivity of certain event records, it is recommended that the transmission protocol supply the necessary confidentiality and integrity measures for the event content and operating environment.

The original Syslog protocol sent event records in plaintext over UDP. This does not provide any security controls. One option is to transmit the Syslog messages using Transport Layer Security, as specified in RFC5425.

Examples

Example 1 - Valid

A valid CEE JSON Event Record embedded within an RFC5424 Syslog transport:

<165>1 2011-12-20T12:38:06Z 10.10.0.1 process - example-event-1 cee:{"Event":{"p_proc":"auth","p_sys":"system.example.com","time":"2011-12-20T12:38:05.123456-05:00"}}

Example 2 - Valid

A valid CEE JSON Event Record with augment modules, used with a "legacy" Syslog transport:

<0>Dec 20 12:42:20 syslog-relay process[35]: cee:{"Event":{"crit":123,"id":"abc","p_app":"application","p_proc":"auth","p_proc_id":123,"p_sys":"system.example.com","pri":10,"time":"2011-12-20T12:38:05.123456-05:00","Type":{"action":"login","domain":"app","object":"account","service":"web","status":"success"}},"Module":[{"Augment":{"p_proc":"my_process","p_proc_id":123,"p_sys":"syslog-relay.example.com","time":"2011-12-20T17:41:12Z"}},{"Augment":{"p_proc":"my_process","p_proc_id":123,"p_sys":"syslog-relay.example.com","time":"2011-12-20T17:41:12Z"}}]}

BACK TO TOP

Page Last Updated: February 21, 2012