File Descriptor
Collect logs from a file descriptor
Configuration
Example configurations
{
"sources": {
"my_source_id": {
"type": "file_descriptor",
"fd": 10
}
}
}
[sources.my_source_id]
type = "file_descriptor"
fd = 10
---
sources:
my_source_id:
type: file_descriptor
fd: 10
{
"sources": {
"my_source_id": {
"type": "file_descriptor",
"decoding": null,
"fd": 10,
"framing": null,
"host_key": null,
"max_length": 102400
}
}
}
[sources.my_source_id]
type = "file_descriptor"
fd = 10
max_length = 102_400
---
sources:
my_source_id:
type: file_descriptor
decoding: null
fd: 10
framing: null
host_key: null
max_length: 102400
decoding
optional objectdecoding.codec
optional string literal enumOption | Description |
---|---|
bytes | Uses the raw bytes as-is. |
gelf | Decodes the raw bytes as a GELF message. |
json | Decodes the raw bytes as JSON. |
native | Decodes the raw bytes as Vector’s native Protocol Buffers format. This codec is experimental. |
native_json | Decodes the raw bytes as Vector’s native JSON format. This codec is experimental. |
syslog | Decodes the raw bytes as a Syslog message. Will decode either as the RFC 3164-style format (“old” style) or the more modern RFC 5424-style format (“new” style, includes structured data). |
bytes
framing
optional objectFraming configuration.
Framing deals with how events are separated when encoded in a raw byte form, where each event is a “frame” that must be prefixed, or delimited, in a way that marks where an event begins and ends within the byte stream.
framing.character_delimited
required objectmethod = "character_delimited"
framing.character_delimited.delimiter
required uintframing.character_delimited.max_length
optional uintThe maximum length of the byte buffer.
This length does not include the trailing delimiter.
framing.method
required string literal enumOption | Description |
---|---|
bytes | Byte frames are passed through as-is according to the underlying I/O boundaries (e.g. split between messages or stream segments). |
character_delimited | Byte frames which are delimited by a chosen character. |
length_delimited | Byte frames which are prefixed by an unsigned big-endian 32-bit integer indicating the length. |
newline_delimited | Byte frames which are delimited by a newline character. |
octet_counting | Byte frames according to the octet counting format. |
framing.newline_delimited
optional objectmethod = "newline_delimited"
framing.newline_delimited.max_length
optional uintThe maximum length of the byte buffer.
This length does not include the trailing delimiter.
framing.octet_counting
optional objectmethod = "octet_counting"
framing.octet_counting.max_length
optional uinthost_key
optional string literalOverrides the name of the log field used to add the current hostname to each event.
By default, the global host_key
option is used.
max_length
optional uintThe maximum buffer size, in bytes, of incoming messages.
Messages larger than this are truncated.
102400
(bytes)Outputs
<component_id>
Output Data
Logs
Line
gethostname
command.my-host.local
2019-02-13T19:48:34+00:00 [info] Started GET "/" for 127.0.0.1
2020-10-10T17:07:36.452332Z
Telemetry
Metrics
linkcomponent_discarded_events_total
countercomponent_id
instead. The value is the same as component_id
.component_errors_total
countercomponent_id
instead. The value is the same as component_id
.component_received_bytes_total
countercomponent_id
instead. The value is the same as component_id
.component_received_event_bytes_total
countercomponent_id
instead. The value is the same as component_id
.component_received_events_total
countercomponent_id
instead. The value is the same as component_id
.component_sent_event_bytes_total
countercomponent_id
instead. The value is the same as component_id
.component_sent_events_total
countercomponent_id
instead. The value is the same as component_id
.events_in_total
countercomponent_received_events_total
instead.component_id
instead. The value is the same as component_id
.events_out_total
countercomponent_sent_events_total
instead.component_id
instead. The value is the same as component_id
.processed_bytes_total
countercomponent_id
instead. The value is the same as component_id
.processed_events_total
countercomponent_received_events_total
and
component_sent_events_total
metrics.component_id
instead. The value is the same as component_id
.source_lag_time_seconds
histogramcomponent_id
instead. The value is the same as component_id
.Examples
Line sent over pipe
Given this event...2019-02-13T19:48:34+00:00 [info] Started GET "/" for 127.0.0.1
[sources.my_source_id]
type = "file_descriptor"
fd = 10
---
sources:
my_source_id:
type: file_descriptor
fd: 10
{
"sources": {
"my_source_id": {
"type": "file_descriptor",
"fd": 10
}
}
}
{
"host": "my-host.local",
"message": "2019-02-13T19:48:34+00:00 [info] Started GET \"/\" for 127.0.0.1",
"timestamp": "2020-10-10T17:07:36.452332Z"
}