log4j2 + logstash - 个人文章 - SegmentFault 思否

安宇雨 - 随手采集
2019-04-23 16:58:43
操作系统
Elasticsearch 专题-elasticsearch-logstash-kibana 0001-整理部分 - 研究ing

log4j2

log4j2 把日志发送到logstash用好几种方式，一个是通过socket appender，这种方式有个缺点是断掉之后不会自动重连；第二种方式就是使用logstash-gelf

<dependency>
    <groupId>biz.paluch.logging</groupId>
    <artifactId>logstash-gelf</artifactId>
    <version>1.11.1</version>
</dependency>

<!-- 添加 log4j2 to elk -->
<dependency>
    <groupId>biz.paluch.logging</groupId>
    <artifactId>logstash-gelf</artifactId>
    <version>1.12.0</version>
</dependency>

log4j2 配置

    <Gelf name="gelf" host="udp:192.168.77.205" port="12201" version="1.1" extractStackTrace="true"
          filterStackTrace="true" mdcProfiling="true" includeFullMdc="true" maximumMessageSize="8192"
          originHost="%host{fqdn}" additionalFieldTypes="fieldName1=String,fieldName2=Double,fieldName3=Long">
      <Field name="timestamp" pattern="%d{dd MMM yyyy HH:mm:ss,SSS}" />
      <Field name="level" pattern="%level" />
      <Field name="simpleClassName" pattern="%C{1}" />
      <Field name="className" pattern="%C" />
      <Field name="server" pattern="%host" />
      <Field name="server.fqdn" pattern="%host{fqdn}" />

      <!-- This is a static field -->
      <Field name="fieldName2" literal="fieldValue2" />

      <!-- This is a field using MDC -->
      <Field name="mdcField2" mdc="mdcField2" />
      <DynamicMdcFields regex="mdc.*" />
      <DynamicMdcFields regex="(mdc|MDC)fields" />
    </Gelf>

我

    <Gelf name="logstash-gelf" host="udp:120.79.80.138" port="4560" version="1.1">
            <Field name="timestamp" pattern="%d{yyyy-MM-dd HH:mm:ss.SSS}"/>
            <Field name="logger" pattern="%logger"/>
            <Field name="level" pattern="%level"/>
            <Field name="simpleClassName" pattern="%C{1}" />
            <Field name="className" pattern="%C"/>
            <Field name="server" pattern="%host"/>
            <!--<Field name="server.fqdn" pattern="%host{fqdn}" />-->

            <!-- This is a static field -->
            <Field name="serviceId" literal="superdesk-app-services-ordermeal-dev" />

            <!-- This is a field using MDC -->
            <Field name="mdcField2" mdc="mdcField2" />

            <Field name="method" pattern="%M"/>
            <Field name="line" pattern="%L"/>
            <DynamicMdcFields regex="mdc.*" />
            <DynamicMdcFields regex="(mdc|MDC)fields" />

        </Gelf>

logstash

配置如下

需要去掉server字段，不然导入elasticsearch 会有问题。
用json格式解开message
es的索引名称可以使用变量。

input {
  gelf {
    port => 12201
  }
}
filter {
  json {
   source => "message"
   add_field => ["type", "%{dtype}"]
   remove_field => [ "server", "server.fqdn", "timestamp" ]
  }
}
output {
  stdout { codec => rubydebug }
  if "_jsonparsefailure" not in [tags] {
  elasticsearch {
    hosts => ["localhost:9200"]
    index => "logstash-%{dtype}-%{action}"
    } 
  }
}

Original url: Access
Created at: 2019-04-23 16:58:43
Category: default
Tags: none

未标明原创文章均为采集，版权归作者所有，转载无需和我联系，请注明原出处，南摩阿彌陀佛，知识，不只知道，要得到

请先后发表评论