Packetbeat Index Template Error

[waltmanwk]

Packetbeat v7.15.2 is installed on a server and is shipping data to NLS v2.1.9 via Logstash. The only issue I seem to be having is getting the Packetbeat Index Template to load correctly in ES. The following curl command is attempting to load the index template with the template file "packetbeat.template.json" - which I've attached in case that is useful (had to add .txt ending to allow attachment). Seems like the more modern Packetbeat index template is not compatible with the ES version running under the hood of NLS v2.1.9.

Code: Select all

[user@nls1 ~]$ curl -XPUT -H 'Content-Type: application/json' http://localhost:9200/_template/packetbeat-7.15.2?pretty=true -d@packetbeat.template.json
{
  "error" : "ElasticsearchIllegalArgumentException[Malformed mappings section for type [date_detection], should include an inner object describing the mapping]",
  "status" : 400
}

[waltmanwk]

The original error about Malformed mappings went away after encapsulating everything within "mappings" inside another field, in my case I called it "packetbeat_doc" :

Code: Select all

  "mappings": {
    "packetbeat_doc" : {
    "_meta": {
      "beat": "packetbeat",
      "version": "7.15.2"

Then trying to load the index template again, got another error message:

Code: Select all

[user@nls1 ~]$ curl -XPUT -H 'Content-Type: application/json' http://localhost:9200/_template/packetbeat-7.15.2?pretty=true -d@packetbeat.template.json
{
  "error" : "ActionRequestValidationException[Validation Failed: 1: template is missing;]",
  "status" : 400
}

The new error was resolved by defining the template field at the top, in my case: "template" : "packetbeat-*"

Code: Select all

{
  "template" : "packetbeat-*",
  "index_patterns": [
    "packetbeat-7.15.2-*"
  ],
  "mappings": {
    "packetbeat_doc" : {
    "_meta": {

After that change, then the index template seemed to load just fine:

Code: Select all

[user@nls1 ~]$ curl -XPUT -H 'Content-Type: application/json' http://localhost:9200/_template/packetbeat-7.15.2?pretty=true -d@packetbeat.template.json
{
  "acknowledged" : true
}

Now the issue appears to be setting an Output correctly to get the Packetbeat data to create indices correctly when going into ES. I've tried the following as an Output under Global Config > Outputs:

Code: Select all

elasticsearch {
    hosts => ['localhost']
    index => '%{[@metadata][beat]}-%{[@metadata][version]}'
}

Now there is an index named appropriately:

Code: Select all

[user@nls1 ~]$ curl -XGET localhost:9200/_cat/indices/packet*
green open packetbeat-7.15.2 5 1 108391 0 157.6mb 76.9mb 

However, I'm unable to figure out how to access that data via Dashboards (only the usual Logstash indices appear viewable). Also, it appears that the Packetbeat data is being duplicated into the usual Logstash indices. Perhaps I need to prepend the Output with an if statement so only the Packetbeat data is processed by that Output and is not also processed by the built-in Output for NLS?

[cdienger]

The dashboard will display data from the indexes matching the pattern found under Dashboard Settings > Index. Dashboard settings can be found by clicking the cog icon in on the dashboard menu.

NLS is hard coded to send data to the default logstash-* indexes. If the packetbeat data has a field that is unique to it, you could periodically remove it from the logstash indexes. For example, if it has a field called 'type' with a value of 'packetbeat':

Code: Select all

curl -XDELETE 'http://localhost:9200/logstash-*/_query?q=type:packetbeat'

[waltmanwk]

Thanks for the Dashboard index setting, that was super helpful. The XDELETE command also works well to clean up the packetbeat data from the logstash indexes; I'll have to cronjob that.

I can't seem to get the right Output syntax to get the indexes named properly. If I use this:

Code: Select all

elasticsearch {
    hosts => ["http://localhost:9200"]
    index => "%{[@metadata][beat]}-%{[@metadata][version]}-%{+YYYY.MM.dd}"
}

Then the index is created as this, notice the date seems to work but not the metadata fields. This worked once before so I'm not sure what's wrong now.

Code: Select all

%{[@metadata][beat]}-%{[@metadata][version]}-2021.11.24

But if I use this Output syntax, no index even gets created:

Code: Select all

elasticsearch {
    hosts => ["http://localhost:9200"]
    index => "packetbeat-%{+YYYY.MM.dd}"
}

Or if I use this Output syntax, then I get just an index just called "packetbeat" , but it appears to be matching all data and not just packetbeat data like what should happen with the metadata fields in the first example.

Code: Select all

elasticsearch {
    hosts => ["http://localhost:9200"]
    index => "packetbeat"
}

What is the syntax used for the hard coded logstash-* indexes? Maybe I can use that to realize where the issue lies.

[cdienger]

Try this variation:

Code: Select all

elasticsearch {
    hosts => ["http://localhost:9200"]
    index => "%{[@metadata][beat]}-%{+YYYY.MM.dd}"
}

which should be similar to what I tested with here:

Code: Select all

elasticsearch {
    hosts => ["http://localhost:9200"]
    index => "packetbeat-%{+YYYY.MM.dd}"
}

and it worked and similar to the hardcoded logstash:

Code: Select all

    elasticsearch {
        hosts => ['localhost']
        document_type => '%{type}'
        workers => 4
    }

Are you seeing anything logged in /var/log/logstash/logstash.log when you use this format? Is the packetbeat data getting dropped altogether when with this format or is it going into the logstash indexes? When you save the changes, make sure the output definition is written to /usr/local/nagioslogserver/logstash/etc/conf.d/999_outputs.conf:

Code: Select all

cat /usr/local/nagioslogserver/logstash/etc/conf.d/999_outputs.conf

[cdienger]

Also, you can use an if statement around the output definition to help direct data to the proper index:

Code: Select all

if [type] == "packetbeat"{
	elasticsearch {
		hosts => ["http://localhost:9200"]
		index => "packetbeat"
	}
}