Customizing Elasticsearch indices using Fluent-Bit in TKG

Fluent-Bit is currently the preferred option for log shipping in TKG and is provided out of the box as a Tanzu package that can be easily deployed on each TKG/Kubernetes cluster.

A recent implementation required shipping all Kubernetes logs to Elasticsearch, complying with a specific naming convention for the Elasticsearch indices.

Applying such customizations requires you to utilize the Lua filter. Using the Lua filter, you can modify incoming records by invoking custom scripts to apply your logic when processing the records.