Integrate applications into LEAF ELK stack

Architecture
1

Hi,

We have bunch of functional services and applications written in C# which are hosted in Kubernetes within the LEAF environment and uses LEAF data services to store data.

I was looking into integrating our logs into the LEAF ELK stack. This would put all logs at one place when we run the demo profile.

I was just wondering if we have any documentation on best practices or how to write logs into LEAF ELK stack.

I know from C# we can write logs directly to Logstash easily using Logstash HTTP plugin but wanted to see how LEAF services do it so I can probably follow same steps for consistency.

Any starting points would help.

Many Thanks

2

Hey Shravan!
We actually use a variation of the ELK stack… EFK where FluentD/Fluentbit are used in place of Logstash. Fluentbit is a logshipper that sits on each node in your Kubernetes cluster and ships all container logs through FluentD and Elastic Search. I believe there actually doesn’t end up being any configuration for your LEAF service other than how and what it logs. Fluentbit simply picks it all up and sends it to FluentD where it’s aggregated. As far as how to deploy a quick EFK stack you have a couple of options:

  1. Deploy and configure our helm chart yourself
  2. Take inspiration from the Reference Implementation’s Skaffold demo profile
  3. Use LEAFctl to quickly deploy a local EFK stack
leafctl deploy efk

We also have this guide in our Platform labs

Some Caveats:
This chart was built some time ago based on a lack of good options in the open source community. It’s very likely that a more well maintained version now exists and it is on our roadmap to investigate potential alternatives or revisit our own log aggregation solution.

3

Hi Josh,

Thank you for the details, I will spend some time over next few weeks and try setting this up. Will get back if I have any issues.