Add JSON parsing to Log View

[r-oldenburg]

Hey there,

I wanted to share this contribution to this amazing project.
First thing: I am not a native GO programmer. SORRY, if I did break anything or violated any best practices / code conventions or whatever. Feel free to give such feedback.

We just changed the logging in our k8s deployments to JSON in order to make use of structured logging. Reading those in k9s is a pain, so I searched for solutions. And only found plugin solutions that are not giving the same UX as the usual Log View.

The current log UX in k8s is amazing, so I started experimenting with "jq" calls from within k9s. And eventually I found "gojq" which now even allows running without a "jq" installation on the system.

The main idea is to extract 3 fields: datetime, log level, message

Features:

• Configure global expressions to be used in expressions (i.e. coloring by log level, padding, etc.)
• Configure templates (collections of jq expression for time / log-level / message)
• Toggle switch for JSON on/off ("J")
• Iterating through the available templates ("Ctrl+J")
• Editing of templates/their expressions while in Log View ("Shift+J")
  • this even with realtime compiling and instant error messages

Example Templates

... that I can provide (as follows):

• "General" (simply selecting typical fields)
• "Spring" / Logback (typical logback logging of Spring Boot Apps, even inserting arguments into placeholders)
• "Spring + Stacktrace" same as above but with traditional Stacktrace visualization
• "Mongo" Typical fields from Mongo DB logs

k9s:
  logger:
    tail: 7000
    buffer: 50000
    sinceSeconds: -1
    textWrap: false
    showTime: true
    decodeJson: true
    json:
      globalExpressions: |
        def lpad(n;p): . | tostring | if (n > length) then ((n - length) * (p|tostring)) + . else . end;
        def lpad(n): lpad(n;" "); 
        def rpad(n;p): . | tostring | if (n > length) then . + ((n - length) * (p|tostring)) else . end;
        def rpad(n): rpad(n;" ");
        def dateTimeFromTsNs($ts;$nanos): if $ts then ($ts/1000 | strflocaltime("%FT%T.")) + 
          (if $nanos then $nanos | lpad(9;0) else $ts%1000 | lpad(3;0) end) else "" end + "Z";
        def color($content;$logLvl): ({
          "ERROR": "\u001b[31m", 
          "E": "\u001b[31m", 
          "F": "\u001b[31m", 
          "WARN":"\u001b[33m", 
          "W":"\u001b[33m", 
          "INFO":"",
          "I":"",
          "DEBUG":"\u001b[34m", 
          "D1":"\u001b[34m",
          "TRACE":"\u361b[0m"
        }) as $lvlColor | $lvlColor.[$logLvl] + ($content | tostring) + "\u001b[0m";
        def insertSpringArgs(args): if (args | not) then . else reduce args[] as $arg (.; . |= sub("{}"; $arg)) end;
        def formatThrowable: .className + (if .message and .message != "null" then ": " + .message else "" end) + "\n" + 
        (.stepArray | [ .[]? | "    at " + .className + "." + .methodName + "(" + .fileName + ":" + (.lineNumber|tostring) + ")" ] | join("\n")) + 
        if .commonFramesCount then "\n    ... " + (.commonFramesCount | tostring) + " common frames omitted" else "" end;
        def formatThrowableWithCause: formatThrowable + (if .cause then "\nCaused by: " + (.cause | formatThrowable) else empty end)
      defaultTemplate: Spring
      templates:
        - name: Generic
          loglevel: '.s // .level // ."log.level"'
          datetime: '."@timestamp" // .t."$date" // .timestamp // .timeMillis'
          message: '.message // "\(.msg) \(.attr)"'
        - name: Spring
          loglevel: '.level // ."log.level" | color("[" + . + "]" | lpad(7); .)'
          datetime: '."@timestamp" // dateTimeFromTsNs(.timestamp//.timeMillis; .nanoseconds) // $k8sts'
          message: '. as $j | .message | color(insertSpringArgs($j.arguments); $j.level // $j."log.level")'
        - name: Spring + Stacktraces
          loglevel: '.level // ."log.level" | color("[" + . + "]" | lpad(7); .)'
          datetime: '."@timestamp" // dateTimeFromTsNs(.timestamp//.timeMillis; .nanoseconds)'
          message: '. as $j | .message | color(insertSpringArgs($j.arguments); $j.level // $j."log.level") +
            (if $j.throwable then "\n" + ($j.throwable | formatThrowableWithCause) else "" end)'
        - name: Mongo
          loglevel: '.s | color("[" + . + "]" | lpad(4); .)'
          datetime: '."@timestamp" // .t."$date" // dateTimeFromTsNs(.timestamp//.timeMillis; .nanoseconds)'
          message: '. as $j | "\(.msg) \(.attr)" | color(. ;$j.s // $j.level // $j."log.level")'

Screenshots

Classic log view

JSON decode turned on

Edit/select templates