fluent-plugin-flatten 0.0.1 Released

I hacked up another fluentd output plugin, fluent-plugin-flatten. It treats a value related to a specified key in config as a JSON-formatted hash string and spread it out to top level of record.

Just to see is more understandable ;)

How it works

When you have a config as below

<match test.**>
  type flatten
  key  foo
</match>

and you feed such a value into fluentd

{
  "foo"  => '{"bar" : {"qux" : "quux", "hoe" : "poe" }, "baz" : "bazz" }',
  "hoge" => "fuga"
}

then you'll get

{
  "foo"  => '{"bar" : {"qux" : "quux", "hoe" : "poe" }, "baz" : "bazz" }',
  "hoge" => "fuga",

  "foo.bar.qux" => "quux",
  "foo.bar.hoe" => "poe",
  "foo.baz"     => "bazz"
}

That is, JSON-formatted hash string in the value of the key foo is flattened recursively and now put into the top level of the hash.

Possible use case

It's possible that you want a value to be structural while being related to one specific key. For example, I want to put some multiple application log data into httpd's log. Apache can handle an environment variable and put it into its access log using %{FOOBAR}e.

This plugin is useful when that value is structural. Fluentd in the middle reads and extract that kind of value to top level of key/value pairs. Once so, we can just do anything as usual.

Enjoy!