Fluentd::Integration: Utilities for Integration Test against Real Fluentd Processes

As all users of fluentd know, it's difficult to test your configurations work well with real fluentd process. Besides, through multiple fluentd processes, it becomes much more. Fluentd::Integration provides some utilities to help you test your fluentd configurations.

• https://github.com/kentaro/fluentd-integration

You can simply write Ruby script to setup fluentd and throw some data into it like below:

#!/usr/bin/env ruby

require 'fluentd/integration'

server = Fluentd::Integration::Server.new(capture_output: true)
server.conf = <<EOS
<source>
  type forward
  port #{server.port}
</source>

<match test>
  type stdout
</match>
EOS
server.start

client = Fluentd::Integration::Client.new(port: server.port)
client.post(:test, { foo: "bar" })

puts server.in.gets #=> <time> test: {"foo":"bar"}

For multiple fluentd processes, write like below:

#!/usr/bin/env ruby

require 'fluentd/integration'

server1 = Fluentd::Integration::Server.new
server2 = Fluentd::Integration::Server.new(capture_output: true)

server1.conf = <<EOS
<source>
  type forward
  port #{server1.port}
</source>

<match test>
  type forward
  flush_interval 1s # DON'T FORGET THIS

  <server>
    name server1
    host 127.0.0.1
    port #{server2.port}
  </server>
</match>
EOS

server2.conf = <<EOS
<source>
  type forward
  port #{server2.port}
</source>

<match test>
  type stdout
</match>
EOS

server2.start
server1.start

client = Fluentd::Integration::Client.new(port: server1.port)
client.post(:test, { foo: "bar" })

# wait until time (`flush_interval`) passes
sleep 2

puts server2.in.first #=> <time> test: {"foo":"bar"}

The examples above are just plain scripts. This library provides simple objects for server and client. I think you can easily use them in RSpec or some test framework.

Enjoy!