Monitoring log files with 'tail'

When you need to look for specific events from logfiles, your first idea is to use 'tail'. Tail is obviously the number one command that any sysadmin knows about.

From the first version of Tail and nowadays, some really nice feature have been implemented, one of those is the "follow=name" feature...

Since your erlang node will stay alive for many days, you'll end up meeting some logrotation tool that will replace the file you're lurking... So "follow=name" is for you !

Extract from a manual page:

     There are two ways to specify how you'd like to track files with
     this option, but that difference is noticeable only when a
     followed file is removed or renamed.  If you'd like to continue to
     track the end of a growing file even after it has been unlinked,
     use `--follow=descriptor'.  This is the default behavior, but it
     is not useful if you're tracking a log file that may be rotated
     (removed or renamed, then reopened).  In that case, use
     `--follow=name' to track the named file by reopening it
     periodically to see if it has been removed and recreated by some
     other program.

Implementing this feature in pure erlang is of course possible, but why loose time when you can directly use the "tail" binary already installed on your system ?

-module(tail).
-export([start/1, start/2, start/3, stop/1, snapshot/1, display/1, init/3]).

start(File) ->
	start(File, fun display/1, "/var/log").

start(File, Callback) ->
	Dir = "/var/log",
	start(File, Callback, Dir).

start(File, Callback, Dir) ->
	spawn_link(?MODULE, init, [File, Callback, Dir]).

snapshot(Pid) ->
	Pid ! {snap, self() },
	receive
		{Port, Callback} ->
			{Port, erlang:fun_info(Callback)};
		_Any ->
			_Any
	end.

stop(Pid) ->
	Pid ! stop.

init(File, Callback, Dir) ->
	Cmd = "/usr/bin/tail --follow=name "++ File,
	Port = open_port({spawn, Cmd}, [ {cd, Dir}, stderr_to_stdout, {line, 256}, exit_status, binary]), 
	tail_loop(Port, Callback).

tail_loop(Port, Callback) ->
	receive
		{Port, {data, {eol, Bin}}} ->
			Callback(Bin),
			tail_loop(Port, Callback);

		{Port, {data, {noeol, Bin}}} ->
			Callback(Bin),
			tail_loop(Port, Callback);

		{Port, {data, Bin}} ->
			Callback(Bin),
			tail_loop(Port, Callback);

		{Port, {exit_status, Status}} ->
			{ok, Status};
			%tail_loop(Port, Callback);
	
		{Port, eof} ->
			port_close(Port),
			{ok, eof};

		{snap, Who} ->
			Who ! { Port, Callback},
			tail_loop(Port, Callback);

		stop ->
			port_close(Port),
			{ok, stop};
		
		_Any ->
			tail_loop(Port, Callback) 
	end.

display(Bin) ->
	Content = iolist_to_binary(Bin),
	io:format("[INFO] ~s~n", [Content]).

Let's say you want to monitor "/var/log/messages", here's how you can do it:

Shell> Tail = tail:start("messages").

This will display every new line (running in background) in your shell session.

Now let's say you want to do some tricky things with every line, you can pass as a parameter a callback fun:

Shell> Pid = logger_new(). % an example
Shell> Callback = fun(X) -> Pid ! {line, X} end. % sending a tuple to Pid
Shell> Tail = tail:start("message", Callback).

Finally, you'll be able to hack the code and transform this method to "tail" multiple files since "tail" is able to watch more than one file...

Quick tip :

init(ListOfFiles, Callback, Dir) ->
	Args = [ [ X, $ ] || X <- ListOfFiles ]
	Cmd = "/usr/bin/tail --follow=name "++ lists:flatten(Args),

Happy Tailing !