Using ports for fast templating...

I find myself many times searching for fast templating system, and I always stop my search by developing one myself. ( the one is done using flex generated code, which means that it is very fast... )

Here, I need some fast (rather simple) template system that gives me some data to send later to an SMTP server (the body of the mail).

First the code:

get(Template, Args) ->
     NewArgs = list_to_shell(Args),
     Cmd = lists:flatten([ ?engine, $ , "conf/template/", Template, $ , NewArgs ]),
     Port = open_port({spawn, Cmd}, [ {cd, code:lib_dir(mmailer)}, stream, binary]),
     process_flag(trap_exit, true),
     loop(Port).

You'll find some unknown notation: '?engine' and $.

• ?engine is a macro, defined like this -define(engine, "templater").
• $ is the character notation, the character that follows the dollar sign is the character I want. Actually I need the space character...
  

Now the function list_to_shell:

list_to_shell(Args) ->
      L = lists:map(fun(X) -> [X, $ ] end, Args),
      lists:flatten(L).

This function add a space after after element of the array passed as parameter. I construct the shell command this way.

The call to open_port is the main part of the code:

Port = open_port({spawn, Cmd}, [ {cd, code:lib_dir(mmailer)}, stream, binary]),

The following describe what the above line does:

• spawn the command 'Cmd'
• in the directory code:lib_dir(mmailer),
• this command will write data as a stream
• and I want erlang to give me a binary stream.

(code:lib_dir(mmailer) gives me the directory where the mmailer module is located)

More about my engine... My engine takes as firts arguments the template file to parse and every other parameter is saved as $1, or $2, or $3 etc.

Example of a template 'test.template'

Hello $1 $2 !
We have a good news for you, join us at url/$3

So calling:

templater test.template Mister John connect?John

Will compute the following:

Hello Mister John !
We have a good news for you, join us at url/connect?John

Conclusion, you can use whatever template system efficiently with the open_port function, since you can dynamically build the command line.

NB: I've not secure the line generated by list_to_shell, any '; rm -rf *' in the command line will do nasty things with your stuff :)

Atoms...

Atoms, Variable, what they are and what you can do with them ?

Atoms are what you'll find strange at the first look, you may ask yourself
continously, but where the hell this thing is defined !!

Relax and just read the following words, atoms are just there to document your code...
So they can appear anywhere, have no special meaning or just may be funny !
You'll find them most of the time tagging tuples, example:

test() ->
        % code that fails
        {error, "Can't find file"}.

This way you can try to match 'error', and bind a Variable to the reason:

case test() of
        {ok, Result} ->
                Result;
        {error, Reason} ->
                io:format("Error: because ~p~n", [Reason])
end;

Simple isn't it ?

Erlang and calling functions

In Erlang, when you want to call a function you can use various notations:

modulename:functionname(arguments).
functionname(arguments).

Calling a function in another process:

NewPid = spawn(modulename, functionname, [arguments]).

Now a simple test module:

-module(test).
-export([test/1]).

test(String) ->
  { erlang:now(), String }.

This module has one function named 'test', its arity is one (one parameter), and this function
returns a tuple that contains the datetime and the 'string' passed as the parameter.

Calling 'erlang:now()':

(master@karoten)52> erlang:now().
{1181,599629,877590}
(master@karoten)53> 

This is some sort of UNIX timestamp (number of seconds since 1970).


Now test our module:

Compiling it, ie compiling 'test.erl' located in the current directory:

(master@karoten)52> c(test).
ok

Calling the funcname 'test':

(master@karoten)53> test:test("test").
{{1181,599702,993670},"test"}
(master@karoten)54> 

Here's we have used the notation 'modulename:functionname(arguments)'.
But within the module itself we could use the notation 'functionname(arguments)'...

-module(test).
-export([test/1, test/0]).

test(String) ->
  { erlang:now(), String }.

test() ->
  test("test").

Here's come what's really important to understand in Erlang, a function is defined by its name AND its arity.
So 'test/0' is NOT 'test/1'.

In our module, what the function 'test/0' do is just calling 'test/1' with a string that contains 'test'.

Erlang is a functionnal language, so the parameter of the function 'test/1' can really be what you want, from simple string to integer to complex list of tuple...

Just try this, and you'll understand :

test:test( [ complex, list, of, atoms, "and a string" ] ).

Shrink or Strip a binary octet-stream easily

I've found a simple way to suppress ending characters of a Binary without parsing it or change it into a list !

I wanted to remove those <<"\r\n">> characters from lines read from a file, and know that my line are made of 29 characters (reading date strings).

So with this function:

shrinkbin(Data, Size) ->
  <<Data:Size/binary-unit:8>>.

I can do:

node> test:shrinkbin(<<"Sun Jun 10 15:20:53 CEST 2007\r\n">>, 29).
<<"Sun Jun 10 15:20:53 CEST 2007">>.

That's it !

Matching Protocol status code using Binary notation .

Once you're succesfully connected to some remote host, you may need to read what this peer sends you...

Sometimes the protocol is using simple integer number to describe what's going on. Let's have a look at the SMTP protocol:
2XX are ok codes,
5XX are error codes.

Testing Binary matching:

-module(bintest).
-export([test/0, check/1]).

test() ->
   Bin = <<"200 OK\r\n">>,
   check(Bin).

check(<<"200", Rest/binary>>) ->
   {200, Rest};
check(<<"300", Rest/binary>>) ->
   {300, Rest};
check(_Bin) ->
   {800, unknown}.

Sorting Mx servers using their preference number...

Once you've retrieved your mx lists, you want to be kind enough to gently contact mx server in their prefered order...

In erlang, sorting a list is done with sort function like 'keysort'.

SortedList = lists:keysort(1, List).

lists:keysort uses as first parameter, the nth element of the tuple to use in the sort, and as second parameter the tuple list you want to sort...

Once you've sorted your tuple list, you may want to remove the key you use to sort, lists:unzip remove this...

Finally, retrieving a mx list, sorting it, and directly using it can be done like this:

mxlist(Domain) ->
 List = get_mx(Domain),
 {_, Hosts} = lists:unzip( lists:keysort(1, List) ),
 Hosts.

Configuring your DNS server for inet_res.

When using inet_res, you must configure a DNS server to query, this could be done like this:

-define(MASTER_DNS, {212,XX,XX,252}).

init() ->
 inet_db:add_ns(?MASTER_DNS).

Once you've called your ?MODULE:init/1 you're able to use your ?MODULE:get_mx/1 function.

node> mail:init().
ok

node> mail:get_mx("yahoo.com").                                                       
[{1,"e.mx.mail.yahoo.com"},
 {1,"a.mx.mail.yahoo.com"},
 {1,"b.mx.mail.yahoo.com"},
 {1,"c.mx.mail.yahoo.com"},
 {1,"d.mx.mail.yahoo.com"},
 {1,"f.mx.mail.yahoo.com"},
 {1,"g.mx.mail.yahoo.com"}]

The result is a list of tuples, the first element is the server weight, the second the server name :p

Retrieve MX DNS record using erlang inet_res, it's easy !

Ever needed to retrieve some MX servers from any big domains out there ?

I'm pretty sure that you've asked yourself this question, right ?

It's EasyErl here, so let's go to the code directly:

get_mx(Domain) ->
       {ok, {hostent, Domain, _, _, _Len, List}} = inet_res:getbyname(Domain, mx),
       List.

Woh ! This wasn't too much difficult !

Okay, inet_res:getbyname isn't really well documented that's right, but Erlang comes with its source code, what's a better Erlang documentation than Erlang himself ?

The next time, I'll show you how you can easily sort the List by using unzip to obtain a correct list of MX servers... (where servers weight are correctly used)