some basic erlang metaprogramming

The basics of erlang metaprogramming are surprisingly simple once you get used to erlang's abstract format. Basic string evaluation can be accomplished using three functions: erl_scan:string/1, erl_parse:parse_exprs/1, and erl_eval:exprs/2, and simple string evaluation can produce some pretty neat results - including a form of function currying.

Basics

Here are some basic functions - I've put them into a module called meta:

run(Bindings,Block) -> 
    evaluate(expressions(Block),Bindings).

expressions(String) ->
    {ok,Tokens,_} = erl_scan:string(String++"."),
    {ok,What} = erl_parse:parse_exprs(Tokens),
    What.

evaluate(Exprs) -> evaluate(Exprs,[]).    
evaluate(Exprs,Bindings) -> element(2,erl_eval:exprs(Exprs,Bindings)).

The run/2 function takes a set of bindings and a block of code that is to be interpreted as a collection of erlang expressions. It then parses the string for expressions and evaluates them with the provided bindings. Here are some basic examples:

1> meta:run([{'X',10}],"X").
10
2> meta:run([{'X',10}],"X+X").
20
3> F = meta:run([{'X',10}],"fun() -> X+X end").
#Fun
4> F().
20
5> meta:run([{'X',10}],"lists:seq(1,X)").      
[1,2,3,4,5,6,7,8,9,10]
6> meta:run([{'Start',10},{'Stop',20}],"lists:seq(Start,Stop)").        
[10,11,12,13,14,15,16,17,18,19,20]

Example: Function Currying

These functions also allow for a surprisingly simple meta:curry/2 function. Here are some examples of currying:

8> Even = meta:curry(fun lists:filter/2,[fun(X) -> X rem 2 == 0 end]).
#Fun
9> Even([1,2,3,4,5]).
[2,4]
10> Foo = meta:curry(fun(A,B,C,D,E) -> A+B+C+D+E end,[1,2,3]).
#Fun
11> Foo(4,5).
15

And here is the code for the curry function:

curry(Fun,Args) ->
    {arity,Arity} = erlang:fun_info(Fun,arity),
    PList = string:join([randvar() || _N<-lists:seq(1,Arity-length(Args))],","),
    run([{'Args',Args},{'Fun',Fun}],
 "fun("++PList++")->apply(Fun,Args++["++PList++"]) end").

Though this isn't true currying, it functions similarly and takes some of the work out of writing wrapper funs by hand. Oh, the randvar/0 function just produces a string that can be interpreted as a variable - it was inspired by lisp's gensym.

Dynamic Modules

Also, with some work, modules can be written and loaded at runtime. The functions of interest are erl_parse:parse_form/1 for parsing function definition strings, compile:forms/1 for actually compiling lists of abstract forms, and finally code:load_binary/3 for loading your compiled forms as a module. I haven't codified much of this yet, mostly because I haven't found a specific use, but here is a basic run:

1> String = "my_function(Var1,Var2) -> Var1*Var2.",
1> {ok,Tokens,_} = erl_scan:string(String),
1> {ok,Form} = erl_parse:parse_form(Tokens),
1> Attributes = [{attribute,1,module,my_module},{attribute,1,compile,[export_all]}],
1> {ok,Module,Binary} = compile:forms(Attributes++[Form]),
1> code:load_binary(Module,atom_to_list(Module)++".erl",Binary).
{module,my_module}
2> my_module:m
module_info/0  module_info/1  my_function/2  
2> my_module:my_function(10,10).
100

This is a pretty fun topic, and I have no doubt that more posts will continue.

Follow Up to Erlang and Inheritance

As a follow up to my last post, I am posting some code that demonstrates how one might do inheritance with erlang. I've also provided some shell work to better elucidate some of the concepts:

The code

The following three modules are used:

1. base.erl : The root interface module, all other interface modules must extend this one, or must be extensions of modules that are themselves valid interface modules.


2. baselib.erl : This module implements the main loop that is responsible for keeping the data. Its state includes the data itself, as well as a possible reference to a "running" base instance acting as the prototype for data inheritance.


3. basedb.erl : Runs a lookup service that is used for finding Pid's of running base instances. It also should (but doesn't do so presently) keep track of "sleeping" or stored process data.

An Example Run

1> basedb:start().
true
2> X = base:new("aprototype").   %We'll use this as a prototype
{base,"aprototype",<0.37.0>}
3> X:proto().
'SELF'
4> X:insert([{"foo","bar"},{"zoo","yar"}]).
ok
5> X:attributes().
["foo","zoo"]
6> Y = base:new("heir","aprototype").   %%Y's uses X's data when it needs to
{base,"heir",<0.44.0>}
7> Y:proto().
{base,"aprototype",<0.37.0>}     %%Y's prototype is just X
8> Y:attributes().               %%Since we haven't set anything for Y yet, it defers to X
["foo","zoo"]
9> Y:select(["foo","zoo"]).
["bar","yar"]
10> Y:select(["moo"]).          %% Oops! this isn't there!
Error: {{case_clause,error},
        [{baselib,selection,3},
         {baselib,handle,2},
         {baselib,safewrap,2},
         {baselib,do_look,4}]}
Error: {{badmatch,{error_with,{select,["moo"]}}},
        [{baselib,selection,3},
         {baselib,handle,2},
         {baselib,safewrap,2},
         {baselib,do_look,4}]}
{error_with,{select,["moo"]}}
11> Y:insert([{"moo","goo"}]).   %% Now Y has something that X lacks
ok
12> Y:select(["moo"]).        
["goo"]

Extending The Interface

Here is a simple extension module:

-module(extension1,[]).
-compile(export_all).

-extends(base).

new(Id) ->
    B = ?BASE_MODULE:new(Id),
    instance(B).

new(Id,Proto) ->
    B = ?BASE_MODULE:new(Id,Proto),
    instance(B).


pretty_print() ->
    Attribs = THIS:attributes(),    %%since attributes/0 isn't defined here
    lists:foreach(                  %%the base module will be consulted
      fun({K,V}) ->
       io:format("~p : ~p ~n",[K,V])
      end,
      lists:zip(Attribs,THIS:select(Attribs))).

And, continuing the shell session above, we see it in action:

17> c(extension1). 
{ok,extension1}
18> Z = extension1:new("an extension","aprototype").
{extension1,{base,"an extension",<0.77.0>}}
19> Z:pretty_print().
"foo" : "bar"  
"zoo" : "yar" 
ok

That about wraps it up. Note that the code posted here is, in truth, very bad code, but you should be able to get the idea - I know that I have.

Erlang and Inheritance

Today I discovered the -extends() module attribute via this presentation , and I must admit, I'm a little overly interested. I have been trying to think of a good way to do objects in erlang, and there is some promise with module extensions + parameterized modules. Because any self respecting implementation of an object requires mutable state, and because parameterized modules do not give you mutable state, we must obviously throw processes into the mix. During the last several hours I have come up with a somewhat convoluted means to do just this. I found, however, that as soon as one manages to slap processes together with extensions of parameterized modules, one finds himself faced with an obnoxiously unmanagable two dimensions of inheritance: data inheritance and interface inheritance.

Starting From Square One

Lets say that you want to create a prototypical inhertiance model. In this case your "objects" may be supplied with a prototype, to which they appeal for default data. This is desirable situation whenever extensibility is required in data-driven applications: to give every "object" of a particular kind a new property, you simply add that property to the prototype from which those objects descend, hopefully without ever having to interrupt the running application.

To accomplish this in erlang, one might uniquely associate data structures with process identifiers, and keep these in some kind of key-value store for quick lookup. When creating a new "object" that should inherit from some other, you simply pass it the pid of the object that it should use as a prototype. This is all well and good - but what if we wish to specialize the interfaces to the data as well as the data themselves?

Extending From A Base

This is where we start to use the -extends module attribute. This is also where things become complicated. I have chosen to create a fundamental module called base that offers a simple create/insert/select interface. The base module is parameterized, and is never meant to be instanced by itself - rather it is always instanced in the MOD:new/K modules of any modules that extend it.

Instances of base look like {base,ID,PID,PROTO}, where ID is the object's unique identifier, and should be a string; PID is the process identifier of the loop that stores its state; and PROTO is either the atom 'PROTO' (signifying that this object is the prototype), or it is another object instance acting as this object's prototype for purposes ofdata inheritance only.

All extending modules have been, for the time being, declared as either

-module(an_extension,[]).
-extends(base).

or as

-module(another_extension,[]).
-extends(an_extension).

so that instances end up looking like {an_extension,{base,ID,PID,PROTO}} or like {another_extension,{an_extension,{base,ID,PID,PROTO}}}. Something that I have had to keep in mind during these trials is that PROTO can itself be an instance of an extension, but that it is only used for data inheritance. This makes things confusing, but also extremely versatile. After a few hours worth of playtime, it seems to me that the approach I have taken will create a large set of "singletons" that are purely useful as prototypes so that data inheritance will probably not span more than a single generation.

The Other Inheritance

Interface inheritance, on the other hand, is accomplished through successive uses of the -extends() module attribute. We end up with a situation in which we can give multiple faces to the same set of data, and have it behave however we like, so long as we have an extension module for it. Having this other "line" of inheritance, however, can get to be confusing. Under this model it is possible to have an object whose interface inheritance is several generations long but whose data is "prototypical" data. We can also get ourselves into situations in which a prototype and it's descendants are using different interfaces.

One note: With some work, this could be made to be RESTful.

I'd like to hear about others' experiences using module extensions, please drop a comment if you have any.

See Also Follow Up Post