Calling Python from Erlang¶
rpc and erpc¶
A Pyrlang node answers Erlang’s remote procedure calls. The module is imported on the Python side if it is not yet, and the function is called with the arguments:
(erl@127.0.0.1)1> rpc:call('py@127.0.0.1', math, sqrt, [16.0]).
4.0
(erl@127.0.0.1)2> erpc:call('py@127.0.0.1', 'os.path', basename, [<<"/a/b.txt">>]).
<<"b.txt">>
Module and function are Python names written as atoms; a dotted module name needs quotes,
'os.path'.rpc:castanderpc:castrun the function and discard the result.If the function raises, or the module or function does not exist, the call returns the exception, as
{<<"ValueError">>, #{<<"traceback">> => "Traceback ..."}}. Erlang does not get{badrpc, _}or anerpcexception (see Known differences).The call runs on the event loop of the node: a function that takes long holds up everything else on it.
The result has to be something that can be sent to Erlang (see Data types). Many Python functions return objects that cannot; for those there are notebook calls.
Notebook calls¶
A Notebook is a process on the Python node
that keeps the results of calls. Erlang gets a reference to each result and
passes it as an argument of later calls, so intermediate values never cross
the wire.
The Erlang side is the module py.erl/py.erl of the repository. It has no
dependencies: copy it into your project.
%% Notebook-style calls from Erlang to Python: results stay on the Python
%% node and are used as arguments of later calls.
%%
%% 1. python examples/e02_registered_process.py (any Pyrlang node will do)
%% 2. erlc -o examples py.erl/py.erl examples/e06_notebook_calls.erl
%% 3. erl -noshell -pa examples -name erl@127.0.0.1 -setcookie COOKIE \
%% -s e06_notebook_calls
-module(e06_notebook_calls).
-export([start/0, start/1]).
start() -> start(['py@127.0.0.1']).
%% `erl -s e06_notebook_calls start Node` passes the node name in a list.
start([Node]) ->
%% A notebook (context) on the Python node keeps the results.
Ctx = py:new_context(Node),
%% datetime.datetime.now(). The first path element is the module to
%% import; binaries, strings and atoms all work as names.
DT1 = py:call(Ctx, [<<"datetime">>, "datetime", now], []),
timer:sleep(100),
%% The same with keyword arguments, here none.
DT2 = py:call(Ctx, [datetime, datetime, <<"now">>], [], #{}),
%% DT2 - DT1: the two results never left Python.
Diff = py:call(Ctx, [DT2, '__sub__'], [DT1], #{}),
%% Diff.total_seconds(), returned at once and not stored.
Result1 = py:call(Ctx, [Diff, <<"total_seconds">>], [], #{},
#{immediate => true}),
io:format("Result1 (immediate) = ~p~n", [Result1]),
%% Or store it on the Python node and fetch it later.
Result2Ref = py:call(Ctx, [Diff, <<"total_seconds">>], []),
Result2 = py:retrieve(Ctx, Result2Ref),
io:format("Result2 (retrieved) = ~p~n", [Result2]),
%% Drop the notebook and everything it holds.
py:destroy(Ctx),
io:format("Finished~n"),
init:stop().
py:new_context(Node), py:new_context(Node, Options)
: Starts a notebook on the node. Options is a map; history (default 50)
is how many results the notebook keeps.
py:call(Context, Path, Args), py:call(Context, Path, Args, KwArgs), py:call(Context, Path, Args, KwArgs, Options)
: Calls a function. The first element of Path is a module name or a
result reference; the rest are attribute names leading to the callable.
Arguments and keyword values may be result references. Options:
timeout in milliseconds (default 5000); immediate (default false),
with true the value itself is returned and nothing is stored.
py:retrieve(Context, ValueRef)
: Fetches a stored result.
py:get_type(ValueRef)
: The name of the Python type of a stored result, known without asking.
py:destroy(Context)
: Ends the notebook and drops what it holds.
Batches¶
A batch is a list of such calls built in Erlang and run on the Python node in one round trip. The same batch can be run on several nodes.
%% A batch of Python calls built in Erlang and run on a Python node in one
%% round trip.
%%
%% 1. python examples/e02_registered_process.py (any Pyrlang node will do)
%% 2. erlc -o examples py.erl/py.erl examples/e07_notebook_batch.erl
%% 3. erl -noshell -pa examples -name erl@127.0.0.1 -setcookie COOKIE \
%% -s e07_notebook_batch
-module(e07_notebook_batch).
-export([start/0, start/1]).
start() -> start(['py@127.0.0.1']).
%% `erl -s e07_notebook_batch start Node` passes the node name in a list.
start([Node]) ->
%% Build the batch; nothing is sent yet. Each call gives a reference
%% to its future result for use in later calls.
S0 = py:batch_new(),
{S1, R1} = py:batch_call(S0, [<<"datetime">>, "datetime", now], []),
{S2, R2} = py:batch_call(S1, [datetime, datetime, <<"now">>], [], #{}),
{S3, Diff} = py:batch_call(S2, [R2, '__sub__'], [R1], #{}),
{S4, _R4} = py:batch_call(S3, [Diff, <<"total_seconds">>], []),
Ctx = py:new_context(Node),
%% Run it: the result of the last call comes back (immediate => true
%% is the default for batches).
Result = py:batch_run(Ctx, S4),
io:format("Result1 (immediate) = ~p~n", [Result]),
%% Run it again, keep the result on the Python node, fetch it later.
Result2Ref = py:batch_run(Ctx, S4, #{immediate => false}),
Result2 = py:retrieve(Ctx, Result2Ref),
io:format("Result2 (retrieved) = ~p~n", [Result2]),
py:destroy(Ctx),
io:format("Finished~n"),
init:stop().
py:batch_new()
: An empty batch.
py:batch_call(Batch, Path, Args), py:batch_call(Batch, Path, Args, KwArgs)
: Appends a call; returns the new batch and a reference to the call’s
future result.
py:batch_run(Context, Batch), py:batch_run(Context, Batch, Options)
: Runs the batch in a notebook. Options as for py:call, except that
immediate defaults to true: the result of the last call is returned.