Known differences

Where Pyrlang knowingly behaves differently from an Erlang/OTP 29 node. One bullet each: what, an example, why. Remove a bullet when the difference is gone.

Node

  • The creation is chosen by the node, not by EPMD. An OTP node uses the creation from EPMD’s ALIVE2_X_RESP. Pyrlang picks a random one in 4..2^32-1 when the Node is created and ignores EPMD’s, as erl_interface nodes do. Why: Rex, NetKernel and user processes get their pids before EPMD has answered. Example: EPMD offers 1794707079, the node’s pids carry another value; both are non-zero and differ per start.

  • A node that cannot register in EPMD keeps running. erl does not start then. Pyrlang logs the error and raises it from await node.wait_started() (EPMDConnectionError); the node object exists and listens, but no other node can find it. EPMD is tried once, with a 5 s timeout.

  • Without EPMD, other nodes are reached only through Node.connect. erl -erl_epmd_port N assumes that every node listens on port N; Node(..., epmd=False) assumes nothing: a send to a node it is not connected to raises NodeException. Example: Node("py@host", cookie, epmd=False, dist_port=4370), then await node.connect("erl@host", "host", 4370).

  • A node’s host part is resolved by the system resolver only. OTP honours inet_db settings (-kernel inet_dist_use_interface, hosts files of its own, IPv6 preference); Pyrlang connects to whatever getaddrinfo returns first that works.

  • A node without a name has no processes. Node(None, cookie) is nonode@nohost until the first node it connects to names it (status named:); creating a Process before that raises NodeException, and rex and net_kernel start when the name arrives. In OTP processes exist before and their pids show the new name afterwards.

  • Pyrlang does not give out dynamic node names. A name message with DFLAG_NAME_ME is answered like any other; such a peer sends only its host, which fails the name check, and the connection is closed. OTP answers named: with a generated name.

  • A dynamic name is kept for the life of the Node, also after the connection to the node that gave it has ended. That OTP node returns the name to its pool then (net_kernel); what a named OTP node does with its name at that point was not probed.

Handshake

  • The old 'n' name message is refused. OTP 29’s dist_util:recv_name still accepts it (with the 'c' complement) for OTP 23/24 peers that do not use EPMD. Example: <<"n", 5:16, Flags:32, "a@b">> → the connection is closed. Why: supported peers are OTP 25–29, which always send 'N'.

  • A peer registered in EPMD with highest version 5 is not contacted. Example: Node.send to such a node raises EPMDConnectionError (“only version 6 …”), without opening a connection. What OTP 29 does with such an EPMD answer was not probed.

  • Node names are not checked when a node is created. OTP refuses to start with a name whose part before @ is outside [0-9A-Za-z_-] (erl -name pyé@127.0.0.1 → “Invalid node name!”); Node("pyé@127.0.0.1", ...) starts, registers in EPMD as UTF-8 and sends the name as Latin-1 in the handshake, like OTP would. A name outside Latin-1 fails only when the first handshake is attempted (DistributionError).

  • A peer must have OTP 26’s mandatory flags, whatever its release. Example: a name message with MANDATORY_DFLAGS_25 only → not_allowed. OTP 25 would accept that peer; OTP 26-29 answer as we do. Real OTP 25 nodes send DFLAG_V4_NC and DFLAG_UNLINK_ID, so they connect.

  • No allow-list. net_kernel:allow/1 has no counterpart: not_allowed is sent only for missing flags, never for a node name.

  • Status alive is always answered true. OTP answers false when its own setup for that node is no longer the pending one. Pyrlang only connects to a node it has no connection to, so the peer’s old connection is dead from our side.

  • Two unfinished incoming handshakes from one node name both go on; the one that finishes last replaces the other. OTP keeps one of them (nok or the older one is killed) as soon as the second name message arrives.

  • Connection setup time is fixed and covers outgoing handshakes only. 7 s, the default of OTP’s net_setuptime, not configurable; an incoming connection that stops talking during the handshake is not timed out. Example: a peer that sends its name and then nothing keeps its socket.

  • Simultaneous connect compares node names as Python strings, OTP as atoms. Same order for ASCII names (all OTP can create); unverified beyond that.

  • Not every optional flag is advertised. DEFAULT_DFLAGS lacks DFLAG_BIG_SEQTRACE_LABELS and DFLAG_UNICODE_IO (trace tokens and I/O are not handled); OTP sets both.

Connected nodes

  • Trace tokens stop at Pyrlang. The _TT forms of send and exit are handled like their plain forms and the token is dropped: nothing a Python process sends in response carries it, and no trace events are produced. Example: a process with a seq_trace token sends to a Python process → the message arrives, the reply is untraced.

  • GROUP_LEADER (7) is ignored (logged). Python processes have no group leader.

  • A message to a registered name sent without a sender carries a pid that belongs to no process. Node.send(None, (ErlNode, Name), Msg) goes out as {6, AnonymousPid, '', Name}; Erlang always has a sending process. A reply to that pid is dropped.

  • Only passive processes can learn a message’s sender. Process.handle_message_from(sender, msg) gets the pid from REG_SEND and SEND_SENDER; a process that reads its inbox with receive gets messages only. Erlang processes cannot see the sender at all.

  • A malformed packet always closes the connection at once. OTP notices some bad messages only when the receiving process looks at them (bad_dist_ext_size); Pyrlang decodes every packet on arrival. Example: a REG_SEND whose message is cut → the connection is closed, linked processes get noconnection.

  • Pyrlang sends no atom cache references. Its distribution headers are 131, 68, 0 (or 131, 69, Seq, Frag, 0) and every atom is written out in full; OTP caches atoms it sends often. What the peer caches is honoured.

  • The fragments of a message are sent one after the other. OTP lets other messages go between the fragments of a large one; nothing overtakes a large message that Pyrlang sends. The sequence id is a counter per connection, not the sender’s pid.

  • Unfinished fragmented messages are limited. A peer whose incomplete fragmented messages on one connection exceed Node(max_fragmented_size=...) (256 MiB) is disconnected. OTP has no limit. Example: fragments 1000, 999, 998 of 8 kB each with a limit of 20 kB → the connection is closed.

  • Framing follows the flags both nodes advertised, like OTP: with DFLAG_DIST_HDR_ATOM_CACHE or DFLAG_FRAGMENTS in common every packet must start with a distribution header and a pass-through packet ('p') closes the connection; without, the reverse.

  • Tick intensity is fixed at 4 and the tick time is set per node only. OTP has net_tickintensity (4..1000) and can change the tick time of a running node (net_kernel:set_net_ticktime/1,2); Pyrlang takes Node(net_ticktime=...) once. A peer is given up after 1.0–1.25 tick times of silence, as in OTP with intensity 4.

  • Reading from a connection pauses while its messages wait in process inboxes. OTP reads distribution input without limit; message queues grow instead. Pyrlang stops reading from a connection once the messages it put into inboxes, and no process has taken out yet, reach Node(recv_backlog_messages=...) (10 000) or recv_backlog_bytes (32 MiB), and reads again below half. Example: an Erlang process floods a Python process that does not read → the Erlang sender is suspended (busy_dist_port), and every other message from that node to any Python process waits too. Why: by design (PYRL-23), so that a slow Python process cannot exhaust the node’s memory. Messages to passive processes do not count: they are handled on arrival.

  • A connection whose reading is paused is never given up for silence. The peer’s ticks wait unread, so its silence is not counted; a peer that dies meanwhile is noticed when reading resumes. OTP never pauses.

  • Only awaited signals wait for a busy connection. OTP suspends every process that sends on a connection whose buffer is over +zdbbl; await Node.send (and link, unlink, dist_command) wait the same way, over Node(dist_buf_busy_limit=...) (1 MiB). send_nowait, link_nowait, exit_process, monitors and everything Pyrlang sends by itself (replies of passive processes, exit signals) never wait: their data is buffered in memory. Why: sync code cannot wait.

  • Priority is not kept. A priority message (erlang:send(Dest, Msg, [priority]), OTP 28) or priority exit signal is taken like an ordinary one: Python processes have no priority aliases and no way to overtake queued messages. Example: three priority messages to a pid, a name and an alias arrive in the order sent, behind whatever was sent before.

  • An exit signal to an alias or a name is acted on. ALTACT_SIG_SEND with the exit flag ends the process behind To whether To is a pid, an active alias or a registered name, with any priority.

Aliases

  • An alias has no options. Process.alias() is alias([explicit_unalias]): it stays active until unalias or the exit of the process. There is no reply alias (removed by the first message) and no alias made by monitor/3 or spawn_request.

  • A message to an alias on another node needs a sender. Node.send(None, AliasOfErlangProcess, Msg) raises BadArgError: the control messages carry the sender’s pid. Erlang always has one.

  • An alias is any reference of the node. Erlang tells alias references from others by their layout; Process.alias() returns an ordinary 3-word reference. Example: PyAlias ! Msg on OTP 25–29 is sent to Pyrlang as an alias message all the same, and a message to any other reference of the node is dropped like one to a removed alias.

Remote calls

  • Module and function are Python names. rpc:call(Py, 'os.path', basename, [<<"/a/b">>]) imports os.path and calls basename. An exception comes back as the call’s result, {<<"TypeError">>, #{...}}, not as {badrpc, _} or an erpc exception; the same for a module or function that does not exist (OTP: {badrpc, {'EXIT', {undef, _}}}).

  • An erpc/rpc call runs inside the node’s event loop, like a call to rex: no process is created for it (the pid in the spawn reply belongs to nothing), other traffic waits until the function returns, and a timeout on the Erlang side does not stop the function.

Spawning Python processes from Erlang

  • Only registered functions can be spawned. spawn(Py, M, F, Args) calls the factory given to Node.register_spawnable(M, F, factory) with Args as positional arguments; the arity is not part of the registration. For anything else Erlang gets a pid whose process is already dead, reason {undef, [{M, F, Args, []}]}, as from an OTP node without that function (no error logger report is produced).

  • A factory that raises gives a dead pid whose exit reason is the exception, e.g. {<<"ValueError">>, #{...}}; one that returns something other than a Process gives {bad_return_value, TypeName}. OTP’s reasons for a crashing initial call are {Reason, Stacktrace}.

  • An argument list that is not a proper list gives exit reason badarg (OTP: {badarg, Stacktrace}), one that cannot be decoded closes the connection at once and starts nothing (OTP starts a process that exits with argument_list_decode_failure).

  • Spawn options other than link and monitor are accepted and ignored (priority, heap sizes, message_queue_data, …), as is the group leader. Options inside {monitor, Opts} and {link, Opts} are not looked at. An option OTP does not know is answered with badopt.

  • Python cannot spawn on Erlang nodes. There is no spawn_request from Pyrlang; an incoming SPAWN_REPLY is logged and dropped.