Changelog¶
Unreleased¶
Documentation rebuilt (Sphinx, Furo, MyST) and rewritten for the current code; it moves to https://pyrlang.clau.se/pyrlang/ and generated HTML is no longer kept in the repository. The reference covers every module.
New examples in
examples/(e01toe07), each run by the tests against real Erlang nodes; the old ones, which predate the current API, are removed (links and monitors per direction, several Python nodes, the Elixir client).Fixed: a notebook batch (
py:batch_run) returnedundefinedinstead of the result of its last call, and a replypy.erldid not understand.Flow control. Reading from a connection pauses while too many of its messages wait in process inboxes (
Node(recv_backlog_messages=10_000, recv_backlog_bytes=32 MiB)), so the sending Erlang processes are suspended instead of this node running out of memory; one slow process holds back every message from that node.await Node.send(andlink,unlink,dist_command) wait while more thanNode(dist_buf_busy_limit=1 MiB)waits to be sent on the connection, as OTP’s+zdbbl; the*_nowaitmethods never wait.Monitors on processes of a node whose connection is lost, or cannot be set up, fire once with
noconnection, as in OTP (links did already); monitors from that node’s processes end.Fixed: signals sent while a connection was being set up could be overtaken by later ones. Signals to a node are now queued in the order sent, also while connecting, and go out first when the connection is up.
Node.send_nowait,link_nowaitand the newunlink_nowaitact at once instead of in a task: a local process has the message, and a remote one’s signal is queued, when they return. Their errors (a remote alias without a sender, a bad receiver) are raised to the caller. Messages to a node that cannot be reached are dropped and logged;await Node.sendraises as before.Incoming signals take effect in the order they arrive, without a task per message.
Node.on_linkis no longer a coroutine.Fixed: a message to a registered Python process was dropped when it held an integer of more than 4300 decimal digits: the message was formatted for an INFO log line, and
str()of such an integer raises. Messages are no longer formatted for logging at all, which also makes large messages cheaper.Fixed:
Node.demonitor_processtowards a process on another node never sentDEMONITOR_P, so the other node kept the monitor (everyGenServerInterface.callto an Erlang process left one behind).Fixed: an exiting process now removes the monitors it holds, as in OTP (a process on another node is sent
DEMONITOR_P); a monitor that fired is removed from its owner; a ‘DOWN’ message arrives only for a monitor that is still active.New:
Process.monitors()andProcess.monitored_by()(reference -> pid), as Erlang’sprocess_info(P, monitors | monitored_by).A monitor from another node on a process that does not exist is answered with
noprocwithout logging an error.Received bytes are no longer copied once per packet: a read holding many packets, or a large message arriving in many reads, took quadratic time.
Nodekeeps the tasks it starts until they are done and cancels them indestroy().Importing
pyrlangno longer touches logging: the loggers ofpyrlang.dist_protowere set to INFO at import, and the environment variablesPYRLANG_LOG_LEVELandPYRLANG_ENABLE_LOG_FORMATwere applied. Callpyrlang.util.start_pyrlang()to apply them.Needs
pyrlang-term2026.1002a0 or newer.Python 3.11 or newer is required (
requires-pythonsaid 3.7; the code needed 3.9 already).Nodemust be created inside a running asyncio loop (it usesasyncio.get_running_loop();get_event_loop()raises on Python 3.14).The EPMD client honours
ERL_EPMD_PORTlikeerl;EPMDClientandEPMDClient.query_nodetake an optionalport.Fixed: a receive pattern whose handler returns a plain value (every
GenServercall,net_kernel’sis_auth) crashed the process, sonet_adm:pingto Pyrlang never answered.Bitstrings from Erlang arrive as
term.BitString(needs pyrlang-term with thebitstringdecode option), so a bitstring sent back stays a bitstring instead of becoming a{Bytes, Bits}tuple.pyrlang.dist_proto.flagsfollows OTP 29’sdist.hrl: new flags up toDFLAG_NATIVE_RECORDS,MANDATORY_DFLAGS_25/_26,DFLAGS_MANDATORY, helpersexpand_mandatory_25andmissing_mandatory. OTP’s namesDFLAG_EXTENDED_REFERENCES,DFLAG_EXTENDED_PIDS_PORTSandDFLAG_HANDSHAKE_23are the main ones; the oldDFLAG_EXT_REFS,DFLAG_EXT_PIDS_PORTS,DFLAGS_HANDSHAKE_23remain as aliases.Distribution protocol version 5 (the
'n'handshake of OTP 22 and older) is no longer spoken; peers must be OTP 23 or newer.EPMDClient.query_nodereturns(host, port)without a version, andDistClientProtocolno longer takesdist_vsn.New module
pyrlang.dist_proto.handshake: the handshake messages as dataclasses with pureencode_*/decode_*functions.A malformed packet from a peer closes that connection with one log line instead of raising out of the event loop callback; the node keeps accepting connections. Node names in handshake messages are checked (
name@host) and bytes after the name are ignored.Handshake statuses are handled as OTP’s
dist_utilandnet_kerneldo. Accepting: a peer without the mandatory flags getsnot_allowed; a second connection from a connected node getsaliveand replaces the old one aftertrueand a correct cookie; with both nodes connecting at once the connection made by the greater node name survives (ok_simultaneous/nok). Connecting:aliveis answeredtrue,nokwaits for the peer’s own connection,not_allowedraises the newConnectionNotAllowedError; the peer’s flags and name are checked.Sending to a node that cannot be reached now raises instead of waiting forever:
NodeExceptionif EPMD does not know it,DistributionError(orConnectionNotAllowedError,SimultaneousConnectError) if the handshake fails or takes more than 7 s. Concurrent sends to a node that is not connected yet share one connection setup.ErlangDistribution.connect_to_nodereturns after the handshake.Fixed: a closed connection stayed registered, so later messages to that node were lost instead of reconnecting.
Handshake challenges come from
secrets, digests are compared in constant time, and challenges, digests and packet contents are no longer written to the log. A cookie may use any Latin-1 character (it was ASCII only); OTP cannot use other cookies, and neither can we.Fixed: pids of
rex,net_kerneland of every process created before EPMD answered had creation 0, which OTP 25 and newer do not accept as a real creation. A node now has a random non-zero creation from the moment it is created (EPMD’s value is no longer used). Pids vary in theirIDfield first, as OTP’s do.The link protocol of OTP 23.3 (mandatory since OTP 26) is implemented and
DFLAG_UNLINK_IDis advertised:Node.unlinkto a remote process sendsUNLINK_ID(it used to send nothing), incomingUNLINK_IDis acknowledged, the oldUNLINKsignal is ignored. NewProcess.links().Exit signals of links follow Erlang: a signal from a process that is not (or no longer) linked is ignored, reason
normalends the link without killing the linked process, and a process that exits normally still tells its remote links. When a connection is lost, processes linked over it exit withnoconnection.Pyrlang connects to Erlang/OTP 26, 27, 28 and 29 (and still to 25):
DFLAG_V4_NCandDFLAG_MANDATORY_25_DIGESTare advertised. Needs pyrlang-term withV4_PORT_EXTand 5-word references.Fixed: an Erlang process monitoring a Python process got the
DOWNtuple as an ordinary message and its monitor stayed; it now getsMONITOR_P_EXIT. Monitoring a Python process that does not exist fires withnoprocinstead of never.rexalso servesrpc:block_call/4. An argument list of small integers ([12, 18]) reached the Python function as a string.rexno longer claims theerpcfeature:rpc:call/4needs spawn requests and answers{badrpc,notsup}for now (userpc:block_call/4).Control messages from a connected node are checked (number, arity, element types, whether a message follows) before they are acted on. A malformed or undecodable packet, or an unknown packet type, closes that connection with one log line; it used to raise inside a background task or be silently half-processed. New module
pyrlang.dist_proto.control.Ticks follow OTP’s
net_ticktime: newNode(..., net_ticktime=60). An idle connection is ticked every quarter of it (it was every 15 s, so Erlang nodes with a tick time under a minute dropped Pyrlang), and a node silent for 1.0 to 1.25 tick times is considered down (it was a fixed 30 to 50 s). Timers of a closed connection are cancelled.DFLAG_SEND_SENDERis advertised: OTP sendsSEND_SENDER, and Pyrlang sends it to peers that have the flag. Messages and exit signals from a process with aseq_tracetoken (the_TTcontrol messages) used to be dropped; they are delivered now (the token is not passed on). NewProcess.handle_message_from(sender, msg)for passive processes.DFLAG_EXIT_PAYLOADis advertised: exit reasons andDOWNreasons travel as payload (PAYLOAD_EXIT,PAYLOAD_EXIT2,PAYLOAD_MONITOR_P_EXIT) in both directions.Fixed: sending
None(the atomundefined) as a message produced a packet without a message.Process aliases (EEP 53):
Process.alias()returns a reference thatNode.sendand Erlang’s!accept as a destination untilProcess.unalias(ref)or the process exits;Node.sendalso sends to aliases of Erlang processes.DFLAG_ALIASis advertised.Alternate action signals of OTP 28 (
ALTACT_SIG_SEND): alias and priority messages and priority exit signals from OTP 28+ are received (priority itself is not kept), and messages to aliases are sent in this form to peers that have it.DFLAG_ALTACT_SIGis advertised.Spawn requests (
DFLAG_SPAWNis advertised):rpc:call/4,5,rpc:cast,erpc:callanderpc:castfrom Erlang work again (they answered{badrpc,notsup}with OTP 23 and newer);rexreports the featureerpc. NewNode.register_spawnable(module, function, factory)lets Erlang’sspawn/4,spawn_link/4,spawn_monitor/4,spawn_opt/5andspawn_request/5start a PythonProcess.Added
KNOWN-DIFFERENCES.md: where Pyrlang differs from an OTP node.Dynamic node names:
Node(None, cookie, host=...)asks the first node it connects to for a name (DFLAG_NAME_ME, aserl -name undefined). Such a node is hidden, does not listen and is not in EPMD; it cannot have processes before it is named. NewNode.adopt_name,pyrlang.dist_proto.handshake.Namedanddecode_named.Fixed:
Node.send(None, (Node, Name), Msg)sentundefinedas the sender ofREG_SEND, over which an Erlang node closes the connection; a pid of the node that belongs to no process is sent instead.EPMD client: a node registers with the right type (77 for a visible node, 72 for
Node(hidden=True); it was always 77), replies are read completely however they are split, host names are resolved without blocking the event loop, and every wait has a 5 s timeout. A failure is anEPMDConnectionErrorwith the address and the cause, also from a send to a node whose EPMD cannot be asked (it was anOSError).EPMDClient.connect()no longer retries for 25 s and returnsFalse: it raises at once.EPMDClient.close()is safe beforeconnect().New
await Node.wait_started(): returns when the node listens and is registered, or raises why it is not.Nodes without EPMD:
Node(..., epmd=False)does not register,Node(..., dist_port=N)listens on a fixed port, andawait Node.connect(name, host, port)connects to a node whose port is known (erl -start_epmd false -erl_epmd_port N).DFLAG_DIST_HDR_ATOM_CACHEandDFLAG_FRAGMENTSare advertised: peers send distribution headers with their atom cache and split large messages into fragments, so a small message is not held up behind a large one; Pyrlang sends payloads above 64 kB in fragments. NewNode(max_fragmented_size=...)(default 256 MiB) limits the unfinished fragmented messages held per connection. Needs pyrlang-term with theatom_cachedecode option.pyrlang.dist_proto.control.split_packetbecamesplit_message.DFLAG_NATIVE_RECORDSis advertised: native records of OTP 29 arrive asterm.Recordand can be sent back.
0.7 (2018-09-11)¶
Added Erlang examples 3 and 4, also makefile target for convenience:
make pynodewhich will run Pyrlang as an idle node.Batch calls added (see docs for calling Python and examples 3, 4).
0.6 (2018-09-07)¶
Notebook-style remote calls (simplified flow for Python calls) which store results remotely until they are needed.
0.5.2 (2018-09-05)¶
Added GenServer class which maps incoming gen_server:calls into Python method calls.
Fixed bug with disconnection being not detected.
0.5 (2018-09-04)¶
Switchable backends created for Gevent and Asyncio.