Known differences¶
Where this codec knowingly behaves differently from OTP 29’s
binary_to_term/term_to_binary, or the Python and native codecs differ
from each other. One bullet each: what, an example, why. Remove a bullet
when the difference is gone.
From OTP 29¶
Resource limits. Decoding and encoding refuse nesting deeper than
max_depth(4096); a compressed term may declare at mostmax_size(256 MiB). OTP has no such limits. Example: 4097 nested lists,<<131, (<<108,0,0,0,1>>)*4097, 106*4098>>→PyCodecError. Why: input comes from the network (TERM-09, TERM-22).References with no ID words are rejected. OTP 29 reads the first word from past the end of the term, so it accepts or rejects depending on the bytes that follow. Example:
<<131,90,0,0,119,3,"a@b",0,0,0,1>>(NEWER_REFERENCE_EXT),<<131,114,0,0,119,3,"a@b",1>>(NEW_REFERENCE_EXT) →PyCodecError(TERM-11).FLOAT_EXTneeds a NUL inside its 31 bytes. OTP reads past the term to find one. Example:<<131,99,"1.",$0*29>>→PyCodecError(TERM-14).LOCAL_EXTis never decoded. OTP decodes it on the node that wrote it. Example:term_to_binary({a,1}, [local])→PyCodecErrornamingLOCAL_EXT(TERM-16).ATOM_CACHE_REF(82) is decoded wherever an atom may stand, given the decode optionatom_cache. OTP accepts it only in terms that follow a distribution header. Example:binary_to_term(<<131,82,0>>, {"atom_cache": ["a"]})→Atom("a"); without the option →PyCodecError, as OTP’sbadarg(TERM-18).Map keys Python cannot hash are rejected. Example:
#{[a] => 1}→PyCodecError(“Map key of type list is not hashable”).Maps with keys that are equal in Python are rejected. Erlang’s
1,1.0andtrueare three keys; in Python1 == 1.0 == True, and a dict would drop entries. Example:#{1 => a, 1.0 => b}(<<131,116,0,0,0,2,97,1,119,1,97,70,63,240,0,0,0,0,0,0,119,1,98>>) →PyCodecError(“Map key of type float equals an earlier key of the map in Python”). The same for0,0.0,-0.0(different keys since OTP 27),false, an atom and a string of the same text (abcand"abc":Atomis astr; not with the optionatom: "StrictAtom"), and tuples of these ({1}and{1.0}). A decode hook that maps two keys to one value is refused the same way. Why: a message that silently loses values is worse than one that is refused (TERM-23).Some values change type on the way through Python.
true,false,undefined↔True,False,None;"abc"(STRING_EXT) →str, but[97,256,98]→list,"aĀb"encodes as that list;[]and""both encode asNIL_EXTand decode as[]. The bytes re-encode the same; the Python types differ.Bitstrings decode to
(bytes, bits)by default, which encodes back as a tuple, not a bitstring. Example:<<5:7>>→(b"\n", 7); pass{"bitstring": "BitString"}to get a value that round-trips (TERM-20).Funs keep their creator pid. OTP 23+ drops it on decoding and writes a dummy one; we decode it and send it back unchanged.
Fun.__eq__ignores it, as Erlang’s=:=does (TERM-13).We encode only OTP 29’s default output. Never
FLOAT_EXT,ATOM_EXT/SMALL_ATOM_EXT,NEW_PORT_EXT/PORT_EXT,REFERENCE_EXT, compressed terms or anyterm_to_binaryoption. Example: a port decoded fromNEW_PORT_EXTre-encodes asV4_PORT_EXT, as OTP 29 does.Records are sent to any peer. OTP before 29 cannot decode
RECORD_EXT; encoding aRecordfor such a peer gives bytes it rejects (TERM-17).Older OTP releases disagree on one bad input; we follow OTP 29.
<<131,77,0,0,0,0,0>>(0 bits in 0 bytes): OTP 25/26 decode<<>>, OTP 27 and 29 reject it, OTP 28.4.1 aborts the VM; we reject it.
Python codec vs native codec¶
Encode hook exceptions. Python lets an exception from
encode_hook,catch_all,__etf__or the generic object encoding escape unchanged; native raisesPyCodecErrorwith it as__cause__. Examples:term_to_binary(object(), {"encode_hook": {"catch_all": f}})withfraisingValueError;term_to_binary(bytearray(b"x"))(AttributeError: no__dict__).