Serialization
serialization
Unified JSON-based serialization for LLMeter objects.
Most of the objects we'd like to save to and load from file in LLMeter are configuration-like and almost JSON-compatible, but with some extensions (like datetimes, binary image/etc payloads, and callback objects). Rather than falling back to Pickle (which is Python-specific, non-human-readable, and Python version fragile) - we implement in this module a JSON-based scheme for saving and loading compatible LLMeter objects.
A warning on security
load_object imports and instantiates whatever class path
is in the __llmeter_class__ field. Like unpickle, this has the potential to run arbitrary
code. Do not load configs from untrusted sources!
Key components
Serializable: Mixin to give any class an automatic state protocol (_get_llmeter_state/_set_llmeter_state) by introspecting__init__. Deliberately distinct from the pickle protocol sopickle/copy/deepcopykeep their native behavior.dump_objectandload_object: Full round-trip persistence forSerializable-compatible objects, using a{"__llmeter_class__": ..., "__llmeter_state__": ...}envelope.
Implementation details to be aware of
State vs. identity: We split an object's serialized form into two layers. The state
(_get_llmeter_state) contains the object's own field values and does not describe which class
those values belong to. The envelope built by
dump_object adds the identity (__llmeter_class__) around
that state. The "state" dictionaries handled by _get_llmeter_state / _set_llmeter_state are
not a fully self-describing representation of the object - only of its state. Keeping identity out
of the state dict avoids polluting the user's field namespace and lets nested polymorphic values
each carry their own envelope.
Serializable
Mixin providing a state extraction protocol compatible with LLMeter serialization.
Serialization in LLMeter uses a state extraction protocol somewhat similar to, but deliberately
separate from, the (__getstate__ / __setstate__) interface used by pickle, copy.copy,
and copy.deepcopy. Subclasses remain natively picklable/copyable, but can also
be saved to and loaded from LLMeter's JSON-based format.
This default implementation works with plain classes, @dataclass, and any class whose
__init__ parameters correspond to instance attributes (self.x or self._x). Nested
Serializable-like objects are recursively persisted and loaded via
dump_object and
load_object.
If your class needs more custom logic to represent and restore its state, customize the provided methods by overriding or implementing your own from scratch.
_get_llmeter_state
_get_llmeter_state()
Extract a JSON-serializable state dict by introspecting __init__ parameters.
Returns the object's constructor arguments (looked up as self.<name> or self._<name>),
recursively serialized. Note that:
- This is state only — it carries no class identity;
dump_objectwraps it with__llmeter_class__. - Any properties not exposed as
__init__arguments will not be persisted. Override your class' state get and set methods if you need different behaviour.
Source code in llmeter/serialization.py
205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 | |
_set_llmeter_state
_set_llmeter_state(state)
Restore this instance from a state dict produced by _get_llmeter_state.
Note this rebuilds the object by calling the constructor with the state as keyword
arguments. load_object first creates a bare instance
with __new__, then calls this method to populate it.
Source code in llmeter/serialization.py
229 230 231 232 233 234 235 236 237 | |
load_from_file
classmethod
load_from_file(path)
Load an object from a JSON file.
Detects the type from the __llmeter_class__ field and reconstructs it.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
path
|
ReadablePathLike
|
(Local or Cloud) path where the object was saved. |
required |
Returns: The loaded instance.
Source code in llmeter/serialization.py
258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 | |
save_to_file
save_to_file(path)
Save this object to a JSON file.
Uses the _get_llmeter_state protocol. Override _get_llmeter_state (not this method) if
custom JSON-based serialization is needed.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
path
|
WritablePathLike
|
(Local or Cloud) path where the object will be saved. |
required |
Returns:
| Type | Description |
|---|---|
UPath
|
The (validated/normalized) path the object was written to. |
Source code in llmeter/serialization.py
239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 | |
_deserialize_value
_deserialize_value(val)
Recursively restore a value from JSON persistence.
Recognizes type-tagged dicts (__llmeter_class__), bytes markers (__llmeter_bytes__),
ISO-8601 datetime strings, and recursively processes nested dicts and lists.
Source code in llmeter/serialization.py
398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 | |
_get_type_args
_get_type_args(tp)
Return the members of a union type (e.g. datetime | None -> (datetime, NoneType)).
Source code in llmeter/serialization.py
141 142 143 144 145 146 147 148 | |
_serialize_value
_serialize_value(val)
Recursively prepare a value for JSON persistence.
Handles primitives, known types (bytes, datetime, PathLike), nested
Serializable objects (via
dump_object), dicts, and lists/tuples.
Raises:
| Type | Description |
|---|---|
TypeError
|
for objects it cannot serialize. |
Source code in llmeter/serialization.py
368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 | |
bytes_decoder
bytes_decoder(dct)
Decode __llmeter_bytes__ marker objects back to Python bytes.
Intended for use as the object_hook argument to json.load or json.loads. Marker objects
produced by json_default are detected and converted
back to bytes; all other dicts pass through unchanged.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
dct
|
dict
|
A dictionary produced by the JSON parser. |
required |
Returns:
| Type | Description |
|---|---|
dict | bytes
|
The original |
Source code in llmeter/serialization.py
123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 | |
datetime_to_str
datetime_to_str(dt)
Convert a datetime to a UTC ISO-8601 string with Z suffix.
Timezone-aware datetimes are converted to UTC first. Naive datetimes are serialized as-is (assumed UTC).
Source code in llmeter/serialization.py
66 67 68 69 70 71 72 73 74 | |
dump_object
dump_object(obj)
Serialize an object to a type-tagged dict for round-trip persistence.
The returned envelope has the form
{"__llmeter_class__": "module.Class", "__llmeter_state__": {...}}.
Serialization strategy (checked in order):
- If the object implements the LLMeter state protocol (as in
Serializable._get_llmeter_state) this will call it to obtain the state dict. - If the object is a dataclass, uses
dataclasses.asdict. - Otherwise, takes all public (non-underscore-prefixed) entries from
__dict__.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
obj
|
Any
|
The object to serialize. |
required |
Returns:
| Name | Type | Description |
|---|---|---|
dct |
dict
|
A JSON-serializable dict that |
Source code in llmeter/serialization.py
280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 | |
json_default
json_default(obj)
Serialize a single non-natively-JSON-serializable object.
Intended for use as the default argument to json.dump or json.dumps. This does not
handle full recursive LLMeter serialization - see
dump_object instead.
Type handling (checked in order):
bytes— wrapped in a{"__llmeter_bytes__": "<base64>"}marker so thatbytes_decodercan restore them on the way back.datetime— converted to a UTC ISO-8601 string with aZsuffix.date/time— converted via.isoformat().os.PathLike— converted to a POSIX path string.- Anything else —
str()fallback (returnsNoneif that also fails).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
obj
|
Any
|
The object that the default JSON encoder could not handle. |
required |
Returns:
| Type | Description |
|---|---|
Any
|
A JSON-serializable representation of obj. |
Source code in llmeter/serialization.py
87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 | |
load_object
load_object(data)
Restore an object from a type-tagged dict produced by dump_object.
Warning
This method imports and instantiates class paths specified by the input data, which (like pickle) can enable arbitrary running arbitrary code. Do not run it on data from unstrusted sources!
Imports the module identified by __llmeter_class__, instantiates the class (bypassing
__init__ via __new__), and restores its state via
Serializable._set_llmeter_state.
Objects that do not implement the protocol are restored by assigning the (deserialized) state
onto their __dict__.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
data
|
dict
|
A dict with |
required |
Returns:
| Type | Description |
|---|---|
Any
|
The reconstructed object instance. |
Source code in llmeter/serialization.py
313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 | |
restore_dataclass_types
restore_dataclass_types(cls, data)
Restore typed fields in a dict destined for a dataclass constructor.
Introspects cls (a dataclass) and converts JSON-native values back to their annotated Python
types. Currently handles:
datetimefields — parses ISO-8601 strings viastr_to_datetime.bytesfields — decodes__llmeter_bytes__markers via base64.
Only fields declared on cls are touched — nested user payloads (e.g. input_payload) are
left unchanged. Mutates data in place.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
cls
|
type
|
A dataclass type to introspect for field type annotations. |
required |
data
|
dict
|
A dictionary of field values (e.g. from |
required |
Source code in llmeter/serialization.py
151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 | |
str_to_datetime
str_to_datetime(s)
Parse an ISO-8601 string (with optional Z suffix) to a datetime.
Source code in llmeter/serialization.py
77 78 79 | |