jsonl.load_archive¶

Load multiple JSON Lines files from a ZIP or TAR archive incrementally.

Function Signature¶

jsonl.load_archive(
    file,
    *,
    pattern=None,
    pwd=None,
    opener=None,
    broken=False,
    cls=None,
    chunk_size=64 * 1024,
    **kwargs,
)

Parameters¶

Parameter	Type	Default	Description
`file`	`str`, `PathLike[str]`, URL, `Request`, file-like	(required)	Archive file to load from
`pattern`	`str` or `None`	`None`	Shell-style wildcard pattern; `None` selects the recognized JSON Lines suffixes automatically
`pwd`	`bytes` or `None`	`None`	Password to decrypt the archive (ZIP only)
`opener`	`Callable` or `None`	`None`	Custom function to open the file (not supported for URLs)
`broken`	`bool`	`False`	If `True`, skip malformed lines and log a warning
`chunk_size`	`int`	64 * 1024	The size (in bytes) of chunks when reading from a URL to avoid loading the entire file into memory at once.
`cls`	`type[json.JSONDecoder]` or `Callable` or `None`	`json.JSONDecoder`	Custom decoder
`**kwargs`			Keyword arguments used to pass the Custom decoder (`cls`)

Returns¶

Iterator[tuple[str, Iterator[Any]]] — An iterator of (filename, items) tuples, where items is an iterator of deserialized objects.

Supported Archive Formats¶

ZIP archives (.zip)
TAR archives (.tar), including compressed variants: .tar.gz, .tar.bz2, .tar.xz, .tar.zst (Python ≥ 3.14)

Key Features¶

Load from local files or remote URLs
Reject ambiguous raw bytes paths and PathLike objects returning bytes with a clear TypeError
Filter files inside the archive using Unix shell-style wildcards
Automatically discover these case-sensitive member suffixes: .jsonl, .ndjson, .jsonl.gz, .jsonl.bz2, .jsonl.xz, .ndjson.gz, .ndjson.bz2, and .ndjson.xz. On Python ≥ 3.14, .jsonl.zst and .ndjson.zst are also recognized.
Support for compressed .jsonl and .ndjson files inside the archive. Check compression detection for details.
ZIP archives with password protection
Graceful handling of malformed lines via the broken parameter

Examples¶

Load from a local archive¶

import jsonl

for filename, items in jsonl.load_archive("archive.zip"):
    print(f"--- {filename} ---")
    for item in items:
        print(item)

Load from a remote archive (URL)¶

You can load archives from a URL. For custom request headers, use urllib.request.Request:

import urllib.request
import jsonl

# Load directly from a URL
for filename, items in jsonl.load_archive("https://example.com/archive.zip"):
    print(f"--- {filename} ---")
    for item in items:
        print(item)

# Load using a custom request with headers
req = urllib.request.Request("https://example.com/archive.zip", headers={"Accept": "application/zip"})
for filename, items in jsonl.load_archive(req, chunk_size=128 * 1024):  # increase chunk size for larger files if needed
    print(f"--- {filename} ---")
    for item in items:
        print(item)

Filter files with pattern matching¶

Use Unix shell-style wildcards to select specific files within the archive:

Pattern	Matches
`*.jsonl`	All `.jsonl` files
`*.ndjson`	All `.ndjson` files
`*.jsonl.gz`	Gzip-compressed `.jsonl` files
`users*.jsonl`	Files starting with `users`
`data/[ab]*.jsonl`	Files in `data/` starting with `a` or `b`
`*`	All files

For more details, see the fnmatch documentation.

import jsonl

# Load only files matching a specific pattern
for filename, items in jsonl.load_archive("archive.zip", pattern="users*.jsonl"):
    print(f"--- {filename} ---")
    for item in items:
        print(item)