[{"data":1,"prerenderedAt":4053},["ShallowReactive",2],{"page-\u002Fadvanced-input-parsing-user-experience\u002Fworking-with-stdin-stdout-and-pipes\u002Fprocessing-large-files-and-ndjson-streams\u002F":3,"content-directory":3209},{"id":4,"title":5,"body":6,"date":3193,"description":3194,"difficulty":3195,"draft":3196,"extension":3197,"meta":3198,"navigation":152,"path":3199,"seo":3200,"stem":3201,"tags":3202,"updated":3193,"__hash__":3208},"content\u002Fadvanced-input-parsing-user-experience\u002Fworking-with-stdin-stdout-and-pipes\u002Fprocessing-large-files-and-ndjson-streams\u002Findex.md","Processing Large Files and NDJSON Streams in Python CLIs",{"type":7,"value":8,"toc":3174},"minimark",[9,48,53,67,71,75,86,89,92,96,111,1232,1239,1789,1792,1881,1886,1904,1914,1933,1942,1948,1952,1955,1970,1974,2038,2042,2049,3012,3026,3030,3045,3049,3053,3067,3071,3084,3088,3099,3103,3121,3125,3137,3141,3170],[10,11,12,13,17,18,21,22,25,26,29,30,33,34,37,38,41,42,47],"p",{},"Event exports, audit logs, API dumps and log archives increasingly arrive as NDJSON — newline-delimited JSON, one object per line, often gzipped and often far larger than the laptop it is being inspected on. A CLI that handles them with ",[14,15,16],"code",{},"json.load(f)"," or ",[14,19,20],{},"f.read().splitlines()"," works on the sample file and then dies on the real one, taking several times the file's size in memory before it prints anything. The alternative costs nothing in code: read one line, handle it, write the result, repeat. Memory stays flat whatever the input size, the first results appear immediately, and the command composes with ",[14,23,24],{},"head",", ",[14,27,28],{},"grep"," and ",[14,31,32],{},"jq"," in a pipeline. This guide builds a small ",[14,35,36],{},"filter"," command that streams NDJSON from files, ",[14,39,40],{},".gz"," archives or stdin, reports malformed lines precisely, offers an explicit lenient mode, and proves with a test that its memory use does not grow with the input. It belongs to the ",[43,44,46],"a",{"href":45},"\u002Fadvanced-input-parsing-user-experience\u002Fworking-with-stdin-stdout-and-pipes\u002F","working with stdin, stdout and pipes topic",".",[49,50,52],"h2",{"id":51},"prerequisites","Prerequisites",[54,55,56,60],"ul",{},[57,58,59],"li",{},"Python 3.10+ and a Typer or Click CLI.",[57,61,62,63,47],{},"Familiarity with ",[43,64,66],{"href":65},"\u002Fadvanced-input-parsing-user-experience\u002Fworking-with-stdin-stdout-and-pipes\u002Freading-piped-input-in-python-clis\u002F","reading piped input in Python CLIs",[49,68,70],{"id":69},"stream-do-not-load","Stream, do not load",[72,73],"inline-diagram",{"name":74},"nd-stream",[10,76,77,78,81,82,85],{},"A Python file object is an iterator over lines, reading from disk in buffered chunks. ",[14,79,80],{},"for line in fh"," therefore holds one line in memory at a time, however big the file. Parse it with ",[14,83,84],{},"json.loads",", decide whether it matches, write the result straight to stdout, and let the object go. Every stage of the command is a generator or a loop, and nothing ever builds a list of all records.",[72,87],{"name":88},"nd-bars",[10,90,91],{},"The difference is not a percentage but orders of magnitude. Loading a file as text costs its size again as a Python string; parsing everything into dictionaries costs several times more, because every key, value and dictionary carries object overhead. Streaming costs the size of the longest line plus buffers. The test at the end of this guide measures it: a peak of about 25 KB while filtering an 18 MB file.",[49,93,95],{"id":94},"the-recipe","The recipe",[10,97,98,99,102,103,106,107,110],{},"The streaming logic lives in a module that knows nothing about the CLI framework. ",[14,100,101],{},"open_input"," handles the three kinds of input, ",[14,104,105],{},"records"," parses lines lazily and deals with bad ones, and ",[14,108,109],{},"filter_records"," connects them to an output stream:",[112,113,118],"pre",{"className":114,"code":115,"language":116,"meta":117,"style":117},"language-python shiki shiki-themes github-light github-dark","# src\u002Fmytool\u002Fstream.py\nfrom __future__ import annotations\n\nimport gzip\nimport io\nimport json\nimport sys\nfrom collections.abc import Iterator\nfrom contextlib import contextmanager\nfrom dataclasses import dataclass, field\nfrom pathlib import Path\nfrom typing import IO, Any\n\n\nclass BadRecord(Exception):\n    def __init__(self, source: str, line_no: int, reason: str) -> None:\n        super().__init__(f\"{source}:{line_no}: {reason}\")\n        self.source, self.line_no, self.reason = source, line_no, reason\n\n\n@dataclass\nclass Stats:\n    read: int = 0\n    written: int = 0\n    skipped: int = 0\n    first_errors: list[str] = field(default_factory=list)\n\n\n@contextmanager\ndef open_input(name: str) -> Iterator[IO[str]]:\n    \"\"\"'-' is stdin; .gz files are decompressed on the fly. Text, UTF-8, read lazily.\"\"\"\n    if name == \"-\":\n        fh = io.TextIOWrapper(sys.stdin.buffer, encoding=\"utf-8\")\n        try:\n            yield fh\n        finally:\n            fh.detach()                     # leave the real stdin open\n    elif name.endswith(\".gz\"):\n        with gzip.open(name, \"rt\", encoding=\"utf-8\") as fh:\n            yield fh\n    else:\n        with Path(name).open(encoding=\"utf-8\") as fh:\n            yield fh\n\n\ndef records(fh: IO[str], source: str, *, skip_invalid: bool, stats: Stats) -> Iterator[dict[str, Any]]:\n    for line_no, line in enumerate(fh, start=1):\n        if not line.strip():\n            continue\n        stats.read += 1\n        try:\n            obj = json.loads(line)\n            if not isinstance(obj, dict):\n                raise ValueError(\"expected a JSON object\")\n        except ValueError as exc:          # json.JSONDecodeError is a ValueError\n            if not skip_invalid:\n                raise BadRecord(source, line_no, str(exc)) from None\n            stats.skipped += 1\n            if len(stats.first_errors) \u003C 5:\n                stats.first_errors.append(f\"{source}:{line_no}: {exc}\")\n            continue\n        yield obj\n\n\ndef filter_records(inputs: list[str], out: IO[str], *, where: dict[str, Any],\n                   fields: list[str] | None, skip_invalid: bool) -> Stats:\n    stats = Stats()\n    for name in inputs:\n        with open_input(name) as fh:\n            for obj in records(fh, name, skip_invalid=skip_invalid, stats=stats):\n                if all(obj.get(k) == v for k, v in where.items()):\n                    if fields:\n                        obj = {k: obj.get(k) for k in fields}\n                    out.write(json.dumps(obj, separators=(\",\", \":\")) + \"\\n\")\n                    stats.written += 1\n    return stats\n","python","",[14,119,120,129,147,154,163,171,179,187,200,213,226,239,255,260,265,284,319,374,400,405,410,416,426,440,452,464,491,496,501,507,535,541,558,579,587,596,604,613,627,656,663,671,691,698,703,708,749,777,789,795,807,814,825,844,860,877,887,905,915,934,971,976,985,990,995,1030,1053,1064,1076,1088,1118,1146,1155,1176,1213,1223],{"__ignoreMap":117},[121,122,125],"span",{"class":123,"line":124},"line",1,[121,126,128],{"class":127},"sJ8bj","# src\u002Fmytool\u002Fstream.py\n",[121,130,132,136,140,143],{"class":123,"line":131},2,[121,133,135],{"class":134},"szBVR","from",[121,137,139],{"class":138},"sj4cs"," __future__",[121,141,142],{"class":134}," import",[121,144,146],{"class":145},"sVt8B"," annotations\n",[121,148,150],{"class":123,"line":149},3,[121,151,153],{"emptyLinePlaceholder":152},true,"\n",[121,155,157,160],{"class":123,"line":156},4,[121,158,159],{"class":134},"import",[121,161,162],{"class":145}," gzip\n",[121,164,166,168],{"class":123,"line":165},5,[121,167,159],{"class":134},[121,169,170],{"class":145}," io\n",[121,172,174,176],{"class":123,"line":173},6,[121,175,159],{"class":134},[121,177,178],{"class":145}," json\n",[121,180,182,184],{"class":123,"line":181},7,[121,183,159],{"class":134},[121,185,186],{"class":145}," sys\n",[121,188,190,192,195,197],{"class":123,"line":189},8,[121,191,135],{"class":134},[121,193,194],{"class":145}," collections.abc ",[121,196,159],{"class":134},[121,198,199],{"class":145}," Iterator\n",[121,201,203,205,208,210],{"class":123,"line":202},9,[121,204,135],{"class":134},[121,206,207],{"class":145}," contextlib ",[121,209,159],{"class":134},[121,211,212],{"class":145}," contextmanager\n",[121,214,216,218,221,223],{"class":123,"line":215},10,[121,217,135],{"class":134},[121,219,220],{"class":145}," dataclasses ",[121,222,159],{"class":134},[121,224,225],{"class":145}," dataclass, field\n",[121,227,229,231,234,236],{"class":123,"line":228},11,[121,230,135],{"class":134},[121,232,233],{"class":145}," pathlib ",[121,235,159],{"class":134},[121,237,238],{"class":145}," Path\n",[121,240,242,244,247,249,252],{"class":123,"line":241},12,[121,243,135],{"class":134},[121,245,246],{"class":145}," typing ",[121,248,159],{"class":134},[121,250,251],{"class":138}," IO",[121,253,254],{"class":145},", Any\n",[121,256,258],{"class":123,"line":257},13,[121,259,153],{"emptyLinePlaceholder":152},[121,261,263],{"class":123,"line":262},14,[121,264,153],{"emptyLinePlaceholder":152},[121,266,268,271,275,278,281],{"class":123,"line":267},15,[121,269,270],{"class":134},"class",[121,272,274],{"class":273},"sScJk"," BadRecord",[121,276,277],{"class":145},"(",[121,279,280],{"class":138},"Exception",[121,282,283],{"class":145},"):\n",[121,285,287,290,293,296,299,302,305,308,310,313,316],{"class":123,"line":286},16,[121,288,289],{"class":134},"    def",[121,291,292],{"class":138}," __init__",[121,294,295],{"class":145},"(self, source: ",[121,297,298],{"class":138},"str",[121,300,301],{"class":145},", line_no: ",[121,303,304],{"class":138},"int",[121,306,307],{"class":145},", reason: ",[121,309,298],{"class":138},[121,311,312],{"class":145},") -> ",[121,314,315],{"class":138},"None",[121,317,318],{"class":145},":\n",[121,320,322,325,328,331,333,336,340,343,346,349,352,354,357,359,362,364,367,369,371],{"class":123,"line":321},17,[121,323,324],{"class":138},"        super",[121,326,327],{"class":145},"().",[121,329,330],{"class":138},"__init__",[121,332,277],{"class":145},[121,334,335],{"class":134},"f",[121,337,339],{"class":338},"sZZnC","\"",[121,341,342],{"class":138},"{",[121,344,345],{"class":145},"source",[121,347,348],{"class":138},"}",[121,350,351],{"class":338},":",[121,353,342],{"class":138},[121,355,356],{"class":145},"line_no",[121,358,348],{"class":138},[121,360,361],{"class":338},": ",[121,363,342],{"class":138},[121,365,366],{"class":145},"reason",[121,368,348],{"class":138},[121,370,339],{"class":338},[121,372,373],{"class":145},")\n",[121,375,377,380,383,386,389,391,394,397],{"class":123,"line":376},18,[121,378,379],{"class":138},"        self",[121,381,382],{"class":145},".source, ",[121,384,385],{"class":138},"self",[121,387,388],{"class":145},".line_no, ",[121,390,385],{"class":138},[121,392,393],{"class":145},".reason ",[121,395,396],{"class":134},"=",[121,398,399],{"class":145}," source, line_no, reason\n",[121,401,403],{"class":123,"line":402},19,[121,404,153],{"emptyLinePlaceholder":152},[121,406,408],{"class":123,"line":407},20,[121,409,153],{"emptyLinePlaceholder":152},[121,411,413],{"class":123,"line":412},21,[121,414,415],{"class":273},"@dataclass\n",[121,417,419,421,424],{"class":123,"line":418},22,[121,420,270],{"class":134},[121,422,423],{"class":273}," Stats",[121,425,318],{"class":145},[121,427,429,432,434,437],{"class":123,"line":428},23,[121,430,431],{"class":145},"    read: ",[121,433,304],{"class":138},[121,435,436],{"class":134}," =",[121,438,439],{"class":138}," 0\n",[121,441,443,446,448,450],{"class":123,"line":442},24,[121,444,445],{"class":145},"    written: ",[121,447,304],{"class":138},[121,449,436],{"class":134},[121,451,439],{"class":138},[121,453,455,458,460,462],{"class":123,"line":454},25,[121,456,457],{"class":145},"    skipped: ",[121,459,304],{"class":138},[121,461,436],{"class":134},[121,463,439],{"class":138},[121,465,467,470,472,475,477,480,484,486,489],{"class":123,"line":466},26,[121,468,469],{"class":145},"    first_errors: list[",[121,471,298],{"class":138},[121,473,474],{"class":145},"] ",[121,476,396],{"class":134},[121,478,479],{"class":145}," field(",[121,481,483],{"class":482},"s4XuR","default_factory",[121,485,396],{"class":134},[121,487,488],{"class":138},"list",[121,490,373],{"class":145},[121,492,494],{"class":123,"line":493},27,[121,495,153],{"emptyLinePlaceholder":152},[121,497,499],{"class":123,"line":498},28,[121,500,153],{"emptyLinePlaceholder":152},[121,502,504],{"class":123,"line":503},29,[121,505,506],{"class":273},"@contextmanager\n",[121,508,510,513,516,519,521,524,527,530,532],{"class":123,"line":509},30,[121,511,512],{"class":134},"def",[121,514,515],{"class":273}," open_input",[121,517,518],{"class":145},"(name: ",[121,520,298],{"class":138},[121,522,523],{"class":145},") -> Iterator[",[121,525,526],{"class":138},"IO",[121,528,529],{"class":145},"[",[121,531,298],{"class":138},[121,533,534],{"class":145},"]]:\n",[121,536,538],{"class":123,"line":537},31,[121,539,540],{"class":338},"    \"\"\"'-' is stdin; .gz files are decompressed on the fly. Text, UTF-8, read lazily.\"\"\"\n",[121,542,544,547,550,553,556],{"class":123,"line":543},32,[121,545,546],{"class":134},"    if",[121,548,549],{"class":145}," name ",[121,551,552],{"class":134},"==",[121,554,555],{"class":338}," \"-\"",[121,557,318],{"class":145},[121,559,561,564,566,569,572,574,577],{"class":123,"line":560},33,[121,562,563],{"class":145},"        fh ",[121,565,396],{"class":134},[121,567,568],{"class":145}," io.TextIOWrapper(sys.stdin.buffer, ",[121,570,571],{"class":482},"encoding",[121,573,396],{"class":134},[121,575,576],{"class":338},"\"utf-8\"",[121,578,373],{"class":145},[121,580,582,585],{"class":123,"line":581},34,[121,583,584],{"class":134},"        try",[121,586,318],{"class":145},[121,588,590,593],{"class":123,"line":589},35,[121,591,592],{"class":134},"            yield",[121,594,595],{"class":145}," fh\n",[121,597,599,602],{"class":123,"line":598},36,[121,600,601],{"class":134},"        finally",[121,603,318],{"class":145},[121,605,607,610],{"class":123,"line":606},37,[121,608,609],{"class":145},"            fh.detach()                     ",[121,611,612],{"class":127},"# leave the real stdin open\n",[121,614,616,619,622,625],{"class":123,"line":615},38,[121,617,618],{"class":134},"    elif",[121,620,621],{"class":145}," name.endswith(",[121,623,624],{"class":338},"\".gz\"",[121,626,283],{"class":145},[121,628,630,633,636,639,641,643,645,647,650,653],{"class":123,"line":629},39,[121,631,632],{"class":134},"        with",[121,634,635],{"class":145}," gzip.open(name, ",[121,637,638],{"class":338},"\"rt\"",[121,640,25],{"class":145},[121,642,571],{"class":482},[121,644,396],{"class":134},[121,646,576],{"class":338},[121,648,649],{"class":145},") ",[121,651,652],{"class":134},"as",[121,654,655],{"class":145}," fh:\n",[121,657,659,661],{"class":123,"line":658},40,[121,660,592],{"class":134},[121,662,595],{"class":145},[121,664,666,669],{"class":123,"line":665},41,[121,667,668],{"class":134},"    else",[121,670,318],{"class":145},[121,672,674,676,679,681,683,685,687,689],{"class":123,"line":673},42,[121,675,632],{"class":134},[121,677,678],{"class":145}," Path(name).open(",[121,680,571],{"class":482},[121,682,396],{"class":134},[121,684,576],{"class":338},[121,686,649],{"class":145},[121,688,652],{"class":134},[121,690,655],{"class":145},[121,692,694,696],{"class":123,"line":693},43,[121,695,592],{"class":134},[121,697,595],{"class":145},[121,699,701],{"class":123,"line":700},44,[121,702,153],{"emptyLinePlaceholder":152},[121,704,706],{"class":123,"line":705},45,[121,707,153],{"emptyLinePlaceholder":152},[121,709,711,713,716,719,721,723,725,728,730,732,735,738,741,744,746],{"class":123,"line":710},46,[121,712,512],{"class":134},[121,714,715],{"class":273}," records",[121,717,718],{"class":145},"(fh: ",[121,720,526],{"class":138},[121,722,529],{"class":145},[121,724,298],{"class":138},[121,726,727],{"class":145},"], source: ",[121,729,298],{"class":138},[121,731,25],{"class":145},[121,733,734],{"class":134},"*",[121,736,737],{"class":145},", skip_invalid: ",[121,739,740],{"class":138},"bool",[121,742,743],{"class":145},", stats: Stats) -> Iterator[dict[",[121,745,298],{"class":138},[121,747,748],{"class":145},", Any]]:\n",[121,750,752,755,758,761,764,767,770,772,775],{"class":123,"line":751},47,[121,753,754],{"class":134},"    for",[121,756,757],{"class":145}," line_no, line ",[121,759,760],{"class":134},"in",[121,762,763],{"class":138}," enumerate",[121,765,766],{"class":145},"(fh, ",[121,768,769],{"class":482},"start",[121,771,396],{"class":134},[121,773,774],{"class":138},"1",[121,776,283],{"class":145},[121,778,780,783,786],{"class":123,"line":779},48,[121,781,782],{"class":134},"        if",[121,784,785],{"class":134}," not",[121,787,788],{"class":145}," line.strip():\n",[121,790,792],{"class":123,"line":791},49,[121,793,794],{"class":134},"            continue\n",[121,796,798,801,804],{"class":123,"line":797},50,[121,799,800],{"class":145},"        stats.read ",[121,802,803],{"class":134},"+=",[121,805,806],{"class":138}," 1\n",[121,808,810,812],{"class":123,"line":809},51,[121,811,584],{"class":134},[121,813,318],{"class":145},[121,815,817,820,822],{"class":123,"line":816},52,[121,818,819],{"class":145},"            obj ",[121,821,396],{"class":134},[121,823,824],{"class":145}," json.loads(line)\n",[121,826,828,831,833,836,839,842],{"class":123,"line":827},53,[121,829,830],{"class":134},"            if",[121,832,785],{"class":134},[121,834,835],{"class":138}," isinstance",[121,837,838],{"class":145},"(obj, ",[121,840,841],{"class":138},"dict",[121,843,283],{"class":145},[121,845,847,850,853,855,858],{"class":123,"line":846},54,[121,848,849],{"class":134},"                raise",[121,851,852],{"class":138}," ValueError",[121,854,277],{"class":145},[121,856,857],{"class":338},"\"expected a JSON object\"",[121,859,373],{"class":145},[121,861,863,866,868,871,874],{"class":123,"line":862},55,[121,864,865],{"class":134},"        except",[121,867,852],{"class":138},[121,869,870],{"class":134}," as",[121,872,873],{"class":145}," exc:          ",[121,875,876],{"class":127},"# json.JSONDecodeError is a ValueError\n",[121,878,880,882,884],{"class":123,"line":879},56,[121,881,830],{"class":134},[121,883,785],{"class":134},[121,885,886],{"class":145}," skip_invalid:\n",[121,888,890,892,895,897,900,902],{"class":123,"line":889},57,[121,891,849],{"class":134},[121,893,894],{"class":145}," BadRecord(source, line_no, ",[121,896,298],{"class":138},[121,898,899],{"class":145},"(exc)) ",[121,901,135],{"class":134},[121,903,904],{"class":138}," None\n",[121,906,908,911,913],{"class":123,"line":907},58,[121,909,910],{"class":145},"            stats.skipped ",[121,912,803],{"class":134},[121,914,806],{"class":138},[121,916,918,920,923,926,929,932],{"class":123,"line":917},59,[121,919,830],{"class":134},[121,921,922],{"class":138}," len",[121,924,925],{"class":145},"(stats.first_errors) ",[121,927,928],{"class":134},"\u003C",[121,930,931],{"class":138}," 5",[121,933,318],{"class":145},[121,935,937,940,942,944,946,948,950,952,954,956,958,960,962,965,967,969],{"class":123,"line":936},60,[121,938,939],{"class":145},"                stats.first_errors.append(",[121,941,335],{"class":134},[121,943,339],{"class":338},[121,945,342],{"class":138},[121,947,345],{"class":145},[121,949,348],{"class":138},[121,951,351],{"class":338},[121,953,342],{"class":138},[121,955,356],{"class":145},[121,957,348],{"class":138},[121,959,361],{"class":338},[121,961,342],{"class":138},[121,963,964],{"class":145},"exc",[121,966,348],{"class":138},[121,968,339],{"class":338},[121,970,373],{"class":145},[121,972,974],{"class":123,"line":973},61,[121,975,794],{"class":134},[121,977,979,982],{"class":123,"line":978},62,[121,980,981],{"class":134},"        yield",[121,983,984],{"class":145}," obj\n",[121,986,988],{"class":123,"line":987},63,[121,989,153],{"emptyLinePlaceholder":152},[121,991,993],{"class":123,"line":992},64,[121,994,153],{"emptyLinePlaceholder":152},[121,996,998,1000,1003,1006,1008,1011,1013,1015,1017,1020,1022,1025,1027],{"class":123,"line":997},65,[121,999,512],{"class":134},[121,1001,1002],{"class":273}," filter_records",[121,1004,1005],{"class":145},"(inputs: list[",[121,1007,298],{"class":138},[121,1009,1010],{"class":145},"], out: ",[121,1012,526],{"class":138},[121,1014,529],{"class":145},[121,1016,298],{"class":138},[121,1018,1019],{"class":145},"], ",[121,1021,734],{"class":134},[121,1023,1024],{"class":145},", where: dict[",[121,1026,298],{"class":138},[121,1028,1029],{"class":145},", Any],\n",[121,1031,1033,1036,1038,1040,1043,1046,1048,1050],{"class":123,"line":1032},66,[121,1034,1035],{"class":145},"                   fields: list[",[121,1037,298],{"class":138},[121,1039,474],{"class":145},[121,1041,1042],{"class":134},"|",[121,1044,1045],{"class":138}," None",[121,1047,737],{"class":145},[121,1049,740],{"class":138},[121,1051,1052],{"class":145},") -> Stats:\n",[121,1054,1056,1059,1061],{"class":123,"line":1055},67,[121,1057,1058],{"class":145},"    stats ",[121,1060,396],{"class":134},[121,1062,1063],{"class":145}," Stats()\n",[121,1065,1067,1069,1071,1073],{"class":123,"line":1066},68,[121,1068,754],{"class":134},[121,1070,549],{"class":145},[121,1072,760],{"class":134},[121,1074,1075],{"class":145}," inputs:\n",[121,1077,1079,1081,1084,1086],{"class":123,"line":1078},69,[121,1080,632],{"class":134},[121,1082,1083],{"class":145}," open_input(name) ",[121,1085,652],{"class":134},[121,1087,655],{"class":145},[121,1089,1091,1094,1097,1099,1102,1105,1107,1110,1113,1115],{"class":123,"line":1090},70,[121,1092,1093],{"class":134},"            for",[121,1095,1096],{"class":145}," obj ",[121,1098,760],{"class":134},[121,1100,1101],{"class":145}," records(fh, name, ",[121,1103,1104],{"class":482},"skip_invalid",[121,1106,396],{"class":134},[121,1108,1109],{"class":145},"skip_invalid, ",[121,1111,1112],{"class":482},"stats",[121,1114,396],{"class":134},[121,1116,1117],{"class":145},"stats):\n",[121,1119,1121,1124,1127,1130,1132,1135,1138,1141,1143],{"class":123,"line":1120},71,[121,1122,1123],{"class":134},"                if",[121,1125,1126],{"class":138}," all",[121,1128,1129],{"class":145},"(obj.get(k) ",[121,1131,552],{"class":134},[121,1133,1134],{"class":145}," v ",[121,1136,1137],{"class":134},"for",[121,1139,1140],{"class":145}," k, v ",[121,1142,760],{"class":134},[121,1144,1145],{"class":145}," where.items()):\n",[121,1147,1149,1152],{"class":123,"line":1148},72,[121,1150,1151],{"class":134},"                    if",[121,1153,1154],{"class":145}," fields:\n",[121,1156,1158,1161,1163,1166,1168,1171,1173],{"class":123,"line":1157},73,[121,1159,1160],{"class":145},"                        obj ",[121,1162,396],{"class":134},[121,1164,1165],{"class":145}," {k: obj.get(k) ",[121,1167,1137],{"class":134},[121,1169,1170],{"class":145}," k ",[121,1172,760],{"class":134},[121,1174,1175],{"class":145}," fields}\n",[121,1177,1179,1182,1185,1187,1189,1192,1194,1197,1200,1203,1206,1209,1211],{"class":123,"line":1178},74,[121,1180,1181],{"class":145},"                    out.write(json.dumps(obj, ",[121,1183,1184],{"class":482},"separators",[121,1186,396],{"class":134},[121,1188,277],{"class":145},[121,1190,1191],{"class":338},"\",\"",[121,1193,25],{"class":145},[121,1195,1196],{"class":338},"\":\"",[121,1198,1199],{"class":145},")) ",[121,1201,1202],{"class":134},"+",[121,1204,1205],{"class":338}," \"",[121,1207,1208],{"class":138},"\\n",[121,1210,339],{"class":338},[121,1212,373],{"class":145},[121,1214,1216,1219,1221],{"class":123,"line":1215},75,[121,1217,1218],{"class":145},"                    stats.written ",[121,1220,803],{"class":134},[121,1222,806],{"class":138},[121,1224,1226,1229],{"class":123,"line":1225},76,[121,1227,1228],{"class":134},"    return",[121,1230,1231],{"class":145}," stats\n",[10,1233,1234,1235,1238],{},"The command is a thin wrapper that parses options, calls the streaming function with ",[14,1236,1237],{},"sys.stdout",", and turns problems into messages and exit codes:",[112,1240,1242],{"className":114,"code":1241,"language":116,"meta":117,"style":117},"# src\u002Fmytool\u002Fcli.py\nfrom __future__ import annotations\n\nimport sys\nfrom typing import Annotated\n\nimport typer\n\nfrom mytool.stream import BadRecord, filter_records\n\napp = typer.Typer()\n\n\n@app.callback()\ndef main() -> None:\n    \"\"\"Work with NDJSON event files.\"\"\"\n\n\n@app.command(\"filter\")\ndef filter_cmd(\n    inputs: Annotated[list[str], typer.Argument(help=\"NDJSON files (.gz ok); '-' for stdin.\")] = None,\n    where: Annotated[list[str], typer.Option(\"--where\", help=\"KEY=VALUE to match.\")] = None,\n    fields: Annotated[str, typer.Option(\"--fields\", help=\"Comma-separated keys to keep.\")] = \"\",\n    skip_invalid: Annotated[bool, typer.Option(\"--skip-invalid\", help=\"Skip malformed lines.\")] = False,\n) -> None:\n    \"\"\"Stream matching records to stdout, one JSON object per line.\"\"\"\n    conditions = dict(w.split(\"=\", 1) for w in (where or []))\n    try:\n        stats = filter_records(inputs or [\"-\"], sys.stdout, where=conditions,\n                               fields=[f for f in fields.split(\",\") if f] or None,\n                               skip_invalid=skip_invalid)\n    except BadRecord as exc:\n        typer.echo(f\"error: {exc} (use --skip-invalid to skip bad lines)\", err=True)\n        raise typer.Exit(65) from None          # EX_DATAERR\n    if stats.skipped:\n        typer.echo(f\"skipped {stats.skipped} of {stats.read} lines:\", err=True)\n        for line in stats.first_errors:\n            typer.echo(f\"  {line}\", err=True)\n",[14,1243,1244,1249,1259,1263,1269,1280,1284,1291,1295,1307,1311,1321,1325,1329,1337,1351,1356,1360,1364,1376,1386,1414,1444,1475,1505,1513,1518,1556,1563,1592,1628,1638,1651,1682,1702,1709,1748,1761],{"__ignoreMap":117},[121,1245,1246],{"class":123,"line":124},[121,1247,1248],{"class":127},"# src\u002Fmytool\u002Fcli.py\n",[121,1250,1251,1253,1255,1257],{"class":123,"line":131},[121,1252,135],{"class":134},[121,1254,139],{"class":138},[121,1256,142],{"class":134},[121,1258,146],{"class":145},[121,1260,1261],{"class":123,"line":149},[121,1262,153],{"emptyLinePlaceholder":152},[121,1264,1265,1267],{"class":123,"line":156},[121,1266,159],{"class":134},[121,1268,186],{"class":145},[121,1270,1271,1273,1275,1277],{"class":123,"line":165},[121,1272,135],{"class":134},[121,1274,246],{"class":145},[121,1276,159],{"class":134},[121,1278,1279],{"class":145}," Annotated\n",[121,1281,1282],{"class":123,"line":173},[121,1283,153],{"emptyLinePlaceholder":152},[121,1285,1286,1288],{"class":123,"line":181},[121,1287,159],{"class":134},[121,1289,1290],{"class":145}," typer\n",[121,1292,1293],{"class":123,"line":189},[121,1294,153],{"emptyLinePlaceholder":152},[121,1296,1297,1299,1302,1304],{"class":123,"line":202},[121,1298,135],{"class":134},[121,1300,1301],{"class":145}," mytool.stream ",[121,1303,159],{"class":134},[121,1305,1306],{"class":145}," BadRecord, filter_records\n",[121,1308,1309],{"class":123,"line":215},[121,1310,153],{"emptyLinePlaceholder":152},[121,1312,1313,1316,1318],{"class":123,"line":228},[121,1314,1315],{"class":145},"app ",[121,1317,396],{"class":134},[121,1319,1320],{"class":145}," typer.Typer()\n",[121,1322,1323],{"class":123,"line":241},[121,1324,153],{"emptyLinePlaceholder":152},[121,1326,1327],{"class":123,"line":257},[121,1328,153],{"emptyLinePlaceholder":152},[121,1330,1331,1334],{"class":123,"line":262},[121,1332,1333],{"class":273},"@app.callback",[121,1335,1336],{"class":145},"()\n",[121,1338,1339,1341,1344,1347,1349],{"class":123,"line":267},[121,1340,512],{"class":134},[121,1342,1343],{"class":273}," main",[121,1345,1346],{"class":145},"() -> ",[121,1348,315],{"class":138},[121,1350,318],{"class":145},[121,1352,1353],{"class":123,"line":286},[121,1354,1355],{"class":338},"    \"\"\"Work with NDJSON event files.\"\"\"\n",[121,1357,1358],{"class":123,"line":321},[121,1359,153],{"emptyLinePlaceholder":152},[121,1361,1362],{"class":123,"line":376},[121,1363,153],{"emptyLinePlaceholder":152},[121,1365,1366,1369,1371,1374],{"class":123,"line":402},[121,1367,1368],{"class":273},"@app.command",[121,1370,277],{"class":145},[121,1372,1373],{"class":338},"\"filter\"",[121,1375,373],{"class":145},[121,1377,1378,1380,1383],{"class":123,"line":407},[121,1379,512],{"class":134},[121,1381,1382],{"class":273}," filter_cmd",[121,1384,1385],{"class":145},"(\n",[121,1387,1388,1391,1393,1396,1399,1401,1404,1407,1409,1411],{"class":123,"line":412},[121,1389,1390],{"class":145},"    inputs: Annotated[list[",[121,1392,298],{"class":138},[121,1394,1395],{"class":145},"], typer.Argument(",[121,1397,1398],{"class":482},"help",[121,1400,396],{"class":134},[121,1402,1403],{"class":338},"\"NDJSON files (.gz ok); '-' for stdin.\"",[121,1405,1406],{"class":145},")] ",[121,1408,396],{"class":134},[121,1410,1045],{"class":138},[121,1412,1413],{"class":145},",\n",[121,1415,1416,1419,1421,1424,1427,1429,1431,1433,1436,1438,1440,1442],{"class":123,"line":418},[121,1417,1418],{"class":145},"    where: Annotated[list[",[121,1420,298],{"class":138},[121,1422,1423],{"class":145},"], typer.Option(",[121,1425,1426],{"class":338},"\"--where\"",[121,1428,25],{"class":145},[121,1430,1398],{"class":482},[121,1432,396],{"class":134},[121,1434,1435],{"class":338},"\"KEY=VALUE to match.\"",[121,1437,1406],{"class":145},[121,1439,396],{"class":134},[121,1441,1045],{"class":138},[121,1443,1413],{"class":145},[121,1445,1446,1449,1451,1454,1457,1459,1461,1463,1466,1468,1470,1473],{"class":123,"line":428},[121,1447,1448],{"class":145},"    fields: Annotated[",[121,1450,298],{"class":138},[121,1452,1453],{"class":145},", typer.Option(",[121,1455,1456],{"class":338},"\"--fields\"",[121,1458,25],{"class":145},[121,1460,1398],{"class":482},[121,1462,396],{"class":134},[121,1464,1465],{"class":338},"\"Comma-separated keys to keep.\"",[121,1467,1406],{"class":145},[121,1469,396],{"class":134},[121,1471,1472],{"class":338}," \"\"",[121,1474,1413],{"class":145},[121,1476,1477,1480,1482,1484,1487,1489,1491,1493,1496,1498,1500,1503],{"class":123,"line":442},[121,1478,1479],{"class":145},"    skip_invalid: Annotated[",[121,1481,740],{"class":138},[121,1483,1453],{"class":145},[121,1485,1486],{"class":338},"\"--skip-invalid\"",[121,1488,25],{"class":145},[121,1490,1398],{"class":482},[121,1492,396],{"class":134},[121,1494,1495],{"class":338},"\"Skip malformed lines.\"",[121,1497,1406],{"class":145},[121,1499,396],{"class":134},[121,1501,1502],{"class":138}," False",[121,1504,1413],{"class":145},[121,1506,1507,1509,1511],{"class":123,"line":454},[121,1508,312],{"class":145},[121,1510,315],{"class":138},[121,1512,318],{"class":145},[121,1514,1515],{"class":123,"line":466},[121,1516,1517],{"class":338},"    \"\"\"Stream matching records to stdout, one JSON object per line.\"\"\"\n",[121,1519,1520,1523,1525,1528,1531,1534,1536,1538,1540,1542,1545,1547,1550,1553],{"class":123,"line":493},[121,1521,1522],{"class":145},"    conditions ",[121,1524,396],{"class":134},[121,1526,1527],{"class":138}," dict",[121,1529,1530],{"class":145},"(w.split(",[121,1532,1533],{"class":338},"\"=\"",[121,1535,25],{"class":145},[121,1537,774],{"class":138},[121,1539,649],{"class":145},[121,1541,1137],{"class":134},[121,1543,1544],{"class":145}," w ",[121,1546,760],{"class":134},[121,1548,1549],{"class":145}," (where ",[121,1551,1552],{"class":134},"or",[121,1554,1555],{"class":145}," []))\n",[121,1557,1558,1561],{"class":123,"line":498},[121,1559,1560],{"class":134},"    try",[121,1562,318],{"class":145},[121,1564,1565,1568,1570,1573,1575,1578,1581,1584,1587,1589],{"class":123,"line":503},[121,1566,1567],{"class":145},"        stats ",[121,1569,396],{"class":134},[121,1571,1572],{"class":145}," filter_records(inputs ",[121,1574,1552],{"class":134},[121,1576,1577],{"class":145}," [",[121,1579,1580],{"class":338},"\"-\"",[121,1582,1583],{"class":145},"], sys.stdout, ",[121,1585,1586],{"class":482},"where",[121,1588,396],{"class":134},[121,1590,1591],{"class":145},"conditions,\n",[121,1593,1594,1597,1599,1602,1604,1607,1609,1612,1614,1616,1619,1622,1624,1626],{"class":123,"line":509},[121,1595,1596],{"class":482},"                               fields",[121,1598,396],{"class":134},[121,1600,1601],{"class":145},"[f ",[121,1603,1137],{"class":134},[121,1605,1606],{"class":145}," f ",[121,1608,760],{"class":134},[121,1610,1611],{"class":145}," fields.split(",[121,1613,1191],{"class":338},[121,1615,649],{"class":145},[121,1617,1618],{"class":134},"if",[121,1620,1621],{"class":145}," f] ",[121,1623,1552],{"class":134},[121,1625,1045],{"class":138},[121,1627,1413],{"class":145},[121,1629,1630,1633,1635],{"class":123,"line":537},[121,1631,1632],{"class":482},"                               skip_invalid",[121,1634,396],{"class":134},[121,1636,1637],{"class":145},"skip_invalid)\n",[121,1639,1640,1643,1646,1648],{"class":123,"line":543},[121,1641,1642],{"class":134},"    except",[121,1644,1645],{"class":145}," BadRecord ",[121,1647,652],{"class":134},[121,1649,1650],{"class":145}," exc:\n",[121,1652,1653,1656,1658,1661,1663,1665,1667,1670,1672,1675,1677,1680],{"class":123,"line":560},[121,1654,1655],{"class":145},"        typer.echo(",[121,1657,335],{"class":134},[121,1659,1660],{"class":338},"\"error: ",[121,1662,342],{"class":138},[121,1664,964],{"class":145},[121,1666,348],{"class":138},[121,1668,1669],{"class":338}," (use --skip-invalid to skip bad lines)\"",[121,1671,25],{"class":145},[121,1673,1674],{"class":482},"err",[121,1676,396],{"class":134},[121,1678,1679],{"class":138},"True",[121,1681,373],{"class":145},[121,1683,1684,1687,1690,1693,1695,1697,1699],{"class":123,"line":581},[121,1685,1686],{"class":134},"        raise",[121,1688,1689],{"class":145}," typer.Exit(",[121,1691,1692],{"class":138},"65",[121,1694,649],{"class":145},[121,1696,135],{"class":134},[121,1698,1045],{"class":138},[121,1700,1701],{"class":127},"          # EX_DATAERR\n",[121,1703,1704,1706],{"class":123,"line":589},[121,1705,546],{"class":134},[121,1707,1708],{"class":145}," stats.skipped:\n",[121,1710,1711,1713,1715,1718,1720,1723,1725,1728,1730,1733,1735,1738,1740,1742,1744,1746],{"class":123,"line":598},[121,1712,1655],{"class":145},[121,1714,335],{"class":134},[121,1716,1717],{"class":338},"\"skipped ",[121,1719,342],{"class":138},[121,1721,1722],{"class":145},"stats.skipped",[121,1724,348],{"class":138},[121,1726,1727],{"class":338}," of ",[121,1729,342],{"class":138},[121,1731,1732],{"class":145},"stats.read",[121,1734,348],{"class":138},[121,1736,1737],{"class":338}," lines:\"",[121,1739,25],{"class":145},[121,1741,1674],{"class":482},[121,1743,396],{"class":134},[121,1745,1679],{"class":138},[121,1747,373],{"class":145},[121,1749,1750,1753,1756,1758],{"class":123,"line":606},[121,1751,1752],{"class":134},"        for",[121,1754,1755],{"class":145}," line ",[121,1757,760],{"class":134},[121,1759,1760],{"class":145}," stats.first_errors:\n",[121,1762,1763,1766,1768,1771,1773,1775,1777,1779,1781,1783,1785,1787],{"class":123,"line":615},[121,1764,1765],{"class":145},"            typer.echo(",[121,1767,335],{"class":134},[121,1769,1770],{"class":338},"\"  ",[121,1772,342],{"class":138},[121,1774,123],{"class":145},[121,1776,348],{"class":138},[121,1778,339],{"class":338},[121,1780,25],{"class":145},[121,1782,1674],{"class":482},[121,1784,396],{"class":134},[121,1786,1679],{"class":138},[121,1788,373],{"class":145},[10,1790,1791],{},"In use, it behaves like any other filter in a pipeline:",[112,1793,1797],{"className":1794,"code":1795,"language":1796,"meta":117,"style":117},"language-bash shiki shiki-themes github-light github-dark","mytool filter events.ndjson.gz --where level=error --fields ts,user | head -20\nkubectl logs deploy\u002Fapi | mytool filter --where level=error\nmytool filter a.ndjson b.ndjson.gz - \u003C c.ndjson > errors.ndjson\n","bash",[14,1798,1799,1831,1854],{"__ignoreMap":117},[121,1800,1801,1804,1807,1810,1813,1816,1819,1822,1825,1828],{"class":123,"line":124},[121,1802,1803],{"class":273},"mytool",[121,1805,1806],{"class":338}," filter",[121,1808,1809],{"class":338}," events.ndjson.gz",[121,1811,1812],{"class":138}," --where",[121,1814,1815],{"class":338}," level=error",[121,1817,1818],{"class":138}," --fields",[121,1820,1821],{"class":338}," ts,user",[121,1823,1824],{"class":134}," |",[121,1826,1827],{"class":273}," head",[121,1829,1830],{"class":138}," -20\n",[121,1832,1833,1836,1839,1842,1844,1847,1849,1851],{"class":123,"line":131},[121,1834,1835],{"class":273},"kubectl",[121,1837,1838],{"class":338}," logs",[121,1840,1841],{"class":338}," deploy\u002Fapi",[121,1843,1824],{"class":134},[121,1845,1846],{"class":273}," mytool",[121,1848,1806],{"class":338},[121,1850,1812],{"class":138},[121,1852,1853],{"class":338}," level=error\n",[121,1855,1856,1858,1860,1863,1866,1869,1872,1875,1878],{"class":123,"line":149},[121,1857,1803],{"class":273},[121,1859,1806],{"class":338},[121,1861,1862],{"class":338}," a.ndjson",[121,1864,1865],{"class":338}," b.ndjson.gz",[121,1867,1868],{"class":338}," -",[121,1870,1871],{"class":134}," \u003C",[121,1873,1874],{"class":338}," c.ndjson",[121,1876,1877],{"class":134}," >",[121,1879,1880],{"class":338}," errors.ndjson\n",[1882,1883,1885],"h3",{"id":1884},"why-it-is-built-this-way","Why it is built this way",[10,1887,1888,1892,1893,1895,1896,1899,1900,47],{},[1889,1890,1891],"strong",{},"Generators all the way down."," ",[14,1894,105],{}," is a generator, so parsing happens only as fast as the consumer asks for records. When ",[14,1897,1898],{},"head -20"," has what it needs and closes the pipe, the command stops reading — a 20 GB file costs the same as a 20 KB one. Handling the resulting broken pipe cleanly is covered in ",[43,1901,1903],{"href":1902},"\u002Fadvanced-input-parsing-user-experience\u002Fworking-with-stdin-stdout-and-pipes\u002Fhandling-broken-pipe-and-sigpipe\u002F","handling broken pipe and SIGPIPE",[10,1905,1906,1909,1910,1913],{},[1889,1907,1908],{},"Explicit UTF-8, and stdin in binary."," NDJSON is UTF-8 by definition, so the files are opened with ",[14,1911,1912],{},"encoding=\"utf-8\""," rather than the platform default, which differs on Windows. Stdin is wrapped from its binary buffer for the same reason, and detached afterwards so the real stdin is not closed.",[10,1915,1916,1892,1919,1922,1923,25,1926,1929,1930,47],{},[1889,1917,1918],{},"Compression is transparent.",[14,1920,1921],{},"gzip.open(..., \"rt\")"," decompresses as it reads, so archives never need to be unpacked to disk first. The same pattern extends to ",[14,1924,1925],{},".bz2",[14,1927,1928],{},".xz"," and, with a third-party package, ",[14,1931,1932],{},".zst",[10,1934,1935,1892,1938,1941],{},[1889,1936,1937],{},"Compact output.",[14,1939,1940],{},"separators=(\",\", \":\")"," writes each object without spaces, which matters when the output is gigabytes, and keeps one object per line so the output is NDJSON too — ready for the next command in the pipeline.",[10,1943,1944,1947],{},[1889,1945,1946],{},"Blank lines are ignored."," Many producers end files with an empty line or separate batches with one; treating those as errors would make the tool fussy for no benefit.",[49,1949,1951],{"id":1950},"bad-records-stop-or-skip","Bad records: stop or skip?",[72,1953],{"name":1954},"nd-errors",[10,1956,1957,1958,1961,1962,1965,1966,1969],{},"A malformed line in the middle of a large file is common: a truncated write, a log line that is not JSON, a stray array. The default here is to ",[1889,1959,1960],{},"stop"," with the file name and line number and exit 65 (",[14,1963,1964],{},"EX_DATAERR","), because silently dropping records is how data goes missing without anybody noticing. When the user knows the input is messy and wants the good records anyway, ",[14,1967,1968],{},"--skip-invalid"," skips bad lines, counts them, and reports the count and the first few locations on stderr so the output stays clean. Leniency is opt-in and visible — never the default.",[49,1971,1973],{"id":1972},"ux-considerations","UX considerations",[54,1975,1976,1990,2004,2015,2028],{},[57,1977,1978,1981,1982,1985,1986,47],{},[1889,1979,1980],{},"Default to stdin."," With no file arguments the command reads stdin, and ",[14,1983,1984],{},"-"," means stdin among other files, following the conventions in ",[43,1987,1989],{"href":1988},"\u002Fmodern-python-cli-frameworks-architecture\u002Fdesigning-cli-interfaces-and-conventions\u002Ffollowing-posix-and-gnu-argument-conventions\u002F","following POSIX and GNU argument conventions",[57,1991,1992,1995,1996,1999,2000,47],{},[1889,1993,1994],{},"Results on stdout, everything else on stderr."," The skip summary and errors go to stderr, so ",[14,1997,1998],{},"> out.ndjson"," captures only records. Output rules are covered in ",[43,2001,2003],{"href":2002},"\u002Fadvanced-input-parsing-user-experience\u002Fworking-with-stdin-stdout-and-pipes\u002Femitting-json-output-for-scripting\u002F","emitting JSON output for scripting",[57,2005,2006,2009,2010,2014],{},[1889,2007,2008],{},"Progress only on a terminal."," For long runs over files, a byte-based progress bar on stderr helps — but only when stderr is a TTY; see ",[43,2011,2013],{"href":2012},"\u002Fadvanced-input-parsing-user-experience\u002Fworking-with-stdin-stdout-and-pipes\u002Fdetecting-tty-and-adapting-output\u002F","detecting TTY and adapting output",". Stdin has no known length, so show a record count instead.",[57,2016,2017,2020,2021,2023,2024,2027],{},[1889,2018,2019],{},"Faster parsing is a drop-in."," If profiling shows ",[14,2022,84],{}," dominating, ",[14,2025,2026],{},"orjson.loads"," is several times faster and accepts the same lines; keep the streaming structure and swap only the parser.",[57,2029,2030,2033,2034,47],{},[1889,2031,2032],{},"Parallelism rarely pays."," Reading and parsing a stream is usually I\u002FO-bound or bottlenecked on one core's JSON parsing; splitting a file across processes adds complexity and reorders output. Measure first, as in ",[43,2035,2037],{"href":2036},"\u002Fcli-runtime-systems-integration\u002Fconcurrency-and-async-in-python-clis\u002Fmultiprocessing-for-cpu-bound-cli-tasks\u002F","multiprocessing for CPU-bound CLI tasks",[49,2039,2041],{"id":2040},"testing-the-behaviour","Testing the behaviour",[10,2043,2044,2045,2048],{},"The tests cover filtering and projection, stdin and gzip input together, both error modes, laziness and — the property that matters most — flat memory, measured with ",[14,2046,2047],{},"tracemalloc"," on an 18 MB file:",[112,2050,2052],{"className":114,"code":2051,"language":116,"meta":117,"style":117},"# tests\u002Ftest_stream.py\nimport gzip\nimport io\nimport json\nimport tracemalloc\n\nimport pytest\nfrom typer.testing import CliRunner\n\nfrom mytool.cli import app\nfrom mytool.stream import BadRecord, Stats, filter_records, records\n\nrunner = CliRunner()\nEVENTS = [{\"level\": \"info\", \"user\": \"ana\"}, {\"level\": \"error\", \"user\": \"bo\"}, {\"level\": \"error\", \"user\": \"cy\"}]\nNDJSON = \"\".join(json.dumps(e) + \"\\n\" for e in EVENTS)\n\n\ndef test_filters_and_projects(tmp_path):\n    path = tmp_path \u002F \"events.ndjson\"\n    path.write_text(NDJSON)\n    result = runner.invoke(app, [\"filter\", str(path), \"--where\", \"level=error\", \"--fields\", \"user\"])\n    assert result.exit_code == 0\n    assert result.stdout.splitlines() == ['{\"user\":\"bo\"}', '{\"user\":\"cy\"}']\n\n\ndef test_reads_stdin_and_gzip(tmp_path):\n    gz = tmp_path \u002F \"events.ndjson.gz\"\n    with gzip.open(gz, \"wt\", encoding=\"utf-8\") as fh:\n        fh.write(NDJSON)\n    result = runner.invoke(app, [\"filter\", \"-\", str(gz), \"--where\", \"user=ana\"], input=NDJSON)\n    assert result.stdout.count('\"ana\"') == 2\n\n\ndef test_bad_line_stops_with_its_line_number(tmp_path):\n    path = tmp_path \u002F \"bad.ndjson\"\n    path.write_text(NDJSON + \"{not json\\n\")\n    result = runner.invoke(app, [\"filter\", str(path)])\n    assert result.exit_code == 65\n    assert f\"{path}:4:\" in result.stderr\n\n\ndef test_skip_invalid_counts_and_reports(tmp_path):\n    path = tmp_path \u002F \"bad.ndjson\"\n    path.write_text(\"[1, 2]\\n\" + NDJSON + \"{oops\\n\")\n    result = runner.invoke(app, [\"filter\", str(path), \"--skip-invalid\"])\n    assert result.exit_code == 0 and len(result.stdout.splitlines()) == 3\n    assert \"skipped 2 of 5 lines\" in result.stderr\n\n\ndef test_records_is_lazy():\n    stats = Stats()\n    gen = records(io.StringIO('{\"a\": 1}\\n{broken\\n'), \"x\", skip_invalid=False, stats=stats)\n    assert next(gen) == {\"a\": 1}            # the bad second line has not been read yet\n    with pytest.raises(BadRecord):\n        next(gen)\n\n\ndef test_memory_stays_flat(tmp_path):\n    path = tmp_path \u002F \"big.ndjson\"\n    with path.open(\"w\") as fh:\n        for i in range(200_000):\n            fh.write(json.dumps({\"i\": i, \"level\": \"info\", \"msg\": \"x\" * 50}) + \"\\n\")\n    tracemalloc.start()\n    filter_records([str(path)], io.StringIO(), where={\"level\": \"none\"}, fields=None, skip_invalid=False)\n    _, peak = tracemalloc.get_traced_memory()\n    tracemalloc.stop()\n    assert path.stat().st_size > 15_000_000\n    assert peak \u003C 1_000_000                 # measured: about 25 KB for an 18 MB file\n",[14,2053,2054,2059,2065,2071,2077,2084,2088,2095,2107,2111,2123,2134,2138,2148,2215,2248,2252,2256,2266,2282,2291,2328,2340,2362,2366,2370,2379,2393,2418,2427,2466,2483,2487,2491,2500,2513,2531,2548,2559,2584,2588,2592,2601,2613,2640,2660,2684,2695,2699,2703,2713,2721,2768,2796,2803,2811,2815,2819,2828,2841,2857,2876,2921,2926,2969,2979,2984,2997],{"__ignoreMap":117},[121,2055,2056],{"class":123,"line":124},[121,2057,2058],{"class":127},"# tests\u002Ftest_stream.py\n",[121,2060,2061,2063],{"class":123,"line":131},[121,2062,159],{"class":134},[121,2064,162],{"class":145},[121,2066,2067,2069],{"class":123,"line":149},[121,2068,159],{"class":134},[121,2070,170],{"class":145},[121,2072,2073,2075],{"class":123,"line":156},[121,2074,159],{"class":134},[121,2076,178],{"class":145},[121,2078,2079,2081],{"class":123,"line":165},[121,2080,159],{"class":134},[121,2082,2083],{"class":145}," tracemalloc\n",[121,2085,2086],{"class":123,"line":173},[121,2087,153],{"emptyLinePlaceholder":152},[121,2089,2090,2092],{"class":123,"line":181},[121,2091,159],{"class":134},[121,2093,2094],{"class":145}," pytest\n",[121,2096,2097,2099,2102,2104],{"class":123,"line":189},[121,2098,135],{"class":134},[121,2100,2101],{"class":145}," typer.testing ",[121,2103,159],{"class":134},[121,2105,2106],{"class":145}," CliRunner\n",[121,2108,2109],{"class":123,"line":202},[121,2110,153],{"emptyLinePlaceholder":152},[121,2112,2113,2115,2118,2120],{"class":123,"line":215},[121,2114,135],{"class":134},[121,2116,2117],{"class":145}," mytool.cli ",[121,2119,159],{"class":134},[121,2121,2122],{"class":145}," app\n",[121,2124,2125,2127,2129,2131],{"class":123,"line":228},[121,2126,135],{"class":134},[121,2128,1301],{"class":145},[121,2130,159],{"class":134},[121,2132,2133],{"class":145}," BadRecord, Stats, filter_records, records\n",[121,2135,2136],{"class":123,"line":241},[121,2137,153],{"emptyLinePlaceholder":152},[121,2139,2140,2143,2145],{"class":123,"line":257},[121,2141,2142],{"class":145},"runner ",[121,2144,396],{"class":134},[121,2146,2147],{"class":145}," CliRunner()\n",[121,2149,2150,2153,2155,2158,2161,2163,2166,2168,2171,2173,2176,2179,2181,2183,2186,2188,2190,2192,2195,2197,2199,2201,2203,2205,2207,2209,2212],{"class":123,"line":262},[121,2151,2152],{"class":138},"EVENTS",[121,2154,436],{"class":134},[121,2156,2157],{"class":145}," [{",[121,2159,2160],{"class":338},"\"level\"",[121,2162,361],{"class":145},[121,2164,2165],{"class":338},"\"info\"",[121,2167,25],{"class":145},[121,2169,2170],{"class":338},"\"user\"",[121,2172,361],{"class":145},[121,2174,2175],{"class":338},"\"ana\"",[121,2177,2178],{"class":145},"}, {",[121,2180,2160],{"class":338},[121,2182,361],{"class":145},[121,2184,2185],{"class":338},"\"error\"",[121,2187,25],{"class":145},[121,2189,2170],{"class":338},[121,2191,361],{"class":145},[121,2193,2194],{"class":338},"\"bo\"",[121,2196,2178],{"class":145},[121,2198,2160],{"class":338},[121,2200,361],{"class":145},[121,2202,2185],{"class":338},[121,2204,25],{"class":145},[121,2206,2170],{"class":338},[121,2208,361],{"class":145},[121,2210,2211],{"class":338},"\"cy\"",[121,2213,2214],{"class":145},"}]\n",[121,2216,2217,2220,2222,2224,2227,2229,2231,2233,2235,2238,2241,2243,2246],{"class":123,"line":267},[121,2218,2219],{"class":138},"NDJSON",[121,2221,436],{"class":134},[121,2223,1472],{"class":338},[121,2225,2226],{"class":145},".join(json.dumps(e) ",[121,2228,1202],{"class":134},[121,2230,1205],{"class":338},[121,2232,1208],{"class":138},[121,2234,339],{"class":338},[121,2236,2237],{"class":134}," for",[121,2239,2240],{"class":145}," e ",[121,2242,760],{"class":134},[121,2244,2245],{"class":138}," EVENTS",[121,2247,373],{"class":145},[121,2249,2250],{"class":123,"line":286},[121,2251,153],{"emptyLinePlaceholder":152},[121,2253,2254],{"class":123,"line":321},[121,2255,153],{"emptyLinePlaceholder":152},[121,2257,2258,2260,2263],{"class":123,"line":376},[121,2259,512],{"class":134},[121,2261,2262],{"class":273}," test_filters_and_projects",[121,2264,2265],{"class":145},"(tmp_path):\n",[121,2267,2268,2271,2273,2276,2279],{"class":123,"line":402},[121,2269,2270],{"class":145},"    path ",[121,2272,396],{"class":134},[121,2274,2275],{"class":145}," tmp_path ",[121,2277,2278],{"class":134},"\u002F",[121,2280,2281],{"class":338}," \"events.ndjson\"\n",[121,2283,2284,2287,2289],{"class":123,"line":407},[121,2285,2286],{"class":145},"    path.write_text(",[121,2288,2219],{"class":138},[121,2290,373],{"class":145},[121,2292,2293,2296,2298,2301,2303,2305,2307,2310,2312,2314,2317,2319,2321,2323,2325],{"class":123,"line":412},[121,2294,2295],{"class":145},"    result ",[121,2297,396],{"class":134},[121,2299,2300],{"class":145}," runner.invoke(app, [",[121,2302,1373],{"class":338},[121,2304,25],{"class":145},[121,2306,298],{"class":138},[121,2308,2309],{"class":145},"(path), ",[121,2311,1426],{"class":338},[121,2313,25],{"class":145},[121,2315,2316],{"class":338},"\"level=error\"",[121,2318,25],{"class":145},[121,2320,1456],{"class":338},[121,2322,25],{"class":145},[121,2324,2170],{"class":338},[121,2326,2327],{"class":145},"])\n",[121,2329,2330,2333,2336,2338],{"class":123,"line":418},[121,2331,2332],{"class":134},"    assert",[121,2334,2335],{"class":145}," result.exit_code ",[121,2337,552],{"class":134},[121,2339,439],{"class":138},[121,2341,2342,2344,2347,2349,2351,2354,2356,2359],{"class":123,"line":428},[121,2343,2332],{"class":134},[121,2345,2346],{"class":145}," result.stdout.splitlines() ",[121,2348,552],{"class":134},[121,2350,1577],{"class":145},[121,2352,2353],{"class":338},"'{\"user\":\"bo\"}'",[121,2355,25],{"class":145},[121,2357,2358],{"class":338},"'{\"user\":\"cy\"}'",[121,2360,2361],{"class":145},"]\n",[121,2363,2364],{"class":123,"line":442},[121,2365,153],{"emptyLinePlaceholder":152},[121,2367,2368],{"class":123,"line":454},[121,2369,153],{"emptyLinePlaceholder":152},[121,2371,2372,2374,2377],{"class":123,"line":466},[121,2373,512],{"class":134},[121,2375,2376],{"class":273}," test_reads_stdin_and_gzip",[121,2378,2265],{"class":145},[121,2380,2381,2384,2386,2388,2390],{"class":123,"line":493},[121,2382,2383],{"class":145},"    gz ",[121,2385,396],{"class":134},[121,2387,2275],{"class":145},[121,2389,2278],{"class":134},[121,2391,2392],{"class":338}," \"events.ndjson.gz\"\n",[121,2394,2395,2398,2401,2404,2406,2408,2410,2412,2414,2416],{"class":123,"line":498},[121,2396,2397],{"class":134},"    with",[121,2399,2400],{"class":145}," gzip.open(gz, ",[121,2402,2403],{"class":338},"\"wt\"",[121,2405,25],{"class":145},[121,2407,571],{"class":482},[121,2409,396],{"class":134},[121,2411,576],{"class":338},[121,2413,649],{"class":145},[121,2415,652],{"class":134},[121,2417,655],{"class":145},[121,2419,2420,2423,2425],{"class":123,"line":503},[121,2421,2422],{"class":145},"        fh.write(",[121,2424,2219],{"class":138},[121,2426,373],{"class":145},[121,2428,2429,2431,2433,2435,2437,2439,2441,2443,2445,2448,2450,2452,2455,2457,2460,2462,2464],{"class":123,"line":509},[121,2430,2295],{"class":145},[121,2432,396],{"class":134},[121,2434,2300],{"class":145},[121,2436,1373],{"class":338},[121,2438,25],{"class":145},[121,2440,1580],{"class":338},[121,2442,25],{"class":145},[121,2444,298],{"class":138},[121,2446,2447],{"class":145},"(gz), ",[121,2449,1426],{"class":338},[121,2451,25],{"class":145},[121,2453,2454],{"class":338},"\"user=ana\"",[121,2456,1019],{"class":145},[121,2458,2459],{"class":482},"input",[121,2461,396],{"class":134},[121,2463,2219],{"class":138},[121,2465,373],{"class":145},[121,2467,2468,2470,2473,2476,2478,2480],{"class":123,"line":537},[121,2469,2332],{"class":134},[121,2471,2472],{"class":145}," result.stdout.count(",[121,2474,2475],{"class":338},"'\"ana\"'",[121,2477,649],{"class":145},[121,2479,552],{"class":134},[121,2481,2482],{"class":138}," 2\n",[121,2484,2485],{"class":123,"line":543},[121,2486,153],{"emptyLinePlaceholder":152},[121,2488,2489],{"class":123,"line":560},[121,2490,153],{"emptyLinePlaceholder":152},[121,2492,2493,2495,2498],{"class":123,"line":581},[121,2494,512],{"class":134},[121,2496,2497],{"class":273}," test_bad_line_stops_with_its_line_number",[121,2499,2265],{"class":145},[121,2501,2502,2504,2506,2508,2510],{"class":123,"line":589},[121,2503,2270],{"class":145},[121,2505,396],{"class":134},[121,2507,2275],{"class":145},[121,2509,2278],{"class":134},[121,2511,2512],{"class":338}," \"bad.ndjson\"\n",[121,2514,2515,2517,2519,2522,2525,2527,2529],{"class":123,"line":598},[121,2516,2286],{"class":145},[121,2518,2219],{"class":138},[121,2520,2521],{"class":134}," +",[121,2523,2524],{"class":338}," \"{not json",[121,2526,1208],{"class":138},[121,2528,339],{"class":338},[121,2530,373],{"class":145},[121,2532,2533,2535,2537,2539,2541,2543,2545],{"class":123,"line":606},[121,2534,2295],{"class":145},[121,2536,396],{"class":134},[121,2538,2300],{"class":145},[121,2540,1373],{"class":338},[121,2542,25],{"class":145},[121,2544,298],{"class":138},[121,2546,2547],{"class":145},"(path)])\n",[121,2549,2550,2552,2554,2556],{"class":123,"line":615},[121,2551,2332],{"class":134},[121,2553,2335],{"class":145},[121,2555,552],{"class":134},[121,2557,2558],{"class":138}," 65\n",[121,2560,2561,2563,2566,2568,2570,2573,2575,2578,2581],{"class":123,"line":629},[121,2562,2332],{"class":134},[121,2564,2565],{"class":134}," f",[121,2567,339],{"class":338},[121,2569,342],{"class":138},[121,2571,2572],{"class":145},"path",[121,2574,348],{"class":138},[121,2576,2577],{"class":338},":4:\"",[121,2579,2580],{"class":134}," in",[121,2582,2583],{"class":145}," result.stderr\n",[121,2585,2586],{"class":123,"line":658},[121,2587,153],{"emptyLinePlaceholder":152},[121,2589,2590],{"class":123,"line":665},[121,2591,153],{"emptyLinePlaceholder":152},[121,2593,2594,2596,2599],{"class":123,"line":673},[121,2595,512],{"class":134},[121,2597,2598],{"class":273}," test_skip_invalid_counts_and_reports",[121,2600,2265],{"class":145},[121,2602,2603,2605,2607,2609,2611],{"class":123,"line":693},[121,2604,2270],{"class":145},[121,2606,396],{"class":134},[121,2608,2275],{"class":145},[121,2610,2278],{"class":134},[121,2612,2512],{"class":338},[121,2614,2615,2617,2620,2622,2624,2626,2629,2631,2634,2636,2638],{"class":123,"line":700},[121,2616,2286],{"class":145},[121,2618,2619],{"class":338},"\"[1, 2]",[121,2621,1208],{"class":138},[121,2623,339],{"class":338},[121,2625,2521],{"class":134},[121,2627,2628],{"class":138}," NDJSON",[121,2630,2521],{"class":134},[121,2632,2633],{"class":338}," \"{oops",[121,2635,1208],{"class":138},[121,2637,339],{"class":338},[121,2639,373],{"class":145},[121,2641,2642,2644,2646,2648,2650,2652,2654,2656,2658],{"class":123,"line":705},[121,2643,2295],{"class":145},[121,2645,396],{"class":134},[121,2647,2300],{"class":145},[121,2649,1373],{"class":338},[121,2651,25],{"class":145},[121,2653,298],{"class":138},[121,2655,2309],{"class":145},[121,2657,1486],{"class":338},[121,2659,2327],{"class":145},[121,2661,2662,2664,2666,2668,2671,2674,2676,2679,2681],{"class":123,"line":710},[121,2663,2332],{"class":134},[121,2665,2335],{"class":145},[121,2667,552],{"class":134},[121,2669,2670],{"class":138}," 0",[121,2672,2673],{"class":134}," and",[121,2675,922],{"class":138},[121,2677,2678],{"class":145},"(result.stdout.splitlines()) ",[121,2680,552],{"class":134},[121,2682,2683],{"class":138}," 3\n",[121,2685,2686,2688,2691,2693],{"class":123,"line":751},[121,2687,2332],{"class":134},[121,2689,2690],{"class":338}," \"skipped 2 of 5 lines\"",[121,2692,2580],{"class":134},[121,2694,2583],{"class":145},[121,2696,2697],{"class":123,"line":779},[121,2698,153],{"emptyLinePlaceholder":152},[121,2700,2701],{"class":123,"line":791},[121,2702,153],{"emptyLinePlaceholder":152},[121,2704,2705,2707,2710],{"class":123,"line":797},[121,2706,512],{"class":134},[121,2708,2709],{"class":273}," test_records_is_lazy",[121,2711,2712],{"class":145},"():\n",[121,2714,2715,2717,2719],{"class":123,"line":809},[121,2716,1058],{"class":145},[121,2718,396],{"class":134},[121,2720,1063],{"class":145},[121,2722,2723,2726,2728,2731,2734,2736,2739,2741,2744,2747,2750,2752,2754,2756,2759,2761,2763,2765],{"class":123,"line":816},[121,2724,2725],{"class":145},"    gen ",[121,2727,396],{"class":134},[121,2729,2730],{"class":145}," records(io.StringIO(",[121,2732,2733],{"class":338},"'{\"a\": 1}",[121,2735,1208],{"class":138},[121,2737,2738],{"class":338},"{broken",[121,2740,1208],{"class":138},[121,2742,2743],{"class":338},"'",[121,2745,2746],{"class":145},"), ",[121,2748,2749],{"class":338},"\"x\"",[121,2751,25],{"class":145},[121,2753,1104],{"class":482},[121,2755,396],{"class":134},[121,2757,2758],{"class":138},"False",[121,2760,25],{"class":145},[121,2762,1112],{"class":482},[121,2764,396],{"class":134},[121,2766,2767],{"class":145},"stats)\n",[121,2769,2770,2772,2775,2778,2780,2783,2786,2788,2790,2793],{"class":123,"line":827},[121,2771,2332],{"class":134},[121,2773,2774],{"class":138}," next",[121,2776,2777],{"class":145},"(gen) ",[121,2779,552],{"class":134},[121,2781,2782],{"class":145}," {",[121,2784,2785],{"class":338},"\"a\"",[121,2787,361],{"class":145},[121,2789,774],{"class":138},[121,2791,2792],{"class":145},"}            ",[121,2794,2795],{"class":127},"# the bad second line has not been read yet\n",[121,2797,2798,2800],{"class":123,"line":846},[121,2799,2397],{"class":134},[121,2801,2802],{"class":145}," pytest.raises(BadRecord):\n",[121,2804,2805,2808],{"class":123,"line":862},[121,2806,2807],{"class":138},"        next",[121,2809,2810],{"class":145},"(gen)\n",[121,2812,2813],{"class":123,"line":879},[121,2814,153],{"emptyLinePlaceholder":152},[121,2816,2817],{"class":123,"line":889},[121,2818,153],{"emptyLinePlaceholder":152},[121,2820,2821,2823,2826],{"class":123,"line":907},[121,2822,512],{"class":134},[121,2824,2825],{"class":273}," test_memory_stays_flat",[121,2827,2265],{"class":145},[121,2829,2830,2832,2834,2836,2838],{"class":123,"line":917},[121,2831,2270],{"class":145},[121,2833,396],{"class":134},[121,2835,2275],{"class":145},[121,2837,2278],{"class":134},[121,2839,2840],{"class":338}," \"big.ndjson\"\n",[121,2842,2843,2845,2848,2851,2853,2855],{"class":123,"line":936},[121,2844,2397],{"class":134},[121,2846,2847],{"class":145}," path.open(",[121,2849,2850],{"class":338},"\"w\"",[121,2852,649],{"class":145},[121,2854,652],{"class":134},[121,2856,655],{"class":145},[121,2858,2859,2861,2864,2866,2869,2871,2874],{"class":123,"line":973},[121,2860,1752],{"class":134},[121,2862,2863],{"class":145}," i ",[121,2865,760],{"class":134},[121,2867,2868],{"class":138}," range",[121,2870,277],{"class":145},[121,2872,2873],{"class":138},"200_000",[121,2875,283],{"class":145},[121,2877,2878,2881,2884,2887,2889,2891,2893,2895,2898,2900,2902,2905,2908,2911,2913,2915,2917,2919],{"class":123,"line":978},[121,2879,2880],{"class":145},"            fh.write(json.dumps({",[121,2882,2883],{"class":338},"\"i\"",[121,2885,2886],{"class":145},": i, ",[121,2888,2160],{"class":338},[121,2890,361],{"class":145},[121,2892,2165],{"class":338},[121,2894,25],{"class":145},[121,2896,2897],{"class":338},"\"msg\"",[121,2899,361],{"class":145},[121,2901,2749],{"class":338},[121,2903,2904],{"class":134}," *",[121,2906,2907],{"class":138}," 50",[121,2909,2910],{"class":145},"}) ",[121,2912,1202],{"class":134},[121,2914,1205],{"class":338},[121,2916,1208],{"class":138},[121,2918,339],{"class":338},[121,2920,373],{"class":145},[121,2922,2923],{"class":123,"line":987},[121,2924,2925],{"class":145},"    tracemalloc.start()\n",[121,2927,2928,2931,2933,2936,2938,2940,2942,2944,2946,2949,2952,2955,2957,2959,2961,2963,2965,2967],{"class":123,"line":992},[121,2929,2930],{"class":145},"    filter_records([",[121,2932,298],{"class":138},[121,2934,2935],{"class":145},"(path)], io.StringIO(), ",[121,2937,1586],{"class":482},[121,2939,396],{"class":134},[121,2941,342],{"class":145},[121,2943,2160],{"class":338},[121,2945,361],{"class":145},[121,2947,2948],{"class":338},"\"none\"",[121,2950,2951],{"class":145},"}, ",[121,2953,2954],{"class":482},"fields",[121,2956,396],{"class":134},[121,2958,315],{"class":138},[121,2960,25],{"class":145},[121,2962,1104],{"class":482},[121,2964,396],{"class":134},[121,2966,2758],{"class":138},[121,2968,373],{"class":145},[121,2970,2971,2974,2976],{"class":123,"line":997},[121,2972,2973],{"class":145},"    _, peak ",[121,2975,396],{"class":134},[121,2977,2978],{"class":145}," tracemalloc.get_traced_memory()\n",[121,2980,2981],{"class":123,"line":1032},[121,2982,2983],{"class":145},"    tracemalloc.stop()\n",[121,2985,2986,2988,2991,2994],{"class":123,"line":1055},[121,2987,2332],{"class":134},[121,2989,2990],{"class":145}," path.stat().st_size ",[121,2992,2993],{"class":134},">",[121,2995,2996],{"class":138}," 15_000_000\n",[121,2998,2999,3001,3004,3006,3009],{"class":123,"line":1066},[121,3000,2332],{"class":134},[121,3002,3003],{"class":145}," peak ",[121,3005,928],{"class":134},[121,3007,3008],{"class":138}," 1_000_000",[121,3010,3011],{"class":127},"                 # measured: about 25 KB for an 18 MB file\n",[10,3013,3014,3017,3018,3021,3022,3025],{},[14,3015,3016],{},"test_records_is_lazy"," proves the generator yields the first record before it has read the broken second line; a version that parsed everything up front would raise immediately. ",[14,3019,3020],{},"test_memory_stays_flat"," is the regression test for \"someone added ",[14,3023,3024],{},"list(...)"," for convenience\": a change that loads the file would push the peak into tens of megabytes and fail. It writes 200,000 records, so it takes about a second — mark it as slow if your suite is strict about speed.",[49,3027,3029],{"id":3028},"conclusion","Conclusion",[10,3031,3032,3033,3035,3036,3038,3039,3041,3042,3044],{},"Handling large inputs in a CLI is a matter of never holding more than one record: iterate over lines, parse each with ",[14,3034,84],{},", write results immediately, and let generators connect the stages so the command stops as soon as its consumer does. Open files with explicit UTF-8, decompress ",[14,3037,40],{}," on the fly, treat ",[14,3040,1984],{}," and no arguments as stdin, stop on malformed lines with a file and line number by default, and make skipping explicit with a counted summary. A ",[14,3043,2047],{}," test keeps memory flat through future changes, and the command stays usable on files far larger than the machine's memory.",[49,3046,3048],{"id":3047},"frequently-asked-questions","Frequently asked questions",[1882,3050,3052],{"id":3051},"what-if-the-input-is-one-big-json-array-rather-than-ndjson","What if the input is one big JSON array rather than NDJSON?",[10,3054,3055,3058,3059,3062,3063,3066],{},[14,3056,3057],{},"json.load"," must then read the whole document. For arrays too large for memory, use an incremental parser such as ",[14,3060,3061],{},"ijson",", which yields array items one at a time; or convert once with ",[14,3064,3065],{},"jq -c '.[]' big.json > big.ndjson"," and stream from then on.",[1882,3068,3070],{"id":3069},"should-the-output-preserve-key-order-and-formatting","Should the output preserve key order and formatting?",[10,3072,3073,3074,3076,3077,3080,3081,3083],{},"Key order is preserved (",[14,3075,841],{}," keeps insertion order and ",[14,3078,3079],{},"json.dumps"," writes it). Formatting is normalised to compact JSON; if byte-for-byte passthrough of matching lines matters, write the original ",[14,3082,123],{}," instead of re-serialising the parsed object.",[1882,3085,3087],{"id":3086},"how-do-i-handle-lines-with-invalid-utf-8","How do I handle lines with invalid UTF-8?",[10,3089,3090,3091,3094,3095,3098],{},"By default decoding raises ",[14,3092,3093],{},"UnicodeDecodeError",", which stops the command. For dirty sources, open with ",[14,3096,3097],{},"errors=\"replace\""," and let the JSON parser decide whether the result is still valid; report the replacement in the skip summary.",[1882,3100,3102],{"id":3101},"can-the-command-read-from-s3-or-http-directly","Can the command read from S3 or HTTP directly?",[10,3104,3105,3106,3109,3110,3113,3114,3117,3118,3120],{},"Keep the command reading streams and let the transport produce one: ",[14,3107,3108],{},"aws s3 cp s3:\u002F\u002Fbucket\u002Fkey - | mytool filter",", or an ",[14,3111,3112],{},"httpx"," streaming response whose ",[14,3115,3116],{},"iter_lines()"," feeds ",[14,3119,105],{},". The streaming core stays the same.",[1882,3122,3124],{"id":3123},"how-do-i-show-which-file-a-bad-line-came-from-when-reading-several-files","How do I show which file a bad line came from when reading several files?",[10,3126,3127,3128,3130,3131,3133,3134,3136],{},"Pass the file name into ",[14,3129,105],{}," as ",[14,3132,345],{},", as the recipe does, and include it in every error message. With stdin, ",[14,3135,1984],{}," is the name users recognise.",[49,3138,3140],{"id":3139},"related","Related",[54,3142,3143,3149,3154,3159,3164],{},[57,3144,3145,3146],{},"Up: ",[43,3147,3148],{"href":45},"Working with stdin, stdout and pipes",[57,3150,3151],{},[43,3152,3153],{"href":65},"Reading piped input in Python CLIs",[57,3155,3156],{},[43,3157,3158],{"href":1902},"Handling broken pipe and SIGPIPE",[57,3160,3161],{},[43,3162,3163],{"href":2002},"Emitting JSON output for scripting",[57,3165,3166],{},[43,3167,3169],{"href":3168},"\u002Fcli-runtime-systems-integration\u002Fcalling-http-apis-from-python-clis\u002Fdownloading-files-with-progress-in-python\u002F","Downloading files with progress in Python",[3171,3172,3173],"style",{},"html pre.shiki code .sJ8bj, html code.shiki .sJ8bj{--shiki-default:#6A737D;--shiki-dark:#6A737D}html pre.shiki code .szBVR, html code.shiki .szBVR{--shiki-default:#D73A49;--shiki-dark:#F97583}html pre.shiki code .sj4cs, html code.shiki .sj4cs{--shiki-default:#005CC5;--shiki-dark:#79B8FF}html pre.shiki code .sVt8B, html code.shiki .sVt8B{--shiki-default:#24292E;--shiki-dark:#E1E4E8}html pre.shiki code .sScJk, html code.shiki .sScJk{--shiki-default:#6F42C1;--shiki-dark:#B392F0}html pre.shiki code .sZZnC, html code.shiki .sZZnC{--shiki-default:#032F62;--shiki-dark:#9ECBFF}html pre.shiki code .s4XuR, html code.shiki .s4XuR{--shiki-default:#E36209;--shiki-dark:#FFAB70}html .default .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .shiki span {color: var(--shiki-default);background: var(--shiki-default-bg);font-style: var(--shiki-default-font-style);font-weight: var(--shiki-default-font-weight);text-decoration: var(--shiki-default-text-decoration);}html .dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}html.dark .shiki span {color: var(--shiki-dark);background: var(--shiki-dark-bg);font-style: var(--shiki-dark-font-style);font-weight: var(--shiki-dark-font-weight);text-decoration: var(--shiki-dark-text-decoration);}",{"title":117,"searchDepth":131,"depth":131,"links":3175},[3176,3177,3178,3181,3182,3183,3184,3185,3192],{"id":51,"depth":131,"text":52},{"id":69,"depth":131,"text":70},{"id":94,"depth":131,"text":95,"children":3179},[3180],{"id":1884,"depth":149,"text":1885},{"id":1950,"depth":131,"text":1951},{"id":1972,"depth":131,"text":1973},{"id":2040,"depth":131,"text":2041},{"id":3028,"depth":131,"text":3029},{"id":3047,"depth":131,"text":3048,"children":3186},[3187,3188,3189,3190,3191],{"id":3051,"depth":149,"text":3052},{"id":3069,"depth":149,"text":3070},{"id":3086,"depth":149,"text":3087},{"id":3101,"depth":149,"text":3102},{"id":3123,"depth":149,"text":3124},{"id":3139,"depth":131,"text":3140},"2026-09-18","Filter gigabytes of newline-delimited JSON from files, gzip archives or stdin with flat memory, clear errors on bad lines and an explicit --skip-invalid mode.","intermediate",false,"md",{},"\u002Fadvanced-input-parsing-user-experience\u002Fworking-with-stdin-stdout-and-pipes\u002Fprocessing-large-files-and-ndjson-streams",{"title":5,"description":3194},"advanced-input-parsing-user-experience\u002Fworking-with-stdin-stdout-and-pipes\u002Fprocessing-large-files-and-ndjson-streams\u002Findex",[3203,3204,3205,3206,3207],"ndjson","streaming","stdin","performance","typer","wssL5i1dDv1Yehg6HbqgrmThpGR0W6nCQuW91VlV3nc",[3210,3213,3216,3219,3222,3225,3228,3231,3234,3237,3240,3243,3246,3249,3252,3255,3258,3261,3264,3267,3270,3273,3276,3279,3282,3285,3288,3291,3294,3297,3300,3303,3306,3309,3312,3315,3318,3321,3324,3327,3330,3333,3336,3339,3342,3345,3348,3351,3354,3357,3360,3363,3366,3369,3372,3375,3378,3381,3384,3387,3390,3393,3396,3399,3402,3405,3408,3411,3414,3417,3420,3423,3426,3429,3432,3435,3438,3441,3444,3447,3448,3451,3454,3457,3460,3463,3466,3469,3472,3475,3478,3481,3484,3487,3490,3493,3496,3499,3502,3505,3508,3511,3514,3517,3520,3523,3526,3529,3532,3535,3538,3541,3544,3547,3550,3553,3556,3559,3562,3565,3568,3571,3574,3577,3580,3583,3586,3589,3592,3595,3598,3601,3604,3607,3610,3613,3616,3619,3622,3625,3627,3630,3633,3636,3639,3642,3645,3648,3651,3654,3657,3660,3663,3666,3669,3672,3675,3678,3681,3684,3687,3690,3693,3696,3699,3702,3705,3708,3711,3714,3717,3720,3723,3726,3729,3732,3735,3738,3741,3744,3747,3750,3753,3756,3759,3762,3765,3768,3771,3774,3777,3780,3783,3786,3789,3792,3795,3798,3801,3804,3807,3810,3813,3816,3819,3822,3825,3828,3831,3834,3837,3840,3843,3846,3849,3852,3855,3858,3861,3864,3867,3870,3873,3876,3879,3882,3885,3888,3891,3894,3897,3900,3903,3906,3909,3912,3915,3918,3921,3924,3927,3930,3933,3936,3939,3942,3945,3948,3951,3954,3957,3960,3963,3966,3969,3972,3975,3978,3981,3984,3987,3990,3993,3996,3999,4002,4005,4008,4011,4014,4017,4020,4023,4026,4029,4032,4035,4038,4041,4044,4047,4050],{"path":3211,"title":3212},"\u002Fabout","About Python CLI Toolcraft",{"path":3214,"title":3215},"\u002Fadvanced-input-parsing-user-experience\u002Fadvanced-argument-validation-strategies","Advanced Argument Validation Strategies",{"path":3217,"title":3218},"\u002Fadvanced-input-parsing-user-experience\u002Fadvanced-argument-validation-strategies\u002Fparsing-nested-json-arguments-in-python-clis","Parsing Nested JSON Args in Python CLIs",{"path":3220,"title":3221},"\u002Fadvanced-input-parsing-user-experience\u002Fadvanced-argument-validation-strategies\u002Fvalidating-dates-and-durations-in-cli-arguments","Validating Dates and Durations in Python CLI Arguments",{"path":3223,"title":3224},"\u002Fadvanced-input-parsing-user-experience\u002Fadvanced-argument-validation-strategies\u002Fvalidating-dependent-and-conflicting-options","Validating Dependent and Conflicting CLI Options",{"path":3226,"title":3227},"\u002Fadvanced-input-parsing-user-experience\u002Fadvanced-argument-validation-strategies\u002Fvalidating-file-and-directory-paths-in-clis","Validating File and Directory Paths in CLIs",{"path":3229,"title":3230},"\u002Fadvanced-input-parsing-user-experience\u002Fadvanced-argument-validation-strategies\u002Fvalidating-urls-hosts-and-ports","Validating URLs, Hosts and Ports in Python CLI Arguments",{"path":3232,"title":3233},"\u002Fadvanced-input-parsing-user-experience\u002Fadvanced-argument-validation-strategies\u002Fwriting-custom-click-parameter-types","Writing Custom Click Parameter Types",{"path":3235,"title":3236},"\u002Fadvanced-input-parsing-user-experience\u002Fbuilding-terminal-uis-with-textual\u002Fbrowsing-records-with-a-textual-datatable","Browsing Records with a Textual DataTable Picker",{"path":3238,"title":3239},"\u002Fadvanced-input-parsing-user-experience\u002Fbuilding-terminal-uis-with-textual\u002Fbuilding-your-first-textual-app","Building Your First Textual App for a Python CLI",{"path":3241,"title":3242},"\u002Fadvanced-input-parsing-user-experience\u002Fbuilding-terminal-uis-with-textual\u002Fchoosing-between-a-cli-a-prompt-flow-and-a-tui","Choosing Between a CLI, a Prompt Flow and a TUI",{"path":3244,"title":3245},"\u002Fadvanced-input-parsing-user-experience\u002Fbuilding-terminal-uis-with-textual","Building Terminal UIs with Textual for Python CLIs",{"path":3247,"title":3248},"\u002Fadvanced-input-parsing-user-experience\u002Fbuilding-terminal-uis-with-textual\u002Frunning-background-work-in-textual-with-workers","Running Background Work in Textual with Workers",{"path":3250,"title":3251},"\u002Fadvanced-input-parsing-user-experience\u002Fbuilding-terminal-uis-with-textual\u002Fstyling-textual-apps-with-tcss","Styling Textual Apps with TCSS",{"path":3253,"title":3254},"\u002Fadvanced-input-parsing-user-experience\u002Fbuilding-terminal-uis-with-textual\u002Ftesting-textual-apps-with-pilot","Testing Textual Apps with Pilot",{"path":3256,"title":3257},"\u002Fadvanced-input-parsing-user-experience\u002Fcli-help-output-and-documentation\u002Fadding-examples-and-epilogs-to-help-output","Adding Examples and Epilogs to Help Output",{"path":3259,"title":3260},"\u002Fadvanced-input-parsing-user-experience\u002Fcli-help-output-and-documentation\u002Fdocumenting-environment-variables-in-help","Documenting Environment Variables in CLI Help",{"path":3262,"title":3263},"\u002Fadvanced-input-parsing-user-experience\u002Fcli-help-output-and-documentation\u002Fgenerating-man-pages-and-docs-from-a-cli","Generating Man Pages and Docs from a CLI",{"path":3265,"title":3266},"\u002Fadvanced-input-parsing-user-experience\u002Fcli-help-output-and-documentation","CLI Help Output and Documentation",{"path":3268,"title":3269},"\u002Fadvanced-input-parsing-user-experience\u002Fcli-help-output-and-documentation\u002Frich-formatted-help-with-rich-click","Rich-Formatted Help for Click CLIs with rich-click",{"path":3271,"title":3272},"\u002Fadvanced-input-parsing-user-experience\u002Fcli-help-output-and-documentation\u002Fversioning-and-deprecating-cli-flags","Versioning and Deprecating CLI Flags",{"path":3274,"title":3275},"\u002Fadvanced-input-parsing-user-experience\u002Fcli-help-output-and-documentation\u002Fwriting-help-text-users-actually-read","Writing Help Text Users Actually Read",{"path":3277,"title":3278},"\u002Fadvanced-input-parsing-user-experience\u002Fcross-platform-terminal-compatibility\u002Fadapting-output-to-terminal-width","Adapting Python CLI Output to Terminal Width",{"path":3280,"title":3281},"\u002Fadvanced-input-parsing-user-experience\u002Fcross-platform-terminal-compatibility\u002Fdetecting-ci-environments-and-non-interactive-shells","Detecting CI Environments and Non-Interactive Shells",{"path":3283,"title":3284},"\u002Fadvanced-input-parsing-user-experience\u002Fcross-platform-terminal-compatibility\u002Ffixing-unicode-and-encoding-errors-on-windows","Fixing Unicode and Encoding Errors on Windows in Python CLIs",{"path":3286,"title":3287},"\u002Fadvanced-input-parsing-user-experience\u002Fcross-platform-terminal-compatibility\u002Fhandling-ansi-escape-codes-on-windows-consoles","Handling ANSI Escape Codes on Windows Consoles",{"path":3289,"title":3290},"\u002Fadvanced-input-parsing-user-experience\u002Fcross-platform-terminal-compatibility","Cross-Platform Terminal Compatibility for Python CLIs",{"path":3292,"title":3293},"\u002Fadvanced-input-parsing-user-experience\u002Fcross-platform-terminal-compatibility\u002Frespecting-no-color-and-force-color","Respecting NO_COLOR and FORCE_COLOR in Python CLIs",{"path":3295,"title":3296},"\u002Fadvanced-input-parsing-user-experience\u002Fcross-platform-terminal-compatibility\u002Fsupporting-dumb-terminals-and-screen-readers","Supporting Dumb Terminals and Screen Readers in a Python CLI",{"path":3298,"title":3299},"\u002Fadvanced-input-parsing-user-experience\u002Ferror-handling-and-exit-codes\u002Fchoosing-exit-codes-for-cli-tools","Choosing Exit Codes for CLI Tools",{"path":3301,"title":3302},"\u002Fadvanced-input-parsing-user-experience\u002Ferror-handling-and-exit-codes\u002Fdesigning-an-exception-hierarchy-for-a-cli","Designing an Exception Hierarchy for a Python CLI",{"path":3304,"title":3305},"\u002Fadvanced-input-parsing-user-experience\u002Ferror-handling-and-exit-codes\u002Fdid-you-mean-suggestions-for-mistyped-input","Did You Mean…? Suggestions for Mistyped CLI Input",{"path":3307,"title":3308},"\u002Fadvanced-input-parsing-user-experience\u002Ferror-handling-and-exit-codes\u002Ffriendly-error-messages-and-tracebacks","Friendly Error Messages and Tracebacks",{"path":3310,"title":3311},"\u002Fadvanced-input-parsing-user-experience\u002Ferror-handling-and-exit-codes\u002Fhandling-keyboard-interrupt-cleanly","Handling Keyboard Interrupt Cleanly",{"path":3313,"title":3314},"\u002Fadvanced-input-parsing-user-experience\u002Ferror-handling-and-exit-codes","Error Handling and Exit Codes for CLIs",{"path":3316,"title":3317},"\u002Fadvanced-input-parsing-user-experience\u002Ferror-handling-and-exit-codes\u002Freporting-machine-readable-errors-in-json-mode","Reporting Machine-Readable Errors in JSON Mode",{"path":3319,"title":3320},"\u002Fadvanced-input-parsing-user-experience\u002Ferror-handling-and-exit-codes\u002Fwriting-crash-reports-users-can-send","Writing Crash Reports Users Can Send",{"path":3322,"title":3323},"\u002Fadvanced-input-parsing-user-experience\u002Fhandling-configuration-files-env-vars\u002Fconfig-precedence-flags-env-files-defaults","Config Precedence: Flags, Env, Files, Defaults",{"path":3325,"title":3326},"\u002Fadvanced-input-parsing-user-experience\u002Fhandling-configuration-files-env-vars\u002Fdiscovering-project-config-files-by-walking-up-directories","Discovering Project Config Files by Walking Up Directories",{"path":3328,"title":3329},"\u002Fadvanced-input-parsing-user-experience\u002Fhandling-configuration-files-env-vars","Handling Config Files and Env Vars in CLIs",{"path":3331,"title":3332},"\u002Fadvanced-input-parsing-user-experience\u002Fhandling-configuration-files-env-vars\u002Floading-yaml-configs-safely-in-cli-apps","Loading YAML configs safely in CLI apps",{"path":3334,"title":3335},"\u002Fadvanced-input-parsing-user-experience\u002Fhandling-configuration-files-env-vars\u002Freading-toml-config-with-tomllib","Reading TOML Config with tomllib in Python CLIs",{"path":3337,"title":3338},"\u002Fadvanced-input-parsing-user-experience\u002Fhandling-configuration-files-env-vars\u002Ftyped-settings-with-pydantic-settings","Typed Settings with pydantic-settings in Python CLIs",{"path":3340,"title":3341},"\u002Fadvanced-input-parsing-user-experience\u002Fhandling-configuration-files-env-vars\u002Fvalidating-config-files-with-json-schema","Validating Config Files with JSON Schema in a Python CLI",{"path":3343,"title":3344},"\u002Fadvanced-input-parsing-user-experience\u002Fhandling-configuration-files-env-vars\u002Fwriting-a-config-init-and-edit-command","Writing a Config Init and Edit Command for a Python CLI",{"path":3346,"title":3347},"\u002Fadvanced-input-parsing-user-experience","Advanced Input Parsing for Python CLIs",{"path":3349,"title":3350},"\u002Fadvanced-input-parsing-user-experience\u002Finteractive-terminal-ui-with-rich\u002Fadding-progress-bars-and-spinners-to-python-clis","Progress Bars and Spinners for Python CLIs",{"path":3352,"title":3353},"\u002Fadvanced-input-parsing-user-experience\u002Finteractive-terminal-ui-with-rich\u002Fbuilding-interactive-prompts-and-menus","Building Interactive Prompts and Menus in Python CLIs",{"path":3355,"title":3356},"\u002Fadvanced-input-parsing-user-experience\u002Finteractive-terminal-ui-with-rich\u002Fbuilding-tree-views-with-rich","Building Tree Views with Rich in a Python CLI",{"path":3358,"title":3359},"\u002Fadvanced-input-parsing-user-experience\u002Finteractive-terminal-ui-with-rich","Interactive Terminal UI with Rich",{"path":3361,"title":3362},"\u002Fadvanced-input-parsing-user-experience\u002Finteractive-terminal-ui-with-rich\u002Flive-dashboards-with-rich-live","Live Dashboards with Rich Live in Python CLIs",{"path":3364,"title":3365},"\u002Fadvanced-input-parsing-user-experience\u002Finteractive-terminal-ui-with-rich\u002Frendering-markdown-and-syntax-highlighting-with-rich","Rendering Markdown and Syntax Highlighting with Rich",{"path":3367,"title":3368},"\u002Fadvanced-input-parsing-user-experience\u002Finteractive-terminal-ui-with-rich\u002Frendering-tables-and-json-with-rich","Rendering Tables and JSON with Rich",{"path":3370,"title":3371},"\u002Fadvanced-input-parsing-user-experience\u002Finteractive-terminal-ui-with-rich\u002Ftheming-rich-output-consistently","Theming Rich Output Consistently in Python CLIs",{"path":3373,"title":3374},"\u002Fadvanced-input-parsing-user-experience\u002Foutput-formats-for-data-clis\u002Fadding-a-format-flag-for-table-json-and-csv","Adding a Format Flag for Table, JSON and CSV Output",{"path":3376,"title":3377},"\u002Fadvanced-input-parsing-user-experience\u002Foutput-formats-for-data-clis\u002Fcustom-output-templates-with-a-format-string","Custom Output Templates with a Format String in Python CLIs",{"path":3379,"title":3380},"\u002Fadvanced-input-parsing-user-experience\u002Foutput-formats-for-data-clis\u002Fexporting-cli-results-to-files","Exporting CLI Results to Files from a Python CLI",{"path":3382,"title":3383},"\u002Fadvanced-input-parsing-user-experience\u002Foutput-formats-for-data-clis","Output Formats for Data-Heavy Python CLIs",{"path":3385,"title":3386},"\u002Fadvanced-input-parsing-user-experience\u002Foutput-formats-for-data-clis\u002Fselecting-fields-and-columns-from-cli-output","Selecting Fields and Columns from Python CLI Output",{"path":3388,"title":3389},"\u002Fadvanced-input-parsing-user-experience\u002Foutput-formats-for-data-clis\u002Fwriting-csv-and-tsv-output-correctly","Writing CSV and TSV Output Correctly from a Python CLI",{"path":3391,"title":3392},"\u002Fadvanced-input-parsing-user-experience\u002Fshell-completion-for-python-clis\u002Fargcomplete-for-argparse-clis","Tab Completion for argparse CLIs with argcomplete",{"path":3394,"title":3395},"\u002Fadvanced-input-parsing-user-experience\u002Fshell-completion-for-python-clis\u002Fdynamic-completion-values-from-apis-and-files","Dynamic Completion Values from APIs and Files",{"path":3397,"title":3398},"\u002Fadvanced-input-parsing-user-experience\u002Fshell-completion-for-python-clis\u002Fenabling-tab-completion-in-click-and-typer","Enabling Tab Completion in Click and Typer",{"path":3400,"title":3401},"\u002Fadvanced-input-parsing-user-experience\u002Fshell-completion-for-python-clis","Shell Completion for Python CLIs",{"path":3403,"title":3404},"\u002Fadvanced-input-parsing-user-experience\u002Fshell-completion-for-python-clis\u002Finstalling-shell-completion-for-bash-zsh-fish","Installing Shell Completion for bash, zsh, fish",{"path":3406,"title":3407},"\u002Fadvanced-input-parsing-user-experience\u002Fshell-completion-for-python-clis\u002Fshipping-completion-scripts-with-packages","Shipping Shell Completion Scripts with a Python CLI",{"path":3409,"title":3410},"\u002Fadvanced-input-parsing-user-experience\u002Fshell-completion-for-python-clis\u002Ftesting-shell-completion-in-python-clis","Testing Shell Completion in Python CLIs",{"path":3412,"title":3413},"\u002Fadvanced-input-parsing-user-experience\u002Fstructured-logging-for-cli-apps\u002Fadding-trace-ids-and-context-to-cli-logs","Adding Trace IDs and Context to Python CLI Logs",{"path":3415,"title":3416},"\u002Fadvanced-input-parsing-user-experience\u002Fstructured-logging-for-cli-apps\u002Fadding-verbose-and-quiet-logging-flags","Adding Verbose and Quiet Logging Flags",{"path":3418,"title":3419},"\u002Fadvanced-input-parsing-user-experience\u002Fstructured-logging-for-cli-apps","Structured Logging for CLI Apps",{"path":3421,"title":3422},"\u002Fadvanced-input-parsing-user-experience\u002Fstructured-logging-for-cli-apps\u002Flogging-with-structlog-in-a-cli","Logging with structlog in a Python CLI",{"path":3424,"title":3425},"\u002Fadvanced-input-parsing-user-experience\u002Fstructured-logging-for-cli-apps\u002Fseparating-logs-from-program-output","Separating Logs from Program Output in a Python CLI",{"path":3427,"title":3428},"\u002Fadvanced-input-parsing-user-experience\u002Fstructured-logging-for-cli-apps\u002Fstructured-json-logging-in-python-clis","Structured JSON Logging in Python CLIs",{"path":3430,"title":3431},"\u002Fadvanced-input-parsing-user-experience\u002Fstructured-logging-for-cli-apps\u002Fwriting-rotating-log-files-from-a-cli","Writing Rotating Log Files from a Python CLI",{"path":3433,"title":3434},"\u002Fadvanced-input-parsing-user-experience\u002Fworking-with-stdin-stdout-and-pipes\u002Fdetecting-tty-and-adapting-output","Detecting a TTY and Adapting Output",{"path":3436,"title":3437},"\u002Fadvanced-input-parsing-user-experience\u002Fworking-with-stdin-stdout-and-pipes\u002Femitting-json-output-for-scripting","Emitting JSON Output for Scripting",{"path":3439,"title":3440},"\u002Fadvanced-input-parsing-user-experience\u002Fworking-with-stdin-stdout-and-pipes\u002Fhandling-broken-pipe-and-sigpipe","Handling Broken Pipe and SIGPIPE",{"path":3442,"title":3443},"\u002Fadvanced-input-parsing-user-experience\u002Fworking-with-stdin-stdout-and-pipes","Working with stdin, stdout and Pipes",{"path":3445,"title":3446},"\u002Fadvanced-input-parsing-user-experience\u002Fworking-with-stdin-stdout-and-pipes\u002Fnull-delimited-input-and-xargs-compatibility","Null-Delimited Input and xargs Compatibility in Python CLIs",{"path":3199,"title":5},{"path":3449,"title":3450},"\u002Fadvanced-input-parsing-user-experience\u002Fworking-with-stdin-stdout-and-pipes\u002Freading-piped-input-in-python-clis","Reading Piped Input in Python CLIs",{"path":3452,"title":3453},"\u002Fcli-runtime-systems-integration\u002Fcalling-http-apis-from-python-clis\u002Fbuilding-an-api-client-cli-with-httpx","Building an API Client CLI with httpx",{"path":3455,"title":3456},"\u002Fcli-runtime-systems-integration\u002Fcalling-http-apis-from-python-clis\u002Fdownloading-files-with-progress-in-python","Downloading Files with Progress in Python CLIs",{"path":3458,"title":3459},"\u002Fcli-runtime-systems-integration\u002Fcalling-http-apis-from-python-clis","Calling HTTP APIs from Python CLIs",{"path":3461,"title":3462},"\u002Fcli-runtime-systems-integration\u002Fcalling-http-apis-from-python-clis\u002Fmocking-http-in-cli-tests-with-respx","Mocking HTTP in Python CLI Tests with respx",{"path":3464,"title":3465},"\u002Fcli-runtime-systems-integration\u002Fcalling-http-apis-from-python-clis\u002Foauth-device-flow-login-for-clis","OAuth Device Flow Login for Python CLIs",{"path":3467,"title":3468},"\u002Fcli-runtime-systems-integration\u002Fcalling-http-apis-from-python-clis\u002Fpaginating-api-results-in-a-cli","Paginating API Results in a Python CLI",{"path":3470,"title":3471},"\u002Fcli-runtime-systems-integration\u002Fcalling-http-apis-from-python-clis\u002Fretries-and-backoff-for-cli-http-calls","Retries and Backoff for CLI HTTP Calls",{"path":3473,"title":3474},"\u002Fcli-runtime-systems-integration\u002Fcalling-http-apis-from-python-clis\u002Fuploading-files-with-multipart-and-progress","Uploading Files with Multipart and Progress in a Python CLI",{"path":3476,"title":3477},"\u002Fcli-runtime-systems-integration\u002Fconcurrency-and-async-in-python-clis\u002Fcancelling-async-tasks-on-ctrl-c","Cancelling Async Tasks on Ctrl+C in Python CLIs",{"path":3479,"title":3480},"\u002Fcli-runtime-systems-integration\u002Fconcurrency-and-async-in-python-clis","Concurrency and Async in Python CLIs",{"path":3482,"title":3483},"\u002Fcli-runtime-systems-integration\u002Fconcurrency-and-async-in-python-clis\u002Fmultiprocessing-for-cpu-bound-cli-tasks","Multiprocessing for CPU-Bound CLI Tasks",{"path":3485,"title":3486},"\u002Fcli-runtime-systems-integration\u002Fconcurrency-and-async-in-python-clis\u002Fparallelising-cli-work-with-thread-pools","Parallelising CLI Work with Thread Pools",{"path":3488,"title":3489},"\u002Fcli-runtime-systems-integration\u002Fconcurrency-and-async-in-python-clis\u002Frate-limiting-concurrent-requests-in-clis","Rate-Limiting Concurrent Requests in Python CLIs",{"path":3491,"title":3492},"\u002Fcli-runtime-systems-integration\u002Fconcurrency-and-async-in-python-clis\u002Frunning-async-code-in-typer-and-click","Running Async Code in Typer and Click",{"path":3494,"title":3495},"\u002Fcli-runtime-systems-integration\u002Fconcurrency-and-async-in-python-clis\u002Fshowing-progress-for-concurrent-tasks","Showing Progress for Concurrent Tasks in a Python CLI",{"path":3497,"title":3498},"\u002Fcli-runtime-systems-integration\u002Fconcurrency-and-async-in-python-clis\u002Fstructured-concurrency-with-taskgroups-in-clis","Structured Concurrency with TaskGroups in Python CLIs",{"path":3500,"title":3501},"\u002Fcli-runtime-systems-integration\u002Ffilesystem-paths-and-atomic-writes\u002Fcross-platform-paths-with-pathlib","Cross-Platform Paths with pathlib in CLIs",{"path":3503,"title":3504},"\u002Fcli-runtime-systems-integration\u002Ffilesystem-paths-and-atomic-writes\u002Ffile-locking-for-concurrent-cli-runs","File Locking for Concurrent CLI Runs in Python",{"path":3506,"title":3507},"\u002Fcli-runtime-systems-integration\u002Ffilesystem-paths-and-atomic-writes\u002Fhandling-file-permissions-and-umask-in-clis","Handling File Permissions and umask in Python CLIs",{"path":3509,"title":3510},"\u002Fcli-runtime-systems-integration\u002Ffilesystem-paths-and-atomic-writes","Filesystem Paths and Atomic Writes for CLIs",{"path":3512,"title":3513},"\u002Fcli-runtime-systems-integration\u002Ffilesystem-paths-and-atomic-writes\u002Fsafe-temporary-files-and-directories","Safe Temporary Files and Directories in CLIs",{"path":3515,"title":3516},"\u002Fcli-runtime-systems-integration\u002Ffilesystem-paths-and-atomic-writes\u002Fstoring-app-data-with-platformdirs","Storing CLI App Data with platformdirs",{"path":3518,"title":3519},"\u002Fcli-runtime-systems-integration\u002Ffilesystem-paths-and-atomic-writes\u002Fwalking-directory-trees-with-ignore-rules","Walking Directory Trees with Ignore Rules in a Python CLI",{"path":3521,"title":3522},"\u002Fcli-runtime-systems-integration\u002Ffilesystem-paths-and-atomic-writes\u002Fwriting-files-atomically-in-python-clis","Writing Files Atomically in Python CLIs",{"path":3524,"title":3525},"\u002Fcli-runtime-systems-integration","CLI Runtime & Systems Integration for Python",{"path":3527,"title":3528},"\u002Fcli-runtime-systems-integration\u002Flocal-state-and-sqlite-in-python-clis\u002Fcaching-http-responses-on-disk-in-a-cli","Caching HTTP Responses on Disk in a Python CLI",{"path":3530,"title":3531},"\u002Fcli-runtime-systems-integration\u002Flocal-state-and-sqlite-in-python-clis","Local State and SQLite in Python CLIs",{"path":3533,"title":3534},"\u002Fcli-runtime-systems-integration\u002Flocal-state-and-sqlite-in-python-clis\u002Fmigrating-a-cli-sqlite-schema","Migrating a CLI’s SQLite Schema Between Releases",{"path":3536,"title":3537},"\u002Fcli-runtime-systems-integration\u002Flocal-state-and-sqlite-in-python-clis\u002Frecording-and-querying-cli-run-history","Recording and Querying CLI Run History in SQLite",{"path":3539,"title":3540},"\u002Fcli-runtime-systems-integration\u002Flocal-state-and-sqlite-in-python-clis\u002Fstoring-cli-state-in-sqlite","Storing CLI State in SQLite with a Small Repository Class",{"path":3542,"title":3543},"\u002Fcli-runtime-systems-integration\u002Flong-running-and-watch-mode-clis\u002Fbuilding-a-watch-mode-with-watchfiles","Building a Watch Mode with watchfiles in Python",{"path":3545,"title":3546},"\u002Fcli-runtime-systems-integration\u002Flong-running-and-watch-mode-clis\u002Fhandling-sigterm-and-graceful-shutdown","Handling SIGTERM and Graceful Shutdown in CLIs",{"path":3548,"title":3549},"\u002Fcli-runtime-systems-integration\u002Flong-running-and-watch-mode-clis\u002Fhealth-checks-and-heartbeats-for-long-running-clis","Health Checks and Heartbeats for Long-Running CLIs",{"path":3551,"title":3552},"\u002Fcli-runtime-systems-integration\u002Flong-running-and-watch-mode-clis","Long-Running and Watch-Mode Python CLIs",{"path":3554,"title":3555},"\u002Fcli-runtime-systems-integration\u002Flong-running-and-watch-mode-clis\u002Freloading-config-on-sighup","Reloading Configuration on SIGHUP in a Python CLI",{"path":3557,"title":3558},"\u002Fcli-runtime-systems-integration\u002Flong-running-and-watch-mode-clis\u002Frunning-a-cli-as-a-systemd-service","Running a Python CLI as a systemd Service",{"path":3560,"title":3561},"\u002Fcli-runtime-systems-integration\u002Flong-running-and-watch-mode-clis\u002Frunning-a-cli-on-a-schedule-with-cron-and-systemd","Running a Python CLI on a Schedule with cron and systemd",{"path":3563,"title":3564},"\u002Fcli-runtime-systems-integration\u002Flong-running-and-watch-mode-clis\u002Fsending-cli-logs-to-journald-and-syslog","Sending Python CLI Logs to journald and syslog",{"path":3566,"title":3567},"\u002Fcli-runtime-systems-integration\u002Frunning-subprocesses-from-python-clis\u002Favoiding-shell-injection-in-python-clis","Avoiding Shell Injection in Python CLIs",{"path":3569,"title":3570},"\u002Fcli-runtime-systems-integration\u002Frunning-subprocesses-from-python-clis\u002Fcalling-external-commands-safely-with-subprocess","Calling External Commands Safely with subprocess",{"path":3572,"title":3573},"\u002Fcli-runtime-systems-integration\u002Frunning-subprocesses-from-python-clis\u002Fhandling-subprocess-timeouts-and-exit-codes","Handling Subprocess Timeouts and Exit Codes",{"path":3575,"title":3576},"\u002Fcli-runtime-systems-integration\u002Frunning-subprocesses-from-python-clis","Running Subprocesses from Python CLIs",{"path":3578,"title":3579},"\u002Fcli-runtime-systems-integration\u002Frunning-subprocesses-from-python-clis\u002Flaunching-the-users-editor-from-a-cli","Launching the User’s Editor from a Python CLI",{"path":3581,"title":3582},"\u002Fcli-runtime-systems-integration\u002Frunning-subprocesses-from-python-clis\u002Fpiping-between-subprocesses-in-python","Piping Between Subprocesses in Python Without a Shell",{"path":3584,"title":3585},"\u002Fcli-runtime-systems-integration\u002Frunning-subprocesses-from-python-clis\u002Fstreaming-subprocess-output-in-real-time","Streaming Subprocess Output in Real Time",{"path":3587,"title":3588},"\u002Fcli-runtime-systems-integration\u002Frunning-subprocesses-from-python-clis\u002Fwrapping-git-and-other-tools-from-a-python-cli","Wrapping git and Other Tools from a Python CLI",{"path":3590,"title":3591},"\u002Fcli-runtime-systems-integration\u002Fsecrets-and-credentials-in-python-clis","Secrets and Credentials in Python CLIs",{"path":3593,"title":3594},"\u002Fcli-runtime-systems-integration\u002Fsecrets-and-credentials-in-python-clis\u002Fprompting-for-passwords-securely","Prompting for Passwords Securely in Python CLIs",{"path":3596,"title":3597},"\u002Fcli-runtime-systems-integration\u002Fsecrets-and-credentials-in-python-clis\u002Freading-secrets-from-env-and-files","Reading Secrets from Env Vars and Files in CLIs",{"path":3599,"title":3600},"\u002Fcli-runtime-systems-integration\u002Fsecrets-and-credentials-in-python-clis\u002Fredacting-secrets-from-cli-output-and-logs","Redacting Secrets from CLI Output and Logs",{"path":3602,"title":3603},"\u002Fcli-runtime-systems-integration\u002Fsecrets-and-credentials-in-python-clis\u002Frefreshing-expired-tokens-automatically","Refreshing Expired Tokens Automatically in a Python CLI",{"path":3605,"title":3606},"\u002Fcli-runtime-systems-integration\u002Fsecrets-and-credentials-in-python-clis\u002Fstoring-tokens-with-keyring","Storing CLI Tokens Securely with keyring",{"path":3608,"title":3609},"\u002Fcli-runtime-systems-integration\u002Fsecrets-and-credentials-in-python-clis\u002Fsupporting-multiple-profiles-and-accounts","Supporting Multiple Profiles and Accounts in CLIs",{"path":3611,"title":3612},"\u002Fcli-runtime-systems-integration\u002Fupdate-checks-and-self-updating-clis\u002Fchecking-pypi-for-a-newer-version","Checking PyPI for a Newer Version of Your Python CLI",{"path":3614,"title":3615},"\u002Fcli-runtime-systems-integration\u002Fupdate-checks-and-self-updating-clis","Update Checks, Upgrade Commands and Telemetry for Python CLIs",{"path":3617,"title":3618},"\u002Fcli-runtime-systems-integration\u002Fupdate-checks-and-self-updating-clis\u002Fopt-in-usage-telemetry-for-python-clis","Opt-In Usage Telemetry for Python CLIs Done Responsibly",{"path":3620,"title":3621},"\u002Fcli-runtime-systems-integration\u002Fupdate-checks-and-self-updating-clis\u002Fself-upgrading-a-cli-installed-with-pipx-or-uv","Self-Upgrading a Python CLI Installed with pipx or uv",{"path":3623,"title":3624},"\u002Fcli-runtime-systems-integration\u002Fupdate-checks-and-self-updating-clis\u002Fshowing-non-blocking-update-notices","Showing Non-Blocking Update Notices in a Python CLI",{"path":2278,"title":3626},"Python CLI Toolcraft",{"path":3628,"title":3629},"\u002Fmodern-python-cli-frameworks-architecture\u002Falternative-python-cli-frameworks\u002Fbuilding-a-cli-with-cleo","Building a Python CLI with Cleo Command Classes",{"path":3631,"title":3632},"\u002Fmodern-python-cli-frameworks-architecture\u002Falternative-python-cli-frameworks\u002Fbuilding-a-cli-with-cyclopts","Building a Type-Hinted Python CLI with Cyclopts",{"path":3634,"title":3635},"\u002Fmodern-python-cli-frameworks-architecture\u002Falternative-python-cli-frameworks","Beyond Click and Typer: Alternative Python CLI Frameworks",{"path":3637,"title":3638},"\u002Fmodern-python-cli-frameworks-architecture\u002Falternative-python-cli-frameworks\u002Fquick-clis-from-functions-with-python-fire","Quick CLIs from Functions with Python Fire",{"path":3640,"title":3641},"\u002Fmodern-python-cli-frameworks-architecture\u002Falternative-python-cli-frameworks\u002Fusage-string-driven-clis-with-docopt-ng","Usage-String Driven Python CLIs with docopt-ng",{"path":3643,"title":3644},"\u002Fmodern-python-cli-frameworks-architecture\u002Fcli-startup-performance-and-lazy-loading\u002Favoiding-import-time-side-effects","Avoiding Import-Time Side Effects in a Python CLI",{"path":3646,"title":3647},"\u002Fmodern-python-cli-frameworks-architecture\u002Fcli-startup-performance-and-lazy-loading\u002Fcaching-expensive-work-between-cli-runs","Caching Expensive Work Between Python CLI Runs",{"path":3649,"title":3650},"\u002Fmodern-python-cli-frameworks-architecture\u002Fcli-startup-performance-and-lazy-loading\u002Fguarding-startup-with-import-tests","Guarding CLI Startup with Import Tests",{"path":3652,"title":3653},"\u002Fmodern-python-cli-frameworks-architecture\u002Fcli-startup-performance-and-lazy-loading","CLI Startup Performance and Lazy Loading",{"path":3655,"title":3656},"\u002Fmodern-python-cli-frameworks-architecture\u002Fcli-startup-performance-and-lazy-loading\u002Flazy-loading-subcommands-for-faster-startup","Lazy Loading Subcommands for Faster Startup",{"path":3658,"title":3659},"\u002Fmodern-python-cli-frameworks-architecture\u002Fcli-startup-performance-and-lazy-loading\u002Fprofiling-python-cli-startup-time","Profiling Python CLI Startup Time",{"path":3661,"title":3662},"\u002Fmodern-python-cli-frameworks-architecture\u002Fcli-startup-performance-and-lazy-loading\u002Freducing-cli-dependency-weight","Reducing CLI Dependency Weight",{"path":3664,"title":3665},"\u002Fmodern-python-cli-frameworks-architecture\u002Fcommand-line-parsing-with-argparse\u002Fargparse-subparsers-for-subcommands","argparse Subparsers for Subcommands",{"path":3667,"title":3668},"\u002Fmodern-python-cli-frameworks-architecture\u002Fcommand-line-parsing-with-argparse\u002Fargparse-vs-click-vs-typer-comparison","argparse vs Click vs Typer Compared",{"path":3670,"title":3671},"\u002Fmodern-python-cli-frameworks-architecture\u002Fcommand-line-parsing-with-argparse\u002Fargument-groups-and-help-formatting-in-argparse","Argument Groups and Help Formatting in argparse",{"path":3673,"title":3674},"\u002Fmodern-python-cli-frameworks-architecture\u002Fcommand-line-parsing-with-argparse","Command-Line Parsing with argparse",{"path":3676,"title":3677},"\u002Fmodern-python-cli-frameworks-architecture\u002Fcommand-line-parsing-with-argparse\u002Fmigrating-from-argparse-to-typer","Migrating from argparse to Typer",{"path":3679,"title":3680},"\u002Fmodern-python-cli-frameworks-architecture\u002Fcommand-line-parsing-with-argparse\u002Fmutually-exclusive-options-in-argparse","Mutually Exclusive Options in argparse",{"path":3682,"title":3683},"\u002Fmodern-python-cli-frameworks-architecture\u002Fcommand-line-parsing-with-argparse\u002Freading-arguments-from-files-with-fromfile-prefix-chars","Reading Arguments from Files with argparse’s fromfile_prefix_chars",{"path":3685,"title":3686},"\u002Fmodern-python-cli-frameworks-architecture\u002Fcommand-line-parsing-with-argparse\u002Fwriting-custom-argparse-actions","Writing Custom argparse Actions in Python",{"path":3688,"title":3689},"\u002Fmodern-python-cli-frameworks-architecture\u002Fdesigning-cli-interfaces-and-conventions\u002Fadding-dry-run-and-confirmation-to-destructive-commands","Adding Dry-Run and Confirmation to Destructive Commands",{"path":3691,"title":3692},"\u002Fmodern-python-cli-frameworks-architecture\u002Fdesigning-cli-interfaces-and-conventions\u002Fdesigning-idempotent-commands","Designing Idempotent Commands in a Python CLI",{"path":3694,"title":3695},"\u002Fmodern-python-cli-frameworks-architecture\u002Fdesigning-cli-interfaces-and-conventions\u002Ffollowing-posix-and-gnu-argument-conventions","Following POSIX and GNU Argument Conventions in Python",{"path":3697,"title":3698},"\u002Fmodern-python-cli-frameworks-architecture\u002Fdesigning-cli-interfaces-and-conventions\u002Fglobal-options-vs-per-command-options","Global Options vs Per-Command Options in Python CLIs",{"path":3700,"title":3701},"\u002Fmodern-python-cli-frameworks-architecture\u002Fdesigning-cli-interfaces-and-conventions","Designing CLI Interfaces and Conventions in Python",{"path":3703,"title":3704},"\u002Fmodern-python-cli-frameworks-architecture\u002Fdesigning-cli-interfaces-and-conventions\u002Fnaming-commands-and-flags-consistently","Naming Commands and Flags Consistently in Python CLIs",{"path":3706,"title":3707},"\u002Fmodern-python-cli-frameworks-architecture\u002Fdesigning-cli-interfaces-and-conventions\u002Fpositional-arguments-vs-options","Positional Arguments vs Options: Designing a CLI Signature",{"path":3709,"title":3710},"\u002Fmodern-python-cli-frameworks-architecture","Python CLI Frameworks and Architecture",{"path":3712,"title":3713},"\u002Fmodern-python-cli-frameworks-architecture\u002Fplugin-architectures-for-extensible-clis\u002Fdiscovering-plugins-with-entry-points","Discovering Plugins with Entry Points in Python CLIs",{"path":3715,"title":3716},"\u002Fmodern-python-cli-frameworks-architecture\u002Fplugin-architectures-for-extensible-clis\u002Fhook-based-plugins-with-pluggy","Hook-Based Plugins for Python CLIs with pluggy",{"path":3718,"title":3719},"\u002Fmodern-python-cli-frameworks-architecture\u002Fplugin-architectures-for-extensible-clis","Plugin Architectures for Extensible CLIs",{"path":3721,"title":3722},"\u002Fmodern-python-cli-frameworks-architecture\u002Fplugin-architectures-for-extensible-clis\u002Fisolating-plugin-failures","Isolating Plugin Failures in an Extensible Python CLI",{"path":3724,"title":3725},"\u002Fmodern-python-cli-frameworks-architecture\u002Fplugin-architectures-for-extensible-clis\u002Ftesting-plugins-against-the-host-cli","Testing Plugins Against the Host CLI",{"path":3727,"title":3728},"\u002Fmodern-python-cli-frameworks-architecture\u002Fplugin-architectures-for-extensible-clis\u002Fversioning-a-plugin-api","Versioning a Plugin API for a Python CLI",{"path":3730,"title":3731},"\u002Fmodern-python-cli-frameworks-architecture\u002Fplugin-architectures-for-extensible-clis\u002Fwriting-a-plugin-for-an-existing-cli","Writing a Plugin for an Existing CLI",{"path":3733,"title":3734},"\u002Fmodern-python-cli-frameworks-architecture\u002Fstructuring-multi-command-python-clis\u002Fbest-practices-for-python-cli-entry-points","Best practices for Python CLI entry points",{"path":3736,"title":3737},"\u002Fmodern-python-cli-frameworks-architecture\u002Fstructuring-multi-command-python-clis\u002Fdependency-injection-patterns-for-cli-commands","Dependency Injection Patterns for CLI Commands",{"path":3739,"title":3740},"\u002Fmodern-python-cli-frameworks-architecture\u002Fstructuring-multi-command-python-clis\u002Fhow-to-structure-a-large-python-cli-project","Structuring a Large Python CLI Project",{"path":3742,"title":3743},"\u002Fmodern-python-cli-frameworks-architecture\u002Fstructuring-multi-command-python-clis","Structuring Multi-Command Python CLIs",{"path":3745,"title":3746},"\u002Fmodern-python-cli-frameworks-architecture\u002Fstructuring-multi-command-python-clis\u002Foffering-a-python-api-alongside-your-cli","Offering a Python API Alongside Your CLI",{"path":3748,"title":3749},"\u002Fmodern-python-cli-frameworks-architecture\u002Fstructuring-multi-command-python-clis\u002Fregistering-commands-from-modules-automatically","Registering CLI Commands from Modules Automatically",{"path":3751,"title":3752},"\u002Fmodern-python-cli-frameworks-architecture\u002Fstructuring-multi-command-python-clis\u002Fsharing-common-options-across-commands","Sharing Common Options Across Python CLI Commands",{"path":3754,"title":3755},"\u002Fmodern-python-cli-frameworks-architecture\u002Fstructuring-multi-command-python-clis\u002Fsharing-state-with-click-context-objects","Sharing State with Click Context Objects",{"path":3757,"title":3758},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftesting-python-cli-applications\u002Fend-to-end-testing-an-installed-cli","End-to-End Testing an Installed Python CLI",{"path":3760,"title":3761},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftesting-python-cli-applications","Testing Python CLI Applications",{"path":3763,"title":3764},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftesting-python-cli-applications\u002Fmeasuring-cli-test-coverage","Measuring CLI Test Coverage",{"path":3766,"title":3767},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftesting-python-cli-applications\u002Fmocking-filesystem-and-network-in-cli-tests","Mocking the Filesystem and Network in CLI Tests",{"path":3769,"title":3770},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftesting-python-cli-applications\u002Fproperty-based-testing-cli-arguments-with-hypothesis","Property-Based Testing CLI Arguments with Hypothesis",{"path":3772,"title":3773},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftesting-python-cli-applications\u002Fsnapshot-testing-cli-output","Snapshot Testing CLI Output",{"path":3775,"title":3776},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftesting-python-cli-applications\u002Ftesting-click-commands-with-clirunner","Testing Click Commands with CliRunner",{"path":3778,"title":3779},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftesting-python-cli-applications\u002Ftesting-commands-that-call-subprocesses","Testing Python CLI Commands That Call Subprocesses",{"path":3781,"title":3782},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftesting-python-cli-applications\u002Ftesting-interactive-prompts-and-stdin","Testing Interactive Prompts and stdin",{"path":3784,"title":3785},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftesting-python-cli-applications\u002Ftranscript-tests-for-cli-commands","Transcript Tests for Python CLI Commands",{"path":3787,"title":3788},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftyper-vs-click-when-to-use-each\u002Fbuilding-a-cli-with-subcommands-in-click","Building a CLI with subcommands in Click",{"path":3790,"title":3791},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftyper-vs-click-when-to-use-each\u002Fbuilding-dynamic-commands-in-click","Building Dynamic Commands in Click",{"path":3793,"title":3794},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftyper-vs-click-when-to-use-each\u002Fchoices-and-enums-in-typer-and-click","Choices and Enums in Typer and Click Options",{"path":3796,"title":3797},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftyper-vs-click-when-to-use-each\u002Fclick-option-callbacks-and-eager-options","Click Option Callbacks and Eager Options Explained",{"path":3799,"title":3800},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftyper-vs-click-when-to-use-each\u002Fconverting-a-click-app-to-typer","Converting a Click App to Typer",{"path":3802,"title":3803},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftyper-vs-click-when-to-use-each","Typer vs Click: When to Use Each",{"path":3805,"title":3806},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftyper-vs-click-when-to-use-each\u002Frich-markup-and-help-panels-in-typer","Rich Markup and Help Panels in Typer",{"path":3808,"title":3809},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftyper-vs-click-when-to-use-each\u002Ftyper-callback-functions-explained","Typer callback functions explained",{"path":3811,"title":3812},"\u002Fmodern-python-cli-frameworks-architecture\u002Ftyper-vs-click-when-to-use-each\u002Fusing-annotated-options-in-typer","Using Annotated Options in Typer",{"path":3814,"title":3815},"\u002Fproject-setup-dependency-management\u002Fci-cd-pipelines-for-python-clis\u002Fautomating-releases-from-git-tags","Automating Python CLI Releases from Git Tags",{"path":3817,"title":3818},"\u002Fproject-setup-dependency-management\u002Fci-cd-pipelines-for-python-clis\u002Fcaching-uv-dependencies-in-ci","Caching uv Dependencies in CI for Python CLIs",{"path":3820,"title":3821},"\u002Fproject-setup-dependency-management\u002Fci-cd-pipelines-for-python-clis","CI\u002FCD Pipelines for Python CLIs",{"path":3823,"title":3824},"\u002Fproject-setup-dependency-management\u002Fci-cd-pipelines-for-python-clis\u002Fpublishing-a-cli-docker-image-from-ci","Publishing a Python CLI as a Docker Image from CI",{"path":3826,"title":3827},"\u002Fproject-setup-dependency-management\u002Fci-cd-pipelines-for-python-clis\u002Fpublishing-to-pypi-with-trusted-publishing","Publishing a CLI to PyPI with Trusted Publishing",{"path":3829,"title":3830},"\u002Fproject-setup-dependency-management\u002Fci-cd-pipelines-for-python-clis\u002Frunning-cli-tests-on-windows-and-macos-runners","Running Python CLI Tests on Windows and macOS Runners",{"path":3832,"title":3833},"\u002Fproject-setup-dependency-management\u002Fci-cd-pipelines-for-python-clis\u002Fsmoke-testing-the-built-wheel-in-ci","Smoke-Testing the Built Wheel of a Python CLI in CI",{"path":3835,"title":3836},"\u002Fproject-setup-dependency-management\u002Fci-cd-pipelines-for-python-clis\u002Ftesting-a-cli-across-python-versions-with-github-actions","Testing a CLI Across Python Versions in GitHub Actions",{"path":3838,"title":3839},"\u002Fproject-setup-dependency-management\u002Fcli-project-scaffolding-with-cookiecutter\u002Fbuilding-a-cookiecutter-template-for-typer-clis","Building a Cookiecutter Template for Typer CLIs",{"path":3841,"title":3842},"\u002Fproject-setup-dependency-management\u002Fcli-project-scaffolding-with-cookiecutter\u002Fcopier-vs-cookiecutter-for-cli-templates","Copier vs Cookiecutter for CLI Templates",{"path":3844,"title":3845},"\u002Fproject-setup-dependency-management\u002Fcli-project-scaffolding-with-cookiecutter","CLI Project Scaffolding with Cookiecutter",{"path":3847,"title":3848},"\u002Fproject-setup-dependency-management\u002Fcli-project-scaffolding-with-cookiecutter\u002Fpost-generation-hooks-in-cli-templates","Post-Generation Hooks in Python CLI Templates",{"path":3850,"title":3851},"\u002Fproject-setup-dependency-management\u002Fcli-project-scaffolding-with-cookiecutter\u002Ftemplate-variables-and-conditional-files","Template Variables and Conditional Files in CLI Templates",{"path":3853,"title":3854},"\u002Fproject-setup-dependency-management\u002Fcli-project-scaffolding-with-cookiecutter\u002Ftesting-a-project-template-with-pytest","Testing a CLI Project Template with pytest",{"path":3856,"title":3857},"\u002Fproject-setup-dependency-management\u002Fcli-project-scaffolding-with-cookiecutter\u002Fupdating-generated-projects-with-copier-update","Updating Generated CLI Projects with copier update",{"path":3859,"title":3860},"\u002Fproject-setup-dependency-management\u002Fdistributing-clis-as-standalone-binaries\u002Fbuilding-cross-platform-release-binaries-in-ci","Building Cross-Platform Release Binaries in CI",{"path":3862,"title":3863},"\u002Fproject-setup-dependency-management\u002Fdistributing-clis-as-standalone-binaries\u002Fbundling-a-python-cli-with-pyinstaller","Bundling a Python CLI with PyInstaller",{"path":3865,"title":3866},"\u002Fproject-setup-dependency-management\u002Fdistributing-clis-as-standalone-binaries\u002Fcode-signing-and-notarizing-cli-binaries","Code Signing and Notarizing Python CLI Binaries",{"path":3868,"title":3869},"\u002Fproject-setup-dependency-management\u002Fdistributing-clis-as-standalone-binaries\u002Fhomebrew-and-scoop-packaging-for-python-clis","Homebrew and Scoop Packaging for Python CLIs",{"path":3871,"title":3872},"\u002Fproject-setup-dependency-management\u002Fdistributing-clis-as-standalone-binaries","Distributing CLIs as Standalone Binaries",{"path":3874,"title":3875},"\u002Fproject-setup-dependency-management\u002Fdistributing-clis-as-standalone-binaries\u002Fnuitka-vs-pyinstaller-for-python-clis","Nuitka vs PyInstaller for Python CLI Binaries",{"path":3877,"title":3878},"\u002Fproject-setup-dependency-management\u002Fdistributing-clis-as-standalone-binaries\u002Freducing-pyinstaller-binary-size","Reducing the Size of a PyInstaller CLI Binary",{"path":3880,"title":3881},"\u002Fproject-setup-dependency-management\u002Fdistributing-clis-as-standalone-binaries\u002Fshipping-a-cli-as-a-zipapp-with-shiv","Shipping a CLI as a Zipapp with shiv",{"path":3883,"title":3884},"\u002Fproject-setup-dependency-management","Project Setup & Dependency Management",{"path":3886,"title":3887},"\u002Fproject-setup-dependency-management\u002Flinting-and-type-checking-cli-code\u002Fconfiguring-ruff-for-a-cli-project","Configuring Ruff for a Python CLI Project",{"path":3889,"title":3890},"\u002Fproject-setup-dependency-management\u002Flinting-and-type-checking-cli-code\u002Fenforcing-import-boundaries-in-a-cli-codebase","Enforcing Import Boundaries in a Python CLI Codebase",{"path":3892,"title":3893},"\u002Fproject-setup-dependency-management\u002Flinting-and-type-checking-cli-code\u002Ffinding-unused-code-and-dependencies-with-vulture-and-deptry","Finding Unused Code and Dependencies with vulture and deptry",{"path":3895,"title":3896},"\u002Fproject-setup-dependency-management\u002Flinting-and-type-checking-cli-code","Linting and Type-Checking Python CLI Code",{"path":3898,"title":3899},"\u002Fproject-setup-dependency-management\u002Flinting-and-type-checking-cli-code\u002Frunning-pyright-in-strict-mode-on-a-cli","Running Pyright in Strict Mode on a Python CLI",{"path":3901,"title":3902},"\u002Fproject-setup-dependency-management\u002Flinting-and-type-checking-cli-code\u002Frunning-ruff-and-mypy-in-ci-with-annotations","Running Ruff and mypy in CI with Inline Annotations",{"path":3904,"title":3905},"\u002Fproject-setup-dependency-management\u002Flinting-and-type-checking-cli-code\u002Ftype-checking-click-and-typer-code-with-mypy","Type-Checking Click and Typer Code with mypy",{"path":3907,"title":3908},"\u002Fproject-setup-dependency-management\u002Fmanaging-cli-versioning-changelogs\u002Fautomating-changelogs-with-conventional-commits","Automating Changelogs with Conventional Commits",{"path":3910,"title":3911},"\u002Fproject-setup-dependency-management\u002Fmanaging-cli-versioning-changelogs\u002Fbumping-versions-consistently-across-a-cli-project","Bumping Versions Consistently Across a Python CLI Project",{"path":3913,"title":3914},"\u002Fproject-setup-dependency-management\u002Fmanaging-cli-versioning-changelogs\u002Fderiving-versions-from-git-tags-with-hatch-vcs","Deriving CLI Versions from Git Tags with hatch-vcs",{"path":3916,"title":3917},"\u002Fproject-setup-dependency-management\u002Fmanaging-cli-versioning-changelogs\u002Fexposing-version-info-and-build-metadata","Exposing Version Info and Build Metadata",{"path":3919,"title":3920},"\u002Fproject-setup-dependency-management\u002Fmanaging-cli-versioning-changelogs","Managing CLI Versioning & Changelogs",{"path":3922,"title":3923},"\u002Fproject-setup-dependency-management\u002Fmanaging-cli-versioning-changelogs\u002Fsemantic-versioning-policy-for-cli-tools","A Semantic Versioning Policy for CLI Tools",{"path":3925,"title":3926},"\u002Fproject-setup-dependency-management\u002Fmanaging-cli-versioning-changelogs\u002Fshipping-pre-releases-and-release-candidates","Shipping Pre-Releases and Release Candidates of a Python CLI",{"path":3928,"title":3929},"\u002Fproject-setup-dependency-management\u002Fpackaging-python-clis-for-distribution\u002Fbuilding-wheels-and-sdists-for-python-clis","Building Wheels and sdists for Python CLIs",{"path":3931,"title":3932},"\u002Fproject-setup-dependency-management\u002Fpackaging-python-clis-for-distribution\u002Fbundling-data-files-with-importlib-resources","Bundling Data Files with importlib.resources in CLIs",{"path":3934,"title":3935},"\u002Fproject-setup-dependency-management\u002Fpackaging-python-clis-for-distribution\u002Fchoosing-a-build-backend-for-a-python-cli","Choosing a Build Backend for a Python CLI",{"path":3937,"title":3938},"\u002Fproject-setup-dependency-management\u002Fpackaging-python-clis-for-distribution","Packaging Python CLIs for Distribution",{"path":3940,"title":3941},"\u002Fproject-setup-dependency-management\u002Fpackaging-python-clis-for-distribution\u002Finstalling-and-distributing-clis-with-pipx","Installing and Distributing CLIs with pipx",{"path":3943,"title":3944},"\u002Fproject-setup-dependency-management\u002Fpackaging-python-clis-for-distribution\u002Foptional-dependencies-and-extras-for-clis","Optional Dependencies and Extras for Python CLIs",{"path":3946,"title":3947},"\u002Fproject-setup-dependency-management\u002Fpackaging-python-clis-for-distribution\u002Fpublishing-a-python-cli-to-pypi","Publishing a Python CLI to PyPI",{"path":3949,"title":3950},"\u002Fproject-setup-dependency-management\u002Fpackaging-python-clis-for-distribution\u002Fwriting-pyproject-toml-metadata-for-a-cli","Writing pyproject.toml Metadata for a Python CLI",{"path":3952,"title":3953},"\u002Fproject-setup-dependency-management\u002Fpoetry-workflows-for-cli-development\u002Fdynamic-versioning-with-poetry-plugins","Dynamic Versioning for a Poetry CLI from Git Tags",{"path":3955,"title":3956},"\u002Fproject-setup-dependency-management\u002Fpoetry-workflows-for-cli-development","Poetry Workflows for CLI Development",{"path":3958,"title":3959},"\u002Fproject-setup-dependency-management\u002Fpoetry-workflows-for-cli-development\u002Fmanaging-poetry-lock-files-for-cli-tools","Managing Poetry Lock Files for CLI Tools",{"path":3961,"title":3962},"\u002Fproject-setup-dependency-management\u002Fpoetry-workflows-for-cli-development\u002Fmigrating-a-cli-from-poetry-to-uv","Migrating a Python CLI from Poetry to uv",{"path":3964,"title":3965},"\u002Fproject-setup-dependency-management\u002Fpoetry-workflows-for-cli-development\u002Fpoetry-dependency-groups-for-cli-tooling","Poetry Dependency Groups for CLI Tooling",{"path":3967,"title":3968},"\u002Fproject-setup-dependency-management\u002Fpoetry-workflows-for-cli-development\u002Fpoetry-entry-points-and-scripts-for-clis","Poetry Entry Points and Scripts for CLIs",{"path":3970,"title":3971},"\u002Fproject-setup-dependency-management\u002Fpoetry-workflows-for-cli-development\u002Fpublishing-a-cli-with-poetry","Building and Publishing a Python CLI with Poetry",{"path":3973,"title":3974},"\u002Fproject-setup-dependency-management\u002Fpre-commit-hooks-for-cli-projects","Pre-commit Hooks for CLI Projects",{"path":3976,"title":3977},"\u002Fproject-setup-dependency-management\u002Fpre-commit-hooks-for-cli-projects\u002Fkeeping-hook-versions-current-with-autoupdate","Keeping pre-commit Hook Versions Current with autoupdate",{"path":3979,"title":3980},"\u002Fproject-setup-dependency-management\u002Fpre-commit-hooks-for-cli-projects\u002Frunning-pre-commit-in-ci","Running pre-commit in CI for a Python CLI Repository",{"path":3982,"title":3983},"\u002Fproject-setup-dependency-management\u002Fpre-commit-hooks-for-cli-projects\u002Fsetting-up-pre-commit-for-python-cli-repos","Setting up pre-commit for Python CLI repos",{"path":3985,"title":3986},"\u002Fproject-setup-dependency-management\u002Fpre-commit-hooks-for-cli-projects\u002Fshipping-your-cli-as-a-pre-commit-hook","Shipping Your Python CLI as a pre-commit Hook",{"path":3988,"title":3989},"\u002Fproject-setup-dependency-management\u002Fpre-commit-hooks-for-cli-projects\u002Fspeeding-up-slow-pre-commit-hooks","Speeding Up Slow pre-commit Hooks in a CLI Repository",{"path":3991,"title":3992},"\u002Fproject-setup-dependency-management\u002Fpre-commit-hooks-for-cli-projects\u002Fwriting-local-pre-commit-hooks-in-python","Writing Local pre-commit Hooks in Python",{"path":3994,"title":3995},"\u002Fproject-setup-dependency-management\u002Fsecuring-a-python-cli-supply-chain\u002Fauditing-dependencies-with-pip-audit","Auditing a Python CLI’s Dependencies with pip-audit",{"path":3997,"title":3998},"\u002Fproject-setup-dependency-management\u002Fsecuring-a-python-cli-supply-chain\u002Fgenerating-an-sbom-for-a-python-cli","Generating an SBOM for a Python CLI Release",{"path":4000,"title":4001},"\u002Fproject-setup-dependency-management\u002Fsecuring-a-python-cli-supply-chain","Securing the Supply Chain of a Python CLI",{"path":4003,"title":4004},"\u002Fproject-setup-dependency-management\u002Fsecuring-a-python-cli-supply-chain\u002Fpinning-dependencies-with-hashes","Pinning a Python CLI’s Dependencies with Hashes",{"path":4006,"title":4007},"\u002Fproject-setup-dependency-management\u002Fsecuring-a-python-cli-supply-chain\u002Fpublishing-attestations-and-verifying-releases","Publishing Attestations and Verifying Python CLI Releases",{"path":4009,"title":4010},"\u002Fproject-setup-dependency-management\u002Fuv-for-python-cli-dependency-management\u002Fbuilding-and-publishing-a-cli-with-uv","Building and Publishing a Python CLI with uv",{"path":4012,"title":4013},"\u002Fproject-setup-dependency-management\u002Fuv-for-python-cli-dependency-management","uv for Python CLI Dependency Management",{"path":4015,"title":4016},"\u002Fproject-setup-dependency-management\u002Fuv-for-python-cli-dependency-management\u002Flocking-and-syncing-cli-dependencies-with-uv","Locking and Syncing a Python CLI’s Dependencies with uv",{"path":4018,"title":4019},"\u002Fproject-setup-dependency-management\u002Fuv-for-python-cli-dependency-management\u002Frunning-one-off-cli-scripts-with-uv-run","Running One-Off CLI Scripts with uv run and PEP 723",{"path":4021,"title":4022},"\u002Fproject-setup-dependency-management\u002Fuv-for-python-cli-dependency-management\u002Fusing-private-package-indexes-with-uv","Using Private Package Indexes with uv for Internal CLIs",{"path":4024,"title":4025},"\u002Fproject-setup-dependency-management\u002Fuv-for-python-cli-dependency-management\u002Fuv-init-vs-poetry-init-for-cli-tools","uv init vs poetry init for CLI tools",{"path":4027,"title":4028},"\u002Fproject-setup-dependency-management\u002Fuv-for-python-cli-dependency-management\u002Fuv-tool-install-vs-pipx-for-clis","uv tool install vs pipx for CLIs",{"path":4030,"title":4031},"\u002Fproject-setup-dependency-management\u002Fuv-for-python-cli-dependency-management\u002Fuv-workspaces-for-multi-package-clis","uv Workspaces for Multi-Package Python CLIs",{"path":4033,"title":4034},"\u002Fproject-setup-dependency-management\u002Fvirtual-environments-isolation-best-practices\u002Fdebugging-wrong-python-and-wrong-venv-problems","Debugging Wrong-Python and Wrong-Venv Problems in CLIs",{"path":4036,"title":4037},"\u002Fproject-setup-dependency-management\u002Fvirtual-environments-isolation-best-practices\u002Fexternally-managed-environments-and-pep-668","PEP 668 and Python CLIs: the externally-managed-environment Error",{"path":4039,"title":4040},"\u002Fproject-setup-dependency-management\u002Fvirtual-environments-isolation-best-practices","Python CLI Env Isolation Best Practices",{"path":4042,"title":4043},"\u002Fproject-setup-dependency-management\u002Fvirtual-environments-isolation-best-practices\u002Fisolating-cli-tools-from-project-dependencies","Isolating CLI Tools from Your Project’s Dependencies",{"path":4045,"title":4046},"\u002Fproject-setup-dependency-management\u002Fvirtual-environments-isolation-best-practices\u002Fmanaging-virtual-environments-for-cross-platform-clis","Managing Python CLI Virtual Environments",{"path":4048,"title":4049},"\u002Fproject-setup-dependency-management\u002Fvirtual-environments-isolation-best-practices\u002Fpinning-the-python-version-for-a-cli","Pinning the Python Version for a CLI",{"path":4051,"title":4052},"\u002Fproject-setup-dependency-management\u002Fvirtual-environments-isolation-best-practices\u002Fsupporting-multiple-python-versions-with-nox","Supporting Multiple Python Versions with nox",1790967537242]