PyInstaller vs Nuitka vs cx_Freeze: A Measured Packaging Benchmark (Build Time, Size, Launch Time)

PyInstaller vs Nuitka vs cx_Freeze: A Measured Packaging Benchmark

Comparisons of PyInstaller and Nuitka tend to say the same things: PyInstaller is easier, Nuitka compiles to C and is “faster”, cx_Freeze is the old reliable one. This page attaches measurements to those claims. Three small scripts were built five ways on one machine, and each of the fifteen artifacts was launched six times with a stopwatch around it. Everything below comes from that run’s results.json.

Measured on macOS 26.5 (Apple silicon), built from Python 3.13.1. The numbers are not Windows numbers. One caveat found in review: the cx_Freeze artifacts run a different interpreter build than the other two tools’ (its own universal 3.13.15 library, not the venv’s 3.13.1 — see “What all five got right”), so its column is not an apples-to-apples comparison of the packager alone. The two findings that matter most are about mechanisms — onefile builds unpacked themselves on every launch here, and PyInstaller’s matplotlib hook rebuilds the font cache on every launch by design — but the seconds attached to them are this machine’s seconds, and whether the first behaves the same elsewhere was not tested. The limits section at the end is specific about that.

The short version:

• All fifteen builds succeeded and all ninety launches produced the right output. For a stdlib script, a pandas script and a matplotlib script, none of the three tools needed a hidden import, a hook or a flag beyond --onefile / --standalone.
• Onefile is slow on every launch, not just the first. The pandas onefile builds took 32–36 s to start, every time, for both PyInstaller and Nuitka; even the stdlib onefile builds took 3–9 s per launch. The same programs as a folder started in 0.02–0.21 s once the first launch was out of the way. A onefile binary unpacks itself into a fresh temporary directory each run, and on this machine a fresh set of executable files costs seconds the first time it runs.
• PyInstaller’s matplotlib builds pay 8 s per launch for the font cache, even as a folder. Its runtime hook deliberately points matplotlib at a new, empty config directory on every run. Nuitka and cx_Freeze builds of the same script launched in 0.23 s.
• Nuitka’s folder builds take 5–10× longer than PyInstaller’s (75 s vs 14 s for the pandas script; the onefile-to-onefile gap is smaller, 2.5–4.9×) and its standalone folder is larger (100 MB vs 81 MB). Its onefile binary is the smallest of the single-file options (22.6 MB vs 35.9 MB), and its folder launches as fast as PyInstaller’s.
• cx_Freeze ships the most: 149 MB and 2,273 files for the pandas script, nearly double PyInstaller’s folder. It built the stdlib script fastest (1.3 s) but was slower than PyInstaller on the two larger scripts (19 s vs 14 s and 13 s).

The decision table

What was measured, and how

Three probe scripts, each printing a sentinel string on success (SENTINEL_HELLO_OK and so on) so a launch can be checked, not just timed:

Five build configurations, each with its default options plus the one flag that selects the mode:

Per artifact: the build was timed wall-clock in a fresh output directory; the artifact’s size was taken as the single file’s size, or the sum of every regular file in the folder; then the artifact was launched six times in a row, each run timed and its stdout checked for the sentinel. Run 1 is the first launch after the build. “Later launches” below is the median of runs 2–6.

Environment (from the run record, not typed in): macOS 26.5.2, Apple silicon (arm64), 16 cores, Apple clang 21.0.0; Python 3.13.1 from Homebrew in a fresh venv; PyInstaller 6.22.3 with pyinstaller-hooks-contrib 2026.8; Nuitka 4.3.1 (which ccache found nothing on this machine, recorded with the run, so no compiler cache was available to it); cx_Freeze 8.7.2; pandas 3.0.6, numpy 2.5.4, matplotlib 3.11.2. Nuitka’s onefile builds had zstandard 0.25.0 available for compression.

Results

hello.py (stdlib only)

pandas_csv.py (pandas + numpy)

plot.py (matplotlib + numpy)

Three things stand out in those tables, and each got a follow-up measurement.

Onefile is slow on every launch, and the reason is the first-launch cost

Look at the “First launch” column for the folder builds: 6.5 s for a stdlib script that then takes 0.03 s, and 50.6 s for a pandas folder that then takes 0.21 s. Something expensive happens the first time a freshly built set of files is executed.

Now look at the onefile rows: the pandas onefile builds took 32–36 s on every launch, and 35 s is roughly what the equivalent folder cost on its first launch. The hypothesis that follows is that a onefile binary unpacks itself into a new temporary directory on each run, so every run is a first run of brand-new files. Two checks were run on that.

Check 1 — copy the folder, launch again. The already-launched PyInstaller onedir folders were copied to a fresh temporary directory (new files, identical bytes) and launched three times (excerpt of the check’s output):

  hello/pyinstaller-onedir: benchmark cold=6.5462 s warm_median=0.0297 s
     original dir, launch again             : 0.115 s rc=0
     copytree took 0.03 s; fresh copy launches 1/2/3: [6.333, 0.043, 0.032] s rc=[0, 0, 0]
     original dir after touching every file  : 0.408 s rc=0
  pandas_csv/pyinstaller-onedir: benchmark cold=50.5669 s warm_median=0.2111 s
     original dir, launch again             : 0.672 s rc=0
     copytree took 0.171 s; fresh copy launches 1/2/3: [38.671, 0.229, 0.199] s rc=[0, 0, 0]
     original dir after touching every file  : 1.065 s rc=0

Copying took 0.17 s; the first launch of the copy took 38.7 s; the second took 0.23 s. The same thing happened for the cx_Freeze and Nuitka folders (3.0 s and 3.4 s for the fresh copies of the stdlib build). Touching the original files’ modification times cost 0.4–1.1 s against 6–39 s for the fresh copies, so it is new files, not new timestamps, that trigger the cost. This is consistent with macOS checking newly created executables and libraries the first time they run; which subsystem does the checking was not measured, and the page does not claim to know. What is measured is that the cost is per new file set and that it is large: the fresh copy of the 384-file pandas folder cost 39 s and the 45-file stdlib folder about 6 s, although the relationship is not a simple count — see the limits section.

Check 2 — watch the temp directory while a onefile binary runs. The stdlib and pandas onefile artifacts of both tools (four of the six onefile builds) were launched while the system temp directory was polled for new entries:

  hello/pyinstaller-onefile launch 1: 4.477 s rc=0; new temp dirs seen during run: {'_MEI00011410MsKisD': 'first seen at 0.066 s'}; still present after exit: none
  hello/pyinstaller-onefile launch 2: 4.579 s rc=0; new temp dirs seen during run: {'_MEI00011459VVmRzE': 'first seen at 0.005 s'}; still present after exit: none
     extracted tree at the last mid-run snapshot (files, bytes): {'_MEI00011464QaET1k': (44, 16344449)}
  pandas_csv/pyinstaller-onefile launch 1: 35.855 s rc=0; new temp dirs seen during run: {'_MEI0001147cYdQgCL': 'first seen at 0.068 s'}; still present after exit: none
     extracted tree at the last mid-run snapshot (files, bytes): {'_MEI000114dc69oEfd': (383, 69619347)}
  pandas_csv/nuitka-onefile launch 1: 34.269 s rc=0; new temp dirs seen during run: {'onefile_70936_759160_QkAu9Whvxm0': 'first seen at 0.06 s'}; still present after exit: none
     extracted tree at the last mid-run snapshot (files, bytes): {'onefile_71043_551867_BhvzylBBEZ0': (118, 100211476)}

Every launch created a new directory (_MEI… for PyInstaller, onefile_<pid>_… for Nuitka), unpacked the full folder build into it — 383 files and 70 MB for PyInstaller’s pandas build, 118 files and 100 MB for Nuitka’s — and deleted it on exit. The next launch did it again, with a new name. So a onefile binary never gets a second launch in the sense the folder builds do; every launch pays the first-launch cost of its own unpacked tree, plus the unpacking itself.

The practical reading: onefile trades a one-time cost for a per-launch cost. For a stdlib-sized program that is 3–9 s per launch on this machine (3.3–8.6 s across the benchmark and the later checks); for a pandas-sized program it is over 30 s. If your users launch the program more than once, a folder (or a proper installer around a folder) is the faster thing to ship, and the single-file convenience is paid for by every one of them, every time.

Two smaller observations from the same data. PyInstaller’s stdlib onefile launches drifted from 4.4 s up to 8.6 s across the six benchmark runs, then came back at 4.5 s in the later check; the page has no explanation and reports it as noise it could not pin down. And Nuitka’s onefile launches were a little faster than PyInstaller’s on the two larger probes (32.5 s vs 35.4 s pandas, 18.5 s vs 26.5 s matplotlib) — the matplotlib gap is the font cache, explained next; the pandas gap is small enough that this measurement cannot say much about it. On the stdlib probe the order flipped between runs (PyInstaller 4.4–8.6 s, Nuitka 6.2 s), so no ordering is claimed there.

PyInstaller’s matplotlib builds rebuild the font cache every launch

The matplotlib folder build from PyInstaller launched in 7.7 s after the first run, where the Nuitka and cx_Freeze folders launched in 0.23 s. The benchmark records each run’s stderr, and all six PyInstaller runs ended with the same line:

Matplotlib is building the font cache; this may take a moment.

PyInstaller ships a runtime hook for matplotlib, pyi_rth_mplconfig.py, whose comment explains the choice: a onefile build’s font cache would otherwise point at a temporary directory that no longer exists on the next launch, so the hook sets MPLCONFIGDIR to a freshly created temporary directory on every start and deletes it at exit. The hook is applied to onedir builds too, so a folder build pays for it as well.

How much is the font cache on its own? The same plot.py, run from the plain venv with MPLCONFIGDIR pointed at a new empty directory, and then again with the cache in place:

     plain venv, MPLCONFIGDIR=…/w43mpl_0tohrlg_ run 1: 8.293 s rc=0  (run 1 builds the cache, runs 2-3 reuse it)
     plain venv, MPLCONFIGDIR=…/w43mpl_0tohrlg_ run 2: 0.258 s rc=0  (run 1 builds the cache, runs 2-3 reuse it)
     plain venv, MPLCONFIGDIR=…/w43mpl_0tohrlg_ run 3: 0.253 s rc=0  (run 1 builds the cache, runs 2-3 reuse it)
     plain venv, a second fresh MPLCONFIGDIR: 8.177 s rc=0  (cache built again)
     pyinstaller-onedir: 'building the font cache' message in 6 of 6 benchmark runs
     pyinstaller-onefile: 'building the font cache' message in 6 of 6 benchmark runs

Eight seconds to build the cache on this machine, a quarter of a second with it. That accounts for the PyInstaller folder’s 7.7 s, and for most of the gap between its onefile (26.5 s) and Nuitka’s onefile (18.5 s) on the same script.

If you package a matplotlib program with PyInstaller and startup matters, the hook is the thing to look at: it is a plain Python file in PyInstaller/hooks/rthooks/, and all it does is set an environment variable before your code runs. In principle your own code can set MPLCONFIGDIR to a persistent directory before import matplotlib, which would override the hook’s value — but that puts the stale-cache problem the hook was written to avoid back on you, so the directory would need clearing whenever the bundle changes. This page measured the hook’s cost; it did not test a workaround.

Build time and size

Nuitka’s build time is the price of compiling. An inventory of the folders (archived with the run) shows its pandas build produced a 56.7 MB main binary and a pandas/ directory containing 44 .so extension modules and no .py or .pyc files at all — against PyInstaller’s 11.0 MB main binary and a base_library.zip of 154 .pyc files. The pure-Python parts of pandas and numpy were compiled to C and linked into Nuitka’s executable. With no ccache on the machine that took 75 s for pandas and 69 s for matplotlib, against 14 s and 13 s for PyInstaller, which copies the .pyc files instead. (With ccache installed, a second build of the same program would reuse the compiled objects; that was not measured here.)

Compiling did not make the folder smaller. Nuitka’s standalone folders were 100 MB against PyInstaller’s 81 MB for both the pandas and the matplotlib script. Where Nuitka does win on size is onefile: 22.6 MB vs 35.9 MB for pandas, 3.9 MB vs 7.2 MB for the stdlib script, which is the difference between its compressed binary and PyInstaller’s compressed archive.

cx_Freeze ships the most files by far: 2,273 for the pandas script, where PyInstaller had 384 and Nuitka 118. Of those, 1,032 are individual .pyc files (276 of them under pandas/, next to the same 45 .so files), and 149 MB of output is 85% more than PyInstaller’s folder. Part of that is architecture: cx_Freeze’s main executable and its lib/Python are universal binaries (x86_64 + arm64), where PyInstaller’s and Nuitka’s are arm64 only. It was the fastest builder for the stdlib script and slower than PyInstaller for the other two.

All three tools bundled more than each script imports. PyInstaller’s pandas folder contains matplotlib’s 40 .ttf and 61 .afm font files, and its matplotlib folder contains a pandas/ directory; cx_Freeze’s folders do the same and add 69 Tcl files; which is why each tool’s pandas and matplotlib builds are within 0.15 MB of each other. Nuitka’s pandas folder has no font files, but its two folders also land within 0.1 MB of each other. The inventory records what is there; why each tool followed which optional import was not investigated.

For the stdlib script, the ordering is the same with smaller numbers: cx_Freeze 1.3 s, PyInstaller 2.6 s, Nuitka 25.2 s to build; 33 MB, 18 MB, 14 MB of folder.

What all five got right

Every build succeeded on the first attempt with default options, and every one of the ninety launches exited 0 and printed its sentinel. pandas 3.0, numpy 2.5 and matplotlib 3.11 were all found and bundled without a hidden-import or data-file flag.

One thing the sentinel output caught that the timings would not have: the stdlib probe prints sys.version, and the cx_Freeze artifact reported Python 3.13.15, where the PyInstaller and Nuitka artifacts reported 3.13.1, the venv’s interpreter. cx_Freeze’s folder carries its own 13.6 MB lib/Python (a universal x86_64 + arm64 library whose strings say 3.13.15), not the 5.1 MB arm64 framework library the venv was built from, and its freezer source prefers a library shipped with its freeze_core dependency (“use macpython distributed files if available”, resolving to freeze_core/bases/lib/Python) over the running interpreter’s. The script ran correctly on it, but it means the cx_Freeze column measures a different interpreter build as well as a different packager, and its sizes include an x86_64 half the other tools’ artifacts do not carry. Three small scripts are not a test of hook coverage — a program using a package with unusual data files or dynamic imports could still fail on any of the three tools — but for these three, the “PyInstaller just works, Nuitka needs babysitting” folklore did not show up.

Before packaging anything, confirm the script runs in the environment you are packaging from. A ModuleNotFoundError inside a frozen build is a different failure from one inside Jupyter, but it starts the same way, with a package installed into an interpreter other than the one you think; the kernel failure matrix covers how to check that. The venv here was created from a Homebrew Python 3.13, the same install route described in installing Jupyter on macOS.

Re-running this

The harness is about 230 lines of Python and depends on nothing outside the packagers themselves. The shape of it:

# one build + six timed launches, per (probe, config); appended to results.json immediately
cmd = pyinstaller_cmd(script, out, onefile)      # or nuitka_cmd / cxfreeze_cmd
t0 = time.perf_counter()
p = subprocess.run(cmd, cwd=out, capture_output=True, text=True, timeout=3600)
build_seconds = time.perf_counter() - t0
 
exe, root = find_artifact(config, probe, out)    # single file, or folder + its executable
size = root.stat().st_size if root.is_file() else sum(f.stat().st_size for f in root.rglob("*") if f.is_file())
 
runs = []
for i in range(6):
    t0 = time.perf_counter()
    r = subprocess.run([str(exe)], capture_output=True, text=True, timeout=600)
    runs.append({"seconds": time.perf_counter() - t0, "rc": r.returncode,
                 "sentinel": SENTINEL in r.stdout, "stderr_tail": r.stderr[-400:]})

The three probe scripts, verbatim:

"""Probe 1: pure standard library. argparse + json + datetime, no third-party imports."""
import argparse
import json
import sys
from datetime import datetime, timezone
 
 
def main() -> int:
    parser = argparse.ArgumentParser(description="stdlib-only probe")
    parser.add_argument("--n", type=int, default=1000)
    args = parser.parse_args()
    total = sum(i * i for i in range(args.n))
    payload = {
        "probe": "hello",
        "python": sys.version.split()[0],
        "frozen": bool(getattr(sys, "frozen", False)),
        "compiled": "__compiled__" in globals(),
        "n": args.n,
        "sum_sq": total,
        "utc": datetime.now(timezone.utc).strftime("%Y-%m-%d"),
    }
    print(json.dumps(payload))
    print("SENTINEL_HELLO_OK")
    return 0
 
 
if __name__ == "__main__":
    raise SystemExit(main())

"""Probe 2: pandas + numpy. Builds a 10,000-row frame in memory, groups it, writes and re-reads a CSV."""
import io
import sys
 
import numpy as np
import pandas as pd
 
 
def main() -> int:
    rng = np.random.default_rng(42)
    df = pd.DataFrame(
        {
            "group": rng.choice(list("ABCDE"), size=10_000),
            "x": rng.normal(size=10_000),
            "y": rng.integers(0, 100, size=10_000),
        }
    )
    summary = df.groupby("group").agg(x_mean=("x", "mean"), y_sum=("y", "sum"))
    buf = io.StringIO()
    summary.to_csv(buf)
    back = pd.read_csv(io.StringIO(buf.getvalue()), index_col="group")
    assert back.shape == (5, 2), back.shape
    print(f"pandas {pd.__version__} numpy {np.__version__} frozen={bool(getattr(sys, 'frozen', False))}")
    print(back.round(3).to_string())
    print("SENTINEL_PANDAS_OK")
    return 0
 
 
if __name__ == "__main__":
    raise SystemExit(main())

"""Probe 3: matplotlib (Agg backend) + numpy. Renders a line chart to a PNG next to the executable's cwd."""
import os
import sys
import tempfile
 
import matplotlib
 
matplotlib.use("Agg")
import matplotlib.pyplot as plt  # noqa: E402
import numpy as np  # noqa: E402
 
 
def main() -> int:
    x = np.linspace(0, 2 * np.pi, 200)
    fig, ax = plt.subplots(figsize=(4, 3), dpi=100)
    ax.plot(x, np.sin(x), label="sin")
    ax.plot(x, np.cos(x), label="cos")
    ax.set_title("probe: matplotlib Agg")
    ax.legend()
    out = os.path.join(tempfile.gettempdir(), "w43_probe_plot.png")
    fig.savefig(out)
    size = os.path.getsize(out)
    assert size > 5_000, size
    print(f"matplotlib {matplotlib.__version__} numpy {np.__version__} frozen={bool(getattr(sys, 'frozen', False))} png_bytes={size}")
    print("SENTINEL_PLOT_OK")
    return 0
 
 
if __name__ == "__main__":
    raise SystemExit(main())

The commands are the five rows of the configuration table. To reproduce the two mechanism checks: copy a folder build to a new directory and time three launches of the copy; and poll your temp directory for _MEI* or onefile_* entries while a onefile binary runs. To reproduce the font-cache number, run plot.py with MPLCONFIGDIR set to an empty directory, twice.

Three things to keep if you adapt it: write results after every build so a crash keeps the earlier rows; record each launch’s stderr, which is how the font-cache line was caught; and check the output of each launch rather than only its exit code.

Related, for the other benchmark on this site that measures what a conversion tool silently loses rather than what it costs: the Jupyter-to-PDF fidelity benchmark was built the same way, with sentinel strings and a results file that the article is generated from.

Limits of this measurement

• macOS only. Every second on this page was measured on one Apple-silicon Mac. The “new files are slow to launch” cost is a property of this operating system’s behaviour on this machine; Windows and Linux have their own first-launch costs (antivirus scanning of new executables on Windows is the commonly reported one) and nothing here measures them. The onefile mechanism — unpack to a fresh temp directory on every launch — is what both tools did here with default options; whether the per-launch cost looks the same on Windows or Linux was not measured. If you need a Windows .exe, build and time it on Windows, or use a service that does, such as Runcell’s Python to EXE converter , which was not tested here.
• One build per configuration. Build times are single samples. Each artifact was launched six times; the first launch is always separate. Across the five later launches, the rows above 1 s spread by at most 4% (for example 35.1–36.4 s, 26.1–27.1 s), the sub-0.1 s rows spread by up to 0.015 s (0.018–0.033 s), and PyInstaller’s stdlib onefile is the exception at 4.4–8.6 s.
• Nuitka was built without ccache, which Nuitka recommends and which would speed up repeat builds. First-build times are what is reported, for all three tools.
• The probes are small. They test whether the tools bundle three common packages and how the artifacts launch; they do not test runtime speed of real work, GUI toolkits, data files, code signing, or notarisation — all of which affect a real macOS or Windows release.
• The cause of the first-launch cost is inferred, not measured. The data shows it is tied to newly created files (the fresh-copy check). That it scales with the number of files is a hypothesis the data does not cleanly support: cx_Freeze’s 2,273-file pandas folder took 34 s on first launch, PyInstaller’s 384-file one took 51 s. Which part of macOS is doing the work was not instrumented.
• The cx_Freeze column runs a different interpreter build (3.13.15, universal) than the PyInstaller and Nuitka columns (the venv’s 3.13.1, arm64). Its build times are comparable; its sizes and launch times are not a packager-only comparison.
• The tool page in the decision table was not benchmarked. Its page says it assembles a Windows executable in the browser; what it uses to do that, and how the result launches, is outside what this repository can verify.

Related reading

• ModuleNotFoundError in Jupyter: A Reproduced Kernel Failure Matrix — the environment problems to clear up before freezing an environment.
• Install Jupyter Notebook on macOS — the Homebrew Python setup these builds were made from.
• Jupyter to PDF: A Measured Fidelity Benchmark — the same method applied to notebook export: what each route silently drops.
• How to Delete a Column in Pandas — the pandas probe’s starting point, for readers who came for the data side rather than the packaging side.