Shelling Out is Selling Out

Do you write Python code? Do you often need to invoke programs from your Python code that are - shockingly - not written in Python?

Have you written code that looks like this?

def process_data(data: bytes) -> bytes:
    # Create a new directory to store input and output data:
    with tempfile.TemporaryDirectory() as tempdir:

        # Write to disk:
        infile = os.path.join(tempdir, "input.bin")
        with open(infile, "wb") as f:
            f.write(data)

        # Start a subprocess and have it read our input file:
        outfile = os.path.join(tempdir, "output.bin")
        subprocess.check_call(["processor", infile, outfile])

        # Read from disk:
        with open(outfile, "rb") as f:
            return f.read()

I understand. Not everything is written in Python. Do this if you need to get something done right away.

But I've started disallowing code like this in my own codebases, for one main reason (and may other reasons too): it's full of footguns.

Footguns? How?

Let's step through this program line-by-line.

First off, we ask the operating system to provide us with a new temporary directory on the filesystem.

def process_data(data: bytes) -> bytes:
    # Create a new directory to store input and output data:
    with tempfile.TemporaryDirectory() as tempdir:
        # Write to disk:
        infile = os.path.join(tempdir, "input.bin")
        with open(infile, "wb") as f:
            f.write(data)

Your code might be doing everything else in memory; maybe it's serving an HTTP request, or processing data in a batch pipeline, or training an ML model.

When invoking this - seemingly-pure function - now, your code is reaching all the way into the operating system to request disk space to do its operation.

This could go wrong in so many ways:

• The machine could be out of disk space; or could not have enough to store the data that you want.
• The machine's disk may be slow. (The disk cache should help, here, but then performance depends strongly on cache parameters that your program doesn't care about.)
• You could be running in a cloud environment where your "disk" is actually a virtual disk hosted on the network.
• Your code could be on a Raspberry Pi, whose disk is likely an SD card with poor wear leveling. Writing to temp files on disk shortens the life of the card.
• Writes to the filesystem are visible by other programs, including antivirus scanners and similar security tools. Writing temporary files can (at best) expose your program's internal data to them, and (at worst) slow things down by waiting for their approval to write files.
• Your machine's disk may be read-only.

But okay, okay, let's say you have a perfect, fast local disk to use for your temporary directory, and don't need to worry about wear leveling or lifespan. Let's look at the next step:

        # Start a subprocess and have it read our input file:
        outfile = os.path.join(tempdir, "output.bin")
        subprocess.check_call(["processor", infile, outfile])

Spawning the subprocess itself has a number of risks:

• The subprocess must be installed somehow; if it's not a Python package, who knows where it comes from. Does it require a separate install step through a native package manager?
• If you bundle the subprocess with your Python code: is it in the correct executable format? Python works on x86, ARM, macOS, Windows, Linux, and many more platforms; will your subprocess?
• Spinning up a subprocess can take 2-3ms on a good day, and 30+ms on a bad day (Windows). Are you okay adding that much latency to this operation?
• Does your code run in an environment where subprocesses are allowed to be launched? Antivirus scanners and security tools may complain or block this operation.
• Does your subprocess expect to be run in headless mode? Will it ask the user to confirm something on stdin/stdout, blocking your program's execution?
• Does your subprocess spam stderr with useless garbage?
• How does your subprocess communicate failure?

Finally, we need to read the subprocess's output. Simple enough, right?

        with open(outfile, "rb") as f:
            return f.read()

Except:

• Do you have permissions to read the resulting file?
• Can the contents of the resulting file fit into memory?

Cleaning Up the Mess

I think I've done a decent job showing that there are about 20 different kinds of errors that can occur when shelling out to a subprocess, and that it should be avoided as a source of footguns. But what's the alternative?

Well, simply put: it depends. If the program you're shelling out to has Python bindings, great! If it doesn't, and it's open source, you can write your own (or have Claude/ChatGPT/etc write them for you).

By moving the core logic back into your program, you get many benefits:

• Everything stays in userspace: at no point do we need to make unnecessary I/O syscalls into the operating system, which force context switches and add overhead.
• Data structures in Python can be directly accessed or modified by the native code being used, without the need to copy data from one part of memory to another (or between processes). This concept is called zero-copy programming and can make a huge difference when operating on large amounts of data. Different binding libraries will support this to varying degrees.
• There's no need to use additional operating system structures, like additional processes, file descriptors, or pipe buffers.
• Logs, errors, and other diagnostic information can be integrated seamlessly between Python and the non-Python code: no need to parse output files, binary strings or error messages, when the Python bindings can just throw a Python Exception or emit a warning.

A real-world example: ffmpeg

One of the most common use cases I see for shelling out is dealing with media files via ffmpeg, the world's best tool for media encoding and decoding.

Let's look at a simple use case: taking a video file in memory and remuxing it into a different container format. With pure ffmpeg, we'd do something like:

ffmpeg -i myvideo.mp4 -c:v copy output.mkv

The equivalent Python code - to shell out - would look something like:

def remux_video(data: bytes) -> bytes:
    with tempfile.TemporaryDirectory() as tempdir:
        infile = os.path.join(tempdir, "input.mp4")
        with open(infile, "wb") as f:
            f.write(data)

        outfile = os.path.join(tempdir, "output.mkv")
        subprocess.check_call(["ffmpeg", "-i", infile, "-c:v", "copy", outfile])

        # Read from disk:
        with open(outfile, "rb") as f:
            return f.read()

And this works! It's convoluted, dangerous, and error-prone, but it works.

When I run this on my laptop on a 15-second video clip, I get:

ffmpeg version 7.1.1 Copyright (c) 2000-2025 the FFmpeg developers
  built with Apple clang version 16.0.0 (clang-1600.0.26.6)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/7.1.1_3 --enable-shared --enable-pthreads --enable-version3 --cc=clang --host-cflags= --host-ldflags='-Wl,-ld_classic' --enable-ffplay --enable-gnutls --enable-gpl --enable-libaom --enable-libaribb24 --enable-libbluray --enable-libdav1d --enable-libharfbuzz --enable-libjxl --enable-libmp3lame --enable-libopus --enable-librav1e --enable-librist --enable-librubberband --enable-libsnappy --enable-libsrt --enable-libssh --enable-libsvtav1 --enable-libtesseract --enable-libtheora --enable-libvidstab --enable-libvmaf --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx264 --enable-libx265 --enable-libxml2 --enable-libxvid --enable-lzma --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libspeex --enable-libsoxr --enable-libzmq --enable-libzimg --disable-libjack --disable-indev=jack --enable-videotoolbox --enable-audiotoolbox
  libavutil      59. 39.100 / 59. 39.100
  libavcodec     61. 19.101 / 61. 19.101
  libavformat    61.  7.100 / 61.  7.100
  libavdevice    61.  3.100 / 61.  3.100
  libavfilter    10.  4.100 / 10.  4.100
  libswscale      8.  3.100 /  8.  3.100
  libswresample   5.  3.100 /  5.  3.100
  libpostproc    58.  3.100 / 58.  3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/var/folders/0b/1pkl65sd0rx50qry7mndjpqw0000gn/T/tmph7x88sm9/input.mp4':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf61.7.100
  Duration: 00:00:14.50, start: 0.000000, bitrate: 3847 kb/s
  Stream #0:0[0x1](und): Video: h264 (Main) (avc1 / 0x31637661), yuvj420p(pc, bt709, progressive), 2688x1520 [SAR 1:1 DAR 168:95], 3845 kb/s, 25.03 fps, 25 tbr, 12800 tbn (default)
      Metadata:
        handler_name    : VideoHandler
        vendor_id       : [0][0][0][0]
Stream mapping:
  Stream #0:0 -> #0:0 (copy)
Output #0, matroska, to '/var/folders/0b/1pkl65sd0rx50qry7mndjpqw0000gn/T/tmph7x88sm9/output.mkv':
  Metadata:
    major_brand     : isom
    minor_version   : 512
    compatible_brands: isomiso2avc1mp41
    encoder         : Lavf61.7.100
  Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuvj420p(pc, bt709, progressive), 2688x1520 [SAR 1:1 DAR 168:95], q=2-31, 3845 kb/s, 25.03 fps, 25 tbr, 12800 tbn (default)
      Metadata:
        handler_name    : VideoHandler
        vendor_id       : [0][0][0][0]
Press [q] to stop, [?] for help
[out#0/mp4 @ 0x7f9613804c80] video:6807KiB audio:0KiB subtitle:0KiB other streams:0KiB global headers:0KiB muxing overhead: 0.039910%
frame=  363 fps=0.0 q=-1.0 Lsize=    6810KiB time=00:00:14.50 bitrate=3847.2kbits/s speed=2.99e+03x    

Remuxed in 0.17147493362426758 seconds

It took 171.4 milliseconds to remux a 15-second file by shelling out.

I hear the complaints already: "most of that is file I/O time! You should use pipes instead!"

Well, with pipes, on macOS, it's even slower:

Remuxed with ffmpeg and pipes in 0.19284319877624512 seconds

Now, let's try doing this entirely in-process, using PyAV, an incredibly good Python binding for FFmpeg.

def remux_video_pyav(data: bytes) -> bytes:
    input_buffer = io.BytesIO(data)
    input_buffer.name = "input_buffer" # for error reporting
    output_buffer = io.BytesIO()

    # Open the input data as a stream:
    with av.open(input_buffer) as input_container:
        # Open the output container to write into:
        with av.open(output_buffer, mode="w", format="mp4") as output_container:
            stream = input_container.streams.video[0]
            output_container.add_stream_from_template(stream)

            # Copy packet-by-packet from the input to the output:
            for packet in input_container.demux(stream):
                if packet.dts is not None: # Ignore non-data packets:
                    output_container.mux(packet)
    return output_buffer.getvalue()

This is admittedly slightly more code, but the complexity now reflects the problem, rather than the unrelated data management problem of making temporary directories, writing to them, passing the appropriate arguments, and so on. And as a side effect, this code is also faster:

Remuxed with PyAV in 0.043556928634643555 seconds

It took only 44 milliseconds to do this in-process; literally four times faster than shelling out.

One of the big advantages here is that we also no longer need FFmpeg installed - just including av in our requirements list is enough to bundle the relevant FFmpeg libraries in with our project's other dependencies. No need to do hacky brew install ffmpeg || apt-get install ffmpeg tricks to ensure that our deployment environment is set up correctly.

What happens if we get an exception in each of these scenarios if the input data is invalid? Well, compare these two messages:

subprocess.CalledProcessError: Command '['ffmpeg', '-i', '/var/folders/0b/1pkl65sd0rx50qry7mndjpqw0000gn/T/tmpu6485o96/input.mp4', '-c:v', 'copy', '/var/folders/0b/1pkl65sd0rx50qry7mndjpqw0000gn/T/tmpu6485o96/output.mkv']' returned non-zero exit status 234.

Traceback (most recent call last):
  File "/Users/psobot/Code/blog.petersobot.com/example_remux.py", line 51, in <module>
    remuxed = remux_video_pyav(data)
              ^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/psobot/Code/blog.petersobot.com/example_remux.py", line 32, in remux_video_pyav
    with av.open(input_buffer) as input_container, av.open(
         ^^^^^^^^^^^^^^^^^^^^^
  File "av/container/core.pyx", line 418, in av.container.core.open
  File "av/container/core.pyx", line 283, in av.container.core.Container.__cinit__
  File "av/container/core.pyx", line 303, in av.container.core.Container.err_check
  File "av/error.pyx", line 424, in av.error.err_check
av.error.InvalidDataError: [Errno 1094995529] Invalid data found when processing input: 'input_buffer'

The first tells us that something went wrong with FFmpeg. The second tells us that PyAV found invalid data when reading from its input buffer, and points to the exact line of code that caused the issue; the av.open call. Much clearer, more Pythonic, and more intuitive.

Okay, but what's the catch?

So, here's the part where I have to come clean: while avoiding shelling out is conceptually purer and cleaner, you can't do it all the time.

You might still need to shell out if:

• You're calling into a binary that you don't control, and for which you don't have the source code
• You're calling into untrusted code and want to lean on the operating system's process boundary for slightly more security (still risky)
• You're calling into code that has an incompatible license, and invoking it as a separate program does not constitute "linking" (see: the GPL)
• The code you need to call is of poor quality and would compromise the reliability of your Python program (memory leaks, resource leaks, etc).

And, to be frank; if it's so easy to just import subprocess and you can live with the risks; sure, go ahead, do it the easy way.

I, for one, will not approve the PR.