Better alternative to shell scripts with Python, uv and pytest

Performing all sorts of small tasks and automations with shell scripts is a superpower for software developers, IT admins and other people working with computers.

I’ve been writing bash scripts for 15+ years to solve my problems but no matter how much I studied and practiced the craft, I always felt insecure about my scripts, felt they were error-prone and had to do a lot of manual testing to get them right.

With a few new additions to Python language and its tool ecosystem, it’s now better equipped for writing self-contained small scripts to replace shell scripts.

⚠️

That said, Python is not always the right tool for the job: especially if you ever think your scripts need to be able to run in different system and computers. For those, I recommend sticking to technologies that are most likely pre-installed on most systems with as little dependencies as possible.

uv

At the core is the new tool darling uv that enables us to run Python scripts with dependencies by creating a temporary virtual environment for the script, installing dependencies there and running the script.

Let’s say we have a Python script like this:

from bs4 import BeautifulSoup
import requests

response = requests.get('https://example.com')
dom = BeautifulSoup(response.text, 'html.parser')

def get_title(dom):
	return dom.title.text 
	
print(get_title(dom))  # prints Example domain

Since it relies on two dependencies, BeautifulSoup and requests, we’d normally need to create a virtual environment and install the packages there to be able to run it with python script.py.

With uv, we can tell it which dependencies it should include when running the script with --with option:

uv run --with bs4 --with requests script.py

With this one-liner, we no longer need to manually set up any virtual environments or accompany the script with a requirements.txt or pyproject.toml.

We can do a bit better though.

PEP 723 - Inline script metadata

My second favourite PEP (after the one with pattern matching) — PEP 723 — is the next piece of the puzzle towards a more script-like usage.

It introduces a way to include minimum Python version and dependency metadata directly into a Python script rather than a separate pyproject.toml file. In a system where we can have many of these scripts in a single folder, it’s not very handy to need to keep track of dependencies on separate files.

Let’s modify our example script:

# /// script
# requires-python = ">=3.13"
# dependencies = [
#   "bs4",
#   "requests"
# ]
# ///

from bs4 import BeautifulSoup
import requests

response = requests.get("https://example.com")
dom = BeautifulSoup(response.text, "html.parser")


def get_title(dom):
    return dom.title.text


print(get_title(dom))  # prints Example domain

Now, we can run this without all the --with flags:

uv run script.py

Additionally, we can add a shebang at the start of the script to let us skip the entire uv run thing from our commands, making the experience more shell-script-like:

#!/usr/bin/env -S uv run --quiet --script

# /// script
# requires-python = ">=3.13"
# dependencies = [
#   "bs4",
#   "requests"
# ]
# ///

from bs4 import BeautifulSoup
import requests

response = requests.get("https://example.com")
dom = BeautifulSoup(response.text, "html.parser")


def get_title(dom):
    return dom.title.text


print(get_title(dom))  # prints Example domain

With the shebang at the start, we can give the script execution permissions (with chmod +x script.py) and run it with ./script.py. Thanks to Tero for teaching this to me in a lightning talk at an archipylago meetup.

Add tests with pytest

We’re not done yet! One of the aspects of shell scripts that I always felt dreary about was that changing anything felt scary. Adding tests can alliviate some of those concerns and with uv, we can run pytest!

Let’s refactor the code a bit:

#!/usr/bin/env -S uv run --quiet --script

# /// script
# requires-python = ">=3.13"
# dependencies = [
#   "bs4",
#   "requests"
# ]
# ///

from bs4 import BeautifulSoup
import requests

def get_title(dom):
    return dom.title.text

if __name__ == '__main__':
	response = requests.get("https://example.com")
	dom = BeautifulSoup(response.text, "html.parser")
	print(get_title(dom))  # prints Example domain

We moved all the script-running parts into if __name__ == '__main__': block which is only executed if the script is being run as a Python program or script but not when it’s imported to other part scripts.

We can now add a test for our get_title function:

#!/usr/bin/env -S uv run --quiet --script

# /// script
# requires-python = ">=3.13"
# dependencies = [
#   "bs4",
#   "requests"
# ]
# ///

from bs4 import BeautifulSoup
import requests

def get_title(dom):
    return dom.title.text

if __name__ == '__main__':
	response = requests.get("https://example.com")
	dom = BeautifulSoup(response.text, "html.parser")
	print(get_title(dom))  # prints Example domain
	
def test_get_title():
  html = '<title>PEP 723 and uv are great together</title>'
  dom = BeautifulSoup(html, 'html.parser')
  
  assert 'PEP 723 and uv are great together' == get_title(dom)

The command line interface with this combo isn’t quite as elegant as with the previous steps though.

We need to pass all the dependencies to uv by hand as the support for reading them from inline metadata for use cases like calling pytest is not yet supported but it’s very close as there’s already a pull request in progress.

However, for now, to run the tests, we run

uv run --with pytest --with bs4 --with requests pytest script.py

and it should find 1 test and that test should pass. To make sure it’s running it correctly, change the test’s assertion to something that should fail.

I’m excited to see there’s work being done to streamline this into (note, this does not work at the time of publication):

uv run pytest script.py

Another aspect is that I usually would like to keep my script filenames without the .py extension but at the moment pytest only runs tests from files that end with .py.

However, I do find both of these two very minor annoyances not too bad because you’re only running the tests when the file is changed anyway so at that moment, a bit of extra work (like spelling out the dependencies in the uv call or temporarily renaming the file from script to script.py ) is not a big deal.

Conclusion

With the adoption of these tools, my approach to building small scripts for myself has changed a lot and for the better. I still try to keep my shell scripting skills sharp to not fall off it but more and more I find myself reaching for Python and uv and having higher confidence in achieving what I need to achieve without accidentally causing unrecoverable issues.

Introducing uv as a dependency on small scripts is the only thing I think about every time I write one. The more dependencies you have on your scripts, less portable they become.

To manage my scripts, I have a git repository with the following file structure:

.
├── link.sh
├── README.md
├── scripts
    ├── ,label
    ├── ,python
    └── ,server

I create a new script in the scripts/ folder (following the comma-prefix pattern) and a link.sh script:

#!/bin/zsh

script=$1
scriptname=$(basename $script)

echo "Making $scriptname executable..."
chmod +x $script

echo "Linking $scriptname to /usr/local/bin...";
ln -s "$PWD/$script" "/usr/local/bin/$scriptname" # $PWD is needed since it requires an absolute path for the link to work.
rehash # Needed to update zsh cache for new scripts for autocomplete.

Once I’m done with creating a new script (or I clone the repository to a new computer), I run ./link.sh scripts/,new-script and it will make it executable, create a symbolic link to /usr/local/bin and then rehashing zsh cache for autocomplete.

With this script, I don’t have to remember to mark it executable or rehash and I don’t have to remember the syntax to ln.