curl | #!/bin/blog

Nothing serious, just a few notes I like to share with friends and colleagues who, like me, script around curl.

`curl -f / --fail`

I try to use --fail whenever I can, because why would I want to exit zero on server errors?

$ curl -L https://download.grml.org/grml64-small_2024.02.iso.NO
<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML 2.0//EN">
<html><head>
<title>404 Not Found</title>
</head><body>
<h1>Not Found</h1>
<p>The requested URL was not found on this server.</p>
<hr>
<address>Apache/2.4.41 (Ubuntu) Server at ftp.fau.de Port 443</address>
</body></html>
$ echo $?
0
$ curl -f -L https://download.grml.org/grml64-small_2024.02.iso.NO
curl: (22) The requested URL returned error: 404
$ echo $?
22

`curl --fail-with-body`

I have a CI/CD situation where curl calls a webhook and it’s incredibly useful to see its error message in case of failure.

$ curl --fail https://binblog.de/xmlrpc.php
curl: (22) The requested URL returned error: 405
$ curl --fail-with-body https://binblog.de/xmlrpc.php
curl: (22) The requested URL returned error: 405
XML-RPC server accepts POST requests only.

`set -o pipefail`

When curl‘s output gets piped to any other command, I try to remember to set -o pipefail along with curl --fail so if curl fails, the pipe exits non-zero.

#!/usr/bin/env bash

url='https://download.grml.org/grml64-small_2024.02.iso.NONO'

if curl -s -f -L "${url}" | sha256sum
then
        echo "Success."
else
        echo "Failure."
fi

set -o pipefail

if curl -s -f -L "${url}" | sha256sum
then
        echo "Success."
else
        echo "Failure."
fi

`curl --connect-timeout`

Useful to get quicker response in scripts instead of waiting for the system’s default timeouts.

`curl -w / --write-out`

This may be over the top most of the time, but I have one situation that requires extremely detailed error handling. (The reason being a bit of a foul split DNS situation in the environment, long story.) This is where I use --write-out to analyze the server response.

curl_http_status="$(curl -o "${tmpfile}" --write-out '%{http_code}\n' "${url}")"
curl_exit_status=$?

(Would be even nicer if a destination filename could be specified instead of needing to work with stdout only.)

`curl -n / --netrc / [ --netrc-file ]`

Username:password authentication is a thing, no matter how much it’s discouraged. Here’s how to at least hide username and password from the process list.

$ chmod 600 ~/.netrc
$ cat ~/.netrc
machine binblog.de
login foo
password bar
$ curl -v -o /dev/null -n https://binblog.de
...
* Server auth using Basic with user 'foo'
...

To use any other file instead of ~/.netrc, use --netrc-file instead.

#!/bin/blog

Tag Archives: curl

Failsafe curl