JSFeeds: stackabuse.com - Creating Command Line Utilities with Python's argparse

Monday, 9 December, 2019 UTC

Creating Command Line Utilities with Python's argparse

Summary

Introduction

Most of the user-facing software comes with a visually pleasing interface or via a decorated webpage. At other times, a program can be so small that it does not warrant an entire graphical user interface or web application to expose its functionality to the end-user.

In these cases, we can build programs that are accessible via a Command Line Interface, or CLI.

In this post, we will explore Python's argparse module and use it to build a simple command-line tool to help us shorten URLs swiftly.

Command Line Interface

A Command Line Interface is a text-based user interface that provides a means of interacting with a computer through textual commands. The program that facilitates this interaction by exposing the interface is known as a command-line interpreter or a shell.

It takes in commands in the form of text input, executes programs based on the input provided, then displays the output on the interface. There are many shells available, with the first popular ones being the Bourne shell and C shell for Unix-based systems. The Bourne Again Shell (aka bash) is a hugely popular extension of the Bourne Shell, alongside the Korn Shell (ksh).

It is also worth noting that CLIs, like other software, require user accounts to work with them. The system enforces permissions on these users to help control the level of access and limit of what a user can achieve with the system. This distinction is required since the shell is an interface between the user and the operating system kernel that controls all computer operations. Access to this functionality needs to be restricted to prevent malicious use of the command-line interface.

CLIs offer a prompt, which is usually a dollar sign ($) that indicates that you can enter your command. However, this prompt also indicates that the command entered will be performed without root access.

When root access is granted to the current user interacting with the CLI, the prompt changes to a hash sign (#).

While Graphical User Interfaces (GUIs) are easier to learn and more visually intuitive, CLIs allow users to interact with software using only a keyboard, which can result in faster performance. CLIs also use up fewer computer resources as compared to GUIs, making them lighter and faster.

Scripting

Most command-line interpreters ship with some basic commands that can all be invoked through the command-line interface to perform specific tasks. Some common ones include:

uptime: Indicates how long the computer has been on
date: Returns the current date and time
ls: Returns a list of all the files and folders in a directory
cd: Used to move from one directory to another
pwd: Used to show the current working directory
man: Used to show the manual or instructions of any command
touch: Used to create new, empty files
mkdir: Used to create new directories

A shell script is a program designed to be executed by a command-line interpreter. It contains a series of commands, like the ones listed above, coupled with variables and conditions instructing the shell on what task or tasks to undertake.

Through shell scripts, a user can perform multiple commands in quick succession without the need to remember them all. They are mostly used to achieve repetitive operations without the repetitive entry of commands, therefore, reducing the effort required by the end-user.

We can write shell scripts that consist of the shell commands to be executed, but also, we can execute other high-level languages, such as Python and JavaScript.

What is argparse?

A Python shell script is just a normal Python program that is executed by the command-line interpreter. When executing a shell script, arguments are passed into our script through sys.argv. This variable is a list of the arguments passed to our program, including the script name, which is also the first argument.

Normally, we can write a simple script that does not require any extra arguments, like a simple script to display the current date. This will, however, limit the functionality we can offer. To make our scripts more versatile and widen the scope of their usage, we have to facilitate customization through arguments that give the user more control and options in terms of functionality.

The argparse module helps us parse the arguments passed with our script and process them in a more convenient way. It also adds customization features such as naming our program and adding descriptions in a simpler way.

argparse also provides a means for us to customize the usage instructions for our script and indicate which arguments are required and which are optional. To explore all these features and more, we'll build our own Python CLI utility in the next section.

Demo Application

Currently if we want to shorten a URL, we will need to fire up a browser and navigate to a URL shortening site in order to do the task. Our goal is to accelerate and enhance this URL shortening process through a script that we can fire up anytime on our terminal. We will only have to pass the URLs we need to shorten as arguments and receive the shortened URLs in response.

For this demo, we'll use Shorte.st as our provider since its API is simple and straightforward.

After creating an account, we can head over to the Link Tools section and select Developers API. Here we will find our access token and the URL which our script will use to shorten our URLs.

Once the user provides a URL to our command line utility to be shortened, we will send the URL to the Shorte.st API endpoint along with our access token. The response will be our shortened URL and a status message.

Let us start by creating a virtual environment and installing the requests module, which we'll use to send HTTP requests to the API:

$ mkdir pyshortener && cd pyshortener
$ virtualenv --python=python3 env --no-site-packages
$ source env/bin/activate
$ pip install requests

In the first line above we have actually combined two commands in one using the double ampersand (&&). This allows us to execute the commands in sequence, unless the first command fails, which then prevents the second command from being executed.

After creating our virtual environment and activating it, we then install our Python dependency.

For this demo we will first build our shortening function and then wrap its functionality using argparse into the final script:

import requests
from requests.exceptions import ConnectionError
import json

def shorten_url(url):
    try:
        response = requests.put("https://api.shorte.st/v1/data/url",
                                {"urlToShorten": url},
                                headers={"public-api-token": "[MY-API-TOKEN]"})

        api_response = json.loads(response.content)

        return {"status": api_response['status'],
                "shortenedUrl": api_response['shortenedUrl'],
                "message": "URL shortened successfully"}

    except ConnectionError:
        return {"status": "error",
                "shortenedUrl": None,
                "message": "Please ensure you are connected to the internet and try again."}

shorten_url(www.stackabuse.com)

Our function takes in a URL and sends it to the Shorte.st API and returns the shortened URL. Let's shorten www.stackabuse.com by executing our script:

$ python pyshortener.py
{'status': 'ok', 'shortenedUrl': 'http://gestyy.com/w6ph2J', 'message': 'URL shortened successfully'}

As we can see, our function works but the output is less than ideal. We also have to hardcode the URL in the script itself, which gives us a fixed input and output.

We are going to take this a step further and allow users to pass in the URL as an argument when they execute the script. In order to do this we will now introduce argparse to help us parse the arguments provided by the user:

import requests
from requests.exceptions import ConnectionError
import json
import argparse # Add the argparse import

def shorten_url(url):
    # Code stays the same...

# Create a parser
parser = argparse.ArgumentParser(description='Shorten URLs on the terminal')

# Add argument
parser.add_argument('--url', default="google.com", help="The URL to be shortened")
args = vars(parser.parse_args())
print(args)

In this version of the script, we don't call shorten_url, but instead just print out the arguments captured and parsed by argparse.

We start by creating an ArgumentParser object using argparse, which will hold all the info required to transform the arguments passed into Python data types we can work with.

After creating the parser, we can now add arguments using parser.add_argument(). This function allows to specify the following information about our arguments:

The first argument is a name or a flag used to identify our arguments. Optional arguments are identified by the - prefix, in our case --url is an optional argument.
The default option allows to specify a default value when the user has not provided the argument.
The help option briefly describes what the argument is.
We can also use the choice option to specify allowable values for an argument, such as yes and no.
Through a type option we can also specify the type to which our command will be converted to, for instance converting arguments into integers.

When we run our script without providing any arguments, the URL defaults to "google.com", just like we've set in the add_argument method:

$ python pyshortener.py
{'url': 'google.com'}

Now when we pass www.stackabuse.com using the --url flag, it is set as the value to the url key:

$ python pyshortener.py --url www.stackabuse.com
{'url': 'www.stackabuse.com'}

We can now receive a user's URL via the command line and shorten it by modifying our script:

if args.get('url'):
   print(shorten_url(args['url']))

When we run it and pass a URL we should then receive the output from the Shorte.st API:

$ python pyshortener.py --url stackabuse.com
{'status': 'ok', 'shortenedUrl': 'http://gestyy.com/w6pk2R', 'message': 'URL shortened successfully'}

Although our output is not as friendly as we'd want it too, so let us create a function to format our output in a more desirable manner:

def handle_output(result):
   """ Function to format and print the output
   """
   if result["status"] == "ok":
       print(f"{result['message']}. Your shortened URL is:\n"
             f"\t{result['shortenedUrl']}")
   elif result["status"] == "error":
       print(f"{result['message']}")

# Receive and process the argument
args = vars(parser.parse_args())

if args.get('url'):
   result = shorten_url(args['url'])
   handle_output(result)

When we run our script one more time:

$ python pyshortener.py --url www.stackabuse.com
URL shortened successfully. Your shortened URL is:
        http://gestyy.com/w6pk2R

Our output is now more user-friendly due to the addition of the handle_output() function. To see how argparse has generated the help text for our script, we can execute our script with the -h flag to display the help text as follows:

$ python pyshortener.py -h
usage: pyshortener.py [-h] [--url URL]

Shorten URLs on the terminal

optional arguments:
  -h, --help  show this help message and exit
  --url URL   The URL to be shortened

Conclusion

We have built a shell script using Python to help us shorten URLs quickly on the terminal. We have used the argparse module to parse the arguments that are passed into our script and even defined a default value in case the argument is provided.

Our script now also has a beautiful help message that can be displayed using the -h flag which was generated by the argparse module meaning we did not have to write it down manually.

The next step would be to enhance our script to accept a list of URLs or read URLs from a text file to facilitate batch shortening of URLs.

The source code for the script in this project can be found here on Github.

... more @ stackabuse.com

stackabuse.com