Skip to content
Related Articles

Related Articles

Improve Article

Command Line File Downloader in Python

  • Difficulty Level : Medium
  • Last Updated : 24 Jan, 2021
Geek Week

Python is one of the most popular general-purpose programming languages with a wide range of use cases from general coding to complex fields like AI. One of the reasons for such popularity of python as a programming language is the availability of many built-in as well as third-party libraries and packages.

In this article, we are going to build a simple command-line file downloader, which you can download a file if you have the download link.

Approach:

We are going to download files over HTTP instead of FTP. Once we have made the request for the file, we will use the response as an input stream to write or save the file to the file system. While Downloading, details like download speed, time, and amount of file downloaded will be shown.

  • Take the file URL via command line
  • Open the file input or download stream over HTTP via requests.
  • Download the file by writing the file in chunks from the download/input stream into the file system.
  • Display the details
  • Save the file.

Step-by-step implementation:

Step 1: Libraries required

Python3






import requests
import sys
import time

Step 2: a dictionary for detecting file size in proper units

While Downloading the file, parameters like speed, size, time will be shown. So let’s see how to do this. Dictionary for getting file size and download speed unit depending on the amount downloaded takes sizes in bytes.

Python3




units = {
'B':{'size':1, 'speed':'B/s'},
'KB':{'size':1024, 'speed':'KB/s'},
'MB':{'size':1024*1024, 'speed':'MB/s'},
'GB':{'size':1024*1024*1024, 'speed':'GB/s'}
}

Function for checking units of the length of the file downloaded. Length is in bytes so corresponding units are checked.

Python3




def check_unit(length): # length in bytes
  if length < units['KB']['size']:
    return 'B'
  elif length >= units['KB']['size'] and length <= units['MB']['size']:
    return 'KB'
  elif length >= units['MB']['size'] and length <= units['GB']['size']:
    return 'MB'
  elif length > units['GB']['size']:
    return 'GB'

Step 3: Downloading file over HTTP

We will open a file stream over HTTP using requests and then save the data in chunks to the local file. Let’s see how the code will look and then put it all together.

Python3






# Opening file stream
  
r = requests.get(link_to_file, stream = True)  
  
# writing file data in chunks.
# for examples, A file of 10 MB written in
# chunk size of 8096 Bytes.
with open(file_name, 'wb') as f:  
  for chunk in r.iter_content(chunk_size):  
    f.write(chunk)

Step 4: Printing output

The output will consist of a dynamic progress bar with downloading details. For this we use stdout.write() and stdout.flush() methods

Python3




sys.stdout.write(format_string % (list_of_variables))
  
# for printing details in single 
# line which updates without going to the next line
# once the details are updated.
  
sys.stdout.flush() 

Below is the full implementation:

Python3




import requests
import sys
import time
  
units = {    
'B' : {'size':1, 'speed':'B/s'},
'KB' : {'size':1024, 'speed':'KB/s'},
'MB' : {'size':1024*1024, 'speed':'MB/s'},
'GB' : {'size':1024*1024*1024, 'speed':'GB/s'}
}
  
def check_unit(length): # length in bytes
    if length < units['KB']['size']:
        return 'B'
    elif length >= units['KB']['size'] and length <= units['MB']['size']:
        return 'KB'
    elif length >= units['MB']['size'] and length <= units['GB']['size']:
        return 'MB'
    elif length > units['GB']['size']:
        return 'GB'
  
# takes download link and directory where file to be saved.
def downloadFile(url, directory) :
  
    localFilename = url.split('/')[-1] # files name
  
    with open(directory + '/' + localFilename, 'wb') as f:
        print ("Downloading . . .\n")
        start = time.time() # start time
        r = requests.get(url, stream=True)
  
        # total length in bytes of the file
        total_length = float(r.headers.get('content-length')) 
  
        d = 0 # counter for amount downloaded 
  
        # when file is not available
        if total_length is None:
            f.write(r.content)
        else:
            for chunk in r.iter_content(8192):
                  
                d += float(len(chunk))
                f.write(chunk) # writing the file in chunks of 8192 bytes
  
                # amount downloaded in proper units
                downloaded = d/units[check_unit(d)]['size']
                  
                # converting the unit of total length or size of file from bytes.
                tl = total_length / units[check_unit(total_length)]['size'
                  
                trs = d // (time.time() - start) # speed in bytes per sec
                  
                #speed in proper unit
                download_speed = trs/units[check_unit(trs)]['size']
                  
                speed_unit = units[check_unit(trs)]['speed'] # speed in proper units
  
                done = 100 * d / total_length # percentage downloaded or done.
                  
                fmt_string = "\r%6.2f %s [%s%s] %7.2f%s  /  %4.2f %s  %7.2f %s"
                  
                set_of_vars = ( float(done), '%',
                                '*' * int(done/2),  
                                '_' * int(50-done/2),  
                                downloaded, check_unit(d),  
                                tl, check_unit(total_length),  
                                download_speed, speed_unit)
  
                sys.stdout.write(fmt_string % set_of_vars)
  
                sys.stdout.flush()
  
    return (time.time() - start) # total time taken for download
  
def main() :
    directory = '.'
    if len(sys.argv) > 1 :
        url = sys.argv[1] # url from cmd line arg
        if len(sys.argv) > 2:
            directory = sys.argv[2]
          
        total_time = downloadFile(url, directory)
        print ('')
        print ("Download complete...")
        print ("\rTime Elapsed: %.2fs" % total_time)
    else :
        print("No link found!")
  
if __name__ == "__main__" :
    main()

Save the code in a python file and use it as follows

python <program_name>.py <file_link> <save_location(by default '.')>

Output:

 Attention geek! Strengthen your foundations with the Python Programming Foundation Course and learn the basics.  

To begin with, your interview preparations Enhance your Data Structures concepts with the Python DS Course. And to begin with your Machine Learning Journey, join the Machine Learning – Basic Level Course




My Personal Notes arrow_drop_up
Recommended Articles
Page :