Python List All Files In Directory And Subdirectories

Listing all files in a directory and its subdirectories is a common task in Python, often encountered in file management, data processing, or system administration. As developers, having multiple approaches at our disposal allows us to choose the most suitable method based on our specific requirements. In this article, we will see how we can list all files in a directory and subdirectories in Python.

List All Files In Directory And Subdirectories in Python

Below are some of the ways by which we can list all files in directory and subdirectories in Python:

List All Files In Directory And Subdirectories Using os.listdir and Recursion

In this example, the Python function `list_files_recursive` recursively traverses a specified directory (`’./’` by default), printing the full paths of all files within that directory and its subdirectories. The function uses the `os` module to list and navigate directories.

Python

import os

def list_files_recursive(path='.'):
    for entry in os.listdir(path):
        full_path = os.path.join(path, entry)
        if os.path.isdir(full_path):
            list_files_recursive(full_path)
        else:
            print(full_path)

# Specify the directory path you want to start from
directory_path = './'
list_files_recursive(directory_path)

Output:

List All Files In Directory And Subdirectories Using os.walk

In this example, the Python function `list_files_walk` recursively traverses a specified directory (`’./’` by default) using `os.walk`, printing the full paths of all files within that directory and its subdirectories. The function takes advantage of the `os` module’s `walk` function, which efficiently generates file paths during directory traversal.

Python

import os

def list_files_walk(start_path='.'):
    for root, dirs, files in os.walk(start_path):
        for file in files:
            print(os.path.join(root, file))

# Specify the directory path you want to start from
directory_path = './'
list_files_walk(directory_path)

Output:

List All Files In Directory And Subdirectories Using glob Module

In this example, the Python function `list_files_glob` uses the `glob` module to list and print the full paths of files matching a specified pattern (`’./**/*’` by default). The `recursive` parameter is set to `True` by default, allowing it to recursively search subdirectories. The function demonstrates a concise way to obtain file paths using globbing patterns.

Python3

import glob

def list_files_glob(pattern='./**/*', recursive=True):
    files = glob.glob(pattern, recursive=recursive)
    for file in files:
        print(file)

list_files_glob()

Output:

List All Files In Directory And Subdirectories Using os.scandir

In this example, the Python function `list_files_scandir` utilizes the `os.scandir` method to efficiently iterate over entries (files and directories) in a specified directory (`’./’` by default). It prints the full paths of files and recursively explores subdirectories using `os.scandir`’s context manager, enhancing performance compared to using `os.listdir` for large directories.

Python3

import os

def list_files_scandir(path='.'):
    with os.scandir(path) as entries:
        for entry in entries:
            if entry.is_file():
                print(entry.path)
            elif entry.is_dir():
                list_files_scandir(entry.path)

# Specify the directory path you want to start from
directory_path = './'
list_files_scandir(directory_path)

Output:

List All Files In Directory And Subdirectories Using pathlib Module

In this example, the Python function `list_files_pathlib` utilizes the `pathlib` module to list and print the paths of files and directories in a specified directory (`’./’` by default). It uses the `iterdir` method to iterate over entries, and if the entry is a file, its path is printed; if it’s a directory, the function recursively explores its contents. The use of `pathlib` provides a more object-oriented and readable way to work with file paths.

Python3

from pathlib import Path

def list_files_pathlib(path=Path('.')):
    for entry in path.iterdir():
        if entry.is_file():
            print(entry)
        elif entry.is_dir():
            list_files_pathlib(entry)

# Specify the directory path you want to start from
directory_path = Path('./')
list_files_pathlib(directory_path)

Output:

Video Demonstration

Considerations and Best Practices:

When it comes to choosing the best approach for listing files in a directory and its subdirectories, consider the following best practices:

• Readability and Maintainability: Choose an approach that aligns with the readability and maintainability of your code. Code clarity is crucial for collaboration and future modifications.
• Python Version Compatibility: Be aware of the Python version you are using. While older versions may not support the newer modules like pathlib and os.scandir, the os module remains a reliable choice across various Python versions.
• Use os.walk for Simplicity: If simplicity is a priority, especially for basic use cases, the os.walk method is a straightforward and widely used solution. It provides a simple interface for traversing directories and listing files.
• Consider pathlib for Modern Code: If you are working with Python 3.4 or newer, consider using the pathlib module. It offers a more modern and object-oriented approach to file and path manipulation, enhancing code readability and usability.
• Performance Considerations: For large directory structures, performance may be a consideration. In such cases, test different approaches and choose the one that provides the best performance for your specific use case.

Conclusion:

In conclusion, Python offers developers a diverse set of tools to tackle the task of listing files in a directory and its subdirectories. The choice of approach depends on factors such as code readability, Python version compatibility, and specific use case requirements. Whether you opt for the simplicity of os.walk, the flexibility of glob, the efficiency of os.scandir, or the modernity of pathlib, each method empowers you to handle this common task with confidence. By understanding these approaches, you can enhance your coding skills and choose the best tool for the job at hand.