Python Programming File Handling

I/O Operations CSV/JSON

Python Files Complete Guide

Learn Python file handling - read/write operations, file modes, context managers, CSV/JSON processing, and file management best practices with practical examples.

File I/O

Read/Write operations

File Modes

r, w, a, b, + modes

CSV Files

Data import/export

JSON Files

Structured data

What is File Handling in Python?

File handling is a crucial aspect of programming that allows Python programs to read from and write to files on the computer's storage. This enables data persistence between program executions.

Key Concept

Python provides built-in functions for file operations. The open() function is used to open files, and files should always be closed after operations are complete. The with statement provides a better approach for automatic resource management.

Basic File Operations

# Basic file operations in Python

# 1. Writing to a file
with open('example.txt', 'w') as file:
    file.write("Hello, Python File Handling!\n")
    file.write("This is line 2.\n")
    file.write("This is line 3.\n")

# 2. Reading from a file
with open('example.txt', 'r') as file:
    content = file.read()
    print("File Content:")
    print(content)

# 3. Reading line by line
print("\nReading line by line:")
with open('example.txt', 'r') as file:
    for line in file:
        print(f"Line: {line.strip()}")

# 4. Appending to a file
with open('example.txt', 'a') as file:
    file.write("This line was appended.\n")

# 5. Reading specific number of characters
with open('example.txt', 'r') as file:
    first_10_chars = file.read(10)
    print(f"\nFirst 10 characters: {first_10_chars}")

Python File Modes Classification

Python supports various file modes that determine how a file is opened and what operations can be performed on it. Understanding these modes is essential for proper file handling.

Complete File Modes Reference Table

File Mode	Description	File Position	Creates File	Common Use Cases
'r' (Read)	Open file for reading (default mode)	Beginning	No	Reading existing files, data processing
'w' (Write)	Open file for writing, truncates file first	Beginning	Yes	Creating new files, overwriting existing content
'a' (Append)	Open file for writing, appends to end	End	Yes	Log files, adding to existing data
'b' (Binary)	Binary mode - used with other modes	Beginning	No	Images, executables, binary data
'+' (Read/Write)	Update mode - reading and writing	Beginning	Yes	Complex file operations
'x' (Exclusive)	Exclusive creation - fails if file exists	Beginning	Yes	Creating unique files, preventing overwrites

Quick Tip:

Use 'r' for reading files without modification
Use 'w' to create new files or overwrite existing ones
Use 'a' to add data to the end of existing files
Combine modes: 'rb' for reading binary files
Always use with statement for automatic file closing
Use 'x' when you want to ensure you're not overwriting existing files

File Operations Examples with Output

These examples cover the most common text and binary file operations. Each task is isolated so it is easy to learn one concept at a time.

1) Write Text to a File

Use 'w' mode to create a new file or overwrite an existing file.

Write file example

lines = [
    "Python is easy to learn.",
    "File handling stores data permanently.",
    "Use with statement for safety."
]

with open("sample.txt", "w") as file:
    for line in lines:
        file.write(line + "\n")

print("sample.txt created.")

2) Read Entire File

Use read() when you want all content at once.

Read complete file

with open("sample.txt", "r") as file:
    content = file.read()

print(content)

3) Read Line by Line

Iterating on the file object is memory-efficient for large files.

Read with line numbers

with open("sample.txt", "r") as file:
    for line_no, line in enumerate(file, 1):
        print(f"{line_no}: {line.strip()}")

4) Append New Content

Use 'a' mode to add data without deleting existing content.

Append example

with open("sample.txt", "a") as file:
    file.write("This line is appended.\n")

print("New line appended.")

5) Readline and Readlines

readline() gets one line; readlines() returns all lines as a list.

readline() and readlines()

with open("sample.txt", "r") as file:
    first_line = file.readline().strip()
    file.seek(0)
    all_lines = file.readlines()

print("First line:", first_line)
print("Total lines:", len(all_lines))

6) Basic Binary File Handling

Use binary modes (rb, wb, r+b) for images, PDFs, and byte data.

Binary read/write

binary_data = bytes(range(10))

with open("demo.bin", "wb") as file:
    file.write(binary_data)

with open("demo.bin", "rb") as file:
    read_back = file.read()

print(list(read_back))

File Operations with Error Handling

Error handling prevents crashes and gives user-friendly messages. Use try-except for predictable file problems like missing files or permission issues.

1) Safe File Reading

Handle missing file safely

def read_file_safely(filename):
    try:
        with open(filename, "r") as file:
            return file.read()
    except FileNotFoundError:
        return f"Error: {filename} not found."
    except PermissionError:
        return f"Error: permission denied for {filename}."

print(read_file_safely("sample.txt"))
print(read_file_safely("missing.txt"))

2) Safe Write with Backup

Create backup before overwrite

import os
import shutil

def safe_write_with_backup(filename, content):
    if os.path.exists(filename):
        shutil.copy2(filename, filename + ".backup")
    with open(filename, "w") as file:
        file.write(content)
    print("Write completed.")

safe_write_with_backup("sample.txt", "Updated content.")

3) Check File Properties

Validate file before processing

import os

filename = "sample.txt"
if os.path.exists(filename):
    print("Size:", os.path.getsize(filename), "bytes")
    print("Readable:", os.access(filename, os.R_OK))
    print("Writable:", os.access(filename, os.W_OK))
else:
    print("File not found.")

CSV File Handling

CSV stores tabular data in plain text. Python's csv module supports both list-based and dictionary-based workflows.

1) Write Rows to CSV

Write CSV using csv.writer

import csv

rows = [
    ["Name", "Age", "City"],
    ["Alice", 28, "New York"],
    ["Bob", 35, "Los Angeles"]
]

with open("employees.csv", "w", newline="") as file:
    writer = csv.writer(file)
    writer.writerows(rows)

2) Read Rows from CSV

Read CSV row by row

import csv

with open("employees.csv", "r") as file:
    reader = csv.reader(file)
    for row in reader:
        print(row)

3) Read CSV as Dictionary

Read with DictReader

import csv

with open("employees.csv", "r") as file:
    reader = csv.DictReader(file)
    for record in reader:
        print(record["Name"], record["City"])

4) Use Custom Delimiter

Semicolon-separated CSV

import csv

with open("products.csv", "w", newline="") as file:
    writer = csv.writer(file, delimiter=";")
    writer.writerow(["Product", "Price"])
    writer.writerow(["Laptop", "1200.99"])

JSON File Handling

JSON is ideal for nested, structured data. Use json.dump/json.load for files and json.dumps/json.loads for strings.

1) Write Python Dictionary to JSON File

Write JSON file

import json

data = {
    "company": "TechCorp",
    "employees": 3,
    "active": True
}

with open("company.json", "w") as file:
    json.dump(data, file, indent=2)

2) Read JSON File

Read JSON file

import json

with open("company.json", "r") as file:
    data = json.load(file)

print(data["company"])
print(data["employees"])

3) Convert JSON String to Python Object

Parse JSON string

import json

json_text = '{"product": "Laptop", "price": 1299.99}'
obj = json.loads(json_text)

print(obj["product"])
print(obj["price"])

4) Pretty Print JSON for Debugging

Use json.dumps with indent

import json

record = {"name": "Nikhil", "skills": ["Python", "SQL", "Git"]}
print(json.dumps(record, indent=2, sort_keys=True))

Context Managers and 'with' Statement

The with statement provides a way to ensure that resources are properly managed and cleaned up after use, even if errors occur.

with Statement Benefits

Automatic resource management
Files are properly closed even if exceptions occur
Cleaner, more readable code
Reduces risk of resource leaks

# Without 'with' statement
file = open('example.txt', 'r')
try:
    content = file.read()
    # Process content
finally:
    file.close()  # Must close manually

# With 'with' statement
with open('example.txt', 'r') as file:
    content = file.read()
    # Process content
# File automatically closed here

Creating Custom Context Managers

You can create your own context managers using classes or contextlib.

# Custom context manager using class
class Timer:
    def __init__(self, name):
        self.name = name
    
    def __enter__(self):
        import time
        self.start = time.time()
        return self
    
    def __exit__(self, exc_type, exc_val, exc_tb):
        import time
        elapsed = time.time() - self.start
        print(f"{self.name} took {elapsed:.2f} seconds")

# Using custom context manager
with Timer("File Processing"):
    with open('data.txt', 'r') as f:
        data = f.read()
    # Process data here

File and Directory Management

1. File Existence Check

Use os.path.exists().

Example: file existence

import os

if os.path.exists("sample.txt"):
    print("File exists")
else:
    print("File not found")

2. Delete a File

Use os.remove().

Example: delete file

import os

os.remove("sample.txt")

3. Rename a File

Use os.rename().

Example: rename file

import os

os.rename("old.txt", "new.txt")

Directory Management

Directories are folders used to organize files.

4. Get Current Directory

Example: current directory

import os

print(os.getcwd())

5. Change Directory

Example: change directory

import os

os.chdir("Documents")

6. Create Directory

Example: create folder

import os

os.mkdir("NewFolder")

7. Create Nested Directories

Example: nested folders

import os

os.makedirs("A/B/C")

8. List Files and Folders

Example: list directory

import os

print(os.listdir())

9. Remove Directory

Example: remove empty folder

import os

os.rmdir("NewFolder")

Warning: Directory must be empty.

10. Remove Non-Empty Directory

Use shutil.

Example: remove folder tree

import shutil

shutil.rmtree("A")

11. Copy Files

Example: copy file

import shutil

shutil.copy("file1.txt", "file2.txt")

12. Move Files

Example: move file

import shutil

shutil.move("file1.txt", "Documents")

13. File Information

Example: file size

import os

size = os.path.getsize("sample.txt")

print(size)

Common os Functions

Function	Purpose
`getcwd()`	Current directory
`chdir()`	Change directory
`mkdir()`	Create folder
`listdir()`	List contents
`remove()`	Delete file
`rename()`	Rename file

Common shutil Functions

Function	Purpose
`copy()`	Copy file
`move()`	Move file
`rmtree()`	Delete folder tree

Real-World Applications

File handling is used in virtually every Python application. Here are practical real-world scenarios:

Data Logging

Application logs, error tracking, user activity:

import logging
from datetime import datetime

def log_activity(user, action):
    timestamp = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
    log_entry = f"{timestamp} - User: {user}, Action: {action}\n"
    
    with open('app_log.txt', 'a') as log_file:
        log_file.write(log_entry)
    
    print(f"Logged: {log_entry.strip()}")

# Usage
log_activity("alice", "login")
log_activity("bob", "file_upload")

Configuration Files

Reading and writing application settings:

import json

def load_config(filename='config.json'):
    try:
        with open(filename, 'r') as config_file:
            return json.load(config_file)
    except FileNotFoundError:
        # Create default config
        default_config = {
            "app_name": "My Application",
            "version": "1.0.0",
            "debug": False,
            "max_users": 100
        }
        save_config(default_config, filename)
        return default_config

def save_config(config_data, filename='config.json'):
    with open(filename, 'w') as config_file:
        json.dump(config_data, config_file, indent=2)

File Upload Processing

Handling user uploaded files:

def process_uploaded_file(uploaded_file, upload_dir='uploads'):
    import hashlib
    import os
    
    # Create upload directory if it doesn't exist
    os.makedirs(upload_dir, exist_ok=True)
    
    # Read uploaded file content
    content = uploaded_file.read()
    
    # Generate unique filename
    file_hash = hashlib.md5(content).hexdigest()[:8]
    original_name = uploaded_file.filename
    extension = os.path.splitext(original_name)[1]
    new_filename = f"{file_hash}{extension}"
    
    # Save file
    file_path = os.path.join(upload_dir, new_filename)
    with open(file_path, 'wb') as f:
        f.write(content)
    
    return file_path

Data Import/Export

Converting between different file formats:

import csv
import json

def csv_to_json(csv_file, json_file):
    """Convert CSV file to JSON format"""
    data = []
    
    with open(csv_file, 'r') as csv_f:
        csv_reader = csv.DictReader(csv_f)
        for row in csv_reader:
            data.append(row)
    
    with open(json_file, 'w') as json_f:
        json.dump(data, json_f, indent=2)
    
    print(f"Converted {csv_file} to {json_file}")

# Usage
csv_to_json('data.csv', 'data.json')

            
                Key Takeaways
            
            Always use the with statement for file operations to ensure proper closing
Understand different file modes: 'r' (read), 'w' (write), 'a' (append), 'b' (binary)
Use try-except blocks to handle file-related exceptions (FileNotFoundError, PermissionError)
The csv module simplifies working with comma-separated values files
The json module handles JSON data serialization and deserialization
Binary files require 'b' mode and work with bytes instead of strings
Use os and shutil modules for file system operations
glob is useful for pattern-based file searching
Create custom context managers for complex resource management
Always validate file paths and check file existence before operations
Implement proper error handling and logging for file operations
Consider file encoding (UTF-8 is recommended for text files)
For large files, read in chunks instead of loading entire file into memory
Regularly backup important files before performing write operations

        

Previous: Modules Next: Exception Handling