Python - Combined Example of Reading and Writing

Combined Example of Reading and Writing in Python

File operations in Python allow developers to interact with data in external files, enabling persistent storage, data transformation, and reporting. The two primary operations—reading from and writing to files—are crucial for building robust applications. In this guide, we will explore how to perform both reading and writing in a single program, understand different modes and practices, and look at real-world use cases where both actions are required together.

Introduction to File Handling in Python

What is File Handling?

File handling refers to the process of reading data from and writing data to files. Python's built-in functions like open(), along with methods like read(), write(), and readlines(), provide all necessary capabilities for this.

Why Combine Reading and Writing?

• Updating records in-place
• Reading data, processing it, and writing the results
• Copying data from one file to another
• Filtering or transforming content

Understanding File Modes

Modes Used for Reading and Writing

• 'r': Read-only mode (default)
• 'w': Write mode (overwrites file)
• 'a': Append mode
• 'r+': Read and write mode (does not truncate)
• 'w+': Write and read mode (truncates existing file)
• 'a+': Append and read mode

Choosing the Right Mode

Use 'r+' to read and modify existing files, 'w+' to clear and rewrite content, and 'a+' to preserve and extend existing content.

Basic Example: Copying File Content

Step-by-Step Code

# Copy contents from source.txt to destination.txt

with open("source.txt", "r") as src, open("destination.txt", "w") as dest:
    for line in src:
        dest.write(line)

Explanation

This example demonstrates how to open one file for reading and another for writing simultaneously using a single with statement. It reads each line from the source file and writes it into the destination file.

Reading, Modifying, and Writing Back

Problem Statement

Suppose we have a file containing a list of user names. We want to read all names, convert them to uppercase, and save them back to the same file.

Code Example

# Convert names to uppercase and rewrite file

with open("users.txt", "r") as file:
    lines = file.readlines()

lines = [line.upper() for line in lines]

with open("users.txt", "w") as file:
    file.writelines(lines)

Key Concepts

• Reading all content first to avoid overwriting before processing.
• Using readlines() and writelines() for batch operations.
• Transforming data using list comprehensions.

Using r+ Mode for In-Place Modification

What is 'r+'?

It allows reading and writing without truncating the file. However, writing starts from the beginning of the file.

Code Sample

with open("sample.txt", "r+") as f:
    content = f.read()
    f.seek(0)
    f.write(content.upper())

Seek Function

seek(0) resets the cursor to the beginning so that writing starts from the top of the file.

Reading from One File and Writing Filtered Data

Use Case

Filter out lines containing a specific keyword and write the rest to a new file.

Example

with open("input.txt", "r") as infile, open("filtered.txt", "w") as outfile:
    for line in infile:
        if "ERROR" not in line:
            outfile.write(line)

Appending Logs While Reading Input

Use Case

Read data from a file and log results to a separate file.

Example

with open("data.txt", "r") as data, open("log.txt", "a") as log:
    for line in data:
        log.write(f"Processed line: {line}")

Working with Structured Data

Reading from CSV and Writing to JSON

import csv
import json

data = []

with open("people.csv", "r") as csvfile:
    reader = csv.DictReader(csvfile)
    for row in reader:
        data.append(row)

with open("people.json", "w") as jsonfile:
    json.dump(data, jsonfile, indent=4)

Updating Specific Line in a File

How to Modify One Line

with open("config.txt", "r") as file:
    lines = file.readlines()

lines[2] = "timeout=60\n"

with open("config.txt", "w") as file:
    file.writelines(lines)

When to Use This

• Updating config files
• Correcting data in structured formats

Working with Large Files

Line-by-Line Reading

with open("bigfile.txt", "r") as infile, open("output.txt", "w") as outfile:
    for line in infile:
        if "Python" in line:
            outfile.write(line)

Why Avoid read() for Big Files

read() loads the entire file into memory which can cause performance issues with large files. Use line iteration instead.

Reading and Writing Binary Files

Copying Images or Executables

with open("image.jpg", "rb") as src, open("copy.jpg", "wb") as dest:
    while chunk := src.read(1024):
        dest.write(chunk)

Why Binary Mode?

Text mode can alter binary data like newline characters. Binary mode ensures byte-for-byte accuracy.

Common Pitfalls and How to Avoid Them

Overwriting Before Reading

Always read content before opening the same file in write mode to avoid data loss.

File Not Found Errors

try:
    with open("unknown.txt", "r") as f:
        print(f.read())
except FileNotFoundError:
    print("File does not exist.")

Improper Resource Management

Use with blocks to ensure automatic closing of files even if errors occur.

Best Practices

• Use context managers (with) to manage files.
• Prefer line-by-line reading for large files.
• Read before writing if updating existing content.
• Use structured formats like JSON or CSV when applicable.
• Handle exceptions like FileNotFoundError and IOError.

Real-World Scenario: Processing Student Records

Use Case

Read student grades from a CSV, calculate pass/fail, and write to a new report file.

Example

import csv

with open("grades.csv", "r") as infile, open("report.csv", "w", newline='') as outfile:
    reader = csv.reader(infile)
    writer = csv.writer(outfile)

    writer.writerow(["Name", "Grade", "Status"])
    for row in reader:
        name, grade = row
        status = "Pass" if int(grade) >= 50 else "Fail"
        writer.writerow([name, grade, status])

Debugging Tips

Print File Paths

import os

path = os.path.abspath("data.txt")
print("Opening file at:", path)

Use Logs

with open("process.log", "a") as log:
    log.write("Successfully processed 1000 lines.\n")

Combining reading and writing in Python file handling is a common task in real-world software development. Whether you're transforming raw data into formatted reports, parsing logs, or creating backup copies, understanding the correct use of file modes, structure, and error handling is essential. With modes like r+, w+, and a+, and the power of Python’s built-in libraries, developers can create efficient, readable, and safe file-processing scripts.

From reading and modifying config files to processing structured datasets like CSV and JSON, mastering the ability to read from and write to files forms the foundation of data engineering and application development in Python.