Tutorial: CSV to JSON from Scratch in Python

Pandas is an absolute beast of a library, but importing it just to convert a spreadsheet into a JSON array is like using a sledgehammer to crack a nut. Python's standard library already has the csv and json modules, which are more than enough for 90% of these tasks without adding bloated dependencies to your environment.

The Logic Behind the Conversion

The secret weapon here is csv.DictReader. Instead of returning a list of strings for each row, it uses the first line of the CSV as keys for a dictionary. This removes the need to manually track column indices or map headers to values.

Step-by-Step Implementation

1. Extracting the Data
We need to open the file with newline="" as recommended by the Python docs to prevent issues with line endings across different operating systems.

import csv

def read_csv(csv_path):
    with open(csv_path, newline="", encoding="utf-8") as f:
        # DictReader automatically maps the header row to dictionary keys
        reader = csv.DictReader(f)
        return list(reader)

By wrapping the reader in list(), we load the entire dataset into memory. This is a practical tutorial for small to medium files; if you're dealing with a 2GB CSV, you'd want to iterate through the reader and write to the JSON file line-by-line.

2. Serializing to JSON
The json.dump method handles the conversion from a Python list of dictionaries to a valid JSON array.

import json

def write_json(rows, json_path):
    with open(json_path, "w", encoding="utf-8") as f:
        # indent=2 ensures the output isn't one giant, unreadable line
        json.dump(rows, f, indent=2)

3. The Final Pipeline
Connecting these two functions allows us to track the number of records processed, which is a critical sanity check when dealing with data pipelines.

import csv
import json

def read_csv(csv_path):
    with open(csv_path, newline="", encoding="utf-8") as f:
        reader = csv.DictReader(f)
        return list(reader)

def write_json(rows, json_path):
    with open(json_path, "w", encoding="utf-8") as f:
        json.dump(rows, f, indent=2)

def csv_to_json(csv_path, json_path):
    rows = read_csv(csv_path)
    write_json(rows, json_path)
    return len(rows)

if __name__ == "__main__":
    # Example usage
    try:
        count = csv_to_json("contacts.csv", "contacts.json")
        print(f"Successfully converted {count} rows to contacts.json")
    except FileNotFoundError:
        print("Error: The source CSV file was not found.")

Real-World Verification

If you have a contacts.csv file with the following content:

name,email,city
Alice,[email protected],Austin
Bob,[email protected],Denver

The resulting contacts.json will look exactly like this:

[
 {
 "name": "Alice",
 "email": "[email protected]",
 "city": "Austin"
 },
 {
 "name": "Bob",
 "email": "[email protected]",
 "city": "Denver"
 }
]

Performance and Limitations

For those looking for a deep dive into when to use this over a heavy library:

• Memory Footprint: This approach uses significantly less RAM than Pandas because it doesn't create a DataFrame object.
• Speed: For files under 50MB, the difference is negligible.
• Edge Cases: Python's csv module handles quoted fields (e.g., "New York, NY") automatically, so you don't have to worry about commas inside the data breaking your columns.

If you're building a lightweight AI workflow or an LLM agent that needs to preprocess data before feeding it into a prompt, keeping your dependencies zero is the way to go for faster deployment.