Managing Secrets and API Keys in Python Projects (.env Guide)

Image by author

# introduction to keeping secrets

It is dangerous to store sensitive information like API keys, database passwords, or tokens directly in your Python code. If these secrets are leaked, attackers can break into your systems, and your organization may suffer loss of trust, financial, and legal consequences. Instead, you should externalize the secrets so that they are never exposed in code or version control. A common best practice is to store the secrets in an environment variable (outside your code). This way, secrets are never revealed in the codebase. However, since manual environment variables do work, it is convenient to keep all secrets in a single .env file for local development.

This article explains Seven practical techniques for managing secrets in Python projectsWith code examples and explanations of common pitfalls.

# Technique 1: Using a .env file locally (and loading it safely)

A .env file is a text file key=value Add the ones you keep locally (not in version control). It lets you define environment-specific settings and secrets for development. For example, a recommended project layout is:

my_project/
  app/
    main.py
    settings.py
  .env              # NOT committed – contains real secrets
  .env.example      # committed – lists keys without real values
  .gitignore
  pyproject.toml

your real secrets are known .env Locally, for example:

# .env (local only, never commit)
OPENAI_API_KEY=your_real_key_here
DATABASE_URL=postgresql://user:pass@localhost:5432/mydb
DEBUG=true

On the contrary, .env.example This is a template that you commit for other developers to see which keys are needed:

# .env.example (commit this)
OPENAI_API_KEY=
DATABASE_URL=
DEBUG=false

Add pattern to ignore these files in Git:

So that your secret .env never gets checked out accidentally. In Python, it is common to use python-dotenv library that will load .env File at runtime. For example, in app/main.py You can write:

# app/main.py
import os
from dotenv import load_dotenv

load_dotenv()  # reads variables from .env into os.environ

api_key = os.getenv("OPENAI_API_KEY")
if not api_key:
    raise RuntimeError("Missing OPENAI_API_KEY. Set it in your environment or .env file.")

print("App started (key loaded).")

Here, load_dotenv() is found automatically .env in the working directory and sets each key=value In OS.Environment (Unless that variable is already set). This approach protects you from common mistakes like committing an .env or sharing it insecurely, while still providing a clean, reproducible development environment. You can switch between machines or dev setups without changing code, and local secrets remain secure.

# Technique 2: Read secrets from the environment

Some developers put placeholders like API_KEY=’test’ In their code or assume that the variables are always set in development. It may work on their machine but fails in production. If a secret is missing, the placeholder may turn on and create a security risk. Instead, always obtain the secret from an environment variable at runtime. In Python, you can use OS.Environment Or os.getenv To get the values safely. For example:

def require_env(name: str) -> str:
    value = os.getenv(name)
    if not value:
        raise RuntimeError(f"Missing required environment variable: {name}")
    return value

OPENAI_API_KEY = require_env("OPENAI_API_KEY")

If a secret is missing this causes your app to rapidly fail on startup, which is much safer than proceeding with a missing or dummy value.

# Technique 3: Validate the configuration with the Settings module

As projects grew, many fell apart os.getenv Calls become disorganized and error-prone. like using setting class pydentic’s basesettings Centralizes configuration, validates types, and loads values from .env and environment. For example:

# app/settings.py
from pydantic_settings import BaseSettings, SettingsConfigDict
from pydantic import Field

class Settings(BaseSettings):
    model_config = SettingsConfigDict(env_file=".env", extra="ignore")

    openai_api_key: str = Field(min_length=1)
    database_url: str = Field(min_length=1)
    debug: bool = False

settings = Settings()

Then in your app:

# app/main.py
from app.settings import settings

if settings.debug:
    print("Debug mode on")
api_key = settings.openai_api_key

This prevents mistakes like mistyping keys, parsing them incorrectly (“wrong” vs. wrong), or duplicating environment lookups. Using the Settings class ensures that your app fails fast if secrets are missing and avoids “works on my machine” problems.

# Technique 4: Using platform/CI secrets for deployment

When you deploy to production, you should not copy your local .env file. Instead, use secret management of your hosting/CI platform. For example, if you’re using GitHub Actions for CI, you can store encrypted secrets in the repository settings and then inject them into the workflow. This way, your CI or cloud platform injects the real values at runtime, and you never see them in the code or logs.

# Technology 5: Docker

In Docker, avoid hiding secrets in images or using plain ENVs. Docker and Kubernetes provide secrets mechanisms that are more secure than environment variables, which can leak through process listings or logs. For local development, .env plus Python-dotenv works, but in production containers, mount the secret or use the Docker secret. Avoid ENV API_KEY=… Write files with secrets in or to Dockerfiles. Doing this reduces the risk of secrets in the images being permanently exposed and simplifies rotation.

# Technique 6: Adding Railings

Humans make mistakes, so automate secret security. GitHub push security can block commits containing secrets, and CI/CD secret-scanning tools like Trufflehog or Gitleaks detect leaked credentials before a merge. Beginners often rely on memory or speed, leading to accidental commitments. Guardrails stops leaks before they enter your repo, making it safer to work with .env and environment variables during development and deployment.

# Technique 7: Using a Real Secrets Manager

For larger applications, it may make sense to use a proper secrets manager like HashiCorp Vault, AWS Secrets Manager, or Azure Key Vault. These tools control who can access secrets, log each access, and automatically rotate keys. Without it, teams often reuse passwords or forget to rotate them, which is risky. A secret manager keeps everything under control, simplifies rotation, and protects your production system, whether it’s a developer’s computer or a local one. .env The file has been exposed.

# wrapping up

Keeping secrets safe is more than following rules. It’s about building a workflow that makes your projects secure, easy to maintain, and portable across different environments. To make it easier, I’ve created a checklist that you can use in your Python projects.

.env is in .gitignore (Never present actual certificate)
.env.example exists and is committed with empty values
reads code secret only through environment variables (os.getenv, a settings class, etc.)
app fails faster With obvious error if any required secret is missing
you use different secrets For dev, staging and prod (never reuse the same key)
use CI and deployment encrypted secret (GitHub Actions Secrets, AWS Parameter Store, etc.)
Push security and or incognito scanning is enabled on your repo
you have a rotation policy (Rotate keys immediately if they leak and rotate regularly otherwise)

Kanwal Mehreen He is a machine learning engineer and a technical writer with a deep passion for the intersection of AI with data science and medicine. He co-authored the eBook “Maximizing Productivity with ChatGPT”. As a Google Generation Scholar 2022 for APAC, she is an advocate for diversity and academic excellence. She has also been recognized as a Teradata Diversity in Tech Scholar, a Mitex GlobalLink Research Scholar, and a Harvard VCode Scholar. Kanwal is a strong advocate for change, having founded FEMCodes to empower women in STEM fields.