Optimizing Python Virtual Environments

Streamline Your Development Workflow for Enhanced Efficiency and Performance

Introduction

Navigating the complexities of Python development often involves juggling various dependencies and project requirements. Fortunately, Python offers robust solutions for managing these challenges through virtual environments and tools like Miniconda.

Whether you're a seasoned developer or just diving into Python, mastering these environment management techniques is crucial for maintaining clean, organized, and efficient workflows.

Different approach to Environments Creation

  1. Using VirtualEnv

  2. Using MiniConda

NOTE: This tutorial assumes you are a Windows user, and have administrator access, All others cases may have different procedures, but the approach is similar to the one being followed here.

Python Env using VirtualEnv

Managing python virtual environment using virtualenv gives all the powers to the user, and make everything manual, meaning, you need to manually manage different python versions on your system. And, the same goes for python packages.

Configuration

1. Setup Powershell Execution Policy:

This parameter defines whether you are able to run locally created PS scripts or not. Before setting your execution policy to something, you shold be able to check current execution policy. To check enter the code:

Get-ExecutionPolicy

If, you get Restricted, means you need to change it, otherwise you are good to proceed. To change execution policy, run the following code:

Set-ExecutionPolicy RemoteSigned

2. Installation of Base Python Version using installers

Now, here comes the tricky part, where you need to choose a Python package you will install. Though you are going to install many different versions of Python, there shall be one main in particular that will be responsible for installtion of virtualenv python package, which will help further in the process to create virtual environments. In my case, I'll go with the latest version avaliable i.e. Python 3.12. So, I'll simply download the python installer and run it. Here, you now need to note three things.

  1. The installtion directory. (C:/Users/username/AppData/Local/Programs/Python/Python312)

  2. Add to path ( shall be checked )

  3. Install for all users ( recommended )

Take a good look and if necessary, save the location of installation directory, which will be required further in the procedure. 3. Installion of Other versions of Python using installers: This section is based on your use, which ever versions of Python you prefer, such as, 3.11, 3.10, etc. you can download. And, set the installation configuration as follows:

  1. Installation Directory

  2. Add to path ( unchecked )

  3. Install for all users ( recommended ) NOTE: Third option may or may not be avaliable, but that does not affect us.

3. Installtion ofvirtualenv python package

Now, we'll proceed to install the virtualenv python package. But, before that, we'll check the version of python installed on the system. Run the following command in powershell or cmd ( any one of them ) python --version It shall be showing you Python 3.12.x here, x can be any number depending upon when you install. Next, we'll proceed to install virtualenv. To do so, run the command: pip install virtualenv And, this will get installed under the python version 3.12.x

4. Setting up virtual environment inside a directory

Here, comes the important part, first make sure you know which python version you want to work upon, may be 3.11 or 3.10 or any other. Also, make sure, you have installed that version as discussed in Step 3. Let's go with version 3.11 for this tutorial. To do so, open powershell inside the directory you want to. And, run the command python -m virtualenv -p path2python.exe env_name Here, note the two variables

  1. env_name: This shall be the name of the environment you'll create. This won't make much of a difference unless you are working with different python environments within same directory.

  2. path2python.exe: Here, we need to enter the path of directory in which python.exe of that specific version is installed. For ex, C:/Users/username/AppData/Local/Programs/Python/Python311/python.exe this shall be used for python 3.11 if you have choosen to install in the default directory.

    The command will take sometime and, you'll see a new folder, of the name of env_name appear.

5. Activating the environment

Once, environment is initialised, you now need to activate so that you can use it in your terminal. To do so, follow the steps:

  1. Open powershell in the root of directory where you created the enviroment

  2. Run the command: env_name/Scripts/activate. This will activate your environment and you'll now see your env_name in the terminal written before the folder path.

6. Deactivating the environment

You may also need to deactivate the python environment once you are done. You can obviously just close the terminal, but deactivating is a good practice. To do so, run the following command: deactivate This will deactivate the environment.

Python Env using MiniConda

This is my personal favourite, and the advantage this provides over the other is, I don't need to create virtual envioments manually, everything is handled by MiniConda, a lightweight alternative to Anaconda.

Configuration

1. Install MiniConda

You first need to install Miniconda on your system after downloading from its official website. The website shows Python versions on which it is working, but that doesn't really matter. The base python version is of the version mentioned on the website, something we did in Step 2 while using virtualenv. Keep the following things in mind while installation:

  1. Install for all users / just me ( as per your requirement )

  2. Add to path ( checked )

  3. Register Miniconda3 as my default Python 3.11 ( checked )

2. Setting up for the first time

After, Step 1 is complete, you have successfully installed Miniconda. Follow the steps to setup conda such that you can easily access the environments on the go.

  1. Using any terminal, open Anaconda Powershell Prompt ( Miniconda3 )

  2. Conda will be opened in its base environment

  3. Run conda init cmd.exe to configure it for Command Prompt

  4. Run conda init powershell to configure it for Powershell Here, is a list which you can use with conda init name

    • bash

    • cmd.exe

    • powershell

    • fish

    • tcsh

    • zsh

    • xonsh

  5. Congratulations you have successfully setup Miniconda on your device

Few MiniConda Commands

  1. Create new Environment: conda create -n env_name python=version ...additional_packages

  2. Delete a conda environment: conda remove --name ENV_NAME --all

  3. List installed packages in environment: conda list

  4. List all conda environments: conda env list

My Recommendation

I'll recommend to use Miniconda instead of any other solution avaliable, such as virtualenv, and Anaconda

Advantage Miniconda has over virtualenv

  1. Provides a cui based approach where miniconda handles the python package management part, so you don't need to keep track of where you installed python package

  2. You don't need to manually install python versions

Distadvantage Miniconda has over virtualenv

  1. Less control over Python versions

  2. All your packages are installed in working directory

Advantage Miniconda has over Anaconda

  1. Very less in size abt 150 GB in place of 5.7 GB

  2. Many pre-installed packages, some of which you might rarely use

Disadvantage Miniconda has over Anaconda

  1. Anaconda provides a GUI based approach for each package management

  2. Provides pre-installed packages which can facilitate development

Conclusion

Python virtual environments and Miniconda are both tools used in Python development to manage dependencies and isolate project environments. Virtual environments, often created using the built-in venv module, allow developers to create isolated environments for each project, ensuring that dependencies do not conflict across projects and enabling easier package management. On the other hand, Miniconda provides a lightweight distribution of Conda, a powerful package and environment manager, allowing users to create separate Python environments with different dependencies and versions efficiently. While virtual environments offer a native Python solution, Miniconda extends this capability with additional features and broader compatibility for scientific computing and data science tasks. Both tools are essential for maintaining clean and reproducible Python development environments.