Stop manually downloading the historical data of NSE stocks, use Python instead

If you are manually downloading the historical or daily data for the stocks from NSE, you should stop doing that and start using Python to do the same. There are many benefits of using Python to download historical data of NSE stocks like :-

1. It is fast and require minimal efforts
2. It can be automated
3. It can be used to update data daily

How to download the historical NSE stock data using Python

We will be using the following modules to download historical stock data from nse:-

1. nsepy
2. Pandas
3. Numpy
4. Jupyter Notebook
5. Pickle

The only reason, I would be using Jupyter Notebook is that it gives a better visualisation for Pandas Dataframe. However, later on I will be converting the script into function so that I could schedule it to auto update the data daily.

Python code for downloading historical NSE stock data

Enough talking, lets move to the coding part.

Creating a Virtual Environment

Whenever, you create a new project, it is advisable to create a virtual environment to keep all your installations separate. You can read more about virtual environments from here or you can watch the video - Venv in Python | Venv Python Explained | Virtual Environments in Python [MACOS/WINDOWS/LINUX]

So, open a terminal, make directory, create virtual environment and activate it

mkdir nse-data
cd nse-data
python3.10 -m venv .
source bin/activate

This will activate the virtual environment and you can proceed further with installing the modules.

Installing necessary modules

You can install the necessary modules using the following code:-

pip install numpy pandas notebook nsepy

We will be using Jupyter Notebook for initial data visualisation .But for this we will be using the Jupyter Notebook from our Virtual environment.

How to use Jupyter Notebook in Virtual Environment.

Since, we have installed the necessary modules in our Virtual Environment, we will need to use the Jupyter Notebook in the same Virtual Environment so that we do not get any ‘module not found error’. To use jupyter notebook in our virtual environment, we will have to create a kernel inside the virtual environment by using the following command:-

ipython kernel install --user --name=nse-data

# Output 
Installed kernelspec nse-data in /Users/uditvashisht/Library/Jupyter/kernels/nse-data

Here nse-data is the name of our virtual environment since we use ‘.’ while creating the virtual environment, it creates the environment in the same directory, if you have used some other name for your virtual environment, you would need to replace it with the same. Now we will run the Jupyter Notebook using the following command:-

jupyter notebook

And this will open the notebook in the browser window. Here, while creating a new python file, we will be using our virtual environment (nse-data)

Note - If you install some new modules, libraries, you would have to install the kernelspec again, refreshing it didn’t work for me

Download historical NSE stock data using python

Let’s start with doing all the necessary imports:-

from datetime import date
from nsepy import get_history
import numpy as np
import pandas as pd
import pickle

Now, say we need to download the data of SBI, for the period from 01.01.2022 upto 30.05.2022. So, we can do the following :-

symbol = "SBIN"
start = date(2022, 1, 1)
end = date(2022, 5, 30)
data = get_history(symbol=symbol, start=start, end=end)
data

Now, you will see that we got a Pandas Dataframe for the historical stock data of SBIN, with columns - Symbol, Series, Prev Close, Open, High, Low, Last, Close, VWAP, Volume, Turnover, Trades, Deliverable Volume, %Deliverble with Date as Index.

Saving the historical NSE Stock data as pickle in Python

Thereafter, we will save this dataframe as a Python Pickle, so that we can re-use it in future. Another benefit of saving the dataframe as a pickle is that, if we need to update our data, we would not have to download the data from the very beginning, we can program our function to upload the data for the missing dates only. So, let’s convert the dataframe into a pickle file.

with open('SBIN.pickle', 'wb') as f:
    pickle.dump(data, f)

The above code will create a pickle file of the dataframe of SBI’s historical data which can be later used even when you are offline.

Daily update the historical NSE stock data using Python.

Once, we have saved the historical data of the symbol as a pickle file, it is very easy to update it daily, to update it daily we can use the following code:-

symbol = "SBIN"
start = date(2022, 6, 1)
end = date(2022, 6, 1)
data = get_history(symbol=symbol, start=start, end=end)
with open('SBIN.pickle', 'rb') as f:
    old_data = pickle.load(f)
new_data = pd.concat([old_data, data])
with open('SBIN.pickle', 'wb') as f:
    pickle.dump(new_data, f)

So, this is how you can use the power of Python to download the historical data of NSE with just few lines of code. You can easily convert the above codes into functions and then schedule it to run at a specific time to keep a copy of data in your machine. You can use the list of NIFTY50 stocks downloaded in our previous post and loop through it to download and upload all the data of all the NIFTY50 Stocks.