Real-Time Market Data Ingestion & Insider Trading Dashboard: Open-Source NSE API Wrapper

Founder’s TL;DR:

• The Problem: Third-party financial data providers charge prohibitive subscription fees (up to ₹20 Lakhs/year) for exchange feeds, while public endpoints break constantly due to anti-scraping headers and session timeouts.
• The Engineering Fix: Built pnsea—a free, open-source Python library—to automate exchange session warming and format Option Chain, Equity, and Insider Trading disclosures into clean Pandas DataFrames. Layered a private FastAPI microservice on Railway to track promoter insider buying/selling in crores.
• The Bottom Line: Replaced costly commercial APIs with a 100% free open-source core running on a $5–$15/month Railway + Vercel micro-budget infrastructure.

Executive Summary

Accessing real-time National Stock Exchange (NSE) market data—such as Option Chains, implied volatility surface matrixes, equity quotes, and corporate insider trading disclosures—typically requires expensive commercial vendor feeds running up to ₹20 Lakhs ($24,000) per year.

Furthermore, existing PyPI packages for NSE data are frequently broken, abandoned, or fail under market opening volatility (9:15 AM IST) due to missing session warming protocols, dynamic cookie expiry, and strict anti-scraping headers.

To solve this, Anil Sardiwal created pnsea—a free, open-source Python library that handles session cookies, bypasses anti-scraping checks, and structures raw exchange JSON into production-ready Pandas DataFrames.

For internal analytics, Unreal Brains deployed a private FastAPI backend instance on Railway ($5–$15/mo) using pnsea as its core engine. The microservice aggregates promoter insider buying and selling volume in crores over custom timeframes (7d, 30d, 90d) and feeds a live Astro dashboard hosted on Vercel ($0).

Technical Architecture & Pipeline

Architectural Breakdown

1. pnsea Open-Source Core Library:

   • Session Warming Protocol: Automatically fetches root landing endpoints to acquire valid bm_sv and session cookies before querying data routes.
   • Pandas Native Output: Converts complex nested JSON payloads (Option Chains, SAST disclosures, Form 7 filings) directly into clean Pandas DataFrames (df = nse.get_option_chain("NIFTY")).
   • Standalone PyPI Solution: Unlike fragmented or broken scripts, pnsea provides an active, maintained open-source API layer for Python developers.

2. Private FastAPI Ingestion Instance (Railway):

   • Deployed on Railway ($5–$15/mo), the private FastAPI service runs scheduled background tasks using pnsea.
   • Ingests corporate insider trading disclosures, converts share quantities and transaction prices into net INR values, and aggregates promoter accumulation trends in Crores (₹10M INR).

3. High-Speed Analytics Dashboard (Vercel):

   • Renders quantitative visualizations on Vercel ($0 hosting) for zero monthly frontend overhead.

Technical Outcomes & Business Impact

• Commercial Savings: Replaced third-party data vendor subscriptions, saving thousands of dollars per year in third-party API fees.
• Open-Source Contribution: Released pnsea as a free and open-source PyPI package, filling a major gap for Python financial data engineering.
• Micro-Budget Operations: Deployed the complete private analytics engine and live insider tracking dashboard on a $5–$15/month total infrastructure spend (Railway + Vercel).
• Data Structuring Velocity: Enabled instant quantitative manipulation by outputting structured Pandas DataFrames natively out of the box.