Deepak Kumar Jha
LiveHindustan — Holding a 60M+ monthly-user news platform through national events
HT Media Ltd. · Nov 2018 – Sep 2021 · Full role details →
The Problem
LiveHindustan.com is HT Media’s Hindi-language news property. A news site’s traffic is not steady — it spikes hard and without much warning around elections, cricket finals and breaking national news — and the backend needed to stay up and fast through exactly those moments, not just on an average day.
Constraints
- Traffic that could multiply many times over within minutes around a scheduled event (an election count, a match) or with no warning at all (breaking news).
- A news homepage and article page are read-heavy but cannot be purely static — headlines, live scores and breaking banners update continuously and have to reach 60M+ monthly users without stale caches showing yesterday’s news.
- A backend team of 5, responsible for keeping the whole property up, not just building features.
Architecture
The design centres on getting as many requests as possible answered before they reach the application servers at all, with the cache layers invalidated precisely enough that "fast" never meant "stale":
CDN edge layer
Serves cacheable article and homepage content close to the reader, absorbing the bulk of read traffic before it reaches origin.
Redis
Caches hot, frequently-changing data — trending lists, live-score widgets, homepage modules — with short, deliberately-tuned TTLs.
Elasticsearch
Serves search and related-article queries without hitting the primary datastore for every request.
Application layer
Handles cache misses, publishing and the editorial workflow; sized for steady-state load rather than peak, because peak is absorbed above it.
Key Decisions & Trade-offs
Cache for the event, not just the average day
A caching strategy tuned for typical traffic will still fall over on the one day it matters most — an election night is a different load profile from a normal Tuesday, not just a bigger one. I tuned TTLs and invalidation specifically against the spike scenarios (elections, cricket finals, breaking news) rather than against average request volume, which is the more common but weaker default.
Cut response times from 400–600ms to sub-100ms as the headline metric, not uptime alone
Uptime is necessary but not sufficient for a news site under load — a page that eventually loads at 3 seconds during a spike has already lost the reader to a competitor’s app notification. I optimised for response time under peak load specifically, treating "stays up" as the floor and "stays fast" as the actual target.
Build the team’s process alongside the system
Introduced CI/CD, code review and TDD practices across a 5-person backend function. A caching architecture that only one person understands is a liability the first time that person is unavailable during an incident; the process changes were there so the on-call reality matched the architecture’s complexity.
Outcome
- Sustained stability through elections, cricket finals and national breaking news at 60M+ monthly users.
- Cut API response times from 400–600ms to sub-100ms under peak load.
- Directed greenfield development of the Hindustan Smart City platform — architecture, API standards and deployment pipelines from day one.
What I'd Do Differently At Greater Scale
At meaningfully greater scale, the next bottleneck is the CDN/Redis invalidation path itself — a coarse-grained purge that clears more than the changed content is safe but wasteful; a properly tagged, content-addressable invalidation scheme would buy more headroom before the origin tier needs to grow.