Connect with us

blogs Why Business Messaging Apps Slow Down as Teams Grow: Insights from Software Developers
why-business-messaging-apps-slow-down-as-teams-grow

Why Business Messaging Apps Slow Down as Teams Grow: Insights from Software Developers

Author : Apoorva Nayak

The first sign is rarely an outage. Someone posts in the company channel, and a colleague two desks away sees it a second later than she used to. Then it's two seconds. Nobody files a ticket over a one-second delay, so people just start saying the app feels heavy. I've built the backends behind that feeling, and the cause is almost never the messages. Messaging apps rarely slow down because of the messages themselves. They slow down because every message now has to reach, notify, and index for many more people.

A chat message is a few hundred bytes of text. Storing it is cheap. What costs money and milliseconds is everything that happens after the insert: pushing it down every open socket in the channel, bumping each member's unread count, waking phones through APNs and FCM, writing it into a search index, and telling the bots. I call that bill the fan-out tax, and it grows faster than your headcount does.

Here's how the tax shows up, stage by stage, as a workspace grows from a handful of people to a company of ten thousand.

At 10 people, nothing breaks, and that's when the bad habits set in

A ten-person team can run on almost anything. One application server, one database, one WebSocket per client, and a send handler that loops through channel members and writes to each socket. Ten members means ten writes. Unread counts come from a COUNT query when the sidebar loads. Search is a LIKE clause. It all feels instant because it is.

The fan-out tax exists at this size, but it's pocket change. The trouble is that these shortcuts quietly become the architecture. I once inherited a codebase where the message endpoint still looped through recipients synchronously, inside the HTTP request, four years and several thousand users after someone wrote it as a prototype. Every new member made every send a little slower, and nobody had noticed the slope until it got steep.

What fixes it: move fan-out off the request path now, onto a queue, even if a single worker drains it. Store unread counts as counters you increment, not numbers you calculate on read.

At 100 people, presence and typing start to outtalk the humans

Somewhere around a hundred users, the heaviest traffic on your sockets stops being messages. It's presence. Every time someone goes online, idles, locks a laptop, or opens the mobile app, a status event fans out to everyone who might see that person in a list. Typing indicators add more, because a naive client sends one on every burst of keystrokes. Most of these events are never looked at by anyone.

The big platforms learned to stop sending them. In its April 2023 write-up on real-time messaging, Slack Engineering explains that a client receives presence notifications only for

the subset of users visible on screen at that moment, and that transient events like typing travel the same fan-out path as messages but are never written to the database. That's the right instinct. Presence is a view concern, so scope it to the view.

This is also the stage when the backend sprouts its satellite services: a search indexing worker, a notification worker, a webhook relay for the first integrations, a small pipeline that copies messages into a data warehouse for retention or compliance. In the stacks I've worked on, most of those services end up in Python, because the client libraries for Elasticsearch, FCM and every SaaS API you'll want to integrate are mature there. Teams whose in-house people are mostly front-end and mobile engineers often hire remote Python developers specifically for this worker layer, and it's a sensible split. The workers are where the fan-out tax gets paid, so they deserve owners who think about queues and retries all day.

Unread counts and badges get interesting here too. A hundred people in thirty channels each is three thousand counters, and a message in a busy channel touches dozens of them at once. Then the mobile badge has to agree with the desktop sidebar, which has to agree with the web tab. When they disagree, users notice faster than they notice latency.

What fixes it: subscribe clients to presence only for what's on screen, debounce typing to one event every few seconds, and batch badge updates instead of pushing each one.

At 1,000 people, search, files, and push notifications fall behind

At a thousand users, the send path itself may still feel fine. The complaints move to the edges. "I searched for the message I just sent, and it isn't there." "The PDF preview is still spinning." "My phone buzzed four minutes late."

Search lag is usually a design choice nobody remembers making. If you index inside the send request, sends get slower as the index grows. If you index asynchronously, there's a gap. Elasticsearch, which a lot of messaging backends sit on, refreshes indices every second by default, but only on indices that have received at least one search in the last 30 seconds. So a quiet index can make a fresh message invisible for a moment, and a queued indexing worker adds its own backlog on top. Users don't care which one it was.

Files are the noisy neighbor. Thumbnails, PDF previews and video posters are CPU-heavy, and when they share a worker pool with notifications, one person uploading a 200-page contract can delay pushes for the whole office. I've watched exactly that happen on a Monday morning after a quarterly report went out.

Push notifications have hard limits that many teams discover in production. Google's FCM documentation sets a default quota of 600,000 messages per minute per project, allows up to 240 messages per minute and 5,000 per hour to a single Android device, and limits collapsible messages to a burst of 20 per app per device, refilling at one message every three minutes. An active user in forty channels can hit the per-device ceiling during a busy incident. iOS has its own limits on the APNs side, and requests that exceed them return errors.

There's also the cold start. When someone opens the app after a weekend, the client asks for everything it missed. Back in 2017, Slack Engineering described Flannel, an edge cache that lazily loads user and channel data, and reported that it cut the startup payload by 7 times on a 1,500-user team and by 44 times on a 32,000-user team. That gap tells you how much of a "slow app" is really a slow first request.

What fixes it: give indexing, previews and push their own queues, collapse and aggregate notifications server-side before they reach APNs or FCM, and load the sidebar lazily.

At 10,000 and up, the fan-out tax compounds

This is where the math stops being linear. Discord's engineers put it bluntly in an October 2023 post on large-server scaling: the work needed to handle a server grows quadratically with its size. If 1,000 online users each post once, that's roughly a million notifications. At 10,000 users, it's 100 million.

The published numbers are worth reading side by side:

  • Discord, October 2023: around 90% of user-to-server connections in large servers were passive, and treating them that way made fan-out 90% cheaper. Each relay process handles up to 15,000 connected sessions.
  • Slack, April 2023: messages are delivered across the world in 500ms, and at peak a single channel server host serves about 16 million channels.
  • Slack, April 2023: traffic spikes at the top of every hour, driven by reminders, scheduled messages, and calendar events.
  • Discord, March 2023: moving message storage from 177 Cassandra nodes to 72 ScyllaDB nodes brought p99 latency for fetching message history from 40 to 125 milliseconds down to 15.


A company workspace isn't a public community with a million people online, and an all-hands channel is quieter than a busy gaming server. Still, I've seen the same curve bend in announcement channels at a few thousand members, especially when everyone replies with an emoji within the same minute. That comparison is my own reading of Discord's data, not a claim Discord makes about corporate chat.

Integrations and bots become real users at this size. A CI bot that posts every deploy to a 5,000-member engineering channel creates 5,000 deliveries, 5,000 counter updates and a burst of push decisions, all from one HTTP call. Webhooks that retry on timeout make it worse, because the retry arrives exactly when the system is slowest.

For on-premises deployments, hardware sizing is where the fan-out tax turns into a purchase order. Size for peak concurrent connections and the top-of-the-hour burst, not for the number of seats on the license. Memory goes to open sockets and in-memory presence, disk IOPS go to the search index, and file storage grows on its own schedule regardless of how many people are typing. I'd rather see an admin over-provision the search nodes and under-provision the app nodes than the other way around.

What fixes it: separate active from passive sessions, put relays between channels and connections, rate-limit bots on their own budget, and size on-premise clusters from load-test data rather than headcount.

More servers buy time, not speed

"Just add more servers."

It's the line you hear in every capacity meeting, and it works for exactly the parts of the system that were never the problem. Stateless API nodes and preview workers scale out nicely. Fan-out doesn't, because fan-out is coordination, not raw compute.

Look at how real systems route a channel. Slack maps each channel to one channel server using consistent hashing, so a hot channel lands on one host whether the fleet has ten machines or a thousand. Adding hosts spreads the other channels around, but the loudest one stays exactly as loud. Discord hit the database version of the same wall. Its 2023 post on storing trillions of messages explains that lots of concurrent reads on one busy channel could hotspot a partition and drag down every query hitting those nodes.

Their fix was not more machines. It was fewer. They moved to a database that handled the load better and added Rust data services with request coalescing, so that if many users ask for the same row at the same time, the database gets queried once. The cluster shrank from 177 nodes to 72, each with more disk, and got faster.

That's the honest answer to the opponent. More hardware delays the pain. Smarter fan-out removes it.

Four team sizes, four different bottlenecks

If you only remember one part of this article, make it the table. The bottleneck moves as the team grows, so the fix that saved you last year usually isn't the one you need next.

Team size

What breaks

Symptom users notice

Fix

10

Nothing yet, but fan-out runs inside the send request

None, which is the problem

Queue fan-out and store unread counts as counters

100

Presence and typing events, badge sync

Sidebar flicker, badges that disagree across devices

Scope presence to the screen, debounce typing, batch badges

1,000

Search indexing, file previews, push quotas, cold start

Fresh messages missing from search, late pushes, slow app open

Separate queues, collapse notifications, lazy-load the sidebar

10,000+

Quadratic fan-out, hot channels, bots, hardware sized by seats

Messages a second or two late in big channels, top-of-hour lag

Passive sessions, relays, bot rate limits, load-test-based sizing

A load-test checklist before you add the next thousand users

Most load tests I've reviewed simulate an average user sending an average message to an average channel. Real workspaces aren't average. They're spiky, lopsided, and full of bots. Here's what I'd run before the next big onboarding wave.

  • Test with your loudest channel, not your typical one. Build a synthetic channel with the membership of your biggest all-hands or support channel, then have a realistic share of members reply within the same minute. If p99 delivery holds there, the small channels will be fine.
  • Replay the top of the hour. Reminders, scheduled posts, and calendar notifications all fire on the hour, and Slack's own engineers call it out as a traffic spike. Schedule a few thousand of them for the same minute in your test and watch the queues, not just the API.
  • Measure send-to-render latency on a phone. Server-side timings hide the last hop. Put a real device on a mediocre Wi-Fi network in the test and record when the message actually appears, at p99, because the median will always look fine.
  • Count pushes per user per minute. Compare the busiest user's push rate against FCM's 240-per-minute per-device ceiling and the 20-message collapsible burst. If any test user gets close, aggregate before you send rather than hoping the platform will be lenient.
  • Time how long a new message takes to become searchable. Send a unique string, then poll search until it appears. Run this while the indexer is under load. A few seconds is acceptable to most people, but thirty usually isn't.
  • Kill a node mid-test. Take down a channel host or a search node while traffic is flowing. Slack reports that a replacement channel server is ready in under 20 seconds, which gives you a reasonable benchmark for how long users should see higher latency during a failover.
  • Let the bots in. Wire up your real integrations, with their real retry settings, and fire a deploy notification into the biggest channel at peak. Bots are the users nobody remembers to model, and they never get tired.

Headcount is the number on the contract, but fan-out is the number your servers actually pay.

Conclusion

Business messaging apps don't necessarily slow down because teams send more messages; they slow down because the infrastructure behind every message becomes more complex as organizations grow. From presence updates and typing indicators to search indexing, push notifications, file processing, and bot integrations, every additional user increases the workload on the backend.

As software developers, the key is to build for scalability from the beginning. Implementing asynchronous queues, optimizing message fan-out, separating background services, managing notifications, and conducting realistic load tests can prevent performance bottlenecks. Ultimately, adding more servers is not always the solution. Smarter architecture, efficient resource management, and scalable infrastructure are what keep business messaging applications fast and reliable as teams expand.

Frequently Asked Questions 

1. Why do business messaging apps become slower as teams grow?

Business messaging apps become slower as teams grow because the backend must handle more than just sending and storing messages. Each message can trigger multiple operations, including real-time delivery, unread count updates, push notifications, search indexing, and bot integrations. As more employees join channels, these operations create additional server workload. This process is known as message fan-out. Without proper architecture, asynchronous processing, and optimized database operations, the increasing workload can cause message delays, slow notifications, and reduced application performance.

2. What is message fan-out in business messaging applications?

Message fan-out refers to the process of distributing a single message or event to multiple connected users. For example, when someone posts in a company channel with 500 members, the application may need to deliver that message to hundreds of users, update unread counters, and trigger relevant notifications. As channel membership increases, the workload also increases. Developers can optimize fan-out using message queues, relay servers, passive connection handling, and efficient event distribution strategies to maintain consistent messaging performance.

3. How can software developers improve messaging app performance?

Software developers can improve messaging app performance by optimizing backend architecture and reducing unnecessary processing. Implementing asynchronous message queues helps prevent delivery operations from blocking user requests. Developers can also store unread counts as counters, limit presence updates to visible users, debounce typing indicators, and separate search indexing from notification processing. Additionally, load testing, database optimization, caching, and monitoring server performance help identify bottlenecks before they affect users. These practices improve scalability, reduce latency, and support reliable communication across growing teams.

4. Why are push notifications and search features delayed in messaging apps?

Push notifications and search features can become delayed when their background processing systems cannot handle incoming workloads efficiently. Messaging applications rely on notification services such as Firebase Cloud Messaging (FCM) and Apple Push Notification service (APNs) to deliver updates. These services have delivery limits and quotas. Similarly, search indexing may take time because messages are processed asynchronously. Heavy file uploads and overloaded worker queues can increase these delays. Developers can address these issues by separating workloads, aggregating notifications, and optimizing indexing pipelines.

5. Is adding more servers enough to scale a business messaging application?

Adding more servers can improve capacity, but it does not automatically solve every scalability problem. Some messaging bottlenecks involve coordination, database hotspots, and message distribution rather than insufficient computing resources. For example, a highly active channel can overload a particular channel server even when other servers have available capacity. Developers need to evaluate the entire architecture, including connection management, database performance, message relays, and background workers. Effective scaling requires load testing, intelligent workload distribution, request coalescing, and infrastructure designed around actual usage patterns.

Recent blogs
To create a Company Messenger
get started
download mobile app
download pc app
close Quick Intro
close
troop messenger demo
Schedule a Free Personalized Demo
Enter
loading
Header
loading