Similar, but different requirements. All of the messages were grouped by sending process, and when the sending process said its session was over its messages were fed into a post-processing queue. There was support for holding messages in memory or on disk, and data-preserving fallbacks if the post-processing halted or got too backed up. The message rate varied quite a bit; during the US business day the rate was at least doubled. We used that to give the post-processing queue time to catch up off-hours.
This was designed about 16 years ago, so well before the big-data tools, ssds, or powerful machines we have today.
This was designed about 16 years ago, so well before the big-data tools, ssds, or powerful machines we have today.