Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Similar, but different requirements. All of the messages were grouped by sending process, and when the sending process said its session was over its messages were fed into a post-processing queue. There was support for holding messages in memory or on disk, and data-preserving fallbacks if the post-processing halted or got too backed up. The message rate varied quite a bit; during the US business day the rate was at least doubled. We used that to give the post-processing queue time to catch up off-hours.

This was designed about 16 years ago, so well before the big-data tools, ssds, or powerful machines we have today.



Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: