Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

McIlroy's code lends itself very well to literate coding! Here's a try:

# First, tr anslate multiple s queezed occurrences of the c omplement of A-Za-z (non-word-characters) into a line separator

  tr -cs A-Za-z '\n' |
# and then lowercase every word.

  tr A-Z a-z |
# Sort the words with a disk-based mergesort.

  sort |
# Count the unique characters :: [String] -> [(Int,String)]

  uniq -c |
# And do a reverse numerical disk-based merge sort.

  sort -rn |
# And write the first $1 lines and then quit.

  sed ${1}q
Personally, I'd change the last two to be

  sort -rn -k 1,1 | head -n $1
but that's just bikeshedding. And the parts it's made from are so modular, and so focused, that you can wrap your head around all of the problem, without worrying about how sort works, or how tr expands character ranges.

If I gave a "professional software developer" in 2011 a problem to count word frequencies, and I got back 10 pages of Pascal that didn't go much faster than code that fits on a Post-It note for my problem, I wouldn't trust that developer with anything else important.



The reason for preferring sed 42q instead of head -n 42 is because head didn't used to exist. IIRC it was created at Berkeley and for some years there were many systems that didn't have it. It does seem a bit superfluous when all it could do was head -42 (the -n and other options came later). I still write sed 42q to this day. :)


Your code fails for French text.


As did Knuth's; see below the discussion about that.

If you wanted to change it, to split "words" by blanks and punctuation characters, you would have

  tr -s [:blank:][:punct:] "\n"
and then that piece would fit into the pipeline, the rest unscathed. This design is far easier to reason about than a dozen pages of Pascal.


Those globs should be protected from the shell otherwise ./bc is going to alter tr's arguments.


How many modern programs comply with Unicode Text Segmentation http://unicode.org/reports/tr29/ for finding word boundaries or Unicode Collation Algorithm http://unicode.org/reports/tr10/ for sorting the words.


Honestly, that looks more like examples in a MAN page rather than explanations of an algorithm.




Consider applying for YC's Fall 2026 batch! Applications are open till July 27.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: