The orthogonality ship has sailed with Unix, for better or (IMHO) for worse. The Unix Haters were right about this, among other things, and the sections of the handbook about the shell are still valid, even as time and Moore have rendered others quaint.
For worse, I concur. I still fire up my SGI boxes once in a while to remind myself things didn't use to/don't have to be as binary as they mostly are today (Linux vs BSD, iOS vs Android, Intel vs AMD etc.). That, and to run electropaint, of course. =)
That was already fixed with -print0 | xargs -0, but then this solution is dismissed with "The problem is that this is not a portable construct;...".
The -delete isn't either, so this is a straw man argument, although it probably is the most efficient and secure of all.
Because find changes to the directory first (carefully not following symlinks), and then deletes the file from there.
It does not delete the file using the entire path (which may contain a sudden symlink).
It's not possible to do this safely using xargs.
Take a look also at -execdir which does the same thing - changes to the directory first, and runs things from there. -exec is not safe and should not be used.
xargs is not safe if you are running against a directory not your own. You should use find and -execdir instead.
Yes, the original authors of posix made a mistake here.
> Also, rant rant, I really don't understand why find was extended with -delete in the first place.
Rob Pike and Brian Kernighan warned about this trend in their seminal paper "Program Design in the UNIX Environment" (also known as "cat -v considered harmful" which describes how proper unix programs should be designed:
I did a quick look online to see who has it. GNU, DragonFly BSD, NetBSD, and FreeBSD all have a -delete option. OpenBSD seems to be the only one that doesn't (although its online manpage does give examples on how to do it with -exec and piping to xargs).
That's basically two flavours of UNIX (Linux and BSD), but that's OK - others are unfortunately either dead or dying. The UNIX family tree (http://www.levenez.com/unix) has been shrinking so rapidly in the last few years, that the current situation looks like the early years.
Nobody remembers the various command line switches of find or grep. I use xargs quite often because it's a very simple concept and I know how to mix and merge any commands together with it and that's something I've only had to to go ahead and learn once.
I'm not an expert by any means, so please provide correction/feedback if I'm wrong, but I've read that xargs is more efficient because it can send parameters to the cmd in batches. I've read this in contrast to `-exec {}` not `-delete`, however.
For example:
find . -name '*~' -exec 'rm {}'
The statement above executes `rm result` for every result. By contrast:
find . -name '*~' | xargs rm
The example above would group the results and pass them to rm like this:
rm result1 result2 result3 result4
Because I'm not an expert, I don't know how many parmeters it will pass, or if initiation of many new processes has a significant performance impact on newer systems. I would suspect that for anything involving the disk, I/O will be the bottle neck, not process turn-up time.
-delete will be the fastest because find can do everything, and it already has the file loaded, and doesn't have to do a second lookup.
-exec will be slower because for each file it has to spawn a new process (this is where most of the time goes), and then that new process has to look up the file.
xargs will be faster than -exec because it will collect a few hundred, or thousand, filenames, and pass them all to one command (the article claims 4096 is the default, on some systems it may be lower). This means that typically, only 3 processes need to spawn: `find', `xargs', and one `rm'; instead of find, and many, many `rm's.
Now, xargs is still slower than -delete because it will buffer the filenames, either waiting for the list to end, or 4000 filenames to pass to `rm'. Then, rm must look up the file from the filename.
To make my point I just set up a test situation with 1110 files ending with `~', among a total of 2221 files. I tested how fast it is to delete all the files ending with `~' using the 3 following commands:
Most systems have a find command that supports the "-exec command {} +" option, which does not spawn the command for each result, but instead puts as many arguments as the shell will allow on the command line.
You are correct about -delete. You could also use -exec to execute arbitrary commands for each file returned by find. I needed some examples however, and I often see "find | xargs" because people don't know, or forget, the options to find. I should write a follow up post on find.
Well, you want xargs as soon as your project is big, because you'll reach the limits on arguments fairly quickly.
I won't go too deep on the issues this will trigger if your filenames contain special characters. We often think of spaces, but you could have some nastier stuff. If you want to sound smart at your next geeks reunion, simply read http://www.dwheeler.com/essays/fixing-unix-linux-filenames.h...
And I would guesstimate (Linux, kernel >= 2.6.23) to still be a fairly small amount of the machines people interact with professionally through a command line.
And if in a script/snippet, you often want to cover a vast majority of the systems you _could_ end up with. Won't be System III, at least for me, but there has to be a RHEL5 system in a closet right? :)
That's the kernel's limit, Bash (3) has its own limit, which is significantly lower, I've hit it before. I think Bash 4 can do an arbitrary number of arguments, within the kernel's limit.
No, bash doesn't and didn't have its own limit. On an ancient system (Debian Sarge), ARG_MAX=131072:
$ bash --version
GNU bash, version 2.05b.0(1)-release (i386-pc-linux-gnu)
Copyright (C) 2002 Free Software Foundation, Inc.
$ strace bash -c '/bin/echo `seq 1 30000`' 2>&1 | grep exec
execve("/bin/bash", ["bash", "-c", "/bin/echo `seq 1 30000`"], …) = 0
execve("/bin/echo", ["/bin/echo", "1", …) = -1 E2BIG (Argument list too long)
As you can see, the argument list too long error came back from the
execve syscall, i.e., from the kernel. (Note that I shortened the strace output to make it fit the page)
Of course, I meant my version as an alternative to xargs. And I think you have about 128k of space for the filenames, but yeah with large projects that can be a problem.
Thanks for the link, that's more interesting than the submission. :)
Recursively find all Python files and search them for the word ‘import’ find . -name '.py' | xargs grep 'import'*
Hmm. I don't mean to be a tweak, but you don't need xargs to do either of those things. Just:
find . -name '~' -delete
find . -name '.py' | grep 'import'
Note: I can't figure out how to get an asterisk to show up, and don't have time to look it up.