I don't know what you mean by "as a requirement for most datatypes". I work with molecules, where distances are typically measured in Angstroms and masses in amu. I don't want to have factors of 1E−10 and 1.660538922E−27 hanging around my code.
I assume that he means that the type system knows that these are in useful units, and tracks them, so you can have it check that your calculation actually results in a value of the unit you expected.
Yeah. There's really no reason why our hot new scientific computing languages and libraries should all be lacking capabilities that graphing calculators had in the 1980s when they only had 2k of RAM. HP demonstrated that it doesn't even require a CAS to be extremely useful.
I know what unit libraries are, and why they can be useful. There are several units libraries for Python, the Boost C++ library includes one, etc.
I don't know why they should be a 'requirement for most datatypes'. Could someone please explain the requirement part?
As a further clarification, why should people in scientific computing, in fields which use non-SI units like eV, amu, Angstrom, barn, light year, and megaparsec, use a programming language which requires SI units? Quasar 3C 273 is 749 megaparsecs from us, or 2.31E25 meters away. I don't see why SI should be preferred.
Any programming language that makes units a first class part of its type system would allow for defining custom units just as easily as defining other data types. Nobody's saying that SI has to be the only units expressible, just that it has to be the foundation of the unit system. Likewise, unitless numerical quantities will necessarily still be expressible, but using those types for variables that represent a value with a unit should be considered extremely poor practice, just like when communicating those values on paper.
There's really not a good reason to argue against using only SI units for data interchange formats. It's trivial to map to the preferred units on import or display, if and only if you know what units the input data is in. I've dealt with too many bugs where interacting programs have differing assumptions about meters, centimeters, and millimeters to believe that the flexibility of storing different units on disk is ever worth the trouble.
I use Python. If I use one of the third-party packages for units then I can do what you say I can, but at extreme cost. Every single operation checks for unit conversion, on the off-chance that the values aren't compatible. The system, in trying to be nice to me, ends up making things invisibly slow.
(In practice, the performance code runs in C, so the Python/C boundary would have to negotiate the array types for full unit safety.)
In my work the base unit of length is angstroms. I've used nanometers a few times, and never used any other length unit, though I know that GROMAC's xtc format uses picometers. Saying something has a volume of 600 cubic angstroms is much more useful than 6E-28 cubic meters. While I can appreciate that other fields closer to human scale use may like to standardize through SI, I don't want your preferences enforced on my field. All I see is the chance to make things worse, and slower, and don't see any advantages.
One of my data formats has coordinates in angstroms, like "8.420 50.899 85.486". How would you suggest that I write that in an exchange format? As "8.420E-10 50.899E-10 85.486E-10"? (Or the last two normalized to E-11.) At the very least that's a lot of data for very little gain. It gets worse for trajectories, which might save 1 million time steps x 10,000 atoms/time step x 3 coordinates/atom = 3 billion coordinates to an exchange file. I see no advantage to doing that in SI units.
In practice those distance coordinates will likely internally represented in angstroms. Consider that the Lennard-Jones potential is sometimes written as A/r^12 - B/r^6 , with expected values of r around 1E-10m. The denominator of the first will go to 1E-120 in intermediate form, and not be representable in 32-bit float. While not relevant for Python, which uses 64 bit floats, some molecular dynamics programs will use 32 bit float. (Eg, for older GPU machines, or to save space.)
My other example was the atomic mass unit, another non-SI unit. I have only used amu (for chemistry) or dalton (for biology) in my work, not kilograms. It seems pointless to require that I store the mass of a carbon as 1.9926467051999998e-26 kg instead of 12 amu.
I therefore disagree, and believe there are good reasons to argue against SI units for some data interchange formats. I agree that I want to store a single distance unit on disk, only that unit is the non-SI unit angstrom and amu, and not the tremendously huge meter or kg.
The idea that you can export a csv file or excel worksheet full of unitless numbers is scientifically whacky. In the same way most would scoff at an graph without axis labels why too don't we scoff at a data file without units.
What's whacky about implicit units for a given format?
I use data files without explicit units all the time. A format specification, which might be explicit or implicit, might say that a given file is stored in CSV format where the first column contains a molecule representation (in my case as a SMILES string), followed by an identifier, followed by the molecular weight (in amu, which is the only reasonable unit), followed by surface area (in Ų, which is always the case), followed by volume (in ų, again, always the case). You'll note that none of these are SI units.
I have another file containing a molecular structure in SDF format, which starts:
Line 4, the 44 is "number of atoms", the 45 is "number of bonds", and the (4.5411, 4.0194, 0.0000) is a coordinate in angstroms. I see a distinct lack of units in the data file.
I have another file, in PDB format, containing lines like:
ATOM 34 N GLY 1 6 8.420 50.899 85.486 0.50 51.30
The (8.420, 50.899, 85.486) is a coordinate in Å. The other numbers are identifiers of one sort or other, or the unitless occupancy and B-factor.
Are you really going to scoff at my entire field, for working with data files without explicit units since the late 1960s?
When working specifically with Excel files, an organization tends to stabilize on what certain column titles mean, so the a title of "MW" means "molecular weight in AMU", etc. This is more complicated when applied to values which are parameter dependent (charge at pH 7.5 vs. 6.0), or depend on specific models and/or software version. You'll notice that pH is a unitless number.
Units are only a subset of the ontology usually omitted from a given file format. Others include unitless terms like B-factor, pH, "number of atoms", prediction model, and implementation version. This ontology is often instead made explicit in external format documentation or through shared knowledge of the users of that data file.
What advantage there is for me or my field to have 'SI units as a requirement for most datatypes', especially since we often deal with non-SI units like Å and amu? I don't see anything except more confusion, more chances for error, and performance overhead of going from/to SI units instead of staying in the domain-specific preferred system.
The people who I've seen try to use a Semantic Web/Linked Data approach end up bogged down in verbose and slow to parse data formats that make it hard to do real work, because the software has to be wary that the input one moment might be in Å, the next in nm, the third in m, and the fourth in yoctoparsecs.
The PDB format is an agreed-upon format that most fields don't have the luxury of using. And its implicit units keep it from being significantly more useful - outside of its own particular field. But you are correct, it has units - and that is a HUGE advantage over nearly any other scientific data format.
If the PDB format really had explicit units you could start to use it in other fields, easily - without knowing anything about the format itself. But again, PDB is an example of a well-codified format.
It'd be great if every figure/table you saw in a paper had an associated <.xsciformat> which was united (interesting that I meant unit-ed, but that's exactly what it would do - it would unite). That way you could download files from a gel-shift assay and directly and computationally compare the data with the diffusion data from a microscopy assay, and utilize pHs estimated from PDB files, or any other such really interesting co-interactions with the raw data itself. Right now this kind of co-linking of data across disparate fields is impossible. And I think much of it could be clarified if the user couldn't print out a graph/dataset that didn't have units - implicit or otherwise.
You asked "why too don't we scoff at a data file without units". You just answered your own question: because it's an "agreed-upon format."
All of my response was to point out that an Excel spreadsheet, CSV file, etc. can equally be considered an "agreed-upon format" by those who use it, so don't need explicit units.
My original question was a simple one. Why should units be a requirement for most datatypes?
I know all of the reasons for why it's useful. I don't understand why it should be a requirement.
The PDB format is not an easy format to understand. Unit conversion is one of the least of the problems in using it outside of its field. Determining bond assignments is much harder, and bond type assignment harder still. In fact, I have a hard time figuring out an example where an explicit "this is in unit X" would make things appreciably easier, as compared to near useless data taking up space.
Could you give an example of how someone could start to use it in another field, easily, where they couldn't now? I can only see it occurring by completely replacing the format, since adding an "A" after each coordinate, or a comment at the top that the coordinates are in angstroms, can't be what you mean. (Nor would including the PDB spec as a comment in each record be what you mean either - though it would be self-documenting!)
For that matter, the X-ray resolution field in a PDB record contains significant digits, so "2.0Å" and "2.00Å" mean different things. The ontology of units is not easy. 2.00Å is 2.00E-10m, not 2E-10m.
In any case, I deal with a lot of unitless numbers as well: pH, molarity, number of atoms and bonds, number of rotatable bonds, ratio between elongation and fixed elongation, etc. The ontology of values is not easy, and a required SI-unit system looks much more like it would get in the way than be useful.
I agree, but in some fields this is already done. For example, This is exactly why geoscientists have standardized on the fully self-described netcdf file format. With netcdf, you can specify units, axis labels and other metadata very straightforwardly.
