On Fri, Mar 21, 2014 at 10:59 AM, Jamie Nguyen j@jamielinux.com wrote:

I'm not sure I'd use or promote such a feature.

And I don't blame you given I did such a poor job at putting forward my proposal. :-) I'm singing the benefits of the new dependency style without explaining what's so much better about them. Let me try again...

At the moment, most node modules use the tilde form of expressing dependencies. This means they use something like ~1.2.3, which translates to >1.2.3 <1.3. This shorthand was stolen from rubygems.

As it turns out, this highly restrictive form of dependency has been criticized in both the ruby and nodejs communities as being too restrictive. Espeically in the nodejs community it's unwarranted; the semantic versioning standard [1] that node modules are supposed to follow (and many do in practice) dictates that the major version must be bumped when backwards incompatible changes are made to a library. Thus, regardless of what the dependency says it's generally possible to upgrade to a new minor version of a library while keeping its dependents working.

A year ago, npm upstream introduced the caret (^) operator as an alternative to the tilde operator. With this operator, something like ^1.2.3 translates to >1.2.3 <2. This form of dependency much more accurately reflects what will actually work, and is more aligned with typical Fedora practice, where we often want to use newer compatible versions of libraries, First being one of our foundations and all.

In the last month, npm upstream recently switched 'npm install --save' to use carets instead of tildes, given that a year has passed since support for it was introduced so most npm installations can reasonably be expected to support it. Unfortunately, as the recent attempts to kill off the unfortunate use of a self-signed certificate for registry.npmjs.org indicate, there are still a lot of old npm installations in the wild (mostly thanks to PaaS providers), so many module authors are going to choose not to use the caret form to retain compatibility with these ancient npm versions for some time to come.

Of course, in Fedora we control the npm version provided, and indeed we've had an npm version that supports this (even if the dependency generator does not) for a year now as well. So we're free to encourage the use of it whenever we want, and using them in place of a tilde likely reflects what upstream would love to do if they didn't have to support PaaS providers that move at glacial paces.

Let's step back and take a look at what we're doing now. We often want to introduce newer versions of libraries to the package collection than are typically defined in dependents' package.json files. When we do that, we have to bump the dependency in all dependent packages. So if we want to update nodejs-foomodule from 1.2.3 to 1.3.4, we often have to go add '%nodejs_fixdep ~1.3.4' to all dependent packages. For certain core modules used by a lot of dependents, this could be a lot of work for what should be a simple upgrade.

If we instead used something like '%nodejs_fixdep ^1.2.3' in those dependent packages, we eliminate the need to push updates that consist solely of modifying package.json to potentially dozens of dependent modules whenever we update a heavily-depended-upon module. We can instead just update it and everything will Just Work. Much nicer, IMHO.

But doing this with classic %nodejs_fixdep is tedious. Now instead of just updating a handful of %nodejs_fixdep calls in %prep, you'd have to double-check every one against the new package.json and make sure you update the lower bound if it moved up.

Enter '%nodejs_fixdep --caretify'. Instead of manually keeping up with the tilde->caret conversion, it's just done for you automatically at build time. It's explicitly saying "don't allow dependencies that are of a lower version than is defined in package.json, but allow later versions up to but not including the next major version". It does what's right for Fedora without second-guessing upstream too much, unlike our current policy which overrides upstream to a greater degree.

I should note that in practical terms, 'sed -i s/~/^/g package.json' would accomplish the same thing, but by adding a special option to %nodejs_fixdep we can introduce error-checking that makes packagers less likely to shoot themselves in the foot. (I'm not entirely happy with --caretify as the argument, I'm open to better suggestions.)

I don't think it reduces the ability of packagers to know what's going on in a package. With our current practice, you either need to look at the combination of the package.json pre-%prep and the spec file to know how they're going to turn out, or else just look at package.json post-%prep. With my proposal, the vast majority of packages will just use '--tildefy' and nothing else, so you can just look at package.json pre-%prep and know all those tildes are going to be converted to carets later on, or look at package.json post-%prep as per usual.

That being said, a possible alternative would be to just pretend tildes are actually carets in the dependency generator. Then no spec changes are required ever, but package.json won't accurately reflect what could be possibly installed on the system. I rather like that package.json and RPM are in sync at all times when it comes to modules in the package collection. Explicit is better than implicit. ;-)

tl;dr - tilde dependencies are evil - we have something better now with carets - we really should use them in Fedora, it'll make our lives so much easier ;-) - given the above, --tildefy makes sense and eliminates tedious, unnecessary manual labor on our parts

I really think there's no point in us playing dependency cops to the degree we have been with current practices, and adopting the above will allow us to bring much more awesomeness to the distribution much more quickly without sacrificing quality in any way.

-T.C.

[1] http://semver.org/