Comments on: Builds And Transitive Dependencies http://mriou.wordpress.com/2007/02/09/builds-and-transitive-dependencies/ If I only knew what this is about... Thu, 11 Jun 2009 14:28:35 +0000 http://wordpress.com/ hourly 1 By: pyclocioche http://mriou.wordpress.com/2007/02/09/builds-and-transitive-dependencies/#comment-2799 pyclocioche Mon, 25 Feb 2008 02:25:13 +0000 http://mriou.wordpress.com/2007/02/09/builds-and-transitive-dependencies/#comment-2799 To me it is necessary to find To me it is necessary to find

]]> By: Tobias Roeser http://mriou.wordpress.com/2007/02/09/builds-and-transitive-dependencies/#comment-854 Tobias Roeser Mon, 30 Apr 2007 07:32:52 +0000 http://mriou.wordpress.com/2007/02/09/builds-and-transitive-dependencies/#comment-854 Ok, maybe I mixed the terms buildsystem and package management system. Then you are right, a build system does not need to handle transitive dependencies (at least not to resolve foreign packages) but then, it depends itself on another tool (or a human) to provide all these. This is the reason why we want more than make, Ant, Rake, (fill in your tool)... We want the buildsystem to include all these package management capabilities and thus, at least Ant fails in this term. Even Maven sometimes fails as you've described above. I've seen it revolving different packages for the same codebase but on a different computer, depending on the content of the local repository - what a crap. Ok, maybe I mixed the terms buildsystem and package management system. Then you are right, a build system does not need to handle transitive dependencies (at least not to resolve foreign packages) but then, it depends itself on another tool (or a human) to provide all these.

This is the reason why we want more than make, Ant, Rake, (fill in your tool)… We want the buildsystem to include all these package management capabilities and thus, at least Ant fails in this term. Even Maven sometimes fails as you’ve described above. I’ve seen it revolving different packages for the same codebase but on a different computer, depending on the content of the local repository – what a crap.

]]> By: Tobias Roeser http://mriou.wordpress.com/2007/02/09/builds-and-transitive-dependencies/#comment-851 Tobias Roeser Sun, 29 Apr 2007 22:16:49 +0000 http://mriou.wordpress.com/2007/02/09/builds-and-transitive-dependencies/#comment-851 Ingenious question! The term build system has to be defined before discussing what a real build system is. But I think you can agree, that a simple make replacement like Ant is not enough. Or at least that for the most mid-size projects with many packages/modules more is needed. Thats the reason why we want some meta support from out build system like dependency resolution or any other task that need some cleverness to help the developer. Ant is just a DSL for shell scripting. Ingenious question!

The term build system has to be defined before discussing what a real build system is. But I think you can agree, that a simple make replacement like Ant is not enough. Or at least that for the most mid-size projects with many packages/modules more is needed.

Thats the reason why we want some meta support from out build system like dependency resolution or any other task that need some cleverness to help the developer. Ant is just a DSL for shell scripting.

]]> By: mriou http://mriou.wordpress.com/2007/02/09/builds-and-transitive-dependencies/#comment-849 mriou Sun, 29 Apr 2007 15:42:47 +0000 http://mriou.wordpress.com/2007/02/09/builds-and-transitive-dependencies/#comment-849 So you wouldn't even describe Ant as "a real build system"? So you wouldn’t even describe Ant as “a real build system”?

]]> By: Tobias Roeser http://mriou.wordpress.com/2007/02/09/builds-and-transitive-dependencies/#comment-845 Tobias Roeser Sun, 29 Apr 2007 10:05:13 +0000 http://mriou.wordpress.com/2007/02/09/builds-and-transitive-dependencies/#comment-845 Transitive dependencies are an essential requirement dictated by the nature of software development. Each buildsystem which claims to be a real one, has to support transitive dependendies. The question is, at what level this support goes and how clever these dependencies are handled. Working with Maven, I can say that dependency handling cannot going worse. Comming from the Linux distribution Gentoo I'm used to a package managment system (portage) which is more clever than all tools I've found for the whole java and scripting language world in terms of (transitive) dependency management. Please read my complete comment here: https://lepetitfou.dyndns.org/home/node/33 Transitive dependencies are an essential requirement dictated by the nature of software development. Each buildsystem which claims to be a real one, has to support transitive dependendies.

The question is, at what level this support goes and how clever these dependencies are handled. Working with Maven, I can say that dependency handling cannot going worse. Comming from the Linux distribution Gentoo I’m used to a package managment system (portage) which is more clever than all tools I’ve found for the whole java and scripting language world in terms of (transitive) dependency management.

Please read my complete comment here: https://lepetitfou.dyndns.org/home/node/33

]]> By: mriou http://mriou.wordpress.com/2007/02/09/builds-and-transitive-dependencies/#comment-211 mriou Wed, 21 Mar 2007 02:30:44 +0000 http://mriou.wordpress.com/2007/02/09/builds-and-transitive-dependencies/#comment-211 The dependencies should always be downloaded only once on a given computer. The issue is when you wipe out your dependency directory, change your computer or somebody else tries to build. You could also want to build 3 years later and find that nothing's distributed anymore. So depending on your level of paranoia you can either startup with external dependencies handling and then check them in svn or cvs. Or you can stick with an external repository. That's more or less your choice. The dependencies should always be downloaded only once on a given computer. The issue is when you wipe out your dependency directory, change your computer or somebody else tries to build. You could also want to build 3 years later and find that nothing’s distributed anymore.

So depending on your level of paranoia you can either startup with external dependencies handling and then check them in svn or cvs. Or you can stick with an external repository. That’s more or less your choice.

]]> By: Saulius Sinkunas http://mriou.wordpress.com/2007/02/09/builds-and-transitive-dependencies/#comment-210 Saulius Sinkunas Tue, 20 Mar 2007 21:18:49 +0000 http://mriou.wordpress.com/2007/02/09/builds-and-transitive-dependencies/#comment-210 BTW how many time do you have to check dependencies? It was always dangerous for me to depend on online remote repositories. That's why we keep all dependant jars locally. This transative dependency is usefull only at startup or when you want to add some big fat library. Almost 95% of time you will reuse your existing dependant jars. And don't want to worry that at the last stage (before production) somebody changes some jar or dependency. BTW how many time do you have to check dependencies? It was always dangerous for me to depend on online remote repositories. That’s why we keep all dependant jars locally. This transative dependency is usefull only at startup or when you want to add some big fat library. Almost 95% of time you will reuse your existing dependant jars. And don’t want to worry that at the last stage (before production) somebody changes some jar or dependency.

]]> By: mriou http://mriou.wordpress.com/2007/02/09/builds-and-transitive-dependencies/#comment-176 mriou Sat, 10 Feb 2007 21:33:47 +0000 http://mriou.wordpress.com/2007/02/09/builds-and-transitive-dependencies/#comment-176 Tom, I agree transitive dependency is just too useful, but conflict happens much more often than you would think. Turns out there's not much originality in the Java world and everybody use the same libraries (commons-*, XML parsing). And then once you're packaging all those mixed together, surprises happen and it takes time, when you're not focused on it, to realize it's because you have several copies of the same thing. But I also agree that if we carefully think of it, there are solutions to these problems. We can try to be more clever than what other tools (read Maven) offer. You're more than welcome to look at the problem. I think I'll also look into how to improve our Gem repository. Like including dependencies in generated gems and build "grouped" gems, like the rails one for example (that links to ActiveRecord, ActionPack, ...). Because at least RubyGems asks you whether you want to install a dependency or not. Tom, I agree transitive dependency is just too useful, but conflict happens much more often than you would think. Turns out there’s not much originality in the Java world and everybody use the same libraries (commons-*, XML parsing). And then once you’re packaging all those mixed together, surprises happen and it takes time, when you’re not focused on it, to realize it’s because you have several copies of the same thing.

But I also agree that if we carefully think of it, there are solutions to these problems. We can try to be more clever than what other tools (read Maven) offer.

You’re more than welcome to look at the problem. I think I’ll also look into how to improve our Gem repository. Like including dependencies in generated gems and build “grouped” gems, like the rails one for example (that links to ActiveRecord, ActionPack, …). Because at least RubyGems asks you whether you want to install a dependency or not.

]]> By: Tom Ayerst http://mriou.wordpress.com/2007/02/09/builds-and-transitive-dependencies/#comment-175 Tom Ayerst Sat, 10 Feb 2007 09:08:59 +0000 http://mriou.wordpress.com/2007/02/09/builds-and-transitive-dependencies/#comment-175 I suspect that most of the time the there will not be a conflict and automatic transitive dependency management is just too useful to not have it. What I would like (and I'll start looking at how to do it). Is to have transitive dependencies but flag up conflicts for manual resolution and then capture the choice and use it in future. Later it could offer heuristics for "best" choice etc. I suspect that most of the time the there will not be a conflict and automatic transitive dependency management is just too useful to not have it.

What I would like (and I’ll start looking at how to do it). Is to have transitive dependencies but flag up conflicts for manual resolution and then capture the choice and use it in future.

Later it could offer heuristics for “best” choice etc.

]]> By: mriou http://mriou.wordpress.com/2007/02/09/builds-and-transitive-dependencies/#comment-174 mriou Sat, 10 Feb 2007 04:46:02 +0000 http://mriou.wordpress.com/2007/02/09/builds-and-transitive-dependencies/#comment-174 Adam, my worry is not so much for the build system itself, except when dependencies can't be found at all. It's more for the runtime and the expectation that a build environment will be as close as possible to the runtime.A system of isolated classloaders will work well for the build but mostly people won't be able to get anything running out of what it produces. Adam, my worry is not so much for the build system itself, except when dependencies can’t be found at all. It’s more for the runtime and the expectation that a build environment will be as close as possible to the runtime.A system of isolated classloaders will work well for the build but mostly people won’t be able to get anything running out of what it produces.

]]>