From: torvalds@transmeta.com (Linus Torvalds)
Subject: Re: A modest proposal -- We need a patch penguin
Date: 	Tue, 29 Jan 2002 03:23:11 +0000 (UTC)
Newsgroups: fa.linux.kernel

In article <200201290137.g0T1bwB24120@karis.localdomain>,
Francesco Munda  <syylk@libero.it> wrote:
>
>I deeply agree with you, especially in keeping "many eyes" to look at the
>same kernel tree, and not chosing one of the many subtrees; as added bonus,
>this stuff is buzzword compliant! What we can ask more? :)

Some thinking, for one thing.

One "patch penguin" scales no better than I do. In fact, I will claim
that most of them scale a whole lot worse. 

The fact is, we've had "patch penguins" pretty much forever, and they
are called subsystem maintainers.  They maintain their own subsystem, ie
people like David Miller (networking), Kai Germaschewski (ISDN), Greg KH
(USB), Ben Collins (firewire), Al Viro (VFS), Andrew Morton (ext3), Ingo
Molnar (scheduler), Jeff Garzik (network drivers) etc etc. 

If there are problems with certain patches, it tends to be due to one or
more of:

 - the subsystem is badly modularized (quite common, originally. I don't
   think many people realize how _far_ Linux has come in the last five
   years wrt issues like architectures, driver independence, filesystem
   infrastructure etc). And it still happens.

 - lack of maintainer interest. Many "maintainers" are less interested
   in true merging than in trying to just push whatever code they have,
   and only ever grow their patches instead of keeping them in pieces.

   This is usually a matter of getting used to it, and the best people
   get used to it really quickly (Andrea, for example, used to not do
   this well at all, but look at how he does it now! From a merge
   standpoint, his patches have gone from "horrible" to "very good")

 - personality/communication issues. Yes, they happen. I've tried to
   have other people be "filters" for the people I cannot work with, but
   I have to say that most of the time when I can't work with somebody,
   others have real problems with those people too. 

   (An example of a very successful situation: David Miller and Alexey
   Kuznetsov: Alexey used to have these rather uncontrolled patches that
   I couldn't judge or work with but that obviously had a lot of
   potential, and David acting as a filter made them a very successful
   team.)

In short, if you have areas or patches that you feel have had problems,
ask yourself _why_ those areas have problems. 

A word of warning: good maintainers are hard to find.  Getting more of
them helps, but at some point it can actually be more useful to help the
_existing_ ones.  I've got about ten-twenty people I really trust, and
quite frankly, the way people work is hardcoded in our DNA.  Nobody
"really trusts" hundreds of people.  The way to make these things scale
out more is to increase the network of trust not by trying to push it on
me, but by making it more of a _network_, not a star-topology around me. 

In short: don't try to come up with a "patch penguin".  Instead try to
help existing maintainers, or maybe help grow new ones. THAT is the way
to scalability.

		Linus

Date: 	Mon, 28 Jan 2002 22:00:19 -0800 (PST)
From: Linus Torvalds <torvalds@transmeta.com>
Subject: Re: A modest proposal -- We need a patch penguin
Newsgroups: fa.linux.kernel

On Mon, 28 Jan 2002, Rob Landley wrote:
>
> > A word of warning: good maintainers are hard to find.  Getting more of
> > them helps, but at some point it can actually be more useful to help the
> > _existing_ ones.  I've got about ten-twenty people I really trust, and
> > quite frankly, the way people work is hardcoded in our DNA.  Nobody
> > "really trusts" hundreds of people.  The way to make these things scale
> > out more is to increase the network of trust not by trying to push it on
> > me, but by making it more of a _network_, not a star-topology around me.
>
> You don't see an integration maintainer as a step in the right direction?
> (It's not a star topology, it's a tree.)

No, I don't really think an "integration manager" works well.

I think it helps a lot to have people pick up patches that nobody else
wants to maintain, and to gather them up. Andrea does that to some degree.
But it is _much_ better if you have somebody who is a point-man for
specific areas.

The problem with an overall guy is that there can't be too many of them.
The very thing you are _complaining_ about is in fact that there are a
number of over-all guys without clear focus, which only leads to confusion
about who handles what.

Clarity is good.

> Are you saying that Alan Cox's didn't serve a purpose during the 2.2 kernel
> time frame, and that Dave Jones is currently wasting his time?

No, I'm saying that there are not very many people who can do it, and who
can get the kind of trust that they are _everywhere_. Let's face it, Alan
grew to be respected because he did lots of different stuff over many
years, and he proved himself more than capable. And I suspect he's _quite_
happy not being in the middle of it right now.. It's a tough job.

It's a lot more likely to find people who can maintain _parts_. And if
there are patches that fall out of those parts, that tends to indicate a
lack of modularity, and perhaps a lack of maintainer for those parts.

And more likely, even if you _do_ find ten people who can do everything,
you don't want them to.

		Linus

Date: 	Mon, 28 Jan 2002 22:49:04 -0800 (PST)
From: Linus Torvalds <torvalds@transmeta.com>
Subject: Re: A modest proposal -- We need a patch penguin
Newsgroups: fa.linux.kernel

On Mon, 28 Jan 2002, Larry McVoy wrote:
>
> What you didn't do, Linus, is paint a picture which allows development
> to scale up.

Actually, I thought I did.

Basic premise: development is done by humans.

Now, look at how humans work. I don't know _anybody_ who works with
hundreds of people. You work with 5-10 people, out of a pool of maybe
30-50 people. Agreed?

Now, look at the suggested "patch penguin", and realize that the
suggestion doesn't add any scaling at all: it only adds a simple layer
that has all the same scaling problems.

What I'm saying is

 - I'm never going to work with hundreds of people directly, because it is
   fundamentally against my nature, by virtue of being human.

 - adding a "patch penguin" doesn't help, because _he_ (or she, although I
   didn't see any women nominated) is also going to be human. So either
   the patch penguin is going to do a bad job (and I won't start trusting
   him/her), or the patch penguin is going to have all the same issues
   people complain about.

Those are obvious truths. If you don't see them as being obvious truths,
you just haven't been thinking things through.

Did Alan do a good job? Yes. He did a _great_ job. But let's face it: (a)
he got really tired of doing it and (b) it really works only with one or
two Alan's, not with more - because with more you get people complaining
about the -aa tree vs the -dj tree vs the -marcelo tree vs the -linus
tree.

So Alan doesn't scale up either - I doubt you'll find a "better Alan", and
I _seriously_ doubt you'll be able to have multiple Alan's.

Does anybody really doubt this?

Now, if you've read this far, and you agree with these issues, I suspect
you know the solution as well as I do. It's the setup I already mentioned:
a network of maintainers, each of whom knows other maintainers.

And there's overlap. I'm not talking about a hierarchy here: it's not the
"general at the top" kind of tree-like setup. The network driver people
are in the same general vicinity as the people doing network protocols,
and there is obviously a lot of overlap.

Are there problems with maintainership? Yes. The main one being that it's
too damn easy to step on any toes. Which is why you want the modularity,
ie you do NOT want to have files issues that slash across different
maintenance boundaries (unless they have clearly defined interfaces:
that's where the importance of a clear VFS interface design comes in etc)

For an example of this, look at init/main.c in 2.2.x, in 2.4.x, and in
2.5.x. BIG changes. And most of the changes have very little to do with
what that file actually does (relatively little), and _everything_ to do
with the fact that it is the file that "instantiates" almost everything
and thus crosses all boundaries.

And notice how the "initcall" etc setup has changed, and cut a lot of
those dependencies. That's very much by design: look at what a device
driver had to do (and know) to be either a compiled-in driver or a modular
driver a few years ago. And look at a driver today.

In short, I'm saying that the true path to scalability is:

 - lack of dependencies on a source level

   This implies good interfaces that do NOT have common source files for
   different projects. In particular, it implies dynamic add/remove
   without the rest of the system having to know at all.

 - lack of people (whether patch-penguins or me) who have to follow
   everything.

   This, in turn, implies two things:

   (a) it is not the maintainers who pull things into their trees (because
       they aren't always there, and they don't know everything). It is
       the developers who _push_ onto maintainers.

   (b) if you as a developer cannot find a maintainer who you know, it is
       YOUR problem, and you cannot blame some super-penguin in the blue
       yonder for not caring about you. You may have to maintain your
       patch-set yourself. Or you should find a maintainer who cares about
       the work you do, and who helps feed it forward.

I know, I know. It's easier to whine about other people than it is to take
responsibility for your own actions. It's so easy to complain about "Linus
doesn't apply my patches", and so hard to just face the fact that Linus
never _will_ care about all patches, and that if you cannot find anybody
else to care about them either, then maybe they should die or you should
take care and feed them yourself.

I'm a bastard. I'm spending a lot of time applying patches, but it's not
my whole life. Never has been, never will be. I'll always be better at
applying patches from those ten people that I know and I trust than I'll
be at applying patches from people I don't trust.

If you're a developer, and you can't get through to me, ask yourself if
you can get through to somebody else.

And if you can't get through to anybody else either, ask yourself whether
maybe the problem is at _your_ end.

			Linus

Date: 	Tue, 29 Jan 2002 09:36:28 -0800 (PST)
From: Linus Torvalds <torvalds@transmeta.com>
Subject: Re: A modest proposal -- We need a patch penguin
Newsgroups: fa.linux.kernel

On Tue, 29 Jan 2002, Skip Ford wrote:
>
> Linus Torvalds wrote:
> [snip]
> > A word of warning: good maintainers are hard to find.  Getting more of
> > them helps, but at some point it can actually be more useful to help the
> > _existing_ ones.  I've got about ten-twenty people I really trust, and
>
> Then why not give the subsystem maintainers patch permissions on your tree.
> Sort of like committers.  The problem people have is that you're dropping
> patches from those ten-twenty people you trust.

No. Ask them, and they will (I bet) pretty uniformly tell you that I'm
_not_ dropping their patches (although I'm sometimes critical of them,
and will tell them that they do not get applied).

Sure, it happens occasionally that they really do get dropped, just
because I get too much email, but these people know how to re-send every
once in a while, and keep their patches separate.

I think there is some confusion about who I trust. Being listed as
MAINTAINER doesn't mean you are automatically trusted. The MAINTAINERS
list is not meant for me _at_all_ in fact, it's meant more as one of the
places for _others_ to search for a contact with.

Examples of people who I trust: Ingo Molnar, Jeff Garzik, Alan Cox, Al
Viro, David Miller, Greg KH, Andrew Morton etc. They've shown what I call
"good taste" for a long time. But it's not always a long process - some
of you may remember Bill Hawes, for example, who came out of nowhere
rather quickly.

There are other categories: Andrea, for example, is in a category all of
his own under "absolutely stunning, but sometimes somewhat erratic", which
just means that I have to think a lot more about his patches. I love his
experimentation, especially now that he maintains separate patches (and
I'd also love for him to be more active in pushing the non-experimental
parts towards me, hint hint, Andrea)

And there are categories of people who just own a big enough chunk that is
separate enough that I trust them for that area: architecture maintainers
etc tend to be here, as long as they only affect their own architecture.

But you have to realize that there are a _lot_ of people on the
maintainers list that I don't implicitly trust. And being loud and
wellknown on the mailing lists or IRC channels doesn't make them any more
trusted.

			Linus

From: torvalds@transmeta.com (Linus Torvalds)
Subject: Re: A modest proposal -- We need a patch penguin
Date: 	Tue, 29 Jan 2002 22:22:46 +0000 (UTC)
Newsgroups: fa.linux.kernel

In article <200201292208.g0TM8ql17622@ns.caldera.de>,
Christoph Hellwig  <hch@ns.caldera.de> wrote:
>In article <p73aduwddni.fsf@oldwotan.suse.de> you wrote:
>> "Most times". For example the EA patches have badly failed so far, just because
>> Linus ignored all patches to add sys call numbers for a repeatedly discussed 
>> and stable API and nobody else can add syscall numbers on i386. 
>
>There still seems to be a lot of discussion vs EAs and ACLs.
>Setting the suboptimal XFS APIs in stone doesn't make the discussion
>easier.

In fact, every time I thought that the extended attributes had reached
some kind of consensus, somebody piped up with some apparently major
complaint. 

I think last time it was Al Viro.  Admittedly (_very_ much admittedly),
making Al happy is really really hard.  His perfectionism makes his
patches very easy to accept, but they make it hard for others to try to
make _him_ accept patches.  But since he effectively is the VFS
maintainer whether he wants it to be written down in MAINTAINERS or not,
a comment from him on VFS interfaces makes me jump. 

The last discussion over EA's in my mailbox was early-mid December, and
there were worries from Al and Stephen Tweedie.  I never heard from the
worried people whether their worries were calmed.

Maybe they did, and maybe they didn't.  If somebody doesn't tell me that
they are resolved, and that the people who would actually _use_ and
maintain this interface agrees on it, how can you expect me to ever
apply a patch?

			Linus

Date: 	Tue, 29 Jan 2002 15:50:43 -0800 (PST)
From: Linus Torvalds <torvalds@transmeta.com>
Subject: Re: A modest proposal -- We need a patch penguin
Newsgroups: fa.linux.kernel

On Tue, 29 Jan 2002, Rob Landley wrote:
> > >
> > > Then why not give the subsystem maintainers patch permissions on your
> > > tree. Sort of like committers.  The problem people have is that you're
> > > dropping patches from those ten-twenty people you trust.
> >
> > No. Ask them, and they will (I bet) pretty uniformly tell you that I'm
> > _not_ dropping their patches (although I'm sometimes critical of them,
> > and will tell them that they do not get applied).
>
> Andre Hedrick, Eric Raymond, Rik van Riel, Michael Elizabeth Chastain, Axel
> Boldt...

NONE of those are in the ten-twenty people group.

How many people do you think fits in a small group? Hint. It sure isn't
all 300 on the maintainers list.

> Ah.  So being listed in the maintainers list doesn't mean someone is actually
> a maintainer it makes sense to forward patches to?

Sure it does.

It just doesn't mean that they should send stuff to _me_.

Did you not understand my point about scalability?  I can work with a
limited number of people, and those people can work with _their_ limited
number of people etc etc.

The MAINTAINERS file is _not_ a list of people I work with on a daily
basis. In fact, I don't necessarily even recognize the names of all those
people.

Let's take an example. Let's say that you had a patch for ppp. You'd send
the patch to Paul Mackerras. He, in turn, would send his patches to David
Miller (who knows a hell of a lot better what it's all about than I do).
And he in turn sends them to me.

They are both maintainers. That doesn't mean that I necessarily work with
every maintainer directly.

Or look at USB: I get the USB patches from Greg, and he gets them from
various different people. Johannes Erdfelt is the maintainer for uhci.c,
and he sends them to Greg, not to me.

Why? Because having hundreds of people emailing me _obviously_ doesn't
scale. Never has, never will. It may work over short timeperiods wih lots
of energy, but it obviously isn't a stable setup.

			Linus

Date: 	Tue, 29 Jan 2002 15:59:57 -0800 (PST)
From: Linus Torvalds <torvalds@transmeta.com>
Subject: Re: A modest proposal -- We need a patch penguin
Newsgroups: fa.linux.kernel

On Wed, 30 Jan 2002, Nathan Scott wrote:
>
> Al had several (additional) issues with the original patch, but I
> think we progressively worked through them - Al stopped suggesting
> changes at one point anyway, and the level of abuse died away ;),
> so I guess he became more satisfied with them.

I think you can safely assume that if Al doesn't curse you to hell, he can
be considered happy.

> Not much point apportioning blame - its as much my fault - I
> hadn't heard back from you at all since day 1, so figured you
> were just not interested in this stuff, so I stopped sending.

Basically, you should always consider email to me to be a unreliable
medium, with no explicit congestion control. So think of an email like a
TCP packet, with exponential backoff - except the times are different (in
TCP, the initial timeout is three seconds, and the max timeout is 2
minutes. In "Linus-lossy-network" it makes sense to use different
default and maximum values ;)

		Linus

Date: 	Tue, 29 Jan 2002 16:39:47 -0800 (PST)
From: Linus Torvalds <torvalds@transmeta.com>
Subject: Re: A modest proposal -- We need a patch penguin
Newsgroups: fa.linux.kernel

On Tue, 29 Jan 2002, Rik van Riel wrote:
>
> That's fine with me, but _who_ do I send VM patches to if
> I can't send them to you ?

The VM stuff right now seems to be Andrea, Dave or you yourself (right now
I just wish you would split up your patches like Andrea does, that way I
can cherry-pick).

> There is no maintainer for mm/* or kernel/*, it's just you.

As to kernel/ there are actually maintainers for some sub-areas, the most
noticeable being Ingo on the scheduler. The rest of kernel/ hasn't ever
been much of a problem, really.

The VM is a big issue, of course. And that one isn't likely to go away
anytime soon as a point of contention. And it's not easy to modularize,
apart from the obvious pieces (ie "filemap.c" vs the rest).

You may not believe me when I say so, but I personally _really_ hope your
rmap patches will work out. I may not have believed in your patches in a
2.4.x kind of timeframe, but for 2.6.x I'm more optimistic. As to how to
actually modularize it better to make points of contention smaller, I
don't know how.

At the same time, while I can understand your personal pain, I don't think
most of the problems have been with the VM (maintenance-wise, that is.
Most of the _technical_ problems really have been with the VM, it's just
the most complex piece).

		Linus

Date: 	Tue, 29 Jan 2002 16:44:12 -0800 (PST)
From: Linus Torvalds <torvalds@transmeta.com>
Subject: Re: A modest proposal -- We need a patch penguin
Newsgroups: fa.linux.kernel

On Tue, 29 Jan 2002, Chris Ricker wrote:
>
> That's fine, but there's a major problem with your scheme.  What happens
> with all the stuff for which no one is listed in MAINTAINERS?

I have to admit that personally I've always found the MAINTAINERS file
more of an irritation than anything else. The first place _I_ tend to look
personally is actually in the source files themselves (although that may
be a false statistic - the kind of people I tend to have to look up aren't
the main maintainers at all, but more single driver people etc).

It might not be a bad idea to just make that "mention maintainer at the
top of the file" the common case.

		Linus

Date: 	Wed, 30 Jan 2002 01:21:09 -0800 (PST)
From: Linus Torvalds <torvalds@transmeta.com>
Subject: Re: A modest proposal -- We need a patch penguin
Newsgroups: fa.linux.kernel

On Wed, 30 Jan 2002, Alexander Viro wrote:
>
> 	Frankly, the only real issue in that thread was that we _do_ need
> a tree specifically for small fixes.  Preferably - quickly getting merged
> into the main tree.

A "small stuff" maintainer may indeed be a good idea. The maintainer could
be the same as somebody who does bigger stuff too, but they should be
clearly different things - trivial one-liners that do not add anything
new, only fix obvious stuff (to the point where nobody even needs to think
about it - if I'd start getting any even halfway questionable patches from
the "small stuff" maintainer, it wouldn't work).

		Linus

From: torvalds@transmeta.com (Linus Torvalds)
Subject: Re: A modest proposal -- We need a patch penguin
Date: 	Wed, 30 Jan 2002 09:33:19 +0000 (UTC)
Newsgroups: fa.linux.kernel

In article <20020130034746.K32317@havoc.gtf.org>,
Jeff Garzik  <garzik@havoc.gtf.org> wrote:
>
>Instead of doing this stuff half-assed, just convince Linus to use BK :)

The thing is, I actually _want_ to use BK (as opposed to CVS, which I
really don't think cuts it). 

I still dislike some things (those SHOUTING SCCS files) in bk, and let's
be honest: I've used CVS, but I've never really used BK. Larry has given
me the demos, and I actually decided to re-do the examples, but it takes
time and effort to get used to new tools, and I'm a bit worried that
I'll find other things to hate than just those loud filenames.

This is partly why I asked how many people use it..

		Linus

Newsgroups: fa.linux.kernel
From: Linus Torvalds <torvalds@transmeta.com>
Subject: Re: A modest proposal -- We need a patch penguin
Original-Message-ID: <Pine.LNX.4.33.0201300002170.1542-100000@penguin.transmeta.com>
Date: Wed, 30 Jan 2002 08:23:36 GMT
Message-ID: <fa.o8ppdmv.h4e4bf@ifi.uio.no>

On Wed, 30 Jan 2002, Alexander Viro wrote:
> On Wed, 30 Jan 2002, Daniel Phillips wrote:
> > Linus just called you the ext2 maintainer.
>
> Message-ID, please?

I called you the VFS maintainer ("whether you like it or not" I think I
said. Although I can't find the message right now).

Now, that obviously does imply a certain control over low-level
filesystems, but it really mainly implies a control over the _interfaces_
used to talk the the filesystem, not the filesystem itself.

I personally really wouldn't mind seeing most filesystem patches coming
through Al (and, in fact, in the inode trimming patches that is partly
what as been happening), but I have this nagging suspicion that some
filesystem maintainers would rather eat barbed wire (*).

		Linus

(*) The discussions between Gooch and Al are always "interesting", to name
some names.

Newsgroups: fa.linux.kernel
From: torvalds@transmeta.com (Linus Torvalds)
Subject: Re: [lkml] Re: A modest proposal -- We need a patch penguin
Original-Message-ID: <a3d979$22g$1@penguin.transmeta.com>
Date: Fri, 1 Feb 2002 05:35:05 GMT
Message-ID: <fa.i9ua0gv.cj27gi@ifi.uio.no>

In article <9cfy9iefvbt.fsf@rogue.ncsl.nist.gov>,
Ian Soboroff  <ian.soboroff@nist.gov> wrote:
>
>or, alternatively, "If you want *BSD, you know where to find it."  The
>funny thing is that the BSDs have all this hierarchy and whatnot, and
>they still fight about it.

I think the fact is that the grass is always greener somewhere else, and
all approaches have their problems.

And it's always easier to point out problems with existing setups than
it is to come up with constructive changes. People end up wanting to
re-design everything, because that way they can make sure the problems
are gone - without really even knowing what new problems will appear
after a re-designed process.

The same thing happens in coding too, of course - you just _know_ you
can solve some problem by changing how something is done, and when you
actually code it up, you notice that "yes, I solved the problem", but
you also notice that "but now I have this other thing..".

This is why trial-and-error is such a powerful way of doing things: try
many things, and yes, they all have their problems, but on the whole you
probably end up selecting the approaches where the problems are the
_least_ irritating.

The BIG problem with things like project management is that you simply
_cannot_ do lots of different trial-and-error things.  Sure, you can
try, and you'll get very Dilbertesque results: "The answer to all
problems: re-organize".

Anyway, I'm actually personally willing to make small trials, and right
now I'm trying to see if it makes any difference if I try to use BK for
a month or two. I seriously doubt it will really "fix" everything, but
neither do I think big re-organizations and patch-lists will. But I'd be
stupid if I wasn't willing to try something.

(So far, trying out BK has only meant that I have spent _none_ of my
time merging patches and reading email, and most of my time writing
helper scripts and emails to Larry to make it possible to use BK in sane
ways that suit me. And I'll doubt you'll see any real productivity
increase from me for a while ;)

			Linus

Newsgroups: fa.linux.kernel
From: Linus Torvalds <torvalds@osdl.org>
Subject: Re: AIC7xxx kernel problem with 2.4.2[234] kernels
Original-Message-ID: <Pine.LNX.4.58.0401192259330.2123@home.osdl.org>
Date: Tue, 20 Jan 2004 07:17:03 GMT
Message-ID: <fa.j4ruqdj.1ci2k8v@ifi.uio.no>

On Mon, 19 Jan 2004, Justin T. Gibbs wrote:
>
> Does the maintainer have the ability to veto changes that harm the
> code they maintain?

Nope. Nobody has that right.

Even _I_ don't veto changes that the right people push (my motto:
"everybody is wrong sometimes: when enough people complain, even I am
wrong").

In particular, maintainers of "conceptually higher" generally always have
priority. If Al Viro says a filesystem is doing something wrong from a VFS
standpoint, then that filesystem is broken - regardless of whether the
filesystem maintainer agrees or not. Because the VFS layer requirements
trump any low-level filesystem issues.

But perhaps more importantly (and it's the reason even _I_ don't have the
right, regardless of how high up in the maintainership chain I am), nobody
has veto-power over anything. That's to keep people honest: nobody should
_ever_ think that they are "in control", and that nobody else can replace
them.

In other words: maintainership is not ownership. It's a stewardship.

End result: maintainership is a nasty and mostly unthankful job. It
doesn't really give many privileges, and most of what it does is just have
people complain to you about bugs. The satisfaction is there, of course,
but

And finally: maintainership is largely about working with people.
There's some code in there too, but people tend to be more important.

		Linus

From: Linus Torvalds <torvalds@linux-foundation.org>
Newsgroups: fa.linux.kernel
Subject: Re: [GIT PATCH] final SCSI pieces for the merge window
Date: Wed, 24 Oct 2007 15:36:09 UTC
Message-ID: <fa.3gjjijKHd8R4yAI3s5uSRmrknZY@ifi.uio.no>

On Wed, 24 Oct 2007, James Bottomley wrote:
>
> OK, so it's no secret that I'm the last of the subsystem maintainers
> whose day job isn't working on the linux kernel.  If you want a full
> time person, who did you have in mind?

Quite frankly, at least for me personally, what I would rather have (in
general: this is really not at all SCSI-specific in any way, shape, or
form, and not directed at James!) is a less rigid maintainership
structure.

Let's face it, we are *all* likely to be overworked at different times,
and even when not overworked, it's just the fact that people need to take
a breather etc. And there is seldom - if ever - a very strong argument for
having one person per subsystem.

I think git is excellent for trying to spreading the "joy" of
maintainership, but even without something like that, I think it's much
better to try to find people you can trust, rather than strict
maintainership boundaries. For example, Andrew certainly seems to be very
productive as a kernel maintainer, and it has nothing to do with git, and
everything to do with trust.

So I'd rather have less recriminations about "xyz is holding up abc", and
have people more open to just trying to help out even across strict
borders. And I don't mean that in a "two fighting people" kind of way
where there are two or more people who maintain things _despite_ each
other, but more in a

  "Hey we know each other, and we trust each other, and no, we won't
   guarantee that we always agree, but we can work on things, and if it
   turns out that one person merged something that the other person
   _really_ doesn't like, we'll revert it and/or work it out some other
   way".

I've personally always been against _strict_ maintainer lines, so I've
always taken stuff "past" the maintainer anyway (and sometimes maintainers
have complained, because they feel like they "own" their subsystem, and I
either tell them to stuff it, or say "my bad", depending on whether they
had a valid _technical_ complaint or not).

So rather than getting into a pissing match of "ok, who would be the best
maintainer", I'd much *much* prefer to take this as another "we really
don't need or even _want_ to have strict maintainer rules" opportunity.

Now, the most important part here is the trust part. I need to be able to
trust maintainers, but when you have multiple people in the same "box",
those people need to trust each other even more than usual, because you
*are* going to get disagreements. And the only way it can work is if you
acknowledge that disagreements will happen, and that any situation is not
a "my way or the highway" kind of thing - the trust also implies a certain
give and take.

And this really *is* an issue that cuts across subsystem boundaries.
Anybody who thinks that SCSI is "unique" in this kind of issues is sadly
mistaken. We've had the exact same thing come up in every single subsystem
over time, and at every single level of maintainership (from individual
drivers all the way right up to me) over time.

So I *really* don't want to throw any stones in a glass house here. Quite
the reverse. I'd like to get rid of some of the glass, and replace it with
padding. Because you all know we'd all fit better in a padded room than a
glass house..

Are there any such people that think there s a sufficient mutual trust
with James that you think a blurring of maintainership lines could work
out? Or in any other subsystem, for that matter? Hmm?

			Linus

From: Linus Torvalds <torvalds@linux-foundation.org>
Newsgroups: fa.linux.kernel
Subject: Re: [git pull] x86 updates
Date: Sun, 10 Feb 2008 08:06:17 UTC
Message-ID: <fa.w+YJ/0EkIjGOP50tNu+KHMzDS8Y@ifi.uio.no>

On Sat, 9 Feb 2008, Randy Dunlap wrote:

> On Sun, 10 Feb 2008 00:24:50 +0100 (CET) Thomas Gleixner wrote:
>
> > Linus,
> >
> > please pull the pending x86 updates from:
> >
> >   ssh://master.kernel.org/pub/scm/linux/kernel/git/x86/linux-2.6-x86.git master
>
> Hi Thomas,
> can we please get diffstats with git pull requests?

Well, not only that, but I actually pulled it once, then ended up undoing
my pull, and had to think twice about whether I want to pull it at all.

I do *not* like it when subsystem maintainers start mixing in stuff into
their subsystem pull that has absolutely _zero_ to do with that subsystem.
In this case, we have commit 9b706aee7d92d6ac3002547aea12e3eaa0a750ae,
which calls itself "x86: trivial printk optimizations", but it really has
nothing AT ALL to do with x86 except that it also did the same things to
the bogus x86 version of those routines.

The thing is, trying to mix up things like that into a subsystem pull
means that now I cannot trust the subsystem maintainer as much! And what
should be a simple "git pull" becomes not just the pull, but also me
having to then scrounge around in the result to see if I really want to do
the pull in the first place.

That's the point where the subsystem maintainer just makes me do
unnecessary extra and stupid work!

In other words: DO NOT DO THINGS LIKE THAT!

That vsnprintf() optimization looks ok, but it really doesn't have
anything what-so-ever to do with x86, and some of it is rather dubious and
too clever by half for its own good, for rather little gain. For example,
we have:

	+/* Works only for digits and letters, but small and fast */
	+#define TOLOWER(x) ((x) | 0x20)

but then later on we have:

	-               if (cp[0] == '0' && toupper(cp[1]) == 'X')
	+               if (cp[0] == '0' && TOLOWER(cp[1]) == 'x')

where we now use that new TOLOWER() macro on a character that is *not*
guaranteed to be a digit or a letter at all!

Does it work? Yes, it does happen to work. If it's a non-character or a
non-letter, it will do essentially random things, but it will never turn
that non-character/letter into 'x' unless it was 'X' or 'x' before. So
yes, it works, but let's face it, that clever trick saved something like
two instructions and a branch from a code-path that wasn't really even
critical!

In general, I don't disagree with being clever. People enjoy tricks like
that, and I don't really disagree with the patch. That's not the problem I
have. The problem I have is that I shouldn't be taken by surprise by these
things, and start having to worry whether I can trust the person who sends
me mis-labeled patches without even warning me.

So I had this thing in my tree, went out to dinner, came back and decided
I don't want to pull it after all. Did some other work, and decided I have
the time to look through it, and then re-pulled it.

But I was *this* close to just deciding that "ok, I'm ready to release the
-rc1 kernel, I don't need this kind of aggravation" and just decided to
not bother pulling something dubious.

So Thomas, don't do this. I don't like it. The same way I didn't like
seeing Ingo trying to mix in a kgdb pull into his x86 pull. Keep these
things separate - git is *really* good at having multiple branches with
different lines of development, use it that way (or send odd-ball misc
patches just as emails).

But it's merged now.

			Linus

PS. And no, maybe I don't always notice. I bet maintainers can slip things
by me all the time without me waking up to it. The fact that it sometimes
works doesn't mean that it's a good idea, though - it just means that if I
catch it, my level of trust goes down.

From: Linus Torvalds <torvalds@linux-foundation.org>
Newsgroups: fa.linux.kernel
Subject: Re: Linux 2.6.25-rc8
Date: Wed, 02 Apr 2008 03:04:47 UTC
Message-ID: <fa.VdBbzr8SLKHLaHbHHRutrWsS2jk@ifi.uio.no>

On Wed, 2 Apr 2008, Stephen Rothwell wrote:
>
> The downside is that at least some of these were already pending in
> maintainers trees for 2.6.26 (among other changes) and so have now caused
> (unnecessary) merge conflicts.  Is there some reason that these fixes
> can't go through the subsystem trees (especially once we get past rc1 (or
> 2) and people are gearing up for the next merge window)?

I don't think it's necessarily a bad idea to go through subsystem trees,
but on the other hand I also don't think it should necessarily be a goal
in itself.

For example, we had a patch-series from Roland McGrath that was apparently
almost entirely based on the fact that going through (and getting
sign-off) from all the architecture maintainers for his ptrace changes was
just painful as hell for him.

At that point, when there is somebody like Roland who knows the rare and
odd ptrace interfaces, having him jump through hoops just to go through
"proper channels" is in my opinion just anti-productive (especially since
I also think the "political" aspect of the problem causes the actual
technical side of the patches to suffer - because they are more about
the politics than about the technology).

So at some point, subsystem maintenance should also be about picking up
and handling the changes that come the other way. The kernel development
isn't a strict hierarchy, and shouldn't be - it's more of a network of
trust.

In other words, there are people I think are generally trusted across most
maintenance borders. Al, as far as I'm concerned, is one of them.
Especially since he is also one of the few people who clearly not only
does run sparse but also looks at the code and actually fixes real bugs
with byte order etc - regardless of where it is (ie he works across
drivers, filesystems, an arch-specific code)

In other words, I don't think the borders are so tightly drawn, and the
same way I trust the individual developers who send me patches (and git
trees) rather than whatever _companies_ they happen to work, I also tend
to trust individual developers rather than the _subsystem_ that they
happen to maintain.

Of course, there's often a rather direct mapping between the two, where
people naturally have the area they work in. But some people cross across
any particular area, and while that tends to be unusual, that very much
includes people like Andrew and Al.

In other words, at least to me it's not about "person X maintains file Y".
It's much more about "I trust person X (perhaps within parameters Z)". And
I don't think that's even unusual or even really unexpected.

And I think that's how we all work (and how we _should_ work), but
sometimes people get so used to the fact that some people are fairly
tightly associated with certain code that they think it's about the
subsystem, not about the person.

			Linus

From: Linus Torvalds <torvalds@linux-foundation.org>
Newsgroups: fa.linux.kernel
Subject: Re: Linux 2.6.25-rc8
Date: Wed, 02 Apr 2008 03:09:52 UTC
Message-ID: <fa.27sfcPZmufACBhRb5lvElTHBsE4@ifi.uio.no>

On Tue, 1 Apr 2008, Linus Torvalds wrote:
>
> And I think that's how we all work (and how we _should_ work), but
> sometimes people get so used to the fact that some people are fairly
> tightly associated with certain code that they think it's about the
> subsystem, not about the person.

Btw - don't get me wrong - we clearly do try to (and should) make merges
easy, and it's why the kernel source code is generally pretty modular and
we try to keep things as independent as possible. I'm not at all arguing
against that.

I'm just trying to explain that at least personally, I just don't see that
"modularity" and maintainership as a _primary_ issue. The primary issue is
just the interpersonal trust people build up over time. The modularity and
trying to keep borders is about practical concerns, and it's important
too. But it is still secondary, I think.

			Linus