Newsgroups: fa.linux.kernel
From: Linus Torvalds <torvalds@osdl.org>
Subject: Re: 2.6.6-rc3: modular DVB tda1004x broken
Original-Message-ID: <Pine.LNX.4.58.0405011722550.18014@ppc970.osdl.org>
Date: Sun, 2 May 2004 00:30:04 GMT
Message-ID: <fa.ito3l7t.16kua3n@ifi.uio.no>

On Sat, 1 May 2004, Chris Wedgwood wrote:
>
> I'm confused.

We all are, don't worry.

> I thought it has been decreed using kernel headers in userspace was a
> bad idea (DONT DO THAT) so in theory we can just ignore this issue?

Absolutely. But "in theory" is a thing we may want to strive for, but not
at the expense of "in practice".

Sadly, we used to encourage (yeah, yeah, I should 'fess up: it was me, I'm
guilty, I was stupid) user-space code to include kernel headers.
Admittedly, that was about ten years ago, and we've tried to fix it ever
since, but the thing is, I think backwards compatibility in the end is
more important than "in theory". And silently breaking things in subtle
ways would be bad.

What we _could_ do is to move _all_ the "_syscallX()" stuff into the
__KERNEL__ define, which would at least break any potential pre-glibc
users in a very visible way. What I _don't_ want to do is to have somebody
by mistake compile against updated kernel headers, and it still compiles,
but just doesn't work at run-time the way it's supposed to. THAT is
confusing and bad.

In short: either we should make non-kernel users break in _really_ obvious
(and hopefully easy-to-fix) ways, or we should keep things compatible.
None of the "oh, the return value changed subtly" thing, please ;)

		Linus

Newsgroups: fa.linux.kernel
From: Linus Torvalds <torvalds@osdl.org>
Subject: Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
Original-Message-ID: <Pine.LNX.4.58.0411281710490.22796@ppc970.osdl.org>
Date: Mon, 29 Nov 2004 01:30:46 GMT
Message-ID: <fa.iv7nn8a.164qbjk@ifi.uio.no>

On Mon, 29 Nov 2004, Paul Mackerras wrote:
>
> uint32_t is defined to be exactly 32 bits wide, so where's the problem
> in using it instead of __u32 in the headers that describe the
> user/kernel interface?  (Ditto for uint{8,16,64}_t, of course.

Ok, this discussion has gone on for too long anyway, but let's make it
easier for everybody. The kernel uses u8/u16/u32 because:

	- the kernel should not depend on, or pollute user-space naming.
	  YOU MUST NOT USE "uint32_t" when that may not be defined, and
	  user-space rules for when it is defined are arcane and totally
	  arbitrary.

	- since the kernel cannot use those types for anything that is
	  visible to user space anyway, there has to be alternate names.
	  The tradition is to prepend two underscores, so the kernel would
	  have to use "__uint32_t" etc for its header files.

	- at that point, there's no longer any valid argument that it's a
	  "standard type" (it ain't), and I personally find it a lot more
	  readable to just use the types that the kernel has always used:
	  __u8/__u16/__u32. For stuff that is only used for the kernel,
	  the shorter "u8/u16/u32" versions may be used.

In short: having the kernel use the same names as user space is ACTIVELY
BAD, exactly because those names have standards-defined visibility, which
means that the kernel _cannot_ use them in all places anyway. So don't
even _try_.

On the bigger question of what to do with kernel headers in general, let's
just make one thing clear: the kernel headers are for the kernel, and big
and painful re-organizations that don't help _existing_ user space are not
going to happen.

In particular, any re-organization that breaks _existing_ uses is totally
pointless. If you break existing uses, you might as well _not_ re-
organize, since if you consider kernel headers to be purely kernel-
internal (like they should be, but hey, reality trumps any wishes we might
have), then the current organization is perfectly fine.

So I think this whole discussion has been largely pointless. We undeniably
have existing users of kernel headers. That's just a fact.  If we break
them, it doesn't _matter_ how the kernel headers look, and then "existing
practice" is about as good an organization as anything, and breaking
things just to break things is not something I'm in the least interested
in. "Beauty"  comes secondary to "usefulness".

So I would encourage people to think about ways to clean up some of the
worst warts, but take into account:
 - testing it out with whatever random collection of old distributions and
   special applications is almost impossible. So every single step of the
   way should be (a) small and (b) obviously not break any current users.
 - cleanup just for the sake of cleanup always needs to take pain into
   account. If you cannot make each small step "worth it", then just don't
   do it. If the "cleanup" just adds another file and doesn't actually
   _help_ anything that you can point to, it's not a cleanup, it's just an
   exercise in wanking.

IOW, I seriously doubt any "let's reorganize header files just to make it
look good" _ever_ accomplished anything. But if there are _specific_
header files that have _specific_ problems, let's look at maybe solving
those. And if you cannot point to a specific problem with a suggested
specific solution, please don't cc me.

		Linus

Newsgroups: fa.linux.kernel
From: Al Viro <viro@parcelfarce.linux.theplanet.co.uk>
Subject: Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
Original-Message-ID: <20041129045705.GM26051@parcelfarce.linux.theplanet.co.uk>
Date: Mon, 29 Nov 2004 04:58:48 GMT
Message-ID: <fa.n6clcqr.1nh2pbf@ifi.uio.no>

On Sun, Nov 28, 2004 at 11:36:22PM -0500, Jeff Garzik wrote:
> If people want to go beyond that, IMHO it would be simple and easy to
> start putting new kernel headers in include/kernel (or somesuch).  That
> way there are no massive reorganizations; kernel-specific stuff gets
> slowly migrated to a kernel-specific area.

ITYM "to areas where it actually gets used".  A _lot_ in include/* is
used only by a couple of drivers and should've been sitting in
drivers/*/* instead.

Newsgroups: fa.linux.kernel
From: Linus Torvalds <torvalds@osdl.org>
Subject: Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
Original-Message-ID: <Pine.LNX.4.58.0411300751570.22796@ppc970.osdl.org>
Date: Tue, 30 Nov 2004 15:59:22 GMT
Message-ID: <fa.iv7jog3.1646art@ifi.uio.no>

On Tue, 30 Nov 2004, David Woodhouse wrote:
>
> On Mon, 2004-11-29 at 09:41 -0800, Linus Torvalds wrote:
> > In fact, in many ways I'd prefer to have source-level annotations like
> > "this is exported to user space" over trying to gather things in one
> > place.
>
> I don't think you would; not once you really tried it.
>
> That's what the littering of ifdef __KERNEL__ attempts to do, and
> there's not really any better way of doing it than that.

The fact is, despite this stupid and way-too-long thread, #ifdef
__KERNEL__ has worked for over a decade, and works damn well, everything
considered.

Remember the second-system-syndrome? It comes from people wanting to fix
problems in the current implementation, without thinking about all the
things that it does _well_ (because the things that work are not on their
radar - they just work). And no, __KERNEL__ may not be pretty, but it's
damn simple to fix up, and one thing it does really well is allow
flexibility in a way that forcing structure does not.

I know people like "structure". But it's _waayyy_ overrated. The reason we
use C instead of some other programming environment is not that C is the
most highly structured language around, but it's the most _flexible_ one,
largely because it says "let's give people rope". It allows structure, but
if you want to cast things around and use gotos, it says "sure, master,
you're the boss".

Same thing here. The __KERNEL__ approach says "whatever you want, boss".
It doesn't get in the way. Maybe it doesn't actively _help_ you either,
but you never have to fight any structure it imposes on you.

Also, there _are_ better ways of annotating it than __KERNEL__. In
particular, if we had a "user annotation", I could make sparse sit up and
take notice, and _complain_ when you use a non-specific-sized type. That's
not just theory - Al Viro was talking about that at some point.

			Linus

Newsgroups: fa.linux.kernel
From: Linus Torvalds <torvalds@osdl.org>
Subject: Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
Original-Message-ID: <Pine.LNX.4.58.0411300846190.22796@ppc970.osdl.org>
Date: Tue, 30 Nov 2004 17:06:54 GMT
Message-ID: <fa.it7po88.1444ajq@ifi.uio.no>

On Tue, 30 Nov 2004, David Woodhouse wrote:
>
> > Same thing here. The __KERNEL__ approach says "whatever you want, boss".
> > It doesn't get in the way. Maybe it doesn't actively _help_ you either,
> > but you never have to fight any structure it imposes on you.
>
> Having to think before adding something that's user visible is a
> _benefit_ not a disadvantage.

I've said this at least three times: if you can point to a _specific_
thing you want to move, go wild. I think the big waste in this discussion
has been that there have _not_ been specific suggestions, just total
sound-bites like "wouldn't it be great to move things to 'include/kapi'".

If you have a specific thing in mind, say instead something like

	"Wouldn't it be great if we moved all the tty layer IOCTL numbers
	 into 'tty-ioctl-nr.h', and made the old header file just include
	 that header file, so that new libc users can get them from just
	 that header? And btw, here's the patch."

then I might listen. Notice how the only really constructive thing to come
out of this flame-fest has been a patch by Al that looked perfectly
reasonable, but that got totally drowned out by the arguing?

Note that even _if_ you have a specific thing in mind, I want to see that
somebody would say "yes, we'd use that organization". I would not be
surprised at all if glibc people said that they can't really use any
re-organization anyway, since they need to support old kernel setups too.

See? Changes that aren't specific enough, or don't actually help things is
what I'm against.

		Linus

Newsgroups: fa.linux.kernel
From: Linus Torvalds <torvalds@osdl.org>
Subject: Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
Original-Message-ID: <Pine.LNX.4.58.0411301020160.22796@ppc970.osdl.org>
Date: Tue, 30 Nov 2004 18:25:06 GMT
Message-ID: <fa.itn3no2.14kab3s@ifi.uio.no>

On Tue, 30 Nov 2004, David Woodhouse wrote:
>
> The idea in the proposal which David posted, which seemed perfectly
> specific enough to me, was to move all the user-visible parts to
> separate header files in a separate directory.

You call _that_ specific?

Hell no. You need to do it without breaking existing uses, as noted
earlier, and it's not specific at all. "all user visible parts" is a big
undertaking, if you can't make it smaller than that, then forget about it.

Basic rule in kernel engineering: you don't just rewrite the world. You do
it in incremental independent steps.

Any mtd-specific rewrite is obviously a go.

		Linus

Newsgroups: fa.linux.kernel
From: hpa@zytor.com (H. Peter Anvin)
Subject:  Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
Original-Message-ID:  <cojm42$sf1$1@terminus.zytor.com>
Date: Wed, 1 Dec 2004 05:52:00 GMT
Message-ID: <fa.hhlslgm.rlslho@ifi.uio.no>

Followup to:  <Pine.LNX.4.58.0411301243000.22796@ppc970.osdl.org>
By author:    Linus Torvalds <torvalds@osdl.org>
In newsgroup: linux.dev.kernel
>
> But claiming that __KERNEL__ doesn't work is clearly a bunch of crapola.
> As is the notion that you can somehow do it all. We do it in small pieces.
>

IMNSHO, based on my experience with klibc, the problem is not #ifdef
__KERNEL__, the *specific* problems are:

- The kernel exporting libc4/5 internals, which are neither what the
  kernel nor any kind of modern userspace wants.  This should be
  possible to get rid of easily enough.

- The kernel exporting things into the userspace namespace.  The
  kernel's definition of "struct stat" isn't usable -- the one that is
  useful is called "struct stat64" on most, but not all,
  architectures, but because it's a structure tag it can't be pulled
  from the exported kernel ABI in such a way that what userspace calls
  "struct stat" is what the kernel currently calls "struct stat64" if
  that exists.

  This one is a bit uglier, since it probably needs something like:

  #ifdef __KERNEL__
  #define __kabi_stat64 stat64
  #endif

  struct __kabi_stat64 { ... };

	-hpa

Newsgroups: fa.linux.kernel
From: Al Viro <viro@parcelfarce.linux.theplanet.co.uk>
Subject: Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
Original-Message-ID: <20041130230325.GY26051@parcelfarce.linux.theplanet.co.uk>
Date: Tue, 30 Nov 2004 23:14:21 GMT
Message-ID: <fa.necjbqe.1thcob2@ifi.uio.no>

On Tue, Nov 30, 2004 at 11:44:21PM +0100, Mariusz Mazur wrote:
> On wtorek 30 listopad 2004 23:33, Sam Ravnborg wrote:
> > On Tue, Nov 30, 2004 at 12:47:50PM -0800, Linus Torvalds wrote:
> >  If that's all that people want, I hereby proclaim that
> >
> > >  include/asm-xxx/user/xxxx.h
> > >  include/user/xxxx.h
> > >
> > > is reserved for user-visible stuff. And now you can send me small and
> > > localized patches that fix a _particular_ gripe.
> >
> > Please use:
> >   include/$arch/user-asm/xxxx.h
> >   include/user/xxxx.h
>
> Wrong. These dirs must be linked to /usr/include so they must have more
> meaningfull names.

WTF?  I've got a dozen kernel trees hanging around, which one (and WTF any,
while we are at it) should be "linked to"?

Newsgroups: fa.linux.kernel
From: Linus Torvalds <torvalds@osdl.org>
Subject: Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
Original-Message-ID: <Pine.LNX.4.58.0411301452180.22796@ppc970.osdl.org>
Date: Tue, 30 Nov 2004 23:02:07 GMT
Message-ID: <fa.itnfog4.14kuaru@ifi.uio.no>

On Tue, 30 Nov 2004, Matt Mackall wrote:
>
> So we follow dhowell's plan with the following additions:

No.

We do _not_ move stuff over that is questionable.

I thought that was clear by now. The rules are:
 - we only move things that _have_ to move
 - we don't break existing programs, and no "but they are broken already"
   is not an excuse.
 - we only move things where that _particular_ move can be shown to be
   beneficial.

No whole-sale moves. No "let's break things that I think are broken". No
"let's change things because we can".

Well-defined moves. Both in content _and_ in reason.

		Linus

Newsgroups: fa.linux.kernel
From: Al Viro <viro@parcelfarce.linux.theplanet.co.uk>
Subject: Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
Original-Message-ID: <20041130175518.GW26051@parcelfarce.linux.theplanet.co.uk>
Date: Tue, 30 Nov 2004 18:14:45 GMT
Message-ID: <fa.ncd1d2g.1qhup34@ifi.uio.no>

On Tue, Nov 30, 2004 at 08:53:34AM -0800, Linus Torvalds wrote:
> then I might listen. Notice how the only really constructive thing to come
> out of this flame-fest has been a patch by Al that looked perfectly
> reasonable, but that got totally drowned out by the arguing?

Frankly, I suspect that amount of effort that went into postings made in
that thread is approaching what it would take to clean the damn thing
up for good.  I wasn't kidding about 10 minutes work to do that patch...

Newsgroups: fa.linux.kernel
From: Linus Torvalds <torvalds@osdl.org>
Original-Message-ID: <Pine.LNX.4.58.0411300744120.22796@ppc970.osdl.org>
Subject: Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
Date: Tue, 30 Nov 2004 15:53:58 GMT
Message-ID: <fa.it79o86.144cajo@ifi.uio.no>

On Tue, 30 Nov 2004, David Howells wrote:
>
> > But in general, I do kind of like the explicit marking. The same way we
> > explicitly mark the functions inside the kernel that we expose to modules,
> > we could try to mark the data structures and values that we expose to user
> > space. That tends to "work".
>
> That gets trickier when it comes to #defines, I think.
>
> Do you really object that much to splitting the header files as I proposed?

I object sternuously to your "the header files". If you can't even say
_which_ header file, I'm not interested.

If you can say "these X header files have this specific problem Y, and if
we move part Z into a common area A, we'd solve it because B", that's a
different matter.

See what I'm saying? Whole-sale "move things around because we want to"
I'm not interested in. Specific problems, yes.

> Personally, I'd prefer us to move to using standard C99 types in lieu of u32
> and co at least for the interface to userspace because they are just that:
> standard.

No. I told you why it cannot and MUST NOT be done. Repeat after me:

	WE MUST NOT POLLUTE THE NAMESPACE!

There are many cases where including a header file means you want the
structure, but it does _not_ mean that you can expose totally unrelated
types, even if those types happen to be used as part of the structure. See
what I'm saying?

In contrast, __u8/__u16/__xxx are explicitly defined by the relevant
standards to be "system name space", and that is _exactly_ why we don't
use the standard namespaces.

And the fact is, the rules for _when_ you export _which_ standard types
are just too damn hard, and more importantly, they are not something that
the kernel headers should even be deciding. They depend on glibc internal
things like __USE_BSD etc, and they depend on which standard you compile
for, and sometimes even on which _version_ of the standard you compile
for.

I will not have that mess in the kernel. Which means that we do _not_
export any of the standard names. Which means that we can't _use_ any of
them. End of story.

		Linus

Newsgroups: fa.linux.kernel
From: Linus Torvalds <torvalds@osdl.org>
Subject: Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
Original-Message-ID: <Pine.LNX.4.58.0411301447570.22796@ppc970.osdl.org>
Date: Tue, 30 Nov 2004 22:58:02 GMT
Message-ID: <fa.ivndo89.16k0ajr@ifi.uio.no>

On Tue, 30 Nov 2004, David Woodhouse wrote:
>
> That depends on your definition of 'break'. It should prevent abuse.

Not really.

It should prevent _future_ abuse.

The notion of "preventing existing xxx" is insane. You can't "prevent"
something that already happened unless you've come up with some new
interesting theory of causality.

> To pick a specific example, since you like them: where userland programs
> are including atomic.h, and hence writing programs which don't compile
> on some architectures, and which compile on others but silently give
> non-atomic results, it's perfectly acceptable and indeed advisable to
> prevent compilation across the board.
>
> Some people might call that breakage; I don't.

I do. The thing is, the people who _notice_ the breakage are often the
people who don't know what the hell to do about it.

The way to prevent _future_ abuse is by adding something like

	#ifndef __KERNEL__
	#warning "This really doesn't work"
	#endif

which does that, and has the advantage of not breaking anything at all.

In other words: if you want to move things around just to break things,
THEN THAT IS INCREDIBLY STUPID. We don't do things to screw our users
over.

Feel free to send a patch.

		Linus

Newsgroups: fa.linux.kernel
From: Linus Torvalds <torvalds@osdl.org>
Subject: Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
Original-Message-ID: <Pine.LNX.4.58.0411301605500.22796@ppc970.osdl.org>
Date: Wed, 1 Dec 2004 00:12:30 GMT
Message-ID: <fa.ivn3n87.16kabjp@ifi.uio.no>

On Tue, 30 Nov 2004, David Woodhouse wrote:
>
> Linus, you're arguing that it's better to let users use something which
> is non-portable and silently does the wrong thing, as long as it
> actually compiles. That this is preferable to making sure it doesn't
> compile.

I'm saying that if it worked before, it should work after. And my
suggestion gives a nice warning.

> But atomic.h isn't an example of that.

Even atomic.h. I could well imagine that somebody includes atomic.h just
to get the thread-safe updates for some architectures. For example,
asm-alpha/atomic.h does it right, and I woul dnot be at all surprised if
somebody had noticed.

And your suggestion has the problem that the people who get bitten by a
non-compiling thing are not necessarily the same people who can fix it.

> software'? Of course not; you have to draw the line somewhere. And I
> would draw it somewhere between atomic.h and byteorder.h -- where would
> _you_ draw it?

I'll draw it at "somebody might validly use it", because the fact is, I
can't test it.

Which is why I want patches to this to be OBVIOUSLY CORRECT, dammit! How
hard is that to understand?

The fact is, the less benefit there is from a change, the more obviously
correct it has to be in all forms. Moving stuff around in header files
basically fixes zero bugs in the kernel, so the benefit for the kernel is
basically zero. Which means that the obviousness factor of the change had
better be pretty close to infinite.

Comprende?

		Linus

Newsgroups: fa.linux.kernel
From: Linus Torvalds <torvalds@osdl.org>
Subject: Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
Original-Message-ID: <Pine.LNX.4.58.0411301612370.22796@ppc970.osdl.org>
Date: Wed, 1 Dec 2004 00:56:43 GMT
Message-ID: <fa.iunhng4.15k4bru@ifi.uio.no>

On Tue, 30 Nov 2004, Linus Torvalds wrote:
>
> Even atomic.h. I could well imagine that somebody includes atomic.h just
> to get the thread-safe updates for some architectures. For example,
> asm-alpha/atomic.h does it right, and I would not be at all surprised if
> somebody had noticed.

In fact, this is not entirely theoretical. I know people _have_ noticed,
because I've gotten queries from some projects that wanted to copy the
definitions for their alpha port. I only got those queries because my name
is on the file, and those people wanted to actually copy them, not just
include the header file. I don't know _how_ many people decided to just
do the #include.

There are probably more people familiar with the kernel source tree than
with GLIB which is probably the preferred way to do those things these
days. And a few years ago that choice wasn't even there.

		Linus

Newsgroups: fa.linux.kernel
From: Linus Torvalds <torvalds@osdl.org>
Subject: Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
Original-Message-ID: <Pine.LNX.4.58.0411301503470.22796@ppc970.osdl.org>
Date: Tue, 30 Nov 2004 23:26:27 GMT
Message-ID: <fa.iv7fn85.1642bjv@ifi.uio.no>

On Tue, 30 Nov 2004, Mariusz Mazur wrote:
>
> Userland? And who's using vanilla kernel headers in userland? Duplicates
> maintainers can always fix stuff independent of kernel (and I do).

Dammit, how hard is it for people to admit that they aren't the only ones
around?

There are still people who use ancient setups, and upgrade the kernel.
They won't complain to _you_ or to redhat when their setup breaks, because
they aren't _using_ your setup. They complain to me.

This is what statisticians call "self-selection". It means that you guys
only see the part of the world that uses your stuff, so you think that
there is nothing else.

> I don't get your point.

Very obviously.

		Linus

Newsgroups: fa.linux.kernel
From: Linus Torvalds <torvalds@osdl.org>
Subject: Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
Original-Message-ID: <Pine.LNX.4.58.0411292019500.22796@ppc970.osdl.org>
Date: Tue, 30 Nov 2004 04:24:23 GMT
Message-ID: <fa.j06nn8k.1746bjs@ifi.uio.no>

On Mon, 29 Nov 2004, Jean Tourrilhes wrote:
>
> 	So, which kernel ABI should be present on my system in
> /usr/include/linux and /usr/include/asm ? Should I use the ABI from
> 2.6.X, 2.4.X or 2.2.X ?

I have always felt (pretty strongly) that the /usr/include/xxx contents
should not be kernel-dependent, but be linked to your glibc version.
That's why the symlink from /usr/include/xxx to /usr/src/linux/include/
has been deprecated for the last, oh about ten years now..

Yes, there are some _very_ specific things which might care about system
calls or ioctl's that have been added later, but let's face it, we don't
actually do that very often. The kernel may change at a rapid pace, but
user interfaces don't, and user interfaces that would bypass the C library
change even less frequently.

		Linus

Newsgroups: fa.linux.kernel
From: Linus Torvalds <torvalds@osdl.org>
Subject: Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
Original-Message-ID: <Pine.LNX.4.58.0412140734340.3279@ppc970.osdl.org>
Date: Tue, 14 Dec 2004 15:50:42 GMT
Message-ID: <fa.gudtvmd.b2gr1v@ifi.uio.no>

On Tue, 14 Dec 2004, Werner Almesberger wrote:
>
> Hmm, so you're predicting problems if user space includes something
> that uses uint32_t ? Wouldn't either of these work (descriptive file
> names for clarity):
>
>  1) include/linux-user/foo.h:
>     #include <stdint.h>
>     #include <linux-kernel-and-user-common/foo.h>

No. Because when you include <sys/ioctl.h> (which includes the
<linux-user/foo.h>, the POSIX/SuS/whatever namespace rules say that YOU
MUST NOT pollute the namespace with the names from stdint.h.

The user is supposed to see "int32_t" and friends _only_ if the user
himself includes <stdint.h> or one of the very specific headers that is
documented by the standard to include it.

Trust me. We are NOT going to use <stdint.h> in the kernel.

This is a common issue with namespace pollution. For example, this program
is perfectly valid afaik (well, except for being _stupid_, but that's
another issue):

	#include <stdio.h>

	const char *int32_t(int i)
	{
		return i ? "non-zero" : "zero";
	}

	int main(int argc, char **argv)
	{
		return printf("argc is %s\n", int32_t(argc));
	}

Get the picture? The user may _use_ names that are defined for the
standard - like "strlen()" or "uint32_t" - for his own nefarious purposes,
because they are _only_ defined if you include the right header files.

And trust me, the rules are really arcane. Not only do you have several
standards, and several versions, you have various local rules too, ie gcc
and glibc make up their own rules about things that depend on compiler
flags etc.

And that is why a kernel header file must not define them, and by
implication, cannot use these names.

Because the whole _point_ of having kernel header files that can be used
in user-space is that they are helpers for things like <sys/ioctl.h> etc,
that would just copy them.

If we are going to pollute the user-visible namespace in random ways, the
whole discussion is pointless. At that point, glibc (or any other libc)
can't use those API headers anyway.

Remember: the _biggest_ reason to make kernel headers available is not to
user programs that want them, but to libc and friends.

		Linus

Newsgroups: fa.linux.kernel
From: Linus Torvalds <torvalds@osdl.org>
Subject: Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
Original-Message-ID: <Pine.LNX.4.58.0412140950520.3279@ppc970.osdl.org>
Date: Tue, 14 Dec 2004 18:17:20 GMT
Message-ID: <fa.gvdu069.820qhr@ifi.uio.no>

On Tue, 14 Dec 2004, Werner Almesberger wrote:
>
> Would you agree that the use of stdint.h types is acceptable for
> new interfaces, provided that these come with the warning that
> their use may pull in all of stdint.h ?

Not really.

Two reasons:
 - The fact is, things are interdependent. And this thing is all about
   _avoiding_ implicit dependencies, and rules like "if you include _this_
   particular header file, then you must include yyy.h first, or you can't
   do zzzz".
 - uint32_t and friends aren't all that nice anyway. Quite frankly, there
   is nada, zilch, _zero_, reason to use them over the __u8/__u16/__u32.
   In contrast, the __u8/__u16/__u32 types have obvious advantages,
   ranging from the _lack_ of namespace collission issue to just plain
   consistency.

iow, using "uint32_t" and friends is _stupid_. The only fathomable
argument for doing so would be consistency, but since BY DEFINITION we
cannot be consistent if we use them (since we cannot and must not use them
anywhere), that _single_ reason for using them is actually the biggest
reason to NOT use them.

See my point?

> If this is as unpredictable as you describe it, it would mean
> that also new interfaces which need to specify an exact integer
> size would require new sets of type names.

No. They require an OLD set of type names, that have existed for years in
the kernel, and that are already used widely.

In other words, they require __u8/__u16/__u32/__u64. That _is_ the way to
specify a sized type in kernel headers. It has been that way for years,
it's extremely readable, it's consistent, and it's compact.

In other words, u8/u16/u32/u64 (and the signed versions: s8 and friends)
_are_ the standard names in the kernel, and the __ versions have always
existed alongside them for things like header files that have namespace
issues.

So forget about that stupid abortion called "uint32_t" already. It buys
you absolutely nothing.

			Linus

Newsgroups: fa.linux.kernel
From: Linus Torvalds <torvalds@osdl.org>
Subject: Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
Original-Message-ID: <Pine.LNX.4.58.0412141351570.3279@ppc970.osdl.org>
Date: Tue, 14 Dec 2004 22:10:47 GMT
Message-ID: <fa.gvts06a.8iuqhq@ifi.uio.no>

On Tue, 14 Dec 2004, Werner Almesberger wrote:
> > So forget about that stupid abortion called "uint32_t" already. It buys
> > you absolutely nothing.
>
> Now, what do we do with them when they are inside the kernel,
> far from any interfaces ? Live and let live ?

Yes. As long as they don't cause problems, I'm a big fan of "let
developers make their own choices". I think that uint32_t and friends are
totally useless in the kernel, but while I have strong opinions, I try to
not force those opinions on others too much.

Quite frankly, it's unlikely to cause problems in practice. Having strong
opinions and enjoying the occasional flame war is one thing. Trying to
rail against other people who don't share those opinions and force them to
change their code is a totally different thing.

[ It has absolutely nothing to do with me being lazy. Oh, no. I'm not
  lazy. Much better to attribute it to some lofty goal like "live and let
  live". If the two just _happen_ to co-incide, all the better ;-]

			Linus

Newsgroups: fa.linux.kernel
From: Linus Torvalds <torvalds@osdl.org>
Subject: Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
Original-Message-ID: <Pine.LNX.4.58.0412141150460.3279@ppc970.osdl.org>
Date: Tue, 14 Dec 2004 20:00:39 GMT
Message-ID: <fa.gvdm069.82oqhr@ifi.uio.no>

On Tue, 14 Dec 2004, Sam Ravnborg wrote:
>
> Today we only pull in stdarg.h from userspace - actually a compiler
> dependent file.

Indeed. You'll notice that gcc doesn't even put stdarg.h in /usr/include,
it's in the compiler-specific header file directory, usually something
like /usr/lib/gcc-lib/<vendor>/<version>/include.

We used to compile with "-nostdinc" to make sure that you couldn't include
user files even by mistake, but that was removed for some reason I can't
for the life of me remember any more.

We probably should do it again. Something vaguely like

	CFLAGS += -nostdinc -I $(CC -print-file-name=include)

should do it (and still pick up "stdarg.h").

		Linus

Newsgroups: fa.linux.kernel
From: Linus Torvalds <torvalds@osdl.org>
Subject: Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
Original-Message-ID: <Pine.LNX.4.58.0412141232480.3279@ppc970.osdl.org>
Date: Tue, 14 Dec 2004 20:35:18 GMT
Message-ID: <fa.gvdrvmb.822r1p@ifi.uio.no>

On Tue, 14 Dec 2004, Andreas Schwab wrote:
>
> ??? We still do that, see NOSTDINC_FLAGS.

Ahh, yup, there is it. Goodie, I was just confused.

		Linus

Newsgroups: fa.linux.kernel
From: Linus Torvalds <torvalds@osdl.org>
Subject: Re: [RFC] Splitting kernel headers and deprecating __KERNEL__
Original-Message-ID: <Pine.LNX.4.58.0412140958320.3279@ppc970.osdl.org>
Date: Tue, 14 Dec 2004 18:08:30 GMT
Message-ID: <fa.gudu06h.b20qhj@ifi.uio.no>

On Tue, 14 Dec 2004, Andreas Schwab wrote:
>
> Linus Torvalds <torvalds@osdl.org> writes:
>
> > This is a common issue with namespace pollution. For example, this program
> > is perfectly valid afaik (well, except for being _stupid_, but that's
> > another issue):
> >
> > 	#include <stdio.h>
> >
> > 	const char *int32_t(int i)
> > 	{
> > 		return i ? "non-zero" : "zero";
> > 	}
>
> Actually this is not allowed in POSIX.  _Any_ header may define any
> identifier ending with "_t".

... and that's POSIX.

It's not true for BSD_SOURCE, for example.

WHICH IS MY POINT, DAMMIT!

How hard is it to understand? There are millions of different standards,
many of them without any standards body at all, and just plain standards
by virtue of "that's how people have done it for years".

uint32_t is totally useless for the kernel. We should never ever use it.

		Linus

From: Linus Torvalds <torvalds@osdl.org>
Newsgroups: fa.linux.kernel
Subject: Re: Simple header cleanups
Date: Thu, 27 Apr 2006 03:00:45 UTC
Message-ID: <fa.RAJrFKW4FNJDs1F1h+GDiqJzm7Q@ifi.uio.no>
Original-Message-ID: <Pine.LNX.4.64.0604261954480.3701@g5.osdl.org>

On Thu, 27 Apr 2006, David Woodhouse wrote:
>
> Well, yes, but we all know that people _have_ to include kernel headers.
> We can't just bury our head in the sand and say "they mustn't do that".
> The kernel headers contain all the juicy stuff like structure
> definitions and ioctls which you _need_ in order to communicate with the
> kernel.

.. and that's generally the job of library maintainers.

I disagree violently with the notion that any normal system should _ever_
have the regular headers have anything to do with "what kernel source is
installed right now". It is untenable to expect kernel developers to have
to worry about user-level header problems.

> The problem is that we don't actually have any _discipline_ about how we
> throw our kernel headers over the wall. We never even _think_ about how
> usable they are in userspace, or how what we're doing will affect
> userspace.

AND WE SHOULD NOT HAVE TO!

Linkages are bad. We _will_ break kernel headers for user space, and
that's just a undeniable fact.

If this is work to try to make kernel headers generally palatable to user
space, I'm not going to apply it at all. Not now, not early in the 2.6.18
sequence, not EVER. Because that's not a goal I believe in for a moment.

In contrast, if this is work to make it eventually _easier_ for _library_
people to decide to upgrade their kernel-related information, I'm ok with
it. But that "target audience" part really is very very important. The
target audience should NOT be applications including kernel headers.

The target audience should be distributions and library maintainers.

			Linus

From: Linus Torvalds <torvalds@osdl.org>
Newsgroups: fa.linux.kernel
Subject: Re: Simple header cleanups
Date: Thu, 27 Apr 2006 03:31:36 UTC
Message-ID: <fa.nJwKoZOXzrjSvWWvfA/Z2ZWBDTs@ifi.uio.no>
Original-Message-ID: <Pine.LNX.4.64.0604262028130.3701@g5.osdl.org>

On Thu, 27 Apr 2006, David Woodhouse wrote:
>
> Agreed. And distributions and library maintainers _will_ fix them. Are
> we to deny those people the tools which will help them to keep track of
> our breakage and submit patches to fix it?

No. As mentioned, as long as the target audience is distributions and
library maintainers, I definitely think we should do help them as much as
possible. Our problems have historically been "random people" who have
/usr/include/linux being the symlink to "kernel source of the day", which
is an unsupportable situation.

(And yes, for a short while back in the early nineties, that symlink was
even the proper thing to do. But exactly because it's unsupportable, it
pretty quickly got to be a "don't do that then", but I still occasionally
hear from people who use bad distributions).

		Linus

From: Linus Torvalds <torvalds@osdl.org>
Newsgroups: fa.linux.kernel
Subject: Re: Simple header cleanups
Date: Thu, 27 Apr 2006 22:03:59 UTC
Message-ID: <fa.xt3NKS+Eqk+QbEkQh3pjBhPnC38@ifi.uio.no>
Original-Message-ID: <Pine.LNX.4.64.0604271439100.3701@g5.osdl.org>

On Thu, 27 Apr 2006, Adrian Bunk wrote:
>
> A definition of the kernel <-> userspace ABI is required.

Well, we can get certain hints by just looking at every single type that
is used as a __user pointer. That should give us a lot of the type
information.

The other big piece ends up being argument values passed in to system
calls, most notably ioctl numbers, but there are certainly others too.

And then there are the system call numbers themselves, and their calling
conventions (fairly small part).

> Create an include/kabi/linux/ with the following properties:

I do hate your naming.

Why is that "linux" there? We're not going to have FreeBSD kabi files. And
what about the (pretty common) architecture-specific ones?

The dependency chain is also quite often nontrivial. The ABI's all end up
depending on the basic types, and often on each other (eg the ioctl
numbers depend on the sizes of all the structures, which in turn depend on
the architecture-specific structure layout and low-level types).

So it's _not_ usually possible to just do one file that does one thing,
because they do actually have linkages.

And the linkages can be nasty, because they can easily be linkages that
POSIX - and other standards - forbid them from being visible (you cannot
expose certain typenames if they weren't _explicitly_ included, regardless
of whether you need the type defines).

This is one reason why we shouldn't even _plan_ on having header files
that can just be _directly_ used by the C libraries etc, even if it's just
a "small" kernel ABI header.

Selling it as that kind of idea will inevitably mean that we then get
blamed for not knowing magic rule #579 for SuS v2.1.6 subsection 8(a).

And if we say "you can use these headers unmodified", that _is_ what we're
going to get blamed for. I'm so _not_ interested in having to care or
worry.

So I seriously think we should aim for making it _easier_ for system
libraries to get the information, but we should at the same time make it
clear that we make it easier for them to get the basic info, BUT WE DO NOT
CARE ABOUT THE RANDOM USER STANDARD OF THE DAY.

Have you looked in /usr/include lately? Have you _looked_ at the "expose
BSD names" vs "GNU extended source" vs "strict POSIX" vs
"_XOPEN_SOURCE==600" vs "_USE_MISC" vs a million random and strange
things?

The day I see somebody adding crap like that to the kernel headers is the
day I pull the plug on any "KABI" interfaces.

And don't tell me this has got nothing to do with the kernel constants. Go
look in something like /usr/include/bits/fcntl.h, and cry. See how it's
using _exactly_ the kernel constants, but it has added all the random
standard-of-the-day #ifdef (whether real standards, or the "GNU standards"
or just "legacy BSD-like" etc).

And THAT is why I don't think the simplistic "kabi" directory approach
that people have brought up many times over many years is actually
realistic. People don't realize that glibcs makes "struct flock" actually
look different in user space depending on whether "__USE_FILE_OFFSET64" is
defined or not.

You just haven't seen just how NASTY those user-space headers are. They
can't use _any_ kernel headers directly, because even when they want a
_raw_ kernel data structure, they actually end up doing things differently
in the _middle_ of that data structure.

Really.

So we should try to help those system libc people perhaps _find_ the
values and structures they need, but no, I will _never_ allow the kernel
headers to be used directly. And it doesn't _matter_ if they've been moved
to a "kabi" subdirectory. That's not the issue. The issue is that user
space does insane things that aren't acceptable in kernel space.

			Linus

From: Linus Torvalds <torvalds@osdl.org>
Newsgroups: fa.linux.kernel
Subject: Re: Simple header cleanups
Date: Fri, 28 Apr 2006 00:05:03 UTC
Message-ID: <fa.lWJRXndKDLDTRyF/fFreKlltdP0@ifi.uio.no>
Original-Message-ID: <Pine.LNX.4.64.0604271656390.3701@g5.osdl.org>

On Fri, 28 Apr 2006, Adrian Bunk wrote:
>
> I do still not get your point.
>
> The ABI headers will be used by C libraries.
>
> And by some programs doing low-level Linux specific things - but these
> are exceptions.
>
> Normal userspace programs will simply not care about the contents of the
> kernel ABI headers - the libc will present them pretty headers adhering
> to all past, current and future standards.

As long as that is clear - that the kernel ABI headers are _never_ seen by
normal programs, even indirectly #include'd from the standard include
headers.

That means that any library implementation still needs to basically
_duplicate_ all the kernel ABI structures etc.

Last I heard, a lot of people wanted to avoid that duplication, and
actually wanted the kabi headers exactly because they wanted just _one_
place for these things.

And I suspect a lot of people still think the kabi headers would be
exactly that: things that get #include'd indirectly from the normal header
files.

My point would be that we can't do that. Ever.

As long as people realize that any kabi headers would only ever be used by
system libraries _internally_ to build themselves (or strange system
tools, of course), then I'm happy. I just get the feeling that people
don't always realize that, and they really want to see it as some kind of
"/usr/include/bits" kind of thing.

			Linus