GNU Gnulib

GNU Gnulib

This manual is for GNU Gnulib (updated 2009-01-09 18:22:24), which is a library of common routines intended to be shared at the source level.

Copyright © 2004, 2005, 2006, 2007, 2008 Free Software Foundation, Inc.

Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled “GNU Free Documentation License”.

• Introduction
• Invoking gnulib-tool
• Miscellaneous Notes
• POSIX Substitutes Library: Building as a separate substitutes library.
• Header File Substitutes: Overriding system headers.
• Function Substitutes: Replacing system functions.
• Legacy Function Substitutes: Replacing system functions.
• Glibc Header File Substitutes: Overriding system headers.
• Glibc Function Substitutes: Replacing system functions.
• Particular Modules: Documentation of individual modules.
• GNU Free Documentation License: Copying and sharing this manual.
• Index

1 Introduction

Gnulib is a source code library. It provides basic functionalities to programs and libraries. Currently (as of October 2006) more than 30 packages make use of Gnulib.

Resources:

• Gnulib is hosted at Savannah: http://savannah.gnu.org/projects/gnulib. Get the sources through Git or CVS from there.
• The Gnulib home page: http://www.gnu.org/software/gnulib/.

• Benefits
• Library vs Reusable Code
• Portability and Application Code
• Modules
• Various Kinds of Modules
• Collaborative Development
• Copyright
• Steady Development
• Openness

1.1 Benefits of using Gnulib

Gnulib is useful to enhance various aspects of a package:

• Portability: With Gnulib, a package maintainer can program against the POSIX and GNU libc APIs and nevertheless expect good portability to platforms that don't implement POSIX.
• Maintainability: When a package uses modules from Gnulib instead of code written specifically for that package, the maintainer has less code to maintain.
• Security: Gnulib provides functions that are immune against vulnerabilities that plague the uses of the corresponding commonplace functions. For example, asprintf, canonicalize_file_name are not affected by buffer sizing problems that affect sprintf, realpath. openat does not have the race conditions that open has. Etc.
• Reliability: Gnulib provides functions that combine a call to a system function with a check of the result. Examples are xalloc, xprintf, xstrtod, xgetcwd.
• Structure: Gnulib offers a way to structure code into modules, typically one include file, one source code file, and one autoconf macro for each functionality. Modularity helps maintainability.

1.2 Library vs. Reusable Code

Classical libraries are installed as binary object code. Gnulib is different: It is used as a source code library. Each package that uses Gnulib thus ships with part of the Gnulib source code. The used portion of Gnulib is tailored to the package: A build tool, called gnulib-tool, is provided that copies a tailored subset of Gnulib into the package.

1.3 Portability and Application Code

One of the goals of Gnulib is to make portable programming easy, on the basis of the standards relevant for GNU (and Unix). The objective behind that is to avoid a fragmentation of the user community into disjoint user communities according to the operating system, and instead allow synergies between users on different operating systems.

Another goal of Gnulib is to provide application code that can be shared between several applications. Some people wonder: "What? glibc doesn't have a function to copy a file?" Indeed, the scope of a system's libc is to implement the relevant standards (ISO C99, POSIX:2001) and to provide access functions to the kernel's system calls, and little more.

There is no clear borderline between both areas.

For example, Gnulib has a facility for generating the name of backup files. While this task is entirely at the application level — no standard specifies an API for it — the naïve code has some portability problems because on some platforms the length of file name components is limited to 30 characters or so. Gnulib handles that.

Similarly, Gnulib has a facility for executing a command in a subprocess. It is at the same time a portability enhancement (it works on GNU, Unix, and Windows, compared to the classical fork/exec idiom which is not portable to Windows), as well as an application aid: it takes care of redirecting stdin and/or stdout if desired, and emits an error message if the subprocess failed.

1.4 Modules

Gnulib is divided into modules. Every module implements a single facility. Modules can depend on other modules.

A module consists of a number of files and a module description. The files are copied by gnulib-tool into the package that will use it, usually verbatim, without changes. Source code files (.h, .c files) reside in the lib/ subdirectory. Autoconf macro files reside in the m4/ subdirectory. Build scripts reside in the build-aux/ subdirectory.

The module description contains the list of files — gnulib-tool copies these files. It contains the module's dependencies — gnulib-tool installs them as well. It also contains the autoconf macro invocation (usually a single line or nothing at all) — gnulib-tool ensures this is invoked from the package's configure.ac file. And also a Makefile.am snippet — gnulib-tool collects these into a Makefile.am for the tailored Gnulib part. The module description and include file specification are for documentation purposes; they are combined into MODULES.html.

The module system serves two purposes:

1. It ensures consistency of the used autoconf macros and Makefile.am rules with the source code. For example, source code which uses the getopt_long function — this is a common way to implement parsing of command line options in a way that complies with the GNU standards — needs the source code (lib/getopt.c and others), the autoconf macro which detects whether the system's libc already has this function (in m4/getopt.m4), and a few Makefile.am lines that create the substitute getopt.h if not. These three pieces belong together. They cannot be used without each other. The module description and gnulib-tool ensure that they are copied altogether into the destination package.
2. It allows for scalability. It is well-known since the inception of the MODULA-2 language around 1978 that dissection into modules with dependencies allows for building large sets of code in a maintainable way. The maintainability comes from the facts that:
   • Every module has a single purpose; you don't worry about other parts of the program while creating, reading or modifying the code of a module.
   • The code you have to read in order to understand a module is limited to the source of the module and the .h files of the modules listed as dependencies. It is for this reason also that we recommend to put the comments describing the functions exported by a module into its .h file.

   In other words, the module is the elementary unit of code in Gnulib, comparable to a class in object-oriented languages like Java or C#.

The module system is the basis of gnulib-tool. When gnulib-tool copies a part of Gnulib into a package, it first compiles a module list, starting with the requested modules and adding all the dependencies, and then collects the files, configure.ac snippets and Makefile.am snippets.

1.5 Various Kinds of Modules

There are modules of various kinds in Gnulib. For a complete list of the modules, see in MODULES.html.

1.5.1 Support for ISO C or POSIX functions.

When a function is not implemented by a system, the Gnulib module provides an implementation under the same name. Examples are the ‘snprintf’ and ‘readlink’ modules.

Similarly, when a function is not correctly implemented by a system, Gnulib provides a replacement. For functions, we use the pattern

     #if !HAVE_WORKING_FOO
     # define foo rpl_foo
     #endif

and implement the foo function under the name rpl_foo. This renaming is needed to avoid conflicts at compile time (in case the system header files declare foo) and at link/run time (because the code making use of foo could end up residing in a shared library, and the executable program using this library could be defining foo itself).

For header files, such as stdbool.h or stdint.h, we provide the substitute only if the system doesn't provide a correct one. The template of this replacement is distributed in a slightly different name, with an added underscore, so that on systems which do provide a correct header file the system's one is used.

1.5.2 Enhancements of ISO C or POSIX functions

These are sometimes POSIX functions with GNU extensions also found in glibc — examples: ‘getopt’, ‘fnmatch’ — and often new APIs — for example, for all functions that allocate memory in one way or the other, we have variants which also include the error checking against the out-of-memory condition.

1.5.3 Portable general use facilities

Examples are a module for copying a file — the portability problems relate to the copying of the file's modification time, access rights, and extended attributes — or a module for extracting the tail component of a file name — here the portability to Woe32 requires a different API than the classical POSIX basename function.

1.5.4 Reusable application code

Examples are an error reporting function, a module that allows output of numbers with K/M/G suffixes, or cryptographic facilities.

1.5.5 Object oriented classes

Examples are data structures like ‘list’, or abstract output stream classes that work around the fact that an application cannot implement an stdio FILE with its logic. Here, while staying in C, we use implementation techniques like tables of function pointers, known from the C++ language or from the Linux kernel.

1.5.6 Interfaces to external libraries

Examples are the ‘iconv’ module, which interfaces to the iconv facility, regardless whether it is contained in libc or in an external libiconv. Or the ‘readline’ module, which interfaces to the GNU readline library.

1.5.7 Build / maintenance infrastructure

An example is the ‘maintainer-makefile’ module, which provides extra Makefile tags for maintaining a package.

1.6 Collaborative Development

Gnulib is maintained collaboratively. The mailing list is <bug-gnulib at gnu dot org>. Be warned that some people on the list may be very active at some times and unresponsive at other times.

Every module has one or more maintainers. While issues are discussed collaboratively on the list, the maintainer of a module nevertheless has a veto right regarding changes in his module.

All patches should be posted the list, regardless whether they are proposed patches or whether they are committed immediately by the maintainer of the particular module. The purpose is not only to inform the other users of the module, but mainly to allow peer review. It is not uncommon that several people contribute comments or spot bugs after a patch was proposed.

Conversely, if you are using Gnulib, and a patch is posted that affects one of the modules that your package uses, you have an interest in proofreading the patch.

1.7 Copyright

Most modules are under the GPL. Some, mostly modules which can reasonably be used in libraries, are under LGPL. The source files always say "GPL", but the real license specification is in the module description file. If the module description file says "GPL", it means "GPLv3+" (GPLv3 or newer, at the licensee's choice); if it says "LGPL", it means "LGPLv3+" (LGPLv3 or newer, at the licensee's choice).

More precisely, the license specification in the module description file applies to the files in lib/ and build-aux/. Different licenses apply to files in special directories:

modules/

Module description files are under this copyright:

Copyright © 200X-200Y Free Software Foundation, Inc.
Copying and distribution of this file, with or without modification, in any medium, are permitted without royalty provided the copyright notice and this notice are preserved.

m4/

Autoconf macro files are under this copyright:

Copyright © 200X-200Y Free Software Foundation, Inc.
This file is free software; the Free Software Foundation gives unlimited permission to copy and/or distribute it, with or without modifications, as long as this notice is preserved.

tests/

If a license statement is not present in a test module, the test files are under GPL. Even if the corresponding source module is under LGPL, this is not a problem, since compiled tests are not installed by “make install”.

doc/

Documentation files are under this copyright:

Copyright © 2004-2008 Free Software Foundation, Inc.
Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled “GNU Free Documentation License”.

If you want to use some Gnulib modules under LGPL, you can do so by passing the option ‘--lgpl’ to gnulib-tool. This will replace the GPL header with an LGPL header while copying the source files to your package. Similarly, if you want some Gnulib modules under LGPLv2+ (Lesser GPL version 2.1 or newer), you can do so by passing the option ‘--lgpl=2’ to gnulib-tool.

Keep in mind that when you submit patches to files in Gnulib, you should license them under a compatible license. This means that sometimes the contribution will have to be LGPL, if the original file is available under LGPL. You can find out about it by looking for a "License: LGPL" information in the corresponding module description.

1.8 Steady Development

Gnulib modules are continually adapted, to match new practices, to be consistent with newly added modules, or simply as a response to build failure reports. We don't make releases, but instead recommend to use the newest version of Gnulib from the Git repository, except in periods of major changes. The source tree can also be fetched from a read-only CVS that mirrors the Git repository.

1.9 Openness

Gnulib is open in the sense that we gladly accept contributions if they are generally useful, well engineered, and if the contributors have signed the obligatory papers with the FSF.

The module system is open in the sense that a package using Gnulib can

1. locally patch or override files in Gnulib,
2. locally add modules that are treated like Gnulib modules by gnulib-tool.

This is achieved by the ‘--local-dir’ option of gnulib-tool.

2 Invoking gnulib-tool

The gnulib-tool command is the recommended way to import Gnulib modules. It is possible to borrow Gnulib modules in a package without using gnulib-tool, relying only on the meta-information stored in the modules/* files, but with a growing number of modules this becomes tedious. gnulib-tool simplifies the management of source files, Makefile.ams and configure.ac in packages incorporating Gnulib modules.

Run ‘gnulib-tool --help’ for information. To get familiar with gnulib-tool without affecting your sources, you can also try some commands with the option ‘--dry-run’; then gnulib-tool will only report which actions it would perform in a real run without changing anything.

• Initial import: First import of Gnulib modules.
• Modified imports: Changing the import specification.
• Simple update: Tracking Gnulib development.
• Source changes: Impact of Gnulib on your source files.
• gettextize and autopoint: Caveat: gettextize and autopoint users!
• Localization: Handling Gnulib's own message translations.
• VCS Issues: Integration with Version Control Systems.
• Unit tests: Bundling the unit tests of the Gnulib modules.

2.1 Initial import

Gnulib assumes your project uses Autoconf and Automake. Invoking ‘gnulib-tool --import’ will copy source files, create a Makefile.am to build them, generate a file gnulib-comp.m4 with Autoconf M4 macro declarations used by configure.ac, and generate a file gnulib-cache.m4 containing the cached specification of how Gnulib is used.

Our example will be a library that uses Autoconf, Automake and Libtool. It calls strdup, and you wish to use gnulib to make the package portable to C89 and C99 (which don't have strdup).

     ~/src/libfoo$ gnulib-tool --import strdup
     Module list with included dependencies:
       absolute-header
       extensions
       strdup
       string
     File list:
       lib/dummy.c
       lib/strdup.c
       lib/string.in.h
       m4/absolute-header.m4
       m4/extensions.m4
       m4/gnulib-common.m4
       m4/strdup.m4
       m4/string_h.m4
     Creating directory ./lib
     Creating directory ./m4
     Copying file lib/dummy.c
     Copying file lib/strdup.c
     Copying file lib/string.in.h
     Copying file m4/absolute-header.m4
     Copying file m4/extensions.m4
     Copying file m4/gnulib-common.m4
     Copying file m4/gnulib-tool.m4
     Copying file m4/strdup.m4
     Copying file m4/string_h.m4
     Creating lib/Makefile.am
     Creating m4/gnulib-cache.m4
     Creating m4/gnulib-comp.m4
     Finished.
     
     You may need to add #include directives for the following .h files.
       #include <string.h>
     
     Don't forget to
       - add "lib/Makefile" to AC_CONFIG_FILES in ./configure.ac,
       - mention "lib" in SUBDIRS in Makefile.am,
       - mention "-I m4" in ACLOCAL_AMFLAGS in Makefile.am,
       - invoke gl_EARLY in ./configure.ac, right after AC_PROG_CC,
       - invoke gl_INIT in ./configure.ac.
     ~/src/libfoo$

By default, the source code is copied into lib/ and the M4 macros in m4/. You can override these paths by using --source-base=DIRECTORY and --m4-base=DIRECTORY. Some modules also provide other files necessary for building. These files are copied into the directory specified by ‘AC_CONFIG_AUX_DIR’ in configure.ac or by the --aux-dir=DIRECTORY option. If neither is specified, the current directory is assumed.

gnulib-tool can make symbolic links instead of copying the source files. The option to specify for this is ‘--symlink’, or ‘-s’ for short. This can be useful to save a few kilobytes of disk space. But it is likely to introduce bugs when gnulib is updated; it is more reliable to use ‘gnulib-tool --update’ (see below) to update to newer versions of gnulib. Furthermore it requires extra effort to create self-contained tarballs, and it may disturb some mechanism the maintainer applies to the sources. For these reasons, this option is generally discouraged.

gnulib-tool will overwrite any pre-existing files, in particular Makefile.am. Unfortunately, separating the generated Makefile.am content (for building the gnulib library) into a separate file, say gnulib.mk, that could be included by your handwritten Makefile.am is not possible, due to how variable assignments are handled by Automake.

Consequently, it is a good idea to choose directories that are not already used by your projects, to separate gnulib imported files from your own files. This approach is also useful if you want to avoid conflicts between other tools (e.g., gettextize that also copy M4 files into your package. Simon Josefsson successfully uses a source base of gl/, and a M4 base of gl/m4/, in several packages.

After the ‘--import’ option on the command line comes the list of Gnulib modules that you want to incorporate in your package. The names of the modules coincide with the filenames in Gnulib's modules/ directory.

Some Gnulib modules depend on other Gnulib modules. gnulib-tool will automatically add the needed modules as well; you need not list them explicitly. gnulib-tool will also memorize which dependent modules it has added, so that when someday a dependency is dropped, the implicitly added module is dropped as well (unless you have explicitly requested that module).

If you want to cut a dependency, i.e., not add a module although one of your requested modules depends on it, you may use the option ‘--avoid=module’ to do so. Multiple uses of this option are possible. Of course, you will then need to implement the same interface as the removed module.

A few manual steps are required to finish the initial import. gnulib-tool printed a summary of these steps.

First, you must ensure Autoconf can find the macro definitions in gnulib-comp.m4. Use the ACLOCAL_AMFLAGS specifier in your top-level Makefile.am file, as in:

     ACLOCAL_AMFLAGS = -I m4

You are now ready to call the M4 macros in gnulib-comp.m4 from configure.ac. The macro gl_EARLY must be called as soon as possible after verifying that the C compiler is working. Typically, this is immediately after AC_PROG_CC, as in:

     ...
     AC_PROG_CC
     gl_EARLY
     ...

The core part of the gnulib checks are done by the macro gl_INIT. Place it further down in the file, typically where you normally check for header files or functions. It must come after other checks which may affect the compiler invocation, such as AC_MINIX. For example:

     ...
     # For gnulib.
     gl_INIT
     ...

gl_INIT will in turn call the macros related with the gnulib functions, be it specific gnulib macros, like gl_FUNC_ALLOCA or autoconf or automake macros like AC_FUNC_ALLOCA or AM_FUNC_GETLINE. So there is no need to call those macros yourself when you use the corresponding gnulib modules.

You must also make sure that the gnulib library is built. Add the Makefile in the gnulib source base directory to AC_CONFIG_FILES, as in:

     AC_CONFIG_FILES(... lib/Makefile ...)

You must also make sure that make will recurse into the gnulib directory. To achieve this, add the gnulib source base directory to a SUBDIRS Makefile.am statement, as in:

     SUBDIRS = lib

or if you, more likely, already have a few entries in SUBDIRS, you can add something like:

     SUBDIRS += lib

Finally, you have to add compiler and linker flags in the appropriate source directories, so that you can make use of the gnulib library. Since some modules (‘getopt’, for example) may copy files into the build directory, top_builddir/lib is needed as well as top_srcdir/lib. For example:

     ...
     AM_CPPFLAGS = -I$(top_builddir)/lib -I$(top_srcdir)/lib
     ...
     LDADD = lib/libgnu.a
     ...

Don't forget to #include the various header files. In this example, you would need to make sure that ‘#include <string.h>’ is evaluated when compiling all source code files, that want to make use of strdup.

In the usual case where Autoconf is creating a config.h file, you should include config.h first, before any other include file. That way, for example, if config.h defines ‘restrict’ to be the empty string on a pre-C99 host, or a macro like ‘_FILE_OFFSET_BITS’ that affects the layout of data structures, the definition is consistent for all include files. Also, on some platforms macros like ‘_FILE_OFFSET_BITS’ and ‘_GNU_SOURCE’ may be ineffective, or may have only a limited effect, if defined after the first system header file is included.

Finally, note that you can not use AC_LIBOBJ or AC_REPLACE_FUNCS in your configure.ac and expect the resulting object files to be automatically added to lib/libgnu.a. This is because your AC_LIBOBJ and AC_REPLACE_FUNCS invocations from configure.ac augment a variable @LIBOBJS@ (and/or @LTLIBOBJS@ if using Libtool), whereas lib/libgnu.a is built from the contents of a different variable, usually @gl_LIBOBJS@ (or @gl_LTLIBOBJS@ if using Libtool).

2.2 Modified imports

You can at any moment decide to use Gnulib differently than the last time.

If you only want to use more Gnulib modules, simply invoke gnulib-tool --import new-modules. gnulib-tool remembers which modules were used last time. The list of modules that you pass after ‘--import’ is added to the previous list of modules.

For most changes, such as added or removed modules, or even different choices of ‘--lib’, ‘--source-base’ or ‘--aux-dir’, there are two ways to perform the change.

The standard way is to modify manually the file gnulib-cache.m4 in the M4 macros directory, then launch ‘gnulib-tool --import’.

The other way is to call gnulib-tool again, with the changed command-line options. Note that this doesn't let you remove modules, because as you just learned, the list of modules is always cumulated. Also this way is often impractical, because you don't remember the way you invoked gnulib-tool last time.

The only change for which this doesn't work is a change of the ‘--m4-base’ directory. Because, when you pass a different value of ‘--m4-base’, gnulib-tool will not find the previous gnulib-cache.m4 file any more... A possible solution is to manually copy the gnulib-cache.m4 into the new M4 macro directory.

In the gnulib-cache.m4, the macros have the following meaning:

gl_MODULES

The argument is a space separated list of the requested modules, not including dependencies.

gl_AVOID

The argument is a space separated list of modules that should not be used, even if they occur as dependencies. Corresponds to the ‘--avoid’ command line argument.

gl_SOURCE_BASE

The argument is the relative file name of the directory containing the gnulib source files (mostly *.c and *.h files). Corresponds to the ‘--source-base’ command line argument.

gl_M4_BASE

The argument is the relative file name of the directory containing the gnulib M4 macros (*.m4 files). Corresponds to the ‘--m4-base’ command line argument.

gl_TESTS_BASE

The argument is the relative file name of the directory containing the gnulib unit test files. Corresponds to the ‘--tests-base’ command line argument.

gl_LIB

The argument is the name of the library to be created. Corresponds to the ‘--lib’ command line argument.

gl_LGPL

The presence of this macro without arguments corresponds to the ‘--lgpl’ command line argument. The presence of this macro with an argument (whose value must be 2 or 3) corresponds to the ‘--lgpl=arg’ command line argument.

gl_LIBTOOL

The presence of this macro corresponds to the ‘--libtool’ command line argument and to the absence of the ‘--no-libtool’ command line argument. It takes no arguments.

gl_MACRO_PREFIX

The argument is the prefix to use for macros in the gnulib-comp.m4 file. Corresponds to the ‘--macro-prefix’ command line argument.

2.3 Simple update

When you want to update to a more recent version of Gnulib, without changing the list of modules or other parameters, a simple call does it:

     $ gnulib-tool --import

This will create, update or remove files, as needed.

Note: From time to time, changes are made in Gnulib that are not backward compatible. When updating to a more recent Gnulib, you should consult Gnulib's NEWS file to check whether the incompatible changes affect your project.

2.4 Changing your sources for use with Gnulib

Gnulib contains some header file overrides. This means that when building on systems with deficient header files in /usr/include/, it may create files named string.h, stdlib.h, stdint.h or similar in the build directory. In the other source directories of your package you will usually pass ‘-I’ options to the compiler, so that these Gnulib substitutes are visible and take precedence over the files in /usr/include/.

These Gnulib substitute header files rely on <config.h> being already included. Furthermore <config.h> must be the first include in every compilation unit. This means that to all your source files and likely also to all your tests source files you need to add an ‘#include <config.h>’ at the top. Which source files are affected? Exactly those whose compilation includes a ‘-I’ option that refers to the Gnulib library directory.

This is annoying, but inevitable: On many systems, <config.h> is used to set system dependent flags (such as _GNU_SOURCE on GNU systems), and these flags have no effect after any system header file has been included.

2.5 Caveat: gettextize and autopoint users

The programs gettextize and autopoint, part of GNU gettext, import or update the internationalization infrastructure. Some of this infrastructure, namely ca. 20 autoconf macro files and the config.rpath file, is also contained in Gnulib and may be imported by gnulib-tool. The use of gettextize or autopoint will therefore overwrite some of the files that gnulib-tool has imported, and vice versa.

Avoiding to use gettextize (manually, as package maintainer) or autopoint (as part of a script like autoreconf or autogen.sh) is not the solution: These programs also import the infrastructure in the po/ and optionally in the intl/ directory.

The copies of the conflicting files in Gnulib are more up-to-date than the copies brought in by gettextize and autopoint. When a new gettext release is made, the copies of the files in Gnulib will be updated immediately.

The solution is therefore:

1. When you run gettextize, always use the gettextize from the matching GNU gettext release. For the most recent Gnulib checkout, this is the newest release found on http://ftp.gnu.org/gnu/gettext/. For an older Gnulib snapshot, it is the release that was the most recent release at the time the Gnulib snapshot was taken. Then, after gettextize, invoke gnulib-tool.
2. When a script of yours run autopoint, invoke gnulib-tool afterwards.
3. If you get an error message like *** error: gettext infrastructure mismatch: using a Makefile.in.in from gettext version ... but the autoconf macros are from gettext version ..., it means that a new GNU gettext release was made, and its autoconf macros were integrated into Gnulib and now mismatch the po/ infrastructure. In this case, fetch and install the new GNU gettext release and run gettextize followed by gnulib-tool.

2.6 Handling Gnulib's own message translations

Gnulib provides some functions that emit translatable messages using GNU gettext. The ‘gnulib’ domain at the Translation Project collects translations of these messages, which you should incorporate into your own programs.

There are two basic ways to achieve this. The first, and older, method is to list all the source files you use from Gnulib in your own po/POTFILES.in file. This will cause all the relevant translatable strings to be included in your POT file. When you send this POT file to the Translation Project, translators will normally fill in the translations of the Gnulib strings from their “translation memory”, and send you back updated PO files.

However, this process is error-prone: you might forget to list some source files, or the translator might not be using a translation memory and provide a different translation than another translator, or the translation might not be kept in sync between Gnulib and your package. It is also slow and causes substantial extra work, because a human translator must be in the loop for each language and you will need to incorporate their work on request.

For these reasons, a new method was designed and is now recommended. If you pass the --po-base=directory and --po-domain=domain options to gnulib-tool, then gnulib-tool will create a separate directory with its own POTFILES.in, and fetch current translations directly from the Translation Project (using rsync or wget, whichever is available). The POT file in this directory will be called domain-gnulib.pot, depending on the domain you gave to the --po-domain option (typically the same as the package name). This causes these translations to reside in a separate message domain, so that they do not clash either with the translations for the main part of your package nor with those of other packages on the system that use possibly different versions of Gnulib. When you use these options, the functions in Gnulib are built in such a way that they will always use this domain regardless of the default domain set by textdomain.

In order to use this method, you must – in each program that might use Gnulib code – add an extra line to the part of the program that initializes locale-dependent behavior. Where you would normally write something like:

       setlocale (LC_ALL, "");
       bindtextdomain (PACKAGE, LOCALEDIR);
       textdomain (PACKAGE);

you should add an additional bindtextdomain call to inform gettext of where the MO files for the extra message domain may be found:

       bindtextdomain (PACKAGE "-gnulib", LOCALEDIR);

(This example assumes that the domain that you specified to gnulib-tool is the same as the value of the PACKAGE preprocessor macro.)

Since you do not change the textdomain call, the default message domain for your program remains the same and your own use of gettext functions will not be affected.

2.7 Issues with Version Control Systems

If a project stores its source files in a version control system (VCS), such as CVS, SVN, or Git, one needs to decide which files to commit.

All files created by gnulib-tool, except gnulib-cache.m4, should be treated like generated source files, like for example a parser.c file is generated from parser.y.

• In projects which commit all source files, whether generated or not, into their VCS, the gnulib-tool generated files should all be committed. In this case, you also pass the option ‘--no-vc-files’ to gnulib-tool.

  Gnulib also contains files generated by make (and removed by make clean), using information determined by configure. They should not be checked into the VCS, but instead added to .gitignore or .cvsignore. When you have a Gnulib source file of the form lib/foo.in.h, the corresponding lib/foo.h is such a file.

• In projects which customarily omit from their VCS all files that are generated from other source files, all these files and directories would not be added into the VCS. The only file that must be added to the VCS is gnulib-cache.m4 in the M4 macros directory. Also, the script for restoring files not in the VCS, customarily called autogen.sh or bootstrap.sh, will typically contain the statement for restoring the omitted files:

            $ gnulib-tool --update
  

  The ‘--update’ option operates much like the ‘--import’ option, but it does not offer the possibility to change the way Gnulib is used. Also it does not report in the ChangeLogs the files that it had to add because they were missing.

2.8 Bundling the unit tests of the Gnulib modules

You can bundle the unit tests of the Gnulib modules together with your package, through the ‘--with-tests’ option. Together with ‘--with-tests’, you also specify the directory for these tests through the ‘--tests-base’ option. Of course, you need to add this directory to the SUBDIRS variable in the Makefile.am of the parent directory.

The advantage of having the unit tests bundled is that when your program has a problem on a particular platform, running the unit tests may help determine quickly if the problem is on Gnulib's side or on your package's side. Also, it helps verifying Gnulib's portability, of course.

The unit tests will be compiled and run when the user runs ‘make check’. When the user runs only ‘make’, the unit tests will not be compiled.

In the SUBDIRS variable, it is useful to put the Gnulib tests directory after the directory containing the other tests, not before:

     SUBDIRS = gnulib-lib src man tests gnulib-tests

This will ensure that on platforms where there are test failures in either directory, users will see and report the failures from the tests of your program.

Note: In packages which use more than one invocation of gnulib-tool in the scope of the same configure.ac, you cannot use ‘--with-tests’. You will have to use a separate configure.ac in this case.

3 Miscellaneous Notes

• Comments
• Header files
• Out of memory handling
• Library version handling
• Windows sockets
• Libtool and Windows
• License Texinfo sources
• Build robot for gnulib

3.1 Comments

Where to put comments describing functions: Because of risk of divergence, we prefer to keep most function describing comments in only one place: just above the actual function definition. Some people prefer to put that documentation in the .h file. In any case, it should appear in just one place unless you can ensure that the multiple copies will always remain identical.

3.2 Header files

It is a tradition to use CPP tricks to avoid parsing the same header file more than once, which might cause warnings. The trick is to wrap the content of the header file (say, foo.h) in a block, as in:

     #ifndef FOO_H
     # define FOO_H
     ...
     body of header file goes here
     ...
     #endif /* FOO_H */

Whether to use FOO_H or _FOO_H is a matter of taste and style. The C89 and C99 standards reserve all identifiers that begin with an underscore and either an uppercase letter or another underscore, for any use. Thus, in theory, an application might not safely assume that _FOO_H has not already been defined by a library. On the other hand, using FOO_H will likely lead the higher risk of collisions with other symbols (e.g., KEY_H, XK_H, BPF_H, which are CPP macro constants, or COFF_LONG_H, which is a CPP macro function). Your preference may depend on whether you consider the header file under discussion as part of the application (which has its own namespace for CPP symbols) or a supporting library (that shouldn't interfere with the application's CPP symbol namespace).

Adapting C header files for use in C++ applications can use another CPP trick, as in:

     # ifdef __cplusplus
     extern "C"
     {
     # endif
     ...
     body of header file goes here
     ...
     # ifdef __cplusplus
     }
     # endif

The idea here is that __cplusplus is defined only by C++ implementations, which will wrap the header file in an ‘extern "C"’ block. Again, whether to use this trick is a matter of taste and style. While the above can be seen as harmless, it could be argued that the header file is written in C, and any C++ application using it should explicitly use the ‘extern "C"’ block itself. Your preference might depend on whether you consider the API exported by your header file as something available for C programs only, or for C and C++ programs alike.

Note that putting a #include in an extern "C" { ... } block yields a syntax error in C++ mode on some platforms (e.g., glibc systems with g++ v3.3 to v4.2, AIX, OSF/1, IRIX). For this reason, it is recommended to place the #include before the extern "C" block.

Include ordering

When writing a gnulib module, or even in general, a good way to order the ‘#include’ directives is the following.

• First comes the #include "..." specifying the module being implemented.
• Then come all the #include <...> of system or system-replacement headers, in arbitrary order.
• Then come all the #include "..." of gnulib and private headers, in arbitrary order.

3.3 Out of memory handling

The GSS API does not have a standard error code for the out of memory error condition. Instead of adding a non-standard error code, this library has chosen to adopt a different strategy. Out of memory handling happens in rare situations, but performing the out of memory error handling after almost all API function invocations pollute your source code and might make it harder to spot more serious problems. The strategy chosen improves code readability and robustness.