Build system tools: make-makefile, file generation

This post will be article #2 in a series about the build-config tool make-makefile.
The series is being written to formally document current tool functionality and
provide a base for future topics related to the container makefile project.
Container structure, functionality and example usage.

Series posts

When makefile generation is needed the tool will perform a few steps.

  1. gmake -f client.mk is invoked. During traversal if gmake detects one
    of two conditions the tool will be invoked to generate a Makefile:
        o $(obj)/Makefile does not exist.
        o $(src)/Makefile.in is newer than $(obj)/Makefile.
  2. Target rules used to invoke mm live in config/rules.mk near line 1342 and 1347
        o 1342 Makefile: Makefile.in
        o 1343 @$(PERL) $(AUTOCONF_TOOLS)/make-makefile -t $(topsrcdir) -d $(DEPTH)
  3. make-makefile will be passed -d and -t command line arguments:
        o -t path to root of a development sandbox ~$(TOPSRCDIR)
        o -d relative path –depth from file to hierarchy root directory.
    Makefile: $(MOZ_OBJDIR), Makefile.in: $( TOPSRCDIR)
  4. When invoked make-makefile is aware of the directory hierarchy and will expect cwd for the shell to be $(MOZ_OBJDIR) or a subdir of it [1]. config/rules.mk logic will place make-makefile in the target directory for each makefile to be generated [2].
  5. Step 1 – obtain values for depth, topsrcdir and objdir.

    Values will be:
        o used to construct directory and file paths.
        o perform text substitution within Makefile.in templates.

    Values can be:
        o explicitly passed on the command line.
        o determined from the filesystem based on arguments and/or cwd.
        o extracted from file arguments.

    $depth will be set by:
        o Explicitly using the -d command line argument.
        o Assignment of DEPTH=../.. specified within an individual Makefile.
        o Assignment of DEPTH=../.. within any other files passed on the command line
          (is this behavior a bug or feature?).
        o Assignment of DEPTH=../.. contained within a parent directory Makefile [3], [4].

  6. Step 2 – Using cwd and arguments, derive paths to the source template
    Makefile.in and generated target file Makefile.
  7. Step 3 – Slurp the source template Makefile.in. Perform value substitutions
    on tokens embedded within the template of the form @token@.

    
    browser/branding/official/Makefile.in
    =====================================
    DEPTH = ../../..
    topsrcdir = @topsrcdir@
    srcdir = @srcdir@
    VPATH = @srcdir@
    

    Some directory/file paths are derived internally by the tool for quick substitution. For any @${unknown}@ tokens make-makefile will delegate expansion with a call to system(“./config.status”) [5] to obtain values, create directories and who knows what else. Shell overhead will be imposed by this step so avoid unnecessary tokens when possible. When only a handful of tokens are in play one optimization to check on is if mm could derive values for the tokens and avoid overhead from calling the config.status script [6].

  8. Step 4 – update or preserve generated Makefile timestamps. If obj/Makefile does not exist create it. If the file exists compare generated content against the original and only update when modified.

footnotes

[1] The requirement of cwd==$(MOZ_OBJDIR)/ can be removed with use of the –enhanced flag. This option and other new flags will be covered in a future post.
[2] Enhancement: a response file could be used to support bulk makefile processing and directory creation (by config.status) saving some shell overhead. Bulk processing might also reduce the number of times config.status must be invoked independently.
[3] Potential bug. DEPTH= searches within multiple files could set $depth incorrectly when it is not explicitly set within a makefile. One ex would be passing browser/…/Makefile and js/src/…/Makefile as command line arguments. Though this error condition exists it is not likely to be triggered as files are processed individually.
[4] Potential bug: checking ../Makefile for ‘DEPTH=’ will fail for makefiles invoked from a parent directory several layers above cwd.
[5] Enhancement – modify make-makefile to parse and extract fully expanded values from config* to avoid invoking ./config.status for a subset of substitution tokens.
[6] make-makefile will issue a warning when the ./config.status script will be launched
WARNING: token SET_MAKE not defined
line 2, src: […..]/js/src/ctypes/libffi/Makefile.in

resources

Tool source

Unit tests

makefiles

Pending enhancements

  • Expand test coverage for make-makefile
  • Add Response file support.
  • Rewrite unit tests in python.
  • Porting make-makefile from perl to python.
  • minimize shell overhead from config.status use

Future Blog Topics

  • make-makefile args and enhancements to generalize logic and support container builds.
  • Container makefiles: scripts & config
    • containermake.py
    • template files
    • non-invasive makefile edits to support container builds
  • Container makefiles: bulk file processing
    • Makefile.in => Makefile
    • *.idl => .h, .xpt, and deps .xpt.pp
    • makefile targets: export, tools, libs, test, *
    • thread mutex from target modifiers and extra container dependencies
  • Makefiles and library logic
    • threadsafe mkdir library function
    • config/rules.mk modularity – isolating logic based on bulk build types
    • thread mutex from target modifiers and container dependencies
Advertisements
This entry was posted in build tools. Bookmark the permalink.

2 Responses to Build system tools: make-makefile, file generation

  1. Manuela says:

    greetings, perfect website, and a decent understand! just one for my bookmarking.

  2. Abdul says:

    Superb post, stylish web page style, keep up the great work

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s