# Autoload Manual

###### [in package AUTOLOAD]

## Links and Systems

Here is the [official
repository](https://github.com/melisgl/autoload/) and the [HTML
documentation](http://melisgl.github.io/mgl-pax-world/autoload-manual.html)
for the latest version.

- [system] "autoload"

    - Version: 0.1.0

    - Description: An ASDF autoloading facility. See
    Autoload Manual.

    - Licence: MIT, see COPYING.

    - Author: Gábor Melis

    - Mailto: [mega@retes.hu](mailto:mega@retes.hu)

    - Homepage: <https://github.com/melisgl/autoload>

    - Bug tracker: <https://github.com/melisgl/autoload/issues>

    - Source control: [GIT](https://github.com/melisgl/autoload.git)

    - Depends on: closer-mop, mgl-pax-bootstrap

- [system] "autoload-doc"

    - Description: Parts of the Autoload library that depend on
    mgl-pax are in this system to avoid the circular
    dependencies that would arise because mgl-pax
    depends on autoload. Note that
    mgl-pax/navigate and mgl-pax/document depend on this system, which renders most of this an
    implementation detail.

    - Depends on: autoload, dref, mgl-pax, named-readtables, pythonic-string-reader

## Introduction

Libraries often choose to limit dependencies, even if it means
sacrificing features or duplicating code, to minimize

- compilation time,

- memory usage in deployment, and

- the risk of breakage through dependencies.

This library mitigates the first two issues by loading heavy
dependencies on demand. The core idea is

    (defmacro autoload (name asdf-system)
      `(defun ,name (&rest args)
         (asdf:load-system ,asdf-system)
         (apply ',name args)))

Suppose we have a library called my-lib that autoloads
my-lib/full. In my-lib, we could use AUTOLOAD
as

    (autoload foo "my-lib/full")

and have

    (defun foo (x)
      "doc"
      (1+ x))

in my-lib/full.

However, manually keeping the loaddefs (e.g. the AUTOLOAD form
above) in sync with the definitions is fragile, so we introduce the
DEFUN/AUTO autodef to mark autoloaded functions in the
my-lib/full system:

    (defun/auto foo (x)
      "doc"
      (1+ x))

<br>

ASDF Integration

To generate loaddefs, we add a few lines to
the system definitions:

    (asdf:defsystem "my-lib"
      :defsystem-depends-on ("autoload")
      :class "autoload:autoload-system"
      :auto-depends-on ("my-lib/full")
      :auto-loaddefs "loaddefs.lisp"
      :components ((:file "loaddefs")
                   ...))

    (asdf:defsystem "my-lib/full"
      :defsystem-depends-on ("autoload")
      :class "autoload:autoload-system"
      :components (...))

Then, the loaddefs can be extracted:

    (extract-loaddefs "my-lib")
    => ((autoload foo "my-lib/full" :arglist "(x)" :docstring "doc"))

This is implemented by loading the :AUTO-DEPENDS-ON of my-lib and
recording DEFUN/AUTOs. EXTRACT-LOADDEFS is a low-level utility used
by RECORD-LOADDEFS, which writes its results to the
system's :AUTO-LOADDEFS, "loaddefs.lisp" in the above example. So,
all we need to do is call RECORD-LOADDEFS to regenerate the loaddefs
file:

    (record-loaddefs "my-lib")

To prevent the loaddefs file from getting out of sync with the
definitions, ASDF:TEST-SYSTEM calls CHECK-LOADDEFS by default.

ASDF, and by extension Quicklisp, doesn't know about the declared
:AUTO-DEPENDS-ON, so (QL:QUICKLOAD "my-lib") does not install the
autoloaded dependencies. They can be installed manually with

    (autodeps "my-lib" :installer #'ql:quickload)

If all the autoloaded dependencies are installed, one can eagerly
load them to ensure that autoloading is not triggered later (e.g.
in deployment):

    (map nil #'asdf:load-system (autodeps "my-lib"))

<br>

Other Features

Autoloading is not only for Functions:

- Autoloading Classes at the time of their first instantiation is
  supported.

- Variables can be marked for early definition and have their
  initial values assigned if the initial value form provably doesn't
  have dependencies. If that's not the case, subject to platform
  support, the definition in the loaded system injects a global
  binding even in the presence of local bindings.

- Multiple Packages can have their final states and
  interdependencies reconstructed before loading their systems even
  if they were mutated operations like IMPORT and EXPORT.

## Basics

- [glossary-term] loaddef

    A loaddef is a preliminary definition that serves as a stand-in
    until the fully-realized implementation is loaded. Accessing it may
    or may not load a system. See
    LOADDEF-FUNCTION-P, LOADDEF-CLASS-P, LOADDEF-VARIABLE-P and
    LOADDEF-PACKAGE-P.

- [glossary-term] autodef

    An autodef (e.g. (DEFUN/AUTO NAME ...)) performs the job of its
    plain counterpart (DEFUN). In addition, it marks the definition (of
    NAME as a function) for Automatically Generating Loaddefs and, at the time of the
    first such autodef, it signals an AUTOLOAD-WARNING if NAME has not
    been declared as a loaddef (has never been LOADDEF-FUNCTION-P). See
    DEFUN/AUTO, DEFCLASS/AUTO, DEFVAR/AUTO and DEFPACKAGE/AUTO.

### Loading Systems

Function and class loaddefs trigger the loading of ASDF:SYSTEMs.
Unlike normal ASDF dependencies (declared in :DEPENDS-ON), autoload
dependencies (which may be declared in :AUTO-DEPENDS-ON) are allowed
to be circular. The rules for loading are as follows.

1. It is an AUTOLOAD-ERROR if loading is triggered during compile
   time or during a LOAD of either a source file or a
   compiled file. This is to prevent infinite autoload
   recursion.

2. It is an AUTOLOAD-ERROR if the system does not exist.

3. The system is loaded under WITH-COMPILATION-UNIT :OVERRIDE T and
   WITH-STANDARD-IO-SYNTAX but with *PRINT-READABLY* NIL. Other
   non-portable measures may be taken to standardize the dynamic
   environment. Errors signalled during the load are not handled or
   resignalled by the Autoload library.

4. It is an AUTOLOAD-ERROR if the loaddef is not replaced by a
   normal definition or deleted by the loaded system, that is, when
   it remains a loaddef (e.g. in terms of LOADDEF-FUNCTION-P).

### Conditions

- [condition] AUTOLOAD-ERROR ERROR

    Signalled for some failures during Loading Systems.

- [condition] AUTOLOAD-WARNING SIMPLE-WARNING

    See AUTOLOAD, autodef and :AUTO-DEPENDS-ON for
    when this is signalled.

### Functions

- [macro] AUTOLOAD NAME SYSTEM-NAME &KEY (ARGLIST NIL) DOCSTRING

    This is the loaddef for autodef DEFUN/AUTO.

    Define a function stub with NAME and return NAME. The arguments are
    not evaluated. If NAME has an FDEFINITION and it is not
    LOADDEF-FUNCTION-P, then this does nothing and returns NIL.

    The stub first loads SYSTEM-NAME, then it APPLYs
    the function NAME to the arguments originally provided to the stub.

    The stub is not defined at compile time, which matches the
    required semantics of DEFUN. NAME is DECLAIMed with FTYPE
    FUNCTION and NOTINLINE.

    - ARGLIST will be installed as the stub's arglist if specified and
      it's supported on the platform (currently only SBCL). If ARGLIST
      is a string, the effective value of ARGLIST is read from it. If
      the read fails, an AUTOLOAD-WARNING is signalled and processing
      continues as if ARGLIST had not been provided.

        Arglists are for interactive purposes only. For example, they
        are shown by SLIME autodoc and returned by DREF:ARGLIST.

    - DOCSTRING, if non-NIL, will be the stub's docstring. If NIL, then
      a generic docstring that says what system it autoloads will be
      used.

    When AUTOLOAD is macroexpanded during the compilation or loading of
    an AUTOLOAD-SYSTEM, it signals an AUTOLOAD-WARNING if SYSTEM-NAME is
    not among those declared in :AUTO-DEPENDS-ON.

- [function] LOADDEF-FUNCTION-P NAME

    See if an AUTOLOAD for NAME was established, and since then it has
    not been redefined (e.g. with DEFUN/AUTO, DEFUN) or made
    FMAKUNBOUND.

- [macro] DEFUN/AUTO NAME LAMBDA-LIST &BODY BODY

    This is the autodef for the loaddef AUTOLOAD.

    Like DEFUN, but also silence redefinition warnings. NAME may be of
    the form (DEFINER NAME). In that case, instead of DEFUN, DEFINER
    is used to establish the underlying function binding.

    Loaddef: The corresponding loaddef is an AUTOLOAD form.
    EXTRACT-LOADDEFS with PROCESS-ARGLIST T installs LAMBDA-LIST as the
    ARGLIST. If PROCESS-ARGLIST is NIL, then ARGLIST will not be passed
    to AUTOLOAD.

- [macro] DEFGENERIC/AUTO NAME LAMBDA-LIST &BODY OPTIONS

    A shorthand for ( DEFUN/AUTO (DEFGENERIC NAME) ...).

### Classes

- [macro] AUTOLOAD-CLASS CLASS-NAME SYSTEM-NAME &KEY DOCSTRING (METACLASS 'STANDARD-CLASS)

    This is the loaddef for autodef DEFCLASS/AUTO.

    Define a dummy class with CLASS-NAME and arrange for SYSTEM-NAME to
    be loaded when the class or any of its
    subclasses are instantiated. Returns the
    class object. The arguments are not evaluated. If CLASS-NAME
    denotes a CLASS and it is not LOADDEF-CLASS-P,
    then it does nothing and returns NIL.

    When AUTOLOAD-CLASS is macroexpanded during the compilation or
    loading of an AUTOLOAD-SYSTEM, it signals an AUTOLOAD-WARNING if
    SYSTEM-NAME is not among those declared in :AUTO-DEPENDS-ON.

    - DOCSTRING, if non-NIL, will be the stub's docstring. If NIL, then
      a generic docstring that says what system it autoloads will be
      used.

    - METACLASS is a symbol denoting STANDARD-CLASS or a subclass of it.
      Also, classes with this metaclass must be allowed to inherit from
      standard classes. In MOP terms, CLOSER-MOP:VALIDATE-SUPERCLASS
      must return true when called with an instance of METACLASS and an
      instance of STANDARD-CLASS.

    The dummy class is also defined at compile time to
    approximate the semantics of DEFCLASS. It class has METACLASS with a
    single superclass and no slots. These are visible through
    introspection (e.g. via CLOSER-MOP:CLASS-DIRECT-SUPERCLASSES), which
    does not trigger autoloading.

    Note that INITIALIZE-INSTANCE :AROUND methods specialized on a
    subclass of CLASS-NAME may run twice in the context of the
    MAKE-INSTANCE that triggers autoloading.

- [function] LOADDEF-CLASS-P NAME

    See if an AUTOLOAD-CLASS for NAME was established, and since then
    it has not been redefined (e.g. with DEFCLASS/AUTO, DEFCLASS) or
    deleted (with (SETF (FIND-CLASS ...) NIL)). Subclasses do not
    inherit this property.

- [macro] DEFCLASS/AUTO NAME DIRECT-SUPERCLASSES DIRECT-SLOTS &REST OPTIONS

    This is the autodef for the loaddef AUTOLOAD-CLASS.

    Like DEFCLASS. NAME may be of the form (DEFINER NAME). In that
    case, instead of DEFCLASS, DEFINER is used to establish the
    underlying class definition.

    Loaddef: The corresponding loaddef is an AUTOLOAD-CLASS form.
    Note that the metaclass of the class NAME must already be defined
    when the loaddef is evaluated.

### Variables

- [function] LOADDEF-VARIABLE-P NAME

    See if a loaddef was generated from a
    DEFVAR/AUTO for NAME, but this autodef has not been evaluated.

- [macro] DEFVAR/AUTO VAR &OPTIONAL (VAL NIL) DOC

    This is an autodef with no public loaddef. See below.

    Unlike DEFVAR, this works with the global binding on Lisps that
    support it (currently Allegro, CCL, ECL, SBCL). This is to handle
    the case when a system that uses DEFVAR with a default value is
    autoloaded while that variable is locally bound:

        ;; Some base system only foreshadows *X*.
        (declaim (special *x*))
        (let ((*x* 1))
          ;; Imagine that the system that defines *X* is autoloaded here.
          (defvar/auto *x* 2)
          *x*)
        => 1

    Loaddef: The corresponding loaddef is not public and must be
    generated. The generated loaddef declaims
    the variable special and maybe sets its initial value and docstring.
    If the initial value form in DEFVAR/AUTO is detected as a simple
    constant form, then it is evaluated and its value is assigned to the
    variable as in DEFVAR. Simple constant forms are strings, numbers,
    characters, keywords, constants in the CL package, and QUOTEd nested
    lists containing any of the previous or any symbol from the CL
    package and other packages for which loaddefs have been generated in
    the same EXTRACT-LOADDEFS call (see DEFPACKAGE/AUTO)).

    In case the global binding of VAR has been set between the
    corresponding loaddef and its first autodef, VAL is evaluated for
    side effect.

### Packages

- [function] LOADDEF-PACKAGE-P NAME

    See if a loaddef was generated from a
    DEFPACKAGE/AUTO for NAME, but this autodef has not been evaluated
    nor has the package been deleted.

- [macro] DEFPACKAGE/AUTO NAME &REST OPTIONS

    This is an autodef with no public loaddef. See below.

    Unlike DEFPACKAGE, if the package is already defined,
    DEFPACKAGE/AUTO extends it additively. The additivity means that
    instead of replacing the package definition or signalling errors on
    redefinition, it expands into individual package-altering operations
    such as SHADOW, USE-PACKAGE and EXPORT. This allows the package
    state to be built incrementally, but it also means that
    the (DEFINER NAME) syntax cannot be supported. DEFPACKAGE/AUTO is
    idempotent.

    In addition, DEFPACKAGE/AUTO deviates from DEFPACKAGE in the
    following ways.

    - The default :USE list is empty.

    - :SIZE is not supported.

    - Implementation-specific extensions such as :LOCAL-NICKNAMES are
      not supported. Use ADD-PACKAGE-LOCAL-NICKNAME after the
      DEFPACKAGE/AUTO.

    Loaddef: The corresponding loaddef is not public and must be
    generated. As in the expansion of
    DEFPACKAGE/AUTO itself, the generated operations are additive.

    - The generated loaddef reconstructs the package states as they
      exist after all :AUTO-DEPENDS-ON systems are loaded. Thus, manual
      modifications after the DEFPACKAGE/AUTO definition (e.g. by
      additional EXPORTs) are reflected in the loaddef.

    - To handle circular dependencies, the loaddefs of all autodef
      packages and those passed in the PACKAGES argument to
      EXTRACT-LOADDEFS are generated in an interleaved manner. First,
      all packages are created, then their state is reconstructed in
      phases following DEFPACKAGE.

    - Any reference to non-existent packages (e.g. in :USE) or symbols
      in non-existent packages (e.g. :IMPORT-FROM) is silently skipped
      when the loaddef is evaluated.

    Instead of DEFPACKAGE/AUTO, one may use, for example, DEFPACKAGE or
    UIOP:DEFINE-PACKAGE and arrange for Automatically Generating Loaddefs for the
    package by listing it in :PACKAGES of :AUTO-LOADDEFS.

## ASDF Integration

- [class] AUTOLOAD-SYSTEM SYSTEM

    Inheriting from this class in an ASDF:DEFSYSTEM
    form enables the features documented in the reader methods. Consider
    the following example.

        (asdf:defsystem "my-system"
          :defsystem-depends-on ("autoload")
          :class "autoload:autoload-system"
          :auto-depends-on ("dyndep")
          :auto-loaddefs "loaddefs.lisp"
          :components ((:file "package")
                       (:file "loaddefs")
                       ...))

    With the above,

    - It is an error if an AUTOLOAD refers to a system other than
      dyndep.

    - (RECORD-LOADDEFS "my-system") will update
      loaddefs.lisp.

    - (ASDF:TEST-SYSTEM "my-system") checks that
      loaddefs.lisp is up-to-date.

    If the package definitions are also generated with
    RECORD-LOADDEFS (e.g. because there is a
    DEFPACKAGE/AUTO in dyndep or :AUTO-LOADDEFS specifies
    :PACKAGES), then we can do without the package.lisp file:

        (asdf:defsystem "my-system"
          :defsystem-depends-on ("autoload")
          :class "autoload:autoload-system"
          :auto-depends-on ("dyndep")
          :auto-loaddefs ("loaddefs.lisp" :packages #:my-pkg)
          :components ((:file "loaddefs")
                       ...))

- [class] AUTOLOAD-CL-SOURCE-FILE CL-SOURCE-FILE

    This is the :DEFAULT-COMPONENT-CLASS of
    AUTOLOAD-SYSTEM. ASDF Integration relies on source files belonging to
    this class. When combining autoload with another ASDF extension that
    has its own ASDF:CL-SOURCE-FILE subclass, define a new class that
    inherits from both, and use that as :DEFAULT-COMPONENT-CLASS.

- [reader] SYSTEM-AUTO-DEPENDS-ON AUTOLOAD-SYSTEM (:AUTO-DEPENDS-ON = NIL)

    This is the list of the names of systems that this
    system may autoload. The names are canonicalized with
    ASDF:COERCE-NAME. It is an AUTOLOAD-WARNING if a loaddef refers
    to a system not listed here. This is also used by
    EXTRACT-LOADDEFS and affects the checks performed by
    Loading Systems.

- [reader] SYSTEM-AUTO-LOADDEFS AUTOLOAD-SYSTEM (:AUTO-LOADDEFS = NIL)

    When non-NIL, this specifies arguments for
    Automatically Generating Loaddefs. It may be a single pathname designator or a
    list of the form

        (loaddefs-file &key (process-arglist t) (process-docstring t)
                            packages (test t))

    - LOADDEFS-FILE designates the pathname where RECORD-LOADDEFS writes the extracted loaddefs.
      The pathname is relative to ASDF:SYSTEM-SOURCE-DIRECTORY of
      SYSTEM and is OPENed with :IF-EXISTS :SUPERSEDE.

    - PROCESS-ARGLIST, PROCESS-DOCSTRING and PACKAGES
      are passed on by RECORD-LOADDEFS to EXTRACT-LOADDEFS.

    - If TEST, then CHECK-LOADDEFS is run by ASDF:TEST-SYSTEM.

    Conditions signalled while ASDF is compiling or loading the file
    given have a RECORD-LOADDEFS restart.

- [function] AUTODEPS SYSTEM &KEY (CROSS-AUTOLOADED T) INSTALLER

    Return the list of system names that may be autoloaded by SYSTEM or
    any of its direct or indirect dependencies. This recursively visits
    systems in the dependency tree, traversing both normal (:DEPENDS-ON)
    and autoloaded (:AUTO-DEPENDS-ON) dependencies. It works even if
    SYSTEM is not an AUTOLOAD-SYSTEM.

    - CROSS-AUTOLOADED controls whether systems only reachable from
      SYSTEM via intermediate autoloaded dependencies are visited. Thus,
      if CROSS-AUTOLOADED is NIL, then the returned list is the first
      boundary of autoloaded systems.

    - If INSTALLER is non-NIL, it is called when an autoloaded system
      that is not installed (i.e. ASDF:FIND-SYSTEM fails) is visited.
      INSTALLER is passed a single argument, the name of the system to
      be installed. It may or may not install the system.

    If an autoloaded system is not installed (i.e. ASDF:FIND-SYSTEM
    fails, even after INSTALLER had a chance), then its dependencies are
    unknown and cannot be traversed. Note that autoloaded systems that
    are not installed are still visited and included in the returned
    list.

    The following example makes sure that all autoloaded dependencies
    (direct or indirect) of my-system are installed:

        (autodeps "my-system" :installer #'ql:quickload)

### Automatically Generating Loaddefs

- [function] EXTRACT-LOADDEFS SYSTEM &KEY (PROCESS-ARGLIST T) (PROCESS-DOCSTRING T) PACKAGES

    List the loaddef forms of the autodef definitions in
    :AUTO-DEPENDS-ON of SYSTEM.

    There is rarely a need to call this function directly, as
    RECORD-LOADDEFS and CHECK-LOADDEFS provide
    ASDF Integration.

    Note that this is an expensive operation, as it loads or reloads the
    direct dependencies listed in :AUTO-DEPENDS-ON one by one with
    ASDF:LOAD-SYSTEM :FORCE T to find the autodefs.

    See the individual autodefs for descriptions of the generated
    loaddefs.

    - If PROCESS-DOCSTRING, then the docstrings extracted from autodef
      definitions will be associated with the definition.

    Note that if a definition is not made with an autodef, then
    EXTRACT-LOADDEFS will not detect it. For such functions, loaddefs
    must be written manually.

- [function] WRITE-LOADDEFS LOADDEFS STREAM

    Write LOADDEFS to STREAM so they can be LOADed or included in an
    ASDF:DEFSYSTEM.

- [function] RECORD-LOADDEFS SYSTEM

    EXTRACT-LOADDEFS from SYSTEM and WRITE-LOADDEFS. The arguments of
    these functions are taken from SYSTEM's :AUTO-LOADDEFS.

    As EXTRACT-LOADDEFS loads the direct autoloaded dependencies,
    compiler warnings (e.g. about undefined specials and functions) may
    occur that go away once the generated loaddefs are in place. The
    easiest way to trigger this is to call RECORD-LOADDEFS before these
    dependencies have been loaded. In this case, temporarily emptying
    the loaddefs file and fixing these warnings is recommended.

    RECORD-LOADDEFS may also be used as a condition handler, in
    which case it invokes the RECORD-LOADDEFS restart.

- [function] CHECK-LOADDEFS SYSTEM &KEY (ERRORP T)

    In the AUTOLOAD-SYSTEM SYSTEM, check that both recorded and manual
    autoload declarations are correct.

    - If :AUTO-LOADDEFS is specified, check that the file generated by
      RECORD-LOADDEFS is up-to-date.

    - Check that all manual (non-generated) loaddefs in SYSTEM are
      resolved (e.g. no longer LOADDEF-FUNCTION-P) by loading
      :AUTO-DEPENDS-ON.

    If ERRORP, then signal an error if a check fails or the loaddefs
    file cannot be read. If :AUTO-LOADDEFS is specified, then the
    RECORD-LOADDEFS restart is provided.

    If ERRORP is NIL, then instead of signalling an error, return NIL.

    This function is called automatically by ASDF:TEST-OP on an
    AUTOLOAD-SYSTEM method if :AUTO-LOADDEFS has :TEST T.

- [restart] RECORD-LOADDEFS

    Provided by CHECK-LOADDEFS and also when the compilation of the
    loaddefs file declared in :AUTO-LOADDEFS fails. The function
    RECORD-LOADDEFS can be used as a condition handler to invoke this
    restart.

* * *
###### \[generated by [MGL-PAX](https://github.com/melisgl/mgl-pax)\]
