This is sbcl.info, produced by gmakeinfo version 7.1 from sbcl.texinfo.

     This manual, for SBCL version 2.6.5.openbsd.sbcl-2.6.5, is part of
     the SBCL software system.  See the ‘README’ file for more
     information.

     This manual is largely derived from the manual for the CMUCL
     system, which was produced at Carnegie Mellon University and later
     released into the public domain.  This manual is in the public
     domain and is provided with absolutely no warranty.  See the
     ‘COPYING’ and ‘CREDITS’ files for more information.
INFO-DIR-SECTION Software development
START-INFO-DIR-ENTRY
* sbcl: (sbcl).           The Steel Bank Common Lisp compiler
END-INFO-DIR-ENTRY


File: sbcl.info,  Node: Top,  Next: Getting Support and Reporting Bugs,  Up: (dir)

sbcl
****

     This manual, for SBCL version 2.6.5.openbsd.sbcl-2.6.5, is part of
     the SBCL software system.  See the ‘README’ file for more
     information.

     This manual is largely derived from the manual for the CMUCL
     system, which was produced at Carnegie Mellon University and later
     released into the public domain.  This manual is in the public
     domain and is provided with absolutely no warranty.  See the
     ‘COPYING’ and ‘CREDITS’ files for more information.

* Menu:

* Getting Support and Reporting Bugs::
* Introduction::
* Starting and Stopping::
* Compiler::
* Debugger::
* Efficiency::
* Beyond the ANSI Standard::
* External Formats::
* Foreign Function Interface::
* Pathnames::
* Streams::
* Package Locks::
* Threading::
* Timers::
* Networking::
* Profiling::
* Contributed Modules::
* Deprecation::
* Concept Index::
* Function Index::
* Variable Index::
* Type Index::
* Colophon::


File: sbcl.info,  Node: Getting Support and Reporting Bugs,  Next: Introduction,  Prev: Top,  Up: Top

1 Getting Support and Reporting Bugs
************************************

* Menu:

* Volunteer Support::
* Commercial Support::
* Reporting Bugs::


File: sbcl.info,  Node: Volunteer Support,  Next: Commercial Support,  Up: Getting Support and Reporting Bugs

1.1 Volunteer Support
=====================

Your primary source of SBCL support should probably be the mailing list
*sbcl-help*: in addition to other users SBCL developers monitor this
list and are available for advice.  As an anti-spam measure subscription
is required for posting:

       <https://lists.sourceforge.net/lists/listinfo/sbcl-help>

   Remember that the people answering your question are volunteers, so
you stand a much better chance of getting a good answer if you ask a
good question.

   Before sending mail, check the list archives at either

      
<http://sourceforge.net/mailarchive/forum.php?forum_name=sbcl-help>

   or

       <http://news.gmane.org/gmane.lisp.steel-bank.general>

   to see if your question has been answered already.  Checking the bug
database is also worth it *Note Reporting Bugs::, to see if the issue is
already known.

   For general advice on asking good questions, see

       <http://www.catb.org/~esr/faqs/smart-questions.html>.


File: sbcl.info,  Node: Commercial Support,  Next: Reporting Bugs,  Prev: Volunteer Support,  Up: Getting Support and Reporting Bugs

1.2 Commercial Support
======================

There is no formal organization developing SBCL, but if you need a paid
support arrangement or custom SBCL development, we maintain the list of
companies and consultants below.  Use it to identify service providers
with appropriate skills and interests, and contact them directly.

   The SBCL project cannot verify the accuracy of the information or the
competence of the people listed, and they have provided their own blurbs
below: you must make your own judgement of suitability from the
available information - refer to the links they provide, the CREDITS
file, mailing list archives, CVS commit messages, and so on.  Please
feel free to ask for advice on the sbcl-help list.

   (At present, no companies or consultants wish to advertise paid
support or custom SBCL development in this manual).


File: sbcl.info,  Node: Reporting Bugs,  Prev: Commercial Support,  Up: Getting Support and Reporting Bugs

1.3 Reporting Bugs
==================

SBCL uses Launchpad to track bugs.  The bug database is available at

       <https://bugs.launchpad.net/sbcl>

   Reporting bugs there requires registering at Launchpad.  However,
bugs can also be reported on the mailing list *sbcl-bugs*, which is
moderated but does _not_ require subscribing.

   Simply send email to <sbcl-bugs@lists.sourceforge.net> and the bug
will be checked and added to Launchpad by SBCL maintainers.

1.3.1 How to Report Bugs Effectively
------------------------------------

Please include enough information in a bug report that someone reading
it can reproduce the problem, i.e.  don't write

     Subject: apparent bug in PRINT-OBJECT (or *PRINT-LENGTH*?)
     PRINT-OBJECT doesn't seem to work with *PRINT-LENGTH*. Is this a bug?

   but instead

     Subject: apparent bug in PRINT-OBJECT (or *PRINT-LENGTH*?)
     In sbcl-1.2.3 running under OpenBSD 4.5 on my Alpha box, when
     I compile and load the file
        (DEFSTRUCT (FOO (:PRINT-OBJECT (LAMBDA (X Y)
                                         (LET ((*PRINT-LENGTH* 4))
                                           (PRINT X Y)))))
          X Y)
     then at the command line type
        (MAKE-FOO)
     the program loops endlessly instead of printing the object.

   A more in-depth discussion on reporting bugs effectively can be found
at

       <http://www.chiark.greenend.org.uk/~sgtatham/bugs.html>.

1.3.2 Signal Related Bugs
-------------------------

If you run into a signal related bug, you are getting fatal errors such
as ‘signal N is [un]blocked’ or just hangs, and you want to send a
useful bug report then:

  1. Compile SBCL with ldb enabled (feature ‘:sb-ldb’, see
     ‘base-target-features.lisp-expr’).

  2. Isolate a smallish test case, run it.

  3. If it just hangs kill it with sigabrt: ‘kill -ABRT <pidof sbcl>’.

  4. Print the backtrace from ldb by typing ‘ba’.

  5. Attach gdb: ‘gdb -p <pidof sbcl>’ and get backtraces for all
     threads: ‘thread apply all ba’.

  6. If multiple threads are in play then still in gdb, try to get Lisp
     backtrace for all threads: ‘thread apply all call
     backtrace_from_fp($ebp, 100, 0)’.  Substitute ‘$ebp’ with ‘$rbp’ on
     x86-64.  The backtraces will appear in the stdout of the SBCL
     process.

  7. Send a report with the backtraces and the output (both stdout and
     stderr) produced by SBCL.

  8. Don't forget to include OS and SBCL version.

  9. If available, include information on outcome of the same test with
     other versions of SBCL, OS, ...


File: sbcl.info,  Node: Introduction,  Next: Starting and Stopping,  Prev: Getting Support and Reporting Bugs,  Up: Top

2 Introduction
**************

SBCL is a mostly-conforming implementation of the ANSI Common Lisp
standard.  This manual focuses on behavior which is specific to SBCL,
not on behavior which is common to all implementations of ANSI Common
Lisp.

* Menu:

* ANSI Conformance::
* Extensions::
* Idiosyncrasies::
* Development Tools::
* More SBCL Information::
* More Common Lisp Information::
* History and Implementation of SBCL::


File: sbcl.info,  Node: ANSI Conformance,  Next: Extensions,  Up: Introduction

2.1 ANSI Conformance
====================

Essentially every type of non-conformance is considered a bug.  (The
exceptions involve internal inconsistencies in the standard.)  *Note
Reporting Bugs::.

2.1.1 Exceptions
----------------

   • ‘prog2’ returns the primary value of its second form, as specified
     in the *Arguments and Values* section of the specification for that
     operator, not that of its first form, as specified in the
     *Description*.

   • The ‘string’ type is considered to be the union of all types
     ‘(array _c_ (_size_))’ for all non-‘nil’ subtypes ‘_c_’ of
     ‘character’, excluding arrays specialized to the empty type.

   • The ‘:order’ long form option in ‘define-method-combination’ method
     group specifiers accepts the value ‘nil’ as well as
     ‘:most-specific-first’ and ‘:most-specific-last’, in order to allow
     programmers to declare that the order of methods playing that role
     in the method combination does not matter.


File: sbcl.info,  Node: Extensions,  Next: Idiosyncrasies,  Prev: ANSI Conformance,  Up: Introduction

2.2 Extensions
==============

SBCL comes with numerous extensions, some in core and some in modules
loadable with ‘require’.  Unfortunately, not all of these extensions
have proper documentation yet.

*System Definition Tool*
     ‘asdf’ is a flexible and popular protocol-oriented system
     definition tool by Daniel Barlow.  *note the asdf manual:
     (asdf)Top, for more information.

*Foreign Function Interface*
     ‘sb-alien’ package allows interfacing with C-code, loading shared
     object files, etc.  *Note Foreign Function Interface::.

     ‘sb-grovel’ can be used to partially automate generation of foreign
     function interface definitions.  *Note sb-grovel::.

*Recursive Event Loop*
     SBCL provides a recursive event loop (‘serve-event’) for doing
     non-blocking IO on multiple streams without using threads.

*Timeouts and Deadlines*
     SBCL allows restricting the execution time of individual operations
     or parts of a computation using ‘:timeout’ arguments to certain
     blocking operations, synchronous timeouts and asynchronous
     timeouts.  The latter two affect operations without explicit
     timeout support (such as standard functions and macros).  *Note
     Timeouts and Deadlines::.

*Metaobject Protocol*
     ‘sb-mop’ package provides a metaobject protocol for the Common Lisp
     Object System as described in ‘Art of Metaobject Protocol’.

*Extensible Sequences*
     SBCL allows users to define subclasses of the ‘sequence’ class.
     *Note Extensible Sequences::.

*Native Threads*
     SBCL has native threads on x86/Linux, capable of taking advantage
     of SMP on multiprocessor machines.  *Note Threading::.

*Network Interface*
     ‘sb-bsd-sockets’ is a low-level networking interface, providing
     both TCP and UDP sockets.  *Note Networking::.

*Introspective Facilities*
     ‘sb-introspect’ module offers numerous introspective extensions,
     including access to function lambda-lists and a cross referencing
     facility.

*Operating System Interface*
     ‘sb-ext’ contains a number of functions for running external
     processes, accessing environment variables, etc.

     ‘sb-posix’ module provides a lispy interface to standard POSIX
     facilities.

*Extensible Streams*
     ‘sb-gray’ is an implementation of _Gray Streams_.  *Note Gray
     Streams::.

     ‘sb-simple-streams’ is an implementation of the _simple streams_
     API proposed by Franz Inc.  *Note Simple Streams::.

*Profiling*
     ‘sb-profile’ is a exact per-function profiler.  *Note Deterministic
     Profiler::.

     ‘sb-sprof’ is a statistical profiler, capable of call-graph
     generation and instruction level profiling, which also supports
     allocation profiling.  *Note Statistical Profiler::.

*Customization Hooks*
     SBCL contains a number of extra-standard customization hooks that
     can be used to tweak the behaviour of the system.  *Note
     Customization Hooks for Users::.

     ‘sb-aclrepl’ provides an Allegro CL -style toplevel for SBCL, as an
     alternative to the classic CMUCL-style one.  *Note sb-aclrepl::.

*CLTL2 Compatibility Layer*
     ‘sb-cltl2’ module provides ‘compiler-let’ and environment access
     functionality described in ‘Common Lisp The Language, 2nd Edition’
     which were removed from the language during the ANSI
     standardization process.

*Executable Delivery*
     The ‘:executable’ argument to *note Function sb-ext
     save-lisp-and-die:: can produce a 'standalone' executable
     containing both an image of the current Lisp session and an SBCL
     runtime.

*Bitwise Rotation*
     ‘sb-rotate-byte’ provides an efficient primitive for bitwise
     rotation of integers, an operation required by e.g.  numerous
     cryptographic algorithms, but not available as a primitive in ANSI
     Common Lisp.  *Note sb-rotate-byte::.

*Test Harness*
     ‘sb-rt’ module is a simple yet attractive regression and unit-test
     framework.

*MD5 Sums*
     ‘sb-md5’ is an implementation of the MD5 message digest algorithm
     for Common Lisp, using the modular arithmetic optimizations
     provided by SBCL. *Note sb-md5::.


File: sbcl.info,  Node: Idiosyncrasies,  Next: Development Tools,  Prev: Extensions,  Up: Introduction

2.3 Idiosyncrasies
==================

The information in this section describes some of the ways that SBCL
deals with choices that the ANSI standard leaves to the implementation.

* Menu:

* Declarations::
* FASL Format::
* Compiler-only Implementation::
* Defining Constants::
* Style Warnings::


File: sbcl.info,  Node: Declarations,  Next: FASL Format,  Up: Idiosyncrasies

2.3.1 Declarations
------------------

Declarations are generally treated as assertions.  This general
principle, and its implications, and the bugs which still keep the
compiler from quite satisfying this principle, are discussed in *note
Declarations as Assertions::.


File: sbcl.info,  Node: FASL Format,  Next: Compiler-only Implementation,  Prev: Declarations,  Up: Idiosyncrasies

2.3.2 FASL Format
-----------------

SBCL fasl-format is binary compatible only with the exact SBCL version
it was generated with.  While this is obviously suboptimal, it has
proven more robust than trying to maintain fasl compatibility across
versions: accidentally breaking things is far too easy, and can lead to
hard to diagnose bugs.

   The following snippet handles fasl recompilation automatically for
ASDF-based systems, and makes a good candidate for inclusion in the user
or system initialization file (*note Initialization Files::.)

     (require :asdf)

     ;;; If a fasl was stale, try to recompile and load (once).
     (defmethod asdf:perform :around ((o asdf:load-op)
                                      (c asdf:cl-source-file))
        (handler-case (call-next-method o c)
           ;; If a fasl was stale, try to recompile and load (once).
           (sb-ext:invalid-fasl ()
              (asdf:perform (make-instance 'asdf:compile-op) c)
              (call-next-method))))


File: sbcl.info,  Node: Compiler-only Implementation,  Next: Defining Constants,  Prev: FASL Format,  Up: Idiosyncrasies

2.3.3 Compiler-only Implementation
----------------------------------

SBCL is essentially a compiler-only implementation of Common Lisp.  That
is, for all but a few special cases, ‘eval’ creates a lambda expression,
calls ‘compile’ on the lambda expression to create a compiled function,
and then calls ‘funcall’ on the resulting function object.  A more
traditional interpreter is also available on default builds; it is
usually only called internally.  This is explicitly allowed by the ANSI
standard, but leads to some oddities; e.g.  at default settings,
‘functionp’ and ‘compiled-function-p’ are equivalent, and they collapse
into the same function when SBCL is built without the interpreter.


File: sbcl.info,  Node: Defining Constants,  Next: Style Warnings,  Prev: Compiler-only Implementation,  Up: Idiosyncrasies

2.3.4 Defining Constants
------------------------

SBCL is quite strict about ANSI's definition of ‘defconstant’.  ANSI
says that doing ‘defconstant’ of the same symbol more than once is
undefined unless the new value is ‘eql’ to the old value.  Conforming to
this specification is a nuisance when the "constant" value is only
constant under some weaker test like ‘string=’ or ‘equal’.

   It's especially annoying because, in SBCL, ‘defconstant’ takes effect
not only at load time but also at compile time, so that just compiling
and loading reasonable code like
     (defconstant +foobyte+ '(1 4))
   runs into this undefined behavior.  Many implementations of Common
Lisp try to help the programmer around this annoyance by silently
accepting the undefined code and trying to do what the programmer
probably meant.

   SBCL instead treats the undefined behavior as an error.  Often such
code can be rewritten in portable ANSI Common Lisp which has the desired
behavior.  E.g., the code above can be given an exactly defined meaning
by replacing ‘defconstant’ either with ‘defparameter’ or with a
customized macro which does the right thing, e.g.
     (defmacro define-constant (name value &optional doc)
       `(defconstant ,name (if (boundp ',name) (symbol-value ',name) ,value)
                           ,@(when doc (list doc))))
   or possibly along the lines of the ‘defconstant-eqx’ macro used
internally in the implementation of SBCL itself.  In circumstances where
this is not appropriate, the programmer can handle the condition type
‘sb-ext:defconstant-uneql’, and choose either the ‘continue’ or ‘abort’
restart as appropriate.


File: sbcl.info,  Node: Style Warnings,  Prev: Defining Constants,  Up: Idiosyncrasies

2.3.5 Style Warnings
--------------------

SBCL gives style warnings about various kinds of perfectly legal code,
e.g.

   • multiple ‘defun’s of the same symbol in different units;

   • special variables not named in the conventional ‘*foo*’ style, and
     lexical variables unconventionally named in the ‘*foo*’ style

   This causes friction with people who point out that other ways of
organizing code (especially avoiding the use of ‘defgeneric’) are just
as aesthetically stylish.  However, these warnings should be read not as
"warning, bad aesthetics detected, you have no style" but "warning, this
style keeps the compiler from understanding the code as well as you
might like."  That is, unless the compiler warns about such conditions,
there's no way for the compiler to warn about some programming errors
which would otherwise be easy to overlook.  (Related bug: The warning
about multiple ‘defun’s is pointlessly annoying when you compile and
then load a function containing ‘defun’ wrapped in ‘eval-when’, and
ideally should be suppressed in that case, but still isn't as of SBCL
0.7.6.)


File: sbcl.info,  Node: Development Tools,  Next: More SBCL Information,  Prev: Idiosyncrasies,  Up: Introduction

2.4 Development Tools
=====================

* Menu:

* Editor Integration::
* Language Reference::
* Generating Executables::


File: sbcl.info,  Node: Editor Integration,  Next: Language Reference,  Up: Development Tools

2.4.1 Editor Integration
------------------------

Though SBCL can be used running "bare", the recommended mode of
development is with an editor connected to SBCL, supporting not only
basic lisp editing (paren-matching, etc), but providing among other
features an integrated debugger, interactive compilation, and automated
documentation lookup.

   Currently “SLIME”(1) (Superior Lisp Interaction Mode for Emacs)
together with Emacs is recommended for use with SBCL, though other
options exist as well.

   SLIME can be downloaded from <https://slime.common-lisp.dev/>.

   ---------- Footnotes ----------

   (1) Historically, the ILISP package at <http://ilisp.cons.org/>
provided similar functionality, but it does not support modern SBCL
versions.


File: sbcl.info,  Node: Language Reference,  Next: Generating Executables,  Prev: Editor Integration,  Up: Development Tools

2.4.2 Language Reference
------------------------

“CLHS” (Common Lisp Hyperspec) is a hypertext version of the ANSI
standard, made freely available by _LispWorks_ - an invaluable
reference.

   See:
<https://www.lispworks.com/documentation/HyperSpec/Front/index.htm>


File: sbcl.info,  Node: Generating Executables,  Prev: Language Reference,  Up: Development Tools

2.4.3 Generating Executables
----------------------------

SBCL can generate stand-alone executables.  The generated executables
include the SBCL runtime itself, so no restrictions are placed on
program functionality.  For example, a deployed program can call
‘compile’ and ‘load’, which requires the compiler to be present in the
executable.  For further information, *Note Function sb-ext
save-lisp-and-die::.


File: sbcl.info,  Node: More SBCL Information,  Next: More Common Lisp Information,  Prev: Development Tools,  Up: Introduction

2.5 More SBCL Information
=========================

* Menu:

* SBCL Homepage::
* Online Documentation::
* Additional Documentation Files::
* Internals Documentation::


File: sbcl.info,  Node: SBCL Homepage,  Next: Online Documentation,  Up: More SBCL Information

2.5.1 SBCL Homepage
-------------------

The SBCL website at <http://www.sbcl.org/> has some general information,
plus links to mailing lists devoted to SBCL, and to archives of these
mailing lists.  Subscribing to the mailing lists ‘sbcl-help’ and
‘sbcl-announce’ is recommended: both are fairly low-volume, and help you
keep abreast with SBCL development.


File: sbcl.info,  Node: Online Documentation,  Next: Additional Documentation Files,  Prev: SBCL Homepage,  Up: More SBCL Information

2.5.2 Online Documentation
--------------------------

Documentation for non-ANSI extensions for various commands is available
online from the SBCL executable itself.  The extensions for functions
which have their own command prompts (e.g.  the debugger, and ‘inspect’)
are documented in text available by typing ‘help’ at their command
prompts.  The extensions for functions which don't have their own
command prompt (such as ‘trace’) are described in their documentation
strings, unless your SBCL was compiled with an option not to include
documentation strings, in which case the documentation strings are only
readable in the source code.


File: sbcl.info,  Node: Additional Documentation Files,  Next: Internals Documentation,  Prev: Online Documentation,  Up: More SBCL Information

2.5.3 Additional Documentation Files
------------------------------------

Besides this user manual both SBCL source and binary distributions
include some other SBCL-specific documentation files, which should be
installed along with this manual on your system, e.g.  in
‘/usr/local/share/doc/sbcl/’.

‘COPYING’
     Licence and copyright summary.

‘CREDITS’
     Authorship information on various parts of SBCL.

‘INSTALL’
     Covers installing SBCL from both source and binary distributions on
     your system, and also has some installation related troubleshooting
     information.

‘NEWS’
     Summarizes changes between various SBCL versions.


File: sbcl.info,  Node: Internals Documentation,  Prev: Additional Documentation Files,  Up: More SBCL Information

2.5.4 Internals Documentation
-----------------------------

If you're interested in the development of the SBCL system itself, then
subscribing to ‘sbcl-devel’ is a good idea.

   SBCL internals documentation - besides comments in the source - is
available in the Web Archive
(https://web.archive.org/web/20120814000933/http://sbcl-internals.cliki.net/index).

   Some low-level information describing the programming details of the
conversion from CMUCL to SBCL is available in the
‘doc/FOR-CMUCL-DEVELOPERS’ file in the SBCL distribution and on
SourceForge
(https://sourceforge.net/p/sbcl/sbcl/ci/master/tree/doc/FOR-CMUCL-DEVELOPERS).


File: sbcl.info,  Node: More Common Lisp Information,  Next: History and Implementation of SBCL,  Prev: More SBCL Information,  Up: Introduction

2.6 More Common Lisp Information
================================

* Menu:

* Internet Community::
* Third-party Libraries::
* Common Lisp Books::


File: sbcl.info,  Node: Internet Community,  Next: Third-party Libraries,  Up: More Common Lisp Information

2.6.1 Internet Community
------------------------

The Common Lisp internet community is fairly diverse:
<https://groups.google.com/g/comp.lang.lisp> is fairly high volume
newsgroup, but has a rather poor signal/noise ratio.  Various special
interest mailing lists and IRC tend to provide more content and less
flames.  <https://www.lisp.org> and <https://cliki.net> contain numerous
pointers places in the net where lispers talks shop.


File: sbcl.info,  Node: Third-party Libraries,  Next: Common Lisp Books,  Prev: Internet Community,  Up: More Common Lisp Information

2.6.2 Third-party Libraries
---------------------------

For a wealth of information about free Common Lisp libraries and tools
we recommend checking out _CLiki_: <https://cliki.net/>.


File: sbcl.info,  Node: Common Lisp Books,  Prev: Third-party Libraries,  Up: More Common Lisp Information

2.6.3 Common Lisp Books
-----------------------

If you're not a programmer and you're trying to learn, many introductory
Lisp books are available.  However, we don't have any standout
favorites.  If you can't decide, try checking the Usenet
<https://groups.google.com/g/comp.lang.lisp> FAQ for recent
recommendations.

   If you are an experienced programmer in other languages but need to
learn about Common Lisp, some books stand out:

‘Practical Common Lisp, by Peter Seibel’
     An excellent introduction to the language, covering both the basics
     and "advanced topics" like macros, CLOS, and packages.  Available
     both in print format and on the web:
     <https://gigamonkeys.com/book/>.

‘Paradigms Of Artificial Intelligence Programming, by Peter Norvig’
     Good information on general Common Lisp programming, and many
     nontrivial examples.  Whether or not your work is AI, it's a very
     good book to look at.

‘On Lisp, by Paul Graham’
     An in-depth treatment of macros, but not recommended as a first
     Common Lisp book, since it is slightly pre-ANSI so you need to be
     on your guard against non-standard usages, and since it doesn't
     really even try to cover the language as a whole, focusing solely
     on macros.  Downloadable from
     <https://www.paulgraham.com/onlisp.html>.

‘Object-Oriented Programming In Common Lisp, by Sonya Keene’
     With the exception of ‘Practical Common Lisp’ most introductory
     books don't emphasize CLOS. This one does.  Even if you're very
     knowledgeable about object oriented programming in the abstract,
     it's worth looking at this book if you want to do any OO in Common
     Lisp.  Some abstractions in CLOS (especially multiple dispatch) go
     beyond anything you'll see in most OO systems, and there are a
     number of lesser differences as well.  This book tends to help with
     the culture shock.

‘Art Of Metaobject Programming, by Gregor Kiczales et al.’
     Currently the prime source of information on the Common Lisp
     Metaobject Protocol, which is supported by SBCL. Section 2
     (Chapters 5 and 6) are freely available at
     <http://mop.lisp.se/www.alu.org/mop/>.


File: sbcl.info,  Node: History and Implementation of SBCL,  Prev: More Common Lisp Information,  Up: Introduction

2.7 History and Implementation of SBCL
======================================

You can work productively with SBCL without knowing or understanding
anything about where it came from, how it is implemented, or how it
extends the ANSI Common Lisp standard.  However, a little knowledge can
be helpful in order to understand error messages, to troubleshoot
problems, to understand why some parts of the system are better debugged
than others, and to anticipate which known bugs, known performance
problems, and missing extensions are likely to be fixed, tuned, or
added.

   SBCL is descended from CMUCL, which is itself descended from Spice
Lisp, including early implementations for the Mach operating system on
the IBM RT, back in the 1980s.  Some design decisions from that time are
still reflected in the current implementation:

   • The system expects to be loaded into a fixed-at-compile-time
     location in virtual memory, and also expects the location of all of
     its heap storage to be specified at compile time.

   • The system overcommits memory, allocating large amounts of address
     space from the system (often more than the amount of virtual memory
     available) and then failing if it ends up using too much of the
     allocated storage.

   • The system is implemented as a C program which is responsible for
     supplying low-level services and loading a Lisp ‘.core’ file.

   SBCL also inherited some newer architectural features from CMUCL. The
most important is that on some architectures it has a generational
garbage collector ("GC"), which has various implications (mostly good)
for performance.  These are discussed in another chapter, *note
Efficiency::.

   SBCL has diverged from CMUCL in that SBCL is now essentially a
"compiler-only implementation" of Common Lisp.  This is a change in
implementation strategy, taking advantage of the freedom "any of these
facilities might share the same execution strategy" guaranteed in the
ANSI specification section 3.1 ("Evaluation").  It does not mean SBCL
can't be used interactively, and in fact the change is largely invisible
to the casual user, since SBCL still can and does execute code
interactively by compiling it on the fly.  (It is visible if you know
how to look, like using ‘compiled-function-p’; and it is visible in the
way that SBCL doesn't have many bugs which behave differently in
interpreted code than in compiled code.)  What it means is that in SBCL,
the ‘eval’ function only truly "interprets" a few easy kinds of forms,
such as symbols which are ‘boundp’.  More complicated forms are
evaluated by calling ‘compile’ and then calling ‘funcall’ on the
returned result.

   The direct ancestor of SBCL is the x86 port of CMUCL. This port was
in some ways the most cobbled-together of all the CMUCL ports, since a
number of strange changes had to be made to support the register-poor
x86 architecture.  Some things (like tracing and debugging) do not work
particularly well there.  SBCL should be able to improve in these areas
(and has already improved in some other areas), but it takes a while.

   On the x86 SBCL - like the x86 port of CMUCL - uses a _conservative_
GC. This means that it doesn't maintain a strict separation between
tagged and untagged data, instead treating some untagged data (e.g.  raw
floating point numbers) as possibly-tagged data and so not collecting
any Lisp objects that they point to.  This has some negative
consequences for average time efficiency (though possibly no worse than
the negative consequences of trying to implement an exact GC on a
processor architecture as register-poor as the X86) and also has
potentially unlimited consequences for worst-case memory efficiency.  In
practice, conservative garbage collectors work reasonably well, not
getting anywhere near the worst case.  But they can occasionally cause
odd patterns of memory usage.

   The fork from CMUCL was based on a major rewrite of the system
bootstrap process.  CMUCL has for many years tolerated a very unusual
"build" procedure which doesn't actually build the complete system from
scratch, but instead progressively overwrites parts of a running system
with new versions.  This quasi-build procedure can cause various bizarre
bootstrapping hangups, especially when a major change is made to the
system.  It also makes the connection between the current source code
and the current executable more tenuous than in other software systems -
it's easy to accidentally "build" a CMUCL system containing
characteristics not reflected in the current version of the source code.

   Other major changes since the fork from CMUCL include

   • SBCL has removed many CMUCL extensions, (e.g.  IP networking,
     remote procedure call, Unix system interface, and X11 interface)
     from the core system.  Most of these are available as contributed
     modules (distributed with SBCL) or third-party modules instead.

   • SBCL has deleted or deprecated some nonstandard features and code
     complexity which helped efficiency at the price of maintainability.
     For example, the SBCL compiler no longer implements memory pooling
     internally (and so is simpler and more maintainable, but generates
     more garbage and runs more slowly).


File: sbcl.info,  Node: Starting and Stopping,  Next: Compiler,  Prev: Introduction,  Up: Top

3 Starting and Stopping
***********************

* Menu:

* Starting SBCL::
* Stopping SBCL::
* Command Line Options::
* Initialization Files::
* Initialization and Exit Hooks::


File: sbcl.info,  Node: Starting SBCL,  Next: Stopping SBCL,  Up: Starting and Stopping

3.1 Starting SBCL
=================

* Menu:

* Running from Shell::
* Running from Emacs::
* Shebang Scripts::


File: sbcl.info,  Node: Running from Shell,  Next: Running from Emacs,  Up: Starting SBCL

3.1.1 From Shell to Lisp
------------------------

To run SBCL type ‘sbcl’ at the command line.

   You should end up in the toplevel “REPL” (read, eval, print -loop),
where you can interact with SBCL by typing expressions.

     $ sbcl
     This is SBCL 0.8.13.60, an implementation of ANSI Common Lisp.
     More information about SBCL is available at <http://www.sbcl.org/>.

     SBCL is free software, provided as is, with absolutely no warranty.
     It is mostly in the public domain; some portions are provided under
     BSD-style licenses.  See the CREDITS and COPYING files in the
     distribution for more information.
     * (+ 2 2)

     4
     * (exit)
     $

   See also *note Command Line Options:: and *note Stopping SBCL::.


File: sbcl.info,  Node: Running from Emacs,  Next: Shebang Scripts,  Prev: Running from Shell,  Up: Starting SBCL

3.1.2 Running from Emacs
------------------------

To run SBCL as an inferior-lisp from Emacs in your ‘.emacs’ do something
like:

     ;;; The SBCL binary and command-line arguments
     (setq inferior-lisp-program "/usr/local/bin/sbcl --noinform")

   For more information on using SBCL with Emacs, see *note Editor
Integration::.


File: sbcl.info,  Node: Shebang Scripts,  Prev: Running from Emacs,  Up: Starting SBCL

3.1.3 Shebang Scripts
---------------------

Standard Unix tools that are interpreters follow a common command line
protocol that is necessary to work with "shebang scripts".  SBCL
supports this via the ‘--script’ command line option.

   Example file (‘hello.lisp’):

     #!/usr/local/bin/sbcl --script
     (write-line "Hello, World!")

   Usage examples:

     $ ./hello.lisp
     Hello, World!

     $ sbcl --script hello.lisp
     Hello, World!


File: sbcl.info,  Node: Stopping SBCL,  Next: Command Line Options,  Prev: Starting SBCL,  Up: Starting and Stopping

3.2 Stopping SBCL
=================

* Menu:

* Exit::
* End of File::
* Saving a Core Image::
* Exit on Errors::


File: sbcl.info,  Node: Exit,  Next: End of File,  Up: Stopping SBCL

3.2.1 Exit
----------

SBCL can be stopped at any time by calling ‘sb-ext:exit’, optionally
returning a specified numeric value to the calling process.  See *note
Threading:: for information about terminating individual threads.

 -- Function: exit &key code abort timeout
     Terminates the process, causing ‘sbcl’ to exit with ‘code’.  ‘code’
     defaults to 0 when ‘abort’ is false, and 1 when it is true.

     When ‘abort’ is false (the default), current thread is first
     unwound, ‘*exit-hooks*’ are run, other threads are terminated, and
     standard output streams are flushed before ‘sbcl’ calls exit(3)
     ‘--’ at which point atexit(3) functions will run.  If multiple
     threads call ‘exit’ with ‘abort’ being false, the first one to call
     it will complete the protocol.

     When ‘abort’ is true, ‘sbcl’ exits immediately by calling _exit(2)
     without unwinding stack, or calling exit hooks.  Note that _exit(2)
     does not call atexit(3) functions unlike exit(3).

     Recursive calls to ‘exit’ cause ‘exit’ to behave as if ‘abort’ was
     true.

     ‘timeout’ controls waiting for other threads to terminate when
     ‘abort’ is ‘nil’.  Once current thread has been unwound and
     ‘*exit-hooks*’ have been run, spawning new threads is prevented and
     all other threads are terminated by calling ‘terminate-thread’ on
     them.  The system then waits for them to finish using
     ‘join-thread’, waiting at most a total ‘timeout’ seconds for all
     threads to join.  Those threads that do not finish in time are
     simply ignored while the exit protocol continues.  ‘timeout’
     defaults to ‘*exit-timeout*’, which in turn defaults to 60.
     ‘timeout’ ‘nil’ means to wait indefinitely.

     Note that ‘timeout’ applies only to ‘join-thread’, not
     ‘*exit-hooks*’.  Since ‘terminate-thread’ is asynchronous, getting
     multithreaded application termination with complex cleanups right
     using it can be tricky.  To perform an orderly synchronous shutdown
     use an exit hook instead of relying on implicit thread termination.

     Consequences are unspecified if serious conditions occur during
     ‘exit’ excepting errors from ‘*exit-hooks*’, which cause warnings
     and stop execution of the hook that signaled, but otherwise allow
     the exit process to continue normally.


File: sbcl.info,  Node: End of File,  Next: Saving a Core Image,  Prev: Exit,  Up: Stopping SBCL

3.2.2 End of File
-----------------

By default SBCL also exits on end of input, caused either by user
pressing ‘Control-D’ on an attached terminal, or end of input when using
SBCL as part of a shell pipeline.


File: sbcl.info,  Node: Saving a Core Image,  Next: Exit on Errors,  Prev: End of File,  Up: Stopping SBCL

3.2.3 Saving a Core Image
-------------------------

SBCL has the ability to save its state as a file for later execution.
This functionality is important for its bootstrapping process, and is
also provided as an extension to the user.

 -- Function: save-lisp-and-die core-file-name &key toplevel executable
          save-runtime-options callable-exports purify root-structures
          environment-name compression
     Save a "core image", i.e.  enough information to restart a Lisp
     process later in the same state, in the file of the specified name.
     Only global state is preserved: the stack is unwound in the
     process.

     The following ‘&key’ arguments are defined:

     _‘:toplevel’_
          The function to run when the created core file is resumed.
          The default function handles command line toplevel option
          processing and runs the top level read-eval-print loop.  This
          function returning is equivalent to (‘sb-ext:exit’ ‘:code’ 0)
          being called.

          ‘toplevel’ functions should always provide an ‘abort’ restart:
          otherwise code they call will run without one.

     _‘:executable’_
          If true, arrange to combine the ‘sbcl’ runtime and the core
          image to create a standalone executable.  If false (the
          default), the core image will not be executable on its own.
          Executable images always behave as if they were passed the
          -noinform runtime option.  If ‘:executable’ is ‘:elf-object’,
          then the resulting core will be wrapped in a .o which requires
          further linking.  (‘experimental’)

     _‘:save-runtime-options’_
          If true, values of runtime options -dynamic-space-size and
          -control-stack-size that were used to start ‘sbcl’ are stored
          in the standalone executable, and restored when the executable
          is run.  This also inhibits normal runtime option processing,
          causing all command line arguments to be passed to the
          toplevel.  If ‘:accept-runtime-options’ then
          -dynamic-space-size and -control-stack-size are still
          processed by the runtime.  Meaningless if ‘:executable’ is
          ‘nil’.

     _‘:callable-exports’_
          This should be a list of symbols to be initialized to the
          appropriate alien callables on startup.  All exported symbols
          should be present as global symbols in the symbol table of the
          runtime before the saved core is loaded.  When this list is
          non-empty, the ‘:toplevel’ argument cannot be supplied.

     _‘:purify’_
          If true (the default), then some objects in the restarted core
          will be memory-mapped as read-only.  Among those objects are
          numeric vectors that were determined to be compile-time
          constants, and any immutable values according to the language
          specification such as symbol names.

     _‘:root-structures’_
          This should be a list of the main entry points in any newly
          loaded systems.  This need not be supplied, but locality
          and/or ‘gc’ performance may be better if they are.  This has
          two different but related meanings: If ‘:purify’ is true ‘-’
          and only for cheneygc ‘-’ the root structures are those which
          anchor the set of objects moved into static space.  On gencgc
          ‘-’ and only on platforms supporting immobile code ‘-’ these
          are the functions and/or function-names which commence a
          depth-first scan of code when reordering based on the
          statically observable call chain.  The complete set of
          reachable objects is not affected per se.  This argument is
          meaningless if neither enabling precondition holds.

     _‘:environment-name’_
          This has no purpose; it is accepted only for legacy
          compatibility.

     _‘:compression’_
          This is only meaningful if the runtime was built with the
          ‘:sb-core-compression’ feature enabled.  If ‘nil’ (the
          default), saves to uncompressed core files.  If
          ‘:sb-core-compression’ was enabled at build-time, the argument
          may also be an integer from -7 to 22, corresponding to zstd
          compression levels, or ‘t’ (which is equivalent to the default
          compression level, 9).

     _‘:application-type’_
          Present only on Windows and is meaningful only with
          ‘:executable’ ‘t’.  Specifies the subsystem of the executable,
          ‘:console’ or ‘:gui’.  The notable difference is that ‘:gui’
          doesn't automatically create a console window.  The default is
          ‘:console’.

     The save/load process changes the values of some global variables:

     _‘*standard-output*’, ‘*debug-io*’, etc._
          Everything related to open streams is necessarily changed,
          since the ‘os’ won't let us preserve a stream across save and
          load.

     _‘*default-pathname-defaults*’_
          This is reinitialized to reflect the working directory where
          the saved core is loaded.

     ‘save-lisp-and-die’ interacts with ‘sb-alien:load-shared-object:’
     see its documentation for details.

     On threaded platforms only a single thread may remain running after
     ‘sb-ext:*save-hooks*’ have run.  Applications using multiple
     threads can be ‘save-lisp-and-die’ friendly by registering a
     save-hook that quits any additional threads, and an init-hook that
     restarts them.

     This implementation is not as polished and painless as you might
     like:
        • It corrupts the current Lisp image enough that the current
          process needs to be killed afterwards.  This can be worked
          around by forking another process that saves the core.
        • There is absolutely no binary compatibility of core images
          between different runtime support programs.  Even runtimes
          built from the same sources at different times are treated as
          incompatible for this purpose.
     This isn't because we like it this way, but just because there
     don't seem to be good quick fixes for either limitation and no one
     has been sufficiently motivated to do lengthy fixes.
 -- Variable: *save-hooks*
     A list of function designators which are called in an unspecified
     order before creating a saved core image.

     Unused by ‘sbcl’ itself: reserved for user and applications.

   In cases where the standard initialization files have already been
loaded into the saved core, and alternative ones should be used (or none
at all), SBCL allows customizing the initfile pathname computation.

 -- Variable: *sysinit-pathname-function*
     Designator for a function of zero arguments called to obtain a
     pathname designator for the default sysinit file, or ‘nil’.  If the
     function returns ‘nil’, no sysinit file is used unless one has been
     specified on the command-line.
 -- Variable: *userinit-pathname-function*
     Designator for a function of zero arguments called to obtain a
     pathname designator or a stream for the default userinit file, or
     ‘nil’.  If the function returns ‘nil’, no userinit file is used
     unless one has been specified on the command-line.

   To facilitate distribution of SBCL applications using external
resources, the filesystem location of the SBCL core file being used is
available from Lisp.

 -- Variable: *core-pathname*
     The absolute pathname of the running ‘sbcl’ core.


File: sbcl.info,  Node: Exit on Errors,  Prev: Saving a Core Image,  Up: Stopping SBCL

3.2.4 Exit on Errors
--------------------

SBCL can also be configured to exit if an unhandled error occurs, which
is mainly useful for acting as part of a shell pipeline; doing so under
most other circumstances would mean giving up large parts of the
flexibility and robustness of Common Lisp.  See *note Debugger Entry::.


File: sbcl.info,  Node: Command Line Options,  Next: Initialization Files,  Prev: Stopping SBCL,  Up: Starting and Stopping

3.3 Command Line Options
========================

Command line options can be considered an advanced topic; for ordinary
interactive use, no command line arguments should be necessary.

   In order to understand the command line argument syntax for SBCL, it
is helpful to understand that the SBCL system is implemented as two
components, a low-level runtime environment written in C and a
higher-level system written in Common Lisp itself.  Some command line
arguments are processed during the initialization of the low-level
runtime environment, some command line arguments are processed during
the initialization of the Common Lisp system, and any remaining command
line arguments are passed on to user code.

   The full, unambiguous syntax for invoking SBCL at the command line
is:

   ‘sbcl’ RUNTIME-OPTION* ‘--end-runtime-options’ TOPLEVEL-OPTION*
‘--end-toplevel-options’ USER-OPTIONS*

   For convenience, the ‘--end-runtime-options’ and
‘--end-toplevel-options’ elements can be omitted.  Omitting these
elements can be convenient when you are running the program
interactively, and you can see that no ambiguities are possible with the
option values you are using.  Omitting these elements is probably a bad
idea for any batch file where any of the options are under user control,
since it makes it impossible for SBCL to detect erroneous command line
input, so that erroneous command line arguments will be passed on to the
user program even if they was intended for the runtime system or the
Lisp system.

* Menu:

* Runtime Options::
* Toplevel Options::


File: sbcl.info,  Node: Runtime Options,  Next: Toplevel Options,  Up: Command Line Options

3.3.1 Runtime Options
---------------------

‘--core COREFILENAME’
     Run the specified Lisp core file instead of the default.  Note that
     if the Lisp core file is a user-created core file, it may run a
     nonstandard toplevel which does not recognize the standard toplevel
     options.

‘--dynamic-space-size MEGABYTES’
     Size of the dynamic space reserved on startup in megabytes.
     Default value is platform dependent.

‘--control-stack-size MEGABYTES’
     Size of control stack reserved for each thread in megabytes.
     Default value is 2.

‘--tls-limit POSITIVE INTEGER’
     Maximum number of thread-local symbols in threaded builds.  Default
     value is 4096.

‘--noinform’
     Suppress the printing of any banner or other informational message
     at startup.  This makes it easier to write Lisp programs which work
     cleanly in Unix pipelines.  See also the ‘--noprint’ and
     ‘--disable-debugger’ options.

‘--disable-ldb’
     Disable the low-level debugger.  Only effective if SBCL is compiled
     with LDB.

‘--lose-on-corruption’
     There are some dangerous low-level errors (for instance, control
     stack exhausted, memory fault) that (or whose handlers) can corrupt
     the image.  By default SBCL prints a warning, then tries to
     continue and handle the error in Lisp, but this will not always
     work and SBCL may malfunction or even hang.  With this option, upon
     encountering such an error SBCL will invoke ldb (if present and
     enabled) or else exit.

‘--script FILENAME’
     As a runtime option this is equivalent to ‘--noinform’
     ‘--disable-ldb’ ‘--lose-on-corruption’ ‘--end-runtime-options’
     ‘--script’ FILENAME.  See the description of ‘--script’ as a
     toplevel option below.  If there are no other command line
     arguments following ‘--script’, the filename argument can be
     omitted.

‘--merge-core-pages’
     When platform support is present, provide hints to the operating
     system that identical pages may be shared between processes until
     they are written to.  This can be useful to reduce the memory usage
     on systems with multiple SBCL processes started from similar but
     differently-named core files, or from compressed cores.  Without
     platform support, do nothing.  By default only compressed cores
     trigger hinting.

‘--no-merge-core-pages’
     Ensures that no sharing hint is provided to the operating system.

‘--help’
     Print some basic information about SBCL, then exit.

‘--version’
     Print SBCL's version information, then exit.

   In the future, runtime options may be added to control behaviour such
as lazy allocation of memory.

   Runtime options, including any -end-runtime-options option, are
stripped out of the command line before the Lisp toplevel logic gets a
chance to see it.


File: sbcl.info,  Node: Toplevel Options,  Prev: Runtime Options,  Up: Command Line Options

3.3.2 Toplevel Options
----------------------

‘--sysinit FILENAME’
     Load filename instead of the default system initialization file
     (*note Initialization Files::.)

‘--no-sysinit’
     Don't load a system-wide initialization file.  If this option is
     given, the ‘--sysinit’ option is ignored.

‘--userinit FILENAME’
     Load filename instead of the default user initialization file
     (*note Initialization Files::.)

‘--no-userinit’
     Don't load a user initialization file.  If this option is given,
     the ‘--userinit’ option is ignored.

‘--eval COMMAND’
     After executing any initialization file, but before starting the
     read-eval-print loop on standard input, read and evaluate the
     command given.  More than one ‘--eval’ option can be used, and all
     will be read and executed, in the order they appear on the command
     line.

‘--load FILENAME’
     This is equivalent to ‘--eval '(load "FILENAME")'’.  The special
     syntax is intended to reduce quoting headaches when invoking SBCL
     from shell scripts.

‘--noprint’
     When ordinarily the toplevel "read-eval-print loop" would be
     executed, execute a "read-eval loop" instead, i.e.  don't print a
     prompt and don't echo results.  Combined with the ‘--noinform’
     runtime option, this makes it easier to write Lisp "scripts" which
     work cleanly in Unix pipelines.

‘--disable-debugger’
     By default when SBCL encounters an error, it enters the builtin
     debugger, allowing interactive diagnosis and possible intercession.
     This option disables the debugger, causing errors to print a
     backtrace and exit with status 1 instead.  When given, this option
     takes effect before loading of initialization files or processing
     ‘--eval’ and ‘--load’ options.  See ‘sb-ext:disable-debugger’ for
     details.  *Note Debugger Entry::.

‘--script FILENAME’
     Implies ‘--no-userinit’ ‘--no-sysinit’ ‘--disable-debugger’
     ‘--end-toplevel-options’.

     Causes the system to load the specified file instead of entering
     the read-eval-print-loop, and exit afterwards.  If the file begins
     with a shebang line, it is ignored.

     If there are no other command line arguments following, the
     filename can be omitted: this causes the script to be loaded from
     standard input instead.  Shebang lines in standard input script are
     currently _not_ ignored.

     In either case, if there is an unhandled error (e.g.  end of file,
     or a broken pipe) on either standard input, standard output, or
     standard error, the script silently exits with code 0.  This allows
     e.g.  safely piping output from SBCL to ‘head -n1’ or similar.

     Additionally, the option sets ‘*compile-verbose*’ and
     ‘*load-verbose*’ to ‘nil’ while loading the file to avoid
     potentially verbose diagnostic messages printed on the standard
     output.


File: sbcl.info,  Node: Initialization Files,  Next: Initialization and Exit Hooks,  Prev: Command Line Options,  Up: Starting and Stopping

3.4 Initialization Files
========================

SBCL processes initialization files with ‘read’ and ‘eval’, not ‘load’;
hence initialization files can be used to set startup ‘*package*’ and
‘*readtable*’, and for proclaiming a global optimization policy.

*System Initialization File*
     Defaults to ‘$SBCL_HOME/sbclrc’, or if that doesn't exist to
     ‘/etc/sbclrc’.  Can be overridden with the command line option
     ‘--sysinit’ or ‘--no-sysinit’ (*note Toplevel Options::).

     The system initialization file is intended for system
     administrators and software packagers to configure locations of
     installed third party modules, etc.

*User Initialization File*
     Defaults to ‘$HOME/.sbclrc’.  Can be overridden with the command
     line option ‘--userinit’ or ‘--no-userinit’ (*note Toplevel
     Options::).

     The user initialization file is intended for personal
     customizations, such as loading certain modules at startup,
     defining convenience functions to use in the REPL, handling
     automatic recompilation of FASLs (*note FASL Format::), etc.

   Neither initialization file is required.


File: sbcl.info,  Node: Initialization and Exit Hooks,  Prev: Initialization Files,  Up: Starting and Stopping

3.5 Initialization and Exit Hooks
=================================

SBCL provides hooks into the system initialization and exit.

 -- Variable: *init-hooks*
     A list of function designators which are called in an unspecified
     order when a saved core image starts up, after the system itself
     has been initialized, but before non-user threads such as the
     finalizer thread have been started.

     Unused by ‘sbcl’ itself: reserved for user and applications.
 -- Variable: *exit-hooks*
     A list of function designators which are called in an unspecified
     order when ‘sbcl’ process exits.

     Unused by ‘sbcl’ itself: reserved for user and applications.

     Using (‘sb-ext:exit’ ‘:abort’ ‘t’), or calling exit(3) directly
     circumvents these hooks.


File: sbcl.info,  Node: Compiler,  Next: Debugger,  Prev: Starting and Stopping,  Up: Top

4 Compiler
**********

This chapter will discuss most compiler issues other than efficiency,
including compiler error messages, the SBCL compiler's unusual approach
to type safety in the presence of type declarations, the effects of
various compiler optimization policies, and the way that inlining and
open coding may cause optimized code to differ from a naive translation.
Efficiency issues are sufficiently varied and separate that they have
their own chapter, *note Efficiency::.

* Menu:

* Diagnostic Messages::
* Handling of Types::
* Compiler Policy::
* Compiler Errors::
* Open Coding and Inline Expansion::
* Interpreter::
* Advanced Compiler Use and Efficiency Hints::


File: sbcl.info,  Node: Diagnostic Messages,  Next: Handling of Types,  Up: Compiler

4.1 Diagnostic Messages
=======================

* Menu:

* Controlling Verbosity::
* Diagnostic Severity::
* Understanding Compiler Diagnostics::


File: sbcl.info,  Node: Controlling Verbosity,  Next: Diagnostic Severity,  Up: Diagnostic Messages

4.1.1 Controlling Verbosity
---------------------------

The compiler can be quite verbose in its diagnostic reporting, rather
more then some users would prefer - the amount of noise emitted can be
controlled, however.

   To control emission of compiler diagnostics (of any severity other
than ‘error’: *note Diagnostic Severity::) use the
‘sb-ext:muffle-conditions’ and ‘sb-ext:unmuffle-conditions’
declarations, specifying the type of condition that is to be muffled
(the muffling is done using an associated ‘muffle-warning’ restart).

   Global control:
     ;;; Muffle compiler-notes globally
     (declaim (sb-ext:muffle-conditions sb-ext:compiler-note))

   Local control:
     ;;; Muffle compiler-notes based on lexical scope
     (defun foo (x)
       (declare (optimize speed) (fixnum x)
                (sb-ext:muffle-conditions sb-ext:compiler-note))
       (values (* x 5) ; no compiler note from this
         (locally
           (declare (sb-ext:unmuffle-conditions sb-ext:compiler-note))
           ;; this one gives a compiler note
           (* x -5))))

 -- Declaration: muffle-conditions
     Syntax: type*

     Muffles the diagnostic messages that would be caused by
     compile-time signals of given types.

 -- Declaration: unmuffle-conditions
     Syntax: type*

     Cancels the effect of a previous ‘sb-ext:muffle-conditions’
     declaration.

   Various details of _how_ the compiler messages are printed can be
controlled via the alist ‘sb-ext:*compiler-print-variable-alist*’.

 -- Variable: *compiler-print-variable-alist*
     an association list describing new bindings for special variables
     to be used by the compiler for error-reporting, etc.  Eg.

           ((*PRINT-LENGTH* . 10) (*PRINT-LEVEL* . 6) (*PRINT-PRETTY* . NIL))

     The variables in the ‘car’ positions are bound to the values in the
     ‘cdr’ during the execution of some debug commands.  When evaluating
     arbitrary expressions in the debugger, the normal values of the
     printer control variables are in effect.

     Initially empty, ‘*compiler-print-variable-alist*’ is Typically
     used to specify bindings for printer control variables.

   For information about muffling warnings signaled outside of the
compiler, see *note Customization Hooks for Users::.


File: sbcl.info,  Node: Diagnostic Severity,  Next: Understanding Compiler Diagnostics,  Prev: Controlling Verbosity,  Up: Diagnostic Messages

4.1.2 Diagnostic Severity
-------------------------

There are four levels of compiler diagnostic severity:

  1. error
  2. warning
  3. style warning
  4. note

   The first three levels correspond to condition classes which are
defined in the ANSI standard for Common Lisp and which have special
significance to the ‘compile’ and ‘compile-file’ functions.  These
levels of compiler error severity occur when the compiler handles
conditions of these classes.

   The fourth level of compiler error severity, _note_, corresponds to
the ‘sb-ext:compiler-note’, and is used for problems which are too mild
for the standard condition classes, typically hints about how efficiency
might be improved.  The ‘sb-ext:code-deletion-note’, a subtype of
‘compiler-note’, is signalled when the compiler deletes user-supplied
code after proving that the code in question is unreachable.

   Future work for SBCL includes expanding this hierarchy of types to
allow more fine-grained control over emission of diagnostic messages.

 -- Condition: compiler-note
     Class precedence list: ‘compiler-note’, ‘condition’, ‘t’

     Root of the hierarchy of conditions representing information
     discovered by the compiler that the user might wish to know, but
     which does not merit a ‘style-warning’ (or any more serious
     condition).
 -- Condition: code-deletion-note
     Class precedence list: ‘code-deletion-note’, ‘compiler-note’,
     ‘condition’, ‘t’

     A condition type signalled when the compiler deletes code that the
     user has written, having proved that it is unreachable.


File: sbcl.info,  Node: Understanding Compiler Diagnostics,  Prev: Diagnostic Severity,  Up: Diagnostic Messages

4.1.3 Understanding Compile Diagnostics
---------------------------------------

The messages emitted by the compiler contain a lot of detail in a terse
format, so they may be confusing at first.  The messages will be
illustrated using this example program:

     (defmacro zoq (x)
       `(roq (ploq (+ ,x 3))))

     (defun foo (y)
       (declare (symbol y))
       (zoq y))

   The main problem with this program is that it is trying to add ‘3’ to
a symbol.  Note also that the functions ‘roq’ and ‘ploq’ aren't defined
anywhere.

* Menu:

* The Parts of a Compiler Diagnostic::
* The Original and Actual Source::
* The Processing Path::


File: sbcl.info,  Node: The Parts of a Compiler Diagnostic,  Next: The Original and Actual Source,  Up: Understanding Compiler Diagnostics

4.1.3.1 The Parts of a Compiler Diagnostic
..........................................

When processing this program, the compiler will produce this warning:

     ; file: /tmp/foo.lisp
     ; in: DEFUN FOO
     ;     (ZOQ Y)
     ; --> ROQ PLOQ
     ; ==>
     ;   (+ Y 3)
     ;
     ; caught WARNING:
     ;   Asserted type NUMBER conflicts with derived type (VALUES SYMBOL &OPTIONAL).

   In this example we see each of the six possible parts of a compiler
diagnostic:

  1. ‘file: /tmp/foo.lisp’ This is the name of the file that the
     compiler read the relevant code from.  The file name is displayed
     because it may not be immediately obvious when there is an error
     during compilation of a large system, especially when
     ‘with-compilation-unit’ is used to delay undefined warnings.

  2. ‘in: DEFUN FOO’ This is the definition top level form responsible
     for the diagnostic.  It is obtained by taking the first two
     elements of the enclosing form whose first element is a symbol
     beginning with "‘def’".  If there is no such enclosing "‘def’"
     form, then the outermost form is used.  If there are multiple ‘def’
     forms, then they are all printed from the outside in, separated by
     ‘=>’'s.  In this example, the problem was in the ‘defun’ for ‘foo’.

  3. ‘(ZOQ Y)’ This is the “original source” form responsible for the
     diagnostic.  Original source means that the form directly appeared
     in the original input to the compiler, i.e.  in the lambda passed
     to ‘compile’ or in the top level form read from the source file.
     In this example, the expansion of the ‘zoq’ macro was responsible
     for the message.

  4. ‘--> ROQ PLOQ’ This is the “processing path” that the compiler used
     to produce the code that caused the message to be emitted.  The
     processing path is a representation of the evaluated forms
     enclosing the actual source that the compiler encountered when
     processing the original source.  The path is the first element of
     each form, or the form itself if the form is not a list.  These
     forms result from the expansion of macros or source-to-source
     transformation done by the compiler.  In this example, the
     enclosing evaluated forms are the calls to ‘roq’ and ‘ploq’.  These
     calls resulted from the expansion of the ‘zoq’ macro.

  5. ‘==> (+ Y 3)’ This is the “actual source” responsible for the
     diagnostic.  If the actual source appears in the explanation, then
     we print the next enclosing evaluated form, instead of printing the
     actual source twice.  (This is the form that would otherwise have
     been the last form of the processing path.)  In this example, the
     problem is with the evaluation of the reference to the variable
     ‘y’.

  6. ‘caught WARNING: Asserted type NUMBER conflicts with derived type
     (VALUES SYMBOL &OPTIONAL).’ This is the “explanation” of the
     problem.  In this example, the problem is that, while the call to
     ‘+’ requires that its arguments are all of type ‘number’, the
     compiler has derived that ‘y’ will evaluate to a ‘symbol’.  Note
     that ‘(VALUES SYMBOL &OPTIONAL)’ expresses that ‘y’ evaluates to
     precisely one value.

   Note that each part of the message is distinctively marked:

   • ‘file:’ and ‘in:’ mark the file and definition, respectively.

   • The original source is an indented form with no prefix.

   • Each line of the processing path is prefixed with ‘-->’

   • The actual source form is indented like the original source, but is
     marked by a preceding ‘==>’ line.

   • The explanation is prefixed with the diagnostic severity, which can
     be ‘caught ERROR:’, ‘caught WARNING:’, ‘caught STYLE-WARNING:’, or
     ‘note:’.

   Each part of the message is more specific than the preceding one.  If
consecutive messages are for nearby locations, then the front part of
the messages would be the same.  In this case, the compiler omits as
much of the second message as in common with the first.  For example:

     ; file: /tmp/foo.lisp
     ; in: DEFUN FOO
     ;     (ZOQ Y)
     ; --> ROQ
     ; ==>
     ;   (PLOQ (+ Y 3))
     ;
     ; caught STYLE-WARNING:
     ;   undefined function: PLOQ

     ; ==>
     ;   (ROQ (PLOQ (+ Y 3)))
     ;
     ; caught STYLE-WARNING:
     ;   undefined function: ROQ

   In this example, the file, definition and original source are
identical for the two messages, so the compiler omits them in the second
message.  If consecutive messages are entirely identical, then the
compiler prints only the first message, followed by: ‘[Last message
occurs REPEATS times]’ where REPEATS is the number of times the message
was given.

   If the source was not from a file, then no file line is printed.  If
the actual source is the same as the original source, then the
processing path and actual source will be omitted.  If no forms
intervene between the original source and the actual source, then the
processing path will also be omitted.


File: sbcl.info,  Node: The Original and Actual Source,  Next: The Processing Path,  Prev: The Parts of a Compiler Diagnostic,  Up: Understanding Compiler Diagnostics

4.1.3.2 The Original and Actual Source
......................................

The _original source_ displayed will almost always be a list.  If the
actual source for an message is a symbol, the original source will be
the immediately enclosing evaluated list form.  So even if the offending
symbol does appear in the original source, the compiler will print the
enclosing list and then print the symbol as the actual source (as though
the symbol were introduced by a macro.)

   When the _actual source_ is displayed (and is not a symbol), it will
always be code that resulted from the expansion of a macro or a
source-to-source compiler optimization.  This is code that did not
appear in the original source program; it was introduced by the
compiler.

   Keep in mind that when the compiler displays a source form in an
diagnostic message, it always displays the most specific (innermost)
responsible form.  For example, compiling this function

     (defun bar (x)
       (let (a)
         (declare (fixnum a))
         (setq a (foo x))
         a))

   gives this error message

     ; file: /tmp/foo.lisp
     ; in: DEFUN BAR
     ;     (LET (A)
     ;     (DECLARE (FIXNUM A))
     ;     (SETQ A (FOO X))
     ;     A)
     ;
     ; caught WARNING:
     ;   Asserted type FIXNUM conflicts with derived type (VALUES NULL &OPTIONAL).

   This message is not saying "there is a problem somewhere in this
‘let’" - it is saying that there is a problem with the ‘let’ itself.  In
this example, the problem is that ‘a’'s ‘nil’ initial value is not a
‘fixnum’.


File: sbcl.info,  Node: The Processing Path,  Prev: The Original and Actual Source,  Up: Understanding Compiler Diagnostics

4.1.3.3 The Processing Path
...........................

The processing path is mainly useful for debugging macros, so if you
don't write macros, you can probably ignore it.  Consider this example:

     (defun foo (n)
       (dotimes (i n *undefined*)))

   Compiling results in this error message:

     ; in: DEFUN FOO
     ;     (DOTIMES (I N *UNDEFINED*))
     ; --> DO BLOCK LET TAGBODY RETURN-FROM
     ; ==>
     ;   (PROGN *UNDEFINED*)
     ;
     ; caught WARNING:
     ;   undefined variable: *UNDEFINED*

   Note that ‘do’ appears in the processing path.  This is because
‘dotimes’ expands into:

     (do ((i 0 (1+ i)) (#:g1 n))
         ((>= i #:g1) *undefined*)
       (declare (type unsigned-byte i)))

   The rest of the processing path results from the expansion of ‘do’:

     (block nil
       (let ((i 0) (#:g1 n))
         (declare (type unsigned-byte i))
         (tagbody (go #:g3)
           #:g2    (psetq i (1+ i))
           #:g3    (unless (>= i #:g1) (go #:g2))
           (return-from nil (progn *undefined*)))))

   In this example, the compiler descended into the ‘block’, ‘let’,
‘tagbody’ and ‘return-from’ to reach the ‘progn’ printed as the actual
source.  This is a place where the "actual source appears in
explanation" rule was applied.  The innermost actual source form was the
symbol ‘*undefined*’ itself, but that also appeared in the explanation,
so the compiler backed out one level.


File: sbcl.info,  Node: Handling of Types,  Next: Compiler Policy,  Prev: Diagnostic Messages,  Up: Compiler

4.2 Handling of Types
=====================

One of the most important features of the SBCL compiler (similar to the
original CMUCL compiler, also known as “Python”) is its fairly
sophisticated understanding of the Common Lisp type system and its
conservative approach to the implementation of type declarations.

   These two features reward the use of type declarations throughout
development, even when high performance is not a concern.  Also, as
discussed in the chapter on performance (*note Efficiency::), the use of
appropriate type declarations can be very important for performance as
well.

   The SBCL compiler also has a greater knowledge of the Common Lisp
type system than other compilers.  Support is incomplete only for types
involving the ‘satisfies’ type specifier.

* Menu:

* Declarations as Assertions::
* Precise Type Checking::
* Getting Existing Programs to Run::
* Implementation Limitations::


File: sbcl.info,  Node: Declarations as Assertions,  Next: Precise Type Checking,  Up: Handling of Types

4.2.1 Declarations as Assertions
--------------------------------

The SBCL compiler treats type declarations differently from most other
Lisp compilers.  Under default compilation policy the compiler doesn't
blindly believe type declarations, but considers them assertions about
the program that should be checked: all type declarations that have not
been proven to always hold are asserted at runtime.

     _Remaining bugs in the compiler's handling of types unfortunately
     provide some exceptions to this rule, see *note Implementation
     Limitations::._

   CLOS slot types form a notable exception.  Types declared using the
‘:type’ slot option in ‘defclass’ are asserted if and only if the class
was defined in _safe code_ and the slot access location is in _safe
code_ as well.  This laxness does not pose any internal consistency
issues, as the CLOS slot types are not available for the type
inferencer, nor do CLOS slot types provide any efficiency benefits.

   There are three type checking policies available in SBCL, selectable
via ‘optimize’ declarations.

*Full Type Checks*
     All declarations are considered assertions to be checked at
     runtime, and all type checks are precise.  The default compilation
     policy provides full type checks.

     Used when ‘(or (>= safety 2) (>= safety speed 1))’.

*Weak Type Checks*
     Declared types may be simplified into faster to check supertypes:
     for example, ‘(or (integer -17 -7) (integer 7 17))’ is simplified
     into ‘(integer -17 17)’.

     *Note*: it is relatively easy to corrupt the heap when weak type
     checks are used if the program contains type-errors.

     Used when ‘(and (< safety 2) (< safety speed))’

*No Type Checks*
     All declarations are believed without assertions.  Also disables
     argument count and array bounds checking.

     *Note*: any type errors in code where type checks are not performed
     are liable to corrupt the heap.

     Used when ‘(= safety 0)’.


File: sbcl.info,  Node: Precise Type Checking,  Next: Getting Existing Programs to Run,  Prev: Declarations as Assertions,  Up: Handling of Types

4.2.2 Precise Type Checking
---------------------------

Precise checking means that the check is done as though ‘typep’ had been
called with the exact type specifier that appeared in the declaration.

   If a variable is declared to be ‘(integer 3 17)’ then its value must
always be an integer between ‘3’ and ‘17’.  If multiple type
declarations apply to a single variable, then all the declarations must
be correct; it is as though all the types were intersected producing a
single ‘and’ type specifier.

   To gain maximum benefit from the compiler's type checking, you should
always declare the types of function arguments and structure slots as
precisely as possible.  This often involves the use of ‘or’, ‘member’,
and other list-style type specifiers.


File: sbcl.info,  Node: Getting Existing Programs to Run,  Next: Implementation Limitations,  Prev: Precise Type Checking,  Up: Handling of Types

4.2.3 Getting Existing Programs to Run
--------------------------------------

Since SBCL's compiler does much more comprehensive type checking than
most Lisp compilers, SBCL may detect type errors in programs that have
been debugged using other compilers.  These errors are mostly incorrect
declarations, although compile-time type errors can find actual bugs if
parts of the program have never been tested.

   Some incorrect declarations can only be detected by run-time type
checking.  It is very important to initially compile a program with full
type checks (high ‘safety’ optimization) and then test this safe
version.  After the checking version has been tested, then you can
consider weakening or eliminating type checks.  _This applies even to
previously debugged programs,_ because the SBCL compiler does much more
type inference than other Common Lisp compilers, so an incorrect
declaration can do more damage.

   The most common problem is with variables whose constant initial
value doesn't match the type declaration.  Incorrect constant initial
values will always be flagged by a compile-time type error, and they are
simple to fix once located.  Consider this code fragment:

     (prog (foo)
       (declare (fixnum foo))
       (setq foo ...)
       ...)

   Here ‘foo’ is given an initial value of ‘nil’, but is declared to be
a ‘fixnum’.  Even if it is never read, the initial value of a variable
must match the declared type.  There are two ways to fix this problem.
Change the declaration

     (prog (foo)
       (declare (type (or fixnum null) foo))
       (setq foo ...)
       ...)

   or change the initial value

     (prog ((foo 0))
       (declare (fixnum foo))
       (setq foo ...)
       ...)

   It is generally preferable to change to a legal initial value rather
than to weaken the declaration, but sometimes it is simpler to weaken
the declaration than to try to make an initial value of the appropriate
type.

   Another declaration problem occasionally encountered is incorrect
declarations on ‘defmacro’ arguments.  This can happen when a function
is converted into a macro.  Consider this macro:

     (defmacro my-1+ (x)
       (declare (fixnum x))
       `(the fixnum (1+ ,x)))

   Although legal and well-defined Common Lisp code, this meaning of
this definition is almost certainly not what the writer intended.  For
example, this call is illegal:

     (my-1+ (+ 4 5))

   This call is illegal because the argument to the macro is ‘(+ 4 5)’,
which is a ‘list’, not a ‘fixnum’.  Because of macro semantics, it is
hardly ever useful to declare the types of macro arguments.  If you
really want to assert something about the type of the result of
evaluating a macro argument, then put a ‘the’ in the expansion:

     (defmacro my-1+ (x)
       `(the fixnum (1+ (the fixnum ,x))))

   In this case, it would be stylistically preferable to change this
macro back to a function and declare it inline.

   Some more subtle problems are caused by incorrect declarations that
can't be detected at compile time.  Consider this code:

     (do ((pos 0 (position #\a string :start (1+ pos))))
       ((null pos))
       (declare (fixnum pos))
       ...)

   Although ‘pos’ is almost always a ‘fixnum’, it is ‘nil’ at the end of
the loop.  If this example is compiled with full type checks (the
default), then running it will signal a type error at the end of the
loop.  If compiled without type checks, the program will go into an
infinite loop (or perhaps ‘position’ will complain because ‘(1+ nil)’
isn't a sensible start.)  Why?  Because if you compile without type
checks, the compiler just quietly believes the type declaration.  Since
the compiler believes that ‘pos’ is always a ‘fixnum’, it believes that
‘pos’ is never ‘nil’, so ‘(null pos)’ is never true, and the loop exit
test is optimized away.  Such errors are sometimes flagged by
unreachable code notes, but it is still important to initially compile
and test any system with full type checks, even if the system works fine
when compiled using other compilers.

   In this case, the fix is to weaken the type declaration to ‘(or
fixnum null)’ (1).

   Note that there is usually little performance penalty for weakening a
declaration in this way.  Any numeric operations in the body can still
assume that the variable is a ‘fixnum’, since ‘nil’ is not a legal
numeric argument.  Another possible fix would be to say:

     (do ((pos 0 (position #\a string :start (1+ pos))))
         ((null pos))
       (let ((pos pos))
         (declare (fixnum pos))
         ...))

   This would be preferable in some circumstances, since it would allow
a non-standard representation to be used for the local ‘pos’ variable in
the loop body.

   ---------- Footnotes ----------

   (1) Actually, this declaration is unnecessary in SBCL, since it
already knows that ‘position’ returns a non-negative ‘fixnum’ or ‘nil’.


File: sbcl.info,  Node: Implementation Limitations,  Prev: Getting Existing Programs to Run,  Up: Handling of Types

4.2.4 Implementation Limitations
--------------------------------

If an FTYPE is placed after the function definition the function won't
perform any type checks, and the calls to the function will blindly
trust the declared types.  ‘(optimize (debug 3))’ will not trust any
FTYPE declarations.


File: sbcl.info,  Node: Compiler Policy,  Next: Compiler Errors,  Prev: Handling of Types,  Up: Compiler

4.3 Compiler Policy
===================

Compiler policy is controlled by the ‘optimize’ declaration, supporting
all ANSI optimization qualities (‘debug’, ‘safety’, ‘space’, and
‘speed’).(1)

   For effects of various optimization qualities on type-safety and
debuggability see *note Declarations as Assertions:: and *note Debugger
Policy Control::.

   Ordinarily, when the ‘speed’ quality is high, the compiler emits
notes to notify the programmer about its inability to apply various
optimizations.  For selective muffling of these notes *Note Controlling
Verbosity::.

   The value of ‘space’ mostly influences the compiler's decision
whether to inline operations, which tend to increase the size of
programs.  Use the value ‘0’ with caution, since it can cause the
compiler to inline operations so indiscriminately that the net effect is
to slow the program by causing cache misses or even swapping.

 -- Function: describe-compiler-policy &optional spec
     Print all global optimization settings, augmented by ‘spec’.
 -- Function: restrict-compiler-policy &optional quality min max
     Assign a minimum value to an optimization quality.  ‘quality’ is
     the name of the optimization quality to restrict, ‘min’ (defaulting
     to zero) is the minimum allowed value, and ‘max’ (defaults to 3) is
     the maximum.

     Returns the alist describing the current policy restrictions.

     If ‘quality’ is ‘nil’ or not given, nothing is done.

     Otherwise, if ‘min’ is zero or ‘max’ is 3 or neither are given, any
     existing restrictions of ‘quality’ are removed.

     See also ‘:policy’ option in ‘with-compilation-unit’.
 -- Macro: with-compilation-unit options &body body
     Affects compilations that take place within its dynamic extent.  It
     is intended to be eg.  wrapped around the compilation of all files
     in the same system.

     Following options are defined:

     _‘:override’ Boolean-Form_
          One of the effects of this form is to delay undefined warnings
          until the end of the form, instead of giving them at the end
          of each compilation.  If ‘override’ is ‘nil’ (the default),
          then the outermost ‘with-compilation-unit’ form grabs the
          undefined warnings.  Specifying ‘override’ true causes that
          form to grab any enclosed warnings, even if it is enclosed by
          another ‘with-compilation-unit’.

     _‘:policy’ Optimize-Declaration-Form_
          Provides dynamic scoping for global compiler optimization
          qualities and restrictions, limiting effects of subsequent
          ‘optimize’ proclamations and calls to
          ‘sb-ext:restrict-compiler-policy’ to the dynamic scope of
          ‘body’.

          If ‘override’ is false, specified ‘policy’ is merged with
          current global policy.  If ‘override’ is true, current global
          policy, including any restrictions, is discarded in favor of
          the specified ‘policy’.

          Supplying ‘policy’ ‘nil’ is equivalent to the option not being
          supplied at all, ie.  dynamic scoping of policy does not take
          place.

          This option is an SBCL-specific experimental extension:
          Interface subject to change.

     _‘:source-namestring’ Namestring-Form_
          Attaches the value returned by the Namestring-Form to the
          internal debug-source information as the namestring of the
          source file.  Normally the namestring of the input-file for
          ‘compile-file’ is used: this option can be used to provide
          source-file information for functions compiled using
          ‘compile’, or to override the input-file of ‘compile-file’.

          If both an outer and an inner ‘with-compilation-unit’ provide
          a ‘source-namestring’, the inner one takes precedence.
          Unaffected by ‘:override’.

          This is an SBCL-specific extension.

     _‘:source-plist’ Plist-Form_
          Attaches the value returned by the Plist-Form to internal
          debug-source information of functions compiled in within the
          dynamic extent of ‘body’.

          Primarily for use by development environments, in order to eg.
          associate function definitions with editor-buffers.  Can be
          accessed using ‘sb-introspect:definition-source-plist’.

          If an outer ‘with-compilation-unit’ form also provide a
          ‘source-plist’, it is appended to the end of the provided
          ‘source-plist’.  Unaffected by ‘:override’.

          This is an SBCL-specific extension.

     Examples:

            ;; Prevent proclamations from the file leaking, and restrict
            ;; SAFETY to 3 -- otherwise uses the current global policy.
            (with-compilation-unit (:policy '(optimize))
              (restrict-compiler-policy 'safety 3)
              (load "foo.lisp"))

            ;; Using default policy instead of the current global one,
            ;; except for DEBUG 3.
            (with-compilation-unit (:policy '(optimize debug)
                                    :override t)
              (load "foo.lisp"))

            ;; Same as if :POLICY had not been specified at all: SAFETY 3
            ;; proclamation leaks out from WITH-COMPILATION-UNIT.
            (with-compilation-unit (:policy nil)
              (declaim (optimize safety))
              (load "foo.lisp"))

   ---------- Footnotes ----------

   (1) A deprecated extension ‘sb-ext:inhibit-warnings’ is still
supported, but liable to go away at any time.


File: sbcl.info,  Node: Compiler Errors,  Next: Open Coding and Inline Expansion,  Prev: Compiler Policy,  Up: Compiler

4.4 Compiler Errors
===================

* Menu:

* Type Errors at Compile Time::
* Errors During Macroexpansion::
* Read Errors::


File: sbcl.info,  Node: Type Errors at Compile Time,  Next: Errors During Macroexpansion,  Up: Compiler Errors

4.4.1 Type Errors at Compile Time
---------------------------------

If the compiler can prove at compile time that some portion of the
program cannot be executed without a type error, then it will give a
warning at compile time.

   It is possible that the offending code would never actually be
executed at run-time due to some higher level consistency constraint
unknown to the compiler, so a type warning doesn't always indicate an
incorrect program.

   For example, consider this code fragment:

     (defun raz (foo)
       (let ((x (case foo
                   (:this 13)
                   (:that 9)
                   (:the-other 42))))
         (declare (fixnum x))
         (foo x)))

   Compilation produces this warning:

     ; in: DEFUN RAZ
     ;     (CASE FOO (:THIS 13) (:THAT 9) (:THE-OTHER 42))
     ; --> LET COND IF COND IF COND IF
     ; ==>
     ;   (COND)
     ;
     ; caught WARNING:
     ;   This is not a FIXNUM:
     ;   NIL

   In this case, the warning means that if ‘foo’ isn't any of ‘:this’,
‘:that’ or ‘:the-other’, then ‘x’ will be initialized to ‘nil’, which
the ‘fixnum’ declaration makes illegal.  The warning will go away if
‘ecase’ is used instead of ‘case’, or if ‘:the-other’ is changed to ‘t’.

   This sort of spurious type warning happens moderately often in the
expansion of complex macros and in inline functions.  In such cases,
there may be dead code that is impossible to correctly execute.  The
compiler can't always prove this code is dead (could never be executed),
so it compiles the erroneous code (which will always signal an error if
it is executed) and gives a warning.


File: sbcl.info,  Node: Errors During Macroexpansion,  Next: Read Errors,  Prev: Type Errors at Compile Time,  Up: Compiler Errors

4.4.2 Errors During Macroexpansion
----------------------------------

The compiler handles errors that happen during macroexpansion, turning
them into compiler errors.  If you want to debug the error (to debug a
macro), you can set ‘*break-on-signals*’ to ‘error’.  For example, this
definition:

     (defun foo (e l)
       (do ((current l (cdr current))
            ((atom current) nil))
           (when (eq (car current) e) (return current))))

   gives this error:

     ; in: DEFUN FOO
     ;     (DO ((CURRENT L (CDR CURRENT))
     ;        ((ATOM CURRENT) NIL))
     ;       (WHEN (EQ (CAR CURRENT) E) (RETURN CURRENT)))
     ;
     ; caught ERROR:
     ;   (in macroexpansion of (DO # #))
     ;   (hint: For more precise location, try *BREAK-ON-SIGNALS*.)
     ;   DO step variable is not a symbol: (ATOM CURRENT)


File: sbcl.info,  Node: Read Errors,  Prev: Errors During Macroexpansion,  Up: Compiler Errors

4.4.3 Read Errors
-----------------

SBCL's compiler does not attempt to recover from read errors when
reading a source file, but instead just reports the offending character
position and gives up on the entire source file.


File: sbcl.info,  Node: Open Coding and Inline Expansion,  Next: Interpreter,  Prev: Compiler Errors,  Up: Compiler

4.5 Open Coding and Inline Expansion
====================================

Since Common Lisp forbids the redefinition of standard functions, the
compiler can have special knowledge of these standard functions embedded
in it.  This special knowledge is used in various ways (open coding,
inline expansion, source transformation), but the implications to the
user are basically the same:

   • Attempts to redefine standard functions may be frustrated, since
     the function may never be called.  Although it is technically
     illegal to redefine standard functions, users sometimes want to
     implicitly redefine these functions when they are debugging using
     the ‘trace’ macro.  Special-casing of standard functions can be
     inhibited using the ‘notinline’ declaration, but even then some
     phases of analysis such as type inferencing are applied by the
     compiler.

   • The compiler can have multiple alternate implementations of
     standard functions that implement different trade-offs of speed,
     space and safety.  This selection is based on the compiler policy,
     *note Compiler Policy::.

   When a function call is _open coded_, inline code whose effect is
equivalent to the function call is substituted for that function call.
When a function call is _closed coded_, it is usually left as is,
although it might be turned into a call to a different function with
different arguments.  As an example, if ‘nthcdr’ were to be open coded,
then

     (nthcdr 4 foobar)

   might turn into

     (cdr (cdr (cdr (cdr foobar))))

   or even

     (do ((i 0 (1+ i))
       (list foobar (cdr foobar)))
       ((= i 4) list))

   If ‘nth’ is closed coded, then

     (nth x l)

   might stay the same, or turn into something like

     (car (nthcdr x l))

   In general, open coding sacrifices space for speed, but some
functions (such as ‘car’) are so simple that they are always open-coded.
Even when not open-coded, a call to a standard function may be
transformed into a different function call (as in the last example) or
compiled as _static call_.  Static function call uses a more efficient
calling convention that forbids redefinition.


File: sbcl.info,  Node: Interpreter,  Next: Advanced Compiler Use and Efficiency Hints,  Prev: Open Coding and Inline Expansion,  Up: Compiler

4.6 Interpreter
===============

By default SBCL implements ‘eval’ by calling the native code compiler.

   SBCL also includes an interpreter for use in special cases where
using the compiler is undesirable, for example due to compilation
overhead.  Unlike in some other Lisp implementations, in SBCL
interpreted code is not safer or more debuggable than compiled code.

 -- Variable: *evaluator-mode*
     Toggle between different evaluator implementations.  If set to
     ‘:compile’, an implementation of ‘eval’ that calls the compiler
     will be used.  If set to ‘:interpret’, an interpreter will be used.


File: sbcl.info,  Node: Advanced Compiler Use and Efficiency Hints,  Prev: Interpreter,  Up: Compiler

4.7 Advanced Compiler Use and Efficiency Hints
==============================================

For more advanced usages of the compiler, please see the chapter of the
same name in the CMUCL manual.  Many aspects of the compiler have stayed
exactly the same, and there is a much more detailed explanation of the
compiler's behavior and how to maximally optimize code in their manual.
In particular, while SBCL no longer supports byte-code compilation, it
does support CMUCL's block compilation facility allowing whole program
optimization and increased use of the local call convention.

   Unlike CMUCL, SBCL is able to open-code forward-referenced type tests
while block compiling.  This helps for mutually referential ‘defstruct’s
in particular.


File: sbcl.info,  Node: Debugger,  Next: Efficiency,  Prev: Compiler,  Up: Top

5 Debugger
**********

This chapter documents the debugging facilities of SBCL, including the
debugger, single-stepper and ‘trace’, and the effect of ‘(optimize
debug)’ declarations.

* Menu:

* Debugger Entry::
* Debugger Command Loop::
* Stack Frames::
* Variable Access::
* Source Location Printing::
* Debugger Policy Control::
* Exiting Commands::
* Information Commands::
* Breakpoint Commands::
* Function Tracing::
* Single Stepping::
* Enabling and Disabling the Debugger::


File: sbcl.info,  Node: Debugger Entry,  Next: Debugger Command Loop,  Up: Debugger

5.1 Debugger Entry
==================

* Menu:

* Debugger Banner::
* Debugger Invocation::


File: sbcl.info,  Node: Debugger Banner,  Next: Debugger Invocation,  Up: Debugger Entry

5.1.1 Debugger Banner
---------------------

When you enter the debugger, it looks something like this:

     debugger invoked on a TYPE-ERROR in thread 11184:
       The value 3 is not of type LIST.

     You can type HELP for debugger help, or (SB-EXT:QUIT) to exit from SBCL.

     restarts (invokable by number or by possibly-abbreviated name):
       0: [ABORT   ] Reduce debugger level (leaving debugger, returning to toplevel).
       1: [TOPLEVEL] Restart at toplevel READ/EVAL/PRINT loop.
     (CAR 1 3)
     0]

   The first group of lines describe what the error was that put us in
the debugger.  In this case ‘car’ was called on ‘3’, causing a
‘type-error’.

   This is followed by the "beginner help line", which appears only if
‘sb-debug:*debug-beginner-help-p*’ is true (default).

   Next comes a listing of the active restart names, along with their
descriptions - the ways we can restart execution after this error.  In
this case, both options return to top-level.  Restarts can be selected
by entering the corresponding number or name.

   The current frame appears right underneath the restarts, immediately
followed by the debugger prompt.


File: sbcl.info,  Node: Debugger Invocation,  Prev: Debugger Banner,  Up: Debugger Entry

5.1.2 Debugger Invocation
-------------------------

The debugger is invoked when:

   • ‘error’ is called, and the condition it signals is not handled.

   • ‘break’ is called, or ‘signal’ is called with a condition that
     matches the current ‘*break-on-signals*’.

   • the debugger is explicitly entered with the ‘invoke-debugger’
     function.

   When the debugger is invoked by a condition, ANSI mandates that the
value of ‘*debugger-hook*’, if any, be called with two arguments: the
condition that caused the debugger to be invoked and the previous value
of ‘*debugger-hook*’.  When this happens, ‘*debugger-hook*’ is bound to
NIL to prevent recursive errors.  However, ANSI also mandates that
‘*debugger-hook*’ not be invoked when the debugger is to be entered by
the ‘break’ function.  For users who wish to provide an alternate
debugger interface (and thus catch ‘break’ entries into the debugger),
SBCL provides ‘sb-ext:*invoke-debugger-hook*’, which is invoked during
any entry into the debugger.

 -- Variable: *invoke-debugger-hook*
     This is either ‘nil’ or a designator for a function of two
     arguments, to be run when the debugger is about to be entered.  The
     function is run with ‘*invoke-debugger-hook*’ bound to ‘nil’ to
     minimize recursive errors, and receives as arguments the condition
     that triggered debugger entry and the previous value of
     ‘*invoke-debugger-hook*’

     This mechanism is an ‘sbcl’ extension similar to the standard
     ‘*debugger-hook*’.  In contrast to ‘*debugger-hook*’, it is
     observed by ‘invoke-debugger’ even when called by ‘break’.


File: sbcl.info,  Node: Debugger Command Loop,  Next: Stack Frames,  Prev: Debugger Entry,  Up: Debugger

5.2 Debugger Command Loop
=========================

The debugger is an interactive read-eval-print loop much like the normal
top level, but some symbols are interpreted as debugger commands instead
of being evaluated.  A debugger command starts with the symbol name of
the command, possibly followed by some arguments on the same line.  Some
commands prompt for additional input.  Debugger commands can be
abbreviated by any unambiguous prefix: ‘help’ can be typed as ‘h’, ‘he’,
etc.

   The package is not significant in debugger commands; any symbol with
the name of a debugger command will work.  If you want to show the value
of a variable that happens also to be the name of a debugger command you
can wrap the variable in a ‘progn’ to hide it from the command loop.

   The debugger prompt is "‘FRAME]’", where FRAME is the number of the
current frame.  Frames are numbered starting from zero at the top (most
recent call), increasing down to the bottom.  The current frame is the
frame that commands refer to.

   It is possible to override the normal printing behaviour in the
debugger by using the ‘sb-ext:*debug-print-variable-alist*’.

 -- Variable: *debug-print-variable-alist*
     an association list describing new bindings for special variables
     to be used within the debugger.  Eg.

           ((*PRINT-LENGTH* . 10) (*PRINT-LEVEL* . 6) (*PRINT-PRETTY* . NIL))

     The variables in the ‘car’ positions are bound to the values in the
     ‘cdr’ during the execution of some debug commands.  When evaluating
     arbitrary expressions in the debugger, the normal values of the
     printer control variables are in effect.

     Initially empty, ‘*debug-print-variable-alist*’ is typically used
     to provide bindings for printer control variables.


File: sbcl.info,  Node: Stack Frames,  Next: Variable Access,  Prev: Debugger Command Loop,  Up: Debugger

5.3 Stack Frames
================

A “stack frame” is the run-time representation of a call to a function;
the frame stores the state that a function needs to remember what it is
doing.  Frames have:

   • “variables” (*note Variable Access::), which are the values being
     operated on.

   • “arguments” to the call (which are really just particularly
     interesting variables).

   • a current source location (*note Source Location Printing::), which
     is the place in the program where the function was running when it
     stopped to call another function, or because of an interrupt or
     error.

* Menu:

* Stack Motion::
* How Arguments are Printed::
* Function Names::
* Debug Tail Recursion::
* Unknown Locations and Interrupts::


File: sbcl.info,  Node: Stack Motion,  Next: How Arguments are Printed,  Up: Stack Frames

5.3.1 Stack Motion
------------------

These commands move to a new stack frame and print the name of the
function and the values of its arguments in the style of a Lisp function
call:

 -- Debugger Command: up
     Move up to the next higher frame.  More recent function calls are
     considered to be higher on the stack.

 -- Debugger Command: down
     Move down to the next lower frame.

 -- Debugger Command: top
     Move to the highest frame, that is, the frame where the debugger
     was entered.

 -- Debugger Command: bottom
     Move to the lowest frame.

 -- Debugger Command: frame [N]
     Move to the frame with the specified number.  Prompts for the
     number if not supplied.  The frame with number 0 is the frame where
     the debugger was entered.


File: sbcl.info,  Node: How Arguments are Printed,  Next: Function Names,  Prev: Stack Motion,  Up: Stack Frames

5.3.2 How Arguments are Printed
-------------------------------

A frame is printed to look like a function call, but with the actual
argument values in the argument positions.  So the frame for this call
in the source:

     (myfun (+ 3 4) 'a)

   would look like this:

     (MYFUN 7 A)

   All keyword and optional arguments are displayed with their actual
values; if the corresponding argument was not supplied, the value will
be the default.  So this call:

     (subseq "foo" 1)

   would look like this:

     (SUBSEQ "foo" 1 3)

   And this call:

     (string-upcase "test case")

   would look like this:

     (STRING-UPCASE "test case" :START 0 :END NIL)

   The arguments to a function call are displayed by accessing the
argument variables.  Although those variables are initialized to the
actual argument values, they can be set inside the function; in this
case the new value will be displayed.

   ‘&rest’ arguments are handled somewhat differently.  The value of the
rest argument variable is displayed as the spread-out arguments to the
call, so:

     (format t "~A is a ~A." "This" 'test)

   would look like this:

     (FORMAT T "~A is a ~A." "This" 'TEST)

   Rest arguments cause an exception to the normal display of keyword
arguments in functions that have both ‘&rest’ and ‘&key’ arguments.  In
this case, the keyword argument variables are not displayed at all; the
rest arg is displayed instead.  So for these functions, only the
keywords actually supplied will be shown, and the values displayed will
be the argument values, not values of the (possibly modified) variables.

   If the variable for an argument is never referenced by the function,
it will be deleted.  The variable value is then unavailable, so the
debugger prints ‘#<unused-arg>’ instead of the value.  Similarly, if for
any of a number of reasons the value of the variable is unavailable or
not known to be available (*note Variable Access::), then
‘#<unavailable-arg>’ will be printed instead of the argument value.

   Note that inline expansion and open-coding affect what frames are
present in the debugger, see *note Debugger Policy Control::.


File: sbcl.info,  Node: Function Names,  Next: Debug Tail Recursion,  Prev: How Arguments are Printed,  Up: Stack Frames

5.3.3 Function Names
--------------------

If a function is defined by ‘defun’ it will appear in backtrace by that
name.  Functions defined by ‘labels’ and ‘flet’ will appear as ‘(FLET
NAME)’ and ‘(LABELS NAME)’ respectively.  Anonymous lambdas will appear
as ‘(LAMBDA LAMBDA-LIST)’.

* Menu:

* Entry Point Details::


File: sbcl.info,  Node: Entry Point Details,  Up: Function Names

5.3.3.1 Entry Point Details
...........................

Sometimes the compiler introduces new functions that are used to
implement a user function, but are not directly specified in the source.
This is mostly done for argument type and count checking.

   With recursive or block compiled functions, an additional ‘external’
frame may appear before the frame representing the first call to the
recursive function or entry to the compiled block.  This is a
consequence of the way the compiler works: there is nothing odd with
your program.  You may also see ‘cleanup’ frames during the execution of
‘unwind-protect’ cleanup code, and ‘optional’ for variable argument
entry points.


File: sbcl.info,  Node: Debug Tail Recursion,  Next: Unknown Locations and Interrupts,  Prev: Function Names,  Up: Stack Frames

5.3.4 Debug Tail Recursion
--------------------------

The compiler is "properly tail recursive."  If a function call is in a
tail-recursive position, the stack frame will be deallocated _at the
time of the call_, rather than after the call returns.  Consider this
backtrace:

     (BAR ...)
     (FOO ...)

   Because of tail recursion, it is not necessarily the case that ‘FOO’
directly called ‘BAR’.  It may be that ‘FOO’ called some other function
‘FOO2’ which then called ‘BAR’ tail-recursively, as in this example:

     (defun foo ()
       ...
       (foo2 ...)
       ...)

     (defun foo2 (...)
       ...
       (bar ...))

     (defun bar (...)
       ...)

   Usually the elimination of tail-recursive frames makes debugging more
pleasant, since these frames are mostly uninformative.  If there is any
doubt about how one function called another, it can usually be
eliminated by finding the source location in the calling frame.  *Note
Source Location Printing::.

   The elimination of tail-recursive frames can be prevented by
disabling tail-recursion optimization, which happens when the ‘debug’
optimization quality is greater than ‘2’.  *Note Debugger Policy
Control::.


File: sbcl.info,  Node: Unknown Locations and Interrupts,  Prev: Debug Tail Recursion,  Up: Stack Frames

5.3.5 Unknown Locations and Interrupts
--------------------------------------

The debugger operates using special debugging information attached to
the compiled code.  This debug information tells the debugger what it
needs to know about the locations in the code where the debugger can be
invoked.  If the debugger somehow encounters a location not described in
the debug information, then it is said to be “unknown”.  If the code
location for a frame is unknown, then some variables may be
inaccessible, and the source location cannot be precisely displayed.

   There are three reasons why a code location could be unknown:

   • There is inadequate debug information due to the value of the
     ‘debug’ optimization quality.  *Note Debugger Policy Control::.

   • The debugger was entered because of an interrupt such as <C-c>.

   • A hardware error such as "‘bus error’" occurred in code that was
     compiled unsafely due to the value of the ‘safety’ optimization
     quality.

   In the last two cases, the values of argument variables are
accessible, but may be incorrect.  For more details on when variable
values are accessible, see *note Variable Value Availability::.

   It is possible for an interrupt to happen when a function call or
return is in progress.  The debugger may then flame out with some
obscure error or insist that the bottom of the stack has been reached,
when the real problem is that the current stack frame can't be located.
If this happens, return from the interrupt and try again.


File: sbcl.info,  Node: Variable Access,  Next: Source Location Printing,  Prev: Stack Frames,  Up: Debugger

5.4 Variable Access
===================

There are two ways to access the current frame's local variables in the
debugger: ‘list-locals’ and ‘sb-debug:var’.

   The debugger doesn't really understand lexical scoping; it has just
one namespace for all the variables in the current stack frame.  If a
symbol is the name of multiple variables in the same function, then the
reference appears ambiguous, even though lexical scoping specifies which
value is visible at any given source location.  If the scopes of the two
variables are not nested, then the debugger can resolve the ambiguity by
observing that only one variable is accessible.

   When there are ambiguous variables, the evaluator assigns each one a
small integer identifier.  The ‘sb-debug:var’ function uses this
identifier to distinguish between ambiguous variables.  The
‘list-locals’ command prints the identifier.  In the following example,
there are two variables named ‘X’.  The first one has identifier 0
(which is not printed), the second one has identifier 1.

     X  =  1
     X#1  =  2

 -- Debugger Command: list-locals [PREFIX]
     This command prints the name and value of all variables in the
     current frame whose name has the specified PREFIX.  PREFIX may be a
     string or a symbol.  If no PREFIX is given, then all available
     variables are printed.  If a variable has a potentially ambiguous
     name, then the name is printed with a "‘#IDENTIFIER’" suffix, where
     IDENTIFIER is the small integer used to make the name unique.

 -- Function: var NAME &optional IDENTIFIER
     This function returns the value of the variable in the current
     frame with the specified NAME.  If supplied, IDENTIFIER determines
     which value to return when there are ambiguous variables.

     When NAME is a symbol, it is interpreted as the symbol name of the
     variable, i.e.  the package is significant.  If NAME is an
     uninterned symbol (gensym), then return the value of the uninterned
     variable with the same name.  If NAME is a string, ‘sb-debug:var’
     interprets it as the prefix of a variable name that must
     unambiguously complete to the name of a valid variable.

     IDENTIFIER is used to disambiguate the variable name; use
     ‘list-locals’ to find out the identifiers.

* Menu:

* Variable Value Availability::
* Note On Lexical Variable Access::


File: sbcl.info,  Node: Variable Value Availability,  Next: Note On Lexical Variable Access,  Up: Variable Access

5.4.1 Variable Value Availability
---------------------------------

The value of a variable may be unavailable to the debugger in portions
of the program where Lisp says that the variable is defined.  If a
variable value is not available, the debugger will not let you read or
write that variable.  With one exception, the debugger will never
display an incorrect value for a variable.  Rather than displaying
incorrect values, the debugger tells you the value is unavailable.

   The one exception is this: if you interrupt (e.g., with <C-c>) or if
there is an unexpected hardware error such as "‘bus error’" (which
should only happen in unsafe code), then the values displayed for
arguments to the interrupted frame might be incorrect.(1)  This
exception applies only to the interrupted frame: any frame farther down
the stack will be fine.

   The value of a variable may be unavailable for these reasons:

   • The value of the ‘debug’ optimization quality may have omitted
     debug information needed to determine whether the variable is
     available.  Unless a variable is an argument, its value will only
     be available when ‘debug’ is at least ‘2’.

   • The compiler did lifetime analysis and determined that the value
     was no longer needed, even though its scope had not been exited.
     Lifetime analysis is inhibited when the ‘debug’ optimization
     quality is ‘3’.

   • The variable's name is an uninterned symbol (gensym).  To save
     space, the compiler only dumps debug information about uninterned
     variables when the ‘debug’ optimization quality is ‘3’.

   • The frame's location is unknown (*note Unknown Locations and
     Interrupts::) because the debugger was entered due to an interrupt
     or unexpected hardware error.  Under these conditions the values of
     arguments will be available, but might be incorrect.  This is the
     exception mentioned above.

   • The variable (or the code referencing it) was optimized out of
     existence.  Variables with no reads are always optimized away.  The
     degree to which the compiler deletes variables will depend on the
     value of the ‘compilation-speed’ optimization quality, but most
     source-level optimizations are done under all compilation policies.

   • The variable is never set and its definition looks like
          (LET ((var1 var2))
             ...)
     In this case, ‘var1’ is substituted with ‘var2’.

   • The variable is never set and is referenced exactly once.  In this
     case, the reference is substituted with the variable initial value.

   Since it is especially useful to be able to get the arguments to a
function, argument variables are treated specially when the ‘speed’
optimization quality is less than ‘3’ and the ‘debug’ quality is at
least ‘1’.  With this compilation policy, the values of argument
variables are almost always available everywhere in the function, even
at unknown locations.  For non-argument variables, ‘debug’ must be at
least ‘2’ for values to be available, and even then, values are only
available at known locations.

   ---------- Footnotes ----------

   (1) Since the location of an interrupt or hardware error will always
be an unknown location, non-argument variable values will never be
available in the interrupted frame.  *Note Unknown Locations and
Interrupts::.


File: sbcl.info,  Node: Note On Lexical Variable Access,  Prev: Variable Value Availability,  Up: Variable Access

5.4.2 Note On Lexical Variable Access
-------------------------------------

When the debugger command loop establishes variable bindings for
available variables, these variable bindings have lexical scope and
dynamic extent.(1)  You can close over them, but such closures can't be
used as upward function arguments.

   You can also set local variables using ‘setq’, but if the variable
was closed over in the original source and never set, then setting the
variable in the debugger may not change the value in all the functions
the variable is defined in.  Another risk of setting variables is that
you may assign a value of a type that the compiler proved the variable
could never take on.  This may result in bad things happening.

   ---------- Footnotes ----------

   (1) The variable bindings are actually created using the Lisp
‘symbol-macrolet’ special form.


File: sbcl.info,  Node: Source Location Printing,  Next: Debugger Policy Control,  Prev: Variable Access,  Up: Debugger

5.5 Source Location Printing
============================

One of the debugger's capabilities is source level debugging of compiled
code.  These commands display the source location for the current frame:

 -- Debugger Command: source [CONTEXT]
     This command displays the file that the current frame's function
     was defined from (if it was defined from a file), and then the
     source form responsible for generating the code that the current
     frame was executing.  If CONTEXT is specified, then it is an
     integer specifying the number of enclosing levels of list structure
     to print.

   The source form for a location in the code is the innermost list
present in the original source that encloses the form responsible for
generating that code.  If the actual source form is not a list, then
some enclosing list will be printed.  For example, if the source form
was a reference to the variable ‘*some-random-special*’, then the
innermost enclosing evaluated form will be printed.  Here are some
possible enclosing forms:

     (let ((a *some-random-special*))
       ...)

     (+ *some-random-special* ...)

   If the code at a location was generated from the expansion of a macro
or a source-level compiler optimization, then the form in the original
source that expanded into that code will be printed.  Suppose the file
‘/usr/me/mystuff.lisp’ looked like this:

     (defmacro mymac ()
       '(myfun))

     (defun foo ()
       (mymac)
       ...)

   If ‘foo’ has called ‘myfun’, and is waiting for it to return, then
the ‘source’ command would print:

     ; File: /usr/me/mystuff.lisp

     (MYMAC)

   Note that the macro use was printed, not the actual function call
form, ‘(myfun)’.

   If enclosing source is printed by giving an argument to ‘source’ or
‘vsource’, then the actual source form is marked by wrapping it in a
list whose first element is ‘#:***HERE***’.  In the previous example,
‘source 1’ would print:

     ; File: /usr/me/mystuff.lisp

     (DEFUN FOO ()
       (#:***HERE***
        (MYMAC))
       ...)

* Menu:

* How the Source is Found::
* Source Location Availability::


File: sbcl.info,  Node: How the Source is Found,  Next: Source Location Availability,  Up: Source Location Printing

5.5.1 How the Source is Found
-----------------------------

If the code was defined from Lisp by ‘compile’ or ‘eval’, then the
source can always be reliably located.  If the code was defined from a
‘fasl’ file created by ‘compile-file’, then the debugger gets the source
forms it prints by reading them from the original source file.  This is
a potential problem, since the source file might have moved or changed
since the time it was compiled.

   The source file is opened using the ‘truename’ of the source file
pathname originally given to the compiler.  This is an absolute pathname
with all logical names and symbolic links expanded.  If the file can't
be located using this name, then the debugger gives up and signals an
error.

   If the source file can be found, but has been modified since the time
it was compiled, the debugger prints this warning:

     ; File has been modified since compilation:
     ;   FILENAME
     ; Using form offset instead of character position.

   where FILENAME is the name of the source file.  It then proceeds
using a robust but not foolproof heuristic for locating the source.
This heuristic works if:

   • No top-level forms before the top-level form containing the source
     have been added or deleted, and

   • The top-level form containing the source has not been modified
     much.  (More precisely, none of the list forms beginning before the
     source form have been added or deleted.)

   If the heuristic doesn't work, the displayed source will be wrong,
but will probably be near the actual source.  If the "shape" of the
top-level form in the source file is too different from the original
form, then an error will be signaled.  When the heuristic is used, the
source location commands are noticeably slowed.

   Source location printing can also be confused if (after the source
was compiled) a read-macro you used in the code was redefined to expand
into something different, or if a read-macro ever returns the same ‘eq’
list twice.  If you don't define read macros and don't use ‘##’ in
perverted ways, you don't need to worry about this.


File: sbcl.info,  Node: Source Location Availability,  Prev: How the Source is Found,  Up: Source Location Printing

5.5.2 Source Location Availability
----------------------------------

Source location information is only available when the ‘debug’
optimization quality is at least ‘2’.  If source location information is
unavailable, the source commands will give an error message.

   If source location information is available, but the source location
is unknown because of an interrupt or unexpected hardware error (*note
Unknown Locations and Interrupts::), then the command will print:

     Unknown location: using block start.

   and then proceed to print the source location for the start of the
_basic block_ enclosing the code location.  It's a bit complicated to
explain exactly what a basic block is, but here are some properties of
the block start location:

   • The block start location may be the same as the true location.

   • The block start location will never be later in the program's flow
     of control than the true location.

   • No conditional control structures (such as ‘if’, ‘cond’, ‘or’) will
     intervene between the block start and the true location (but note
     that some conditionals present in the original source could be
     optimized away.)  Function calls _do not_ end basic blocks.

   • The head of a loop will be the start of a block.

   • The programming language concept of "block structure" and the Lisp
     ‘block’ special form are totally unrelated to the compiler's basic
     block.

   In other words, the true location lies between the printed location
and the next conditional (but watch out because the compiler may have
changed the program on you.)


File: sbcl.info,  Node: Debugger Policy Control,  Next: Exiting Commands,  Prev: Source Location Printing,  Up: Debugger

5.6 Debugger Policy Control
===========================

The compilation policy specified by ‘optimize’ declarations affects the
behavior seen in the debugger.  The ‘debug’ quality directly affects the
debugger by controlling the amount of debugger information dumped.
Other optimization qualities have indirect but observable effects due to
changes in the way compilation is done.

   Unlike the other optimization qualities (which are compared in
relative value to evaluate tradeoffs), the ‘debug’ optimization quality
is directly translated to a level of debug information.  This absolute
interpretation allows the user to count on a particular amount of debug
information being available even when the values of the other qualities
are changed during compilation.  These are the levels of debug
information that correspond to the values of the ‘debug’ quality:

‘0’
     Only the function name and enough information to allow the stack to
     be parsed.

‘> 0’
     Any level greater than ‘0’ gives level ‘0’ plus all argument
     variables.  Values will only be accessible if the argument variable
     is never set and ‘speed’ is not ‘3’.  SBCL allows any real value
     for optimization qualities.  It may be useful to specify ‘0.5’ to
     get backtrace argument display without argument documentation.

‘1’
     Level ‘1’ provides argument documentation (printed argument lists)
     and derived argument/result type information.  This makes
     ‘describe’ more informative, and allows the compiler to do
     compile-time argument count and type checking for any calls
     compiled at run-time.  This is the default.

‘2’
     Level ‘1’ plus all interned local variables, source location
     information, and lifetime information that tells the debugger when
     arguments are available (even when ‘speed’ is ‘3’ or the argument
     is set).

‘> 2’
     Any level greater than ‘2’ gives level ‘2’ and in addition disables
     tail-call optimization, so that the backtrace will contain frames
     for all invoked functions, even those in tail positions.

‘3’
     Level ‘2’ plus all uninterned variables.  In addition, lifetime
     analysis is disabled (even when ‘speed’ is ‘3’), ensuring that all
     variable values are available at any known location within the
     scope of the binding.  This has a speed penalty in addition to the
     obvious space penalty.

     Inlining of local functions is inhibited so that they may be
     ‘trace’d.

‘> (max speed space)’
     If ‘debug’ is greater than both ‘speed’ and ‘space’, the command
     ‘return’ can be used to continue execution by returning a value
     from the current stack frame.

‘> (max speed space compilation-speed)’
     If ‘debug’ is greater than all of ‘speed’, ‘space’ and
     ‘compilation-speed’ the code will be steppable (*note Single
     Stepping::).

   As you can see, if the ‘speed’ quality is ‘3’, debugger performance
is degraded.  This effect comes from the elimination of argument
variable special-casing (*note Variable Value Availability::).  Some
degree of speed/debuggability tradeoff is unavoidable, but the effect is
not too drastic when ‘debug’ is at least ‘2’.

   In addition to ‘inline’ and ‘notinline’ declarations, the relative
values of the ‘speed’ and ‘space’ qualities also change whether
functions are inline expanded.  If a function is inline expanded, then
there will be no frame to represent the call, and the arguments will be
treated like any other local variable.  Functions may also be
"semi-inline", in which case there is a frame to represent the call, but
the call is to an optimized local version of the function, not to the
original function.


File: sbcl.info,  Node: Exiting Commands,  Next: Information Commands,  Prev: Debugger Policy Control,  Up: Debugger

5.7 Exiting Commands
====================

These commands get you out of the debugger.

 -- Debugger Command: toplevel
     Throw to top level.

 -- Debugger Command: restart [N]
     Invokes the Nth restart case as displayed by the ‘error’ command.
     If N is not specified, the available restart cases are reported.

 -- Debugger Command: continue
     Calls ‘continue’ on the condition given to ‘debug’.  If there is no
     restart case named CONTINUE, then an error is signaled.

 -- Debugger Command: abort
     Calls ‘abort’ on the condition given to ‘debug’.  This is useful
     for popping debug command loop levels or aborting to top level, as
     the case may be.

 -- Debugger Command: return VALUE
     Returns VALUE from the current stack frame.  This command is
     available when the ‘debug’ optimization quality is greater than
     both ‘speed’ and ‘space’.  Care must be taken that the value is of
     the same type as SBCL expects the stack frame to return.

 -- Debugger Command: restart-frame
     Restarts execution of the current stack frame.  This command is
     available when the ‘debug’ optimization quality is greater than
     both ‘speed’ and ‘space’ and when the frame is for a global
     function.  If the function is redefined in the debugger before the
     frame is restarted, the new function will be used.


File: sbcl.info,  Node: Information Commands,  Next: Breakpoint Commands,  Prev: Exiting Commands,  Up: Debugger

5.8 Information Commands
========================

Most of these commands print information about the current frame or
function, but a few show general information.

 -- Debugger Command: help
 -- Debugger Command: ?
     Displays a synopsis of debugger commands.

 -- Debugger Command: describe
     Calls ‘describe’ on the current function and displays the number of
     local variables.

 -- Debugger Command: print
     Displays the current function call as it would be displayed by
     moving to this frame.

 -- Debugger Command: error
     Prints the condition given to ‘invoke-debugger’ and the active
     proceed cases.

 -- Debugger Command: backtrace [N]
     Displays all the frames from the current to the bottom.  Only shows
     N frames if specified.  The printing is controlled by
     ‘*debug-print-variable-alist*’.


File: sbcl.info,  Node: Breakpoint Commands,  Prev: Information Commands,  Up: Debugger

5.9 Breakpoint Commands
=======================

SBCL supports setting of breakpoints inside compiled functions and
stepping of compiled code.  Breakpoints can only be set at known
locations (*note Unknown Locations and Interrupts::), so these commands
are largely useless unless the ‘debug’ optimize quality is at least ‘2’
(*note Debugger Policy Control::).  These commands manipulate
breakpoints:

 -- Debugger Command: breakpoint LOCATION [OPTION VALUE]*
     Set a breakpoint in some function.  LOCATION may be an integer code
     location number (as displayed by ‘list-locations’) or a keyword.
     The keyword can be used to indicate setting a breakpoint at the
     function start (‘:start’, ‘:s’) or function end (‘:end’, ‘:e’).
     The ‘breakpoint’ command has ‘:condition’, ‘:break’, ‘:print’ and
     ‘:function’ options which work similarly to the ‘trace’ options.

 -- Debugger Command: list-locations [FUNCTION]
 -- Debugger Command: ll [FUNCTION]
     List all the code locations in the current frame's function, or in
     FUNCTION if it is supplied.  The display format is the code
     location number, a colon and then the source form for that
     location:

          3: (1- N)

     If consecutive locations have the same source, then a numeric range
     like ‘3-5:’ will be printed.  For example, a default function call
     has a known location both immediately before and after the call,
     which would result in two code locations with the same source.  The
     listed function becomes the new default function for breakpoint
     setting (via the ‘breakpoint’) command.

 -- Debugger Command: list-breakpoints
 -- Debugger Command: lb
     List all currently active breakpoints with their breakpoint number.

 -- Debugger Command: delete-breakpoint [NUMBER]
 -- Debugger Command: db [NUMBER]
     Delete a breakpoint specified by its breakpoint number.  If no
     number is specified, delete all breakpoints.

 -- Debugger Command: step*
     Step to the next possible breakpoint location in the current
     function.  This always steps over function calls, instead of
     stepping into them

* Menu:

* Breakpoint Example::


File: sbcl.info,  Node: Breakpoint Example,  Prev: Breakpoint Commands,  Up: Breakpoint Commands

5.9.1 Breakpoint Example
------------------------

Consider this definition of the factorial function:

     (defun ! (n)
       (if (zerop n)
           1
           (* n (! (1- n)))))

   This debugger session demonstrates the use of breakpoints:

     * (break)  ; invoke debugger

     debugger invoked on a SIMPLE-CONDITION in thread 11184: break

     restarts (invokable by number or by possibly-abbreviated name):
       0: [CONTINUE] Return from BREAK.
       1: [ABORT   ] Reduce debugger level (leaving debugger, returning to toplevel).
       2: [TOPLEVEL] Restart at toplevel READ/EVAL/PRINT loop.
     ("varargs entry for top level local call BREAK" "break")
     0] ll #'!

     0-1: (SB-INT:NAMED-LAMBDA ! (N) (BLOCK ! (IF (ZEROP N) 1 (* N (! #)))))
     2: (BLOCK ! (IF (ZEROP N) 1 (* N (! (1- N)))))
     3: (ZEROP N)
     4: (* N (! (1- N)))
     5: (1- N)
     6: (! (1- N))
     7-8: (* N (! (1- N)))
     9-10: (IF (ZEROP N) 1 (* N (! (1- N))))
     0] br 4

     (* N (! (1- N)))
     1: 4 in !
     added
     0] toplevel

     > (! 10) ; Call the function

     *Breakpoint hit*

     Restarts:
       0: [CONTINUE] Return from BREAK.
       1: [ABORT   ] Return to Top-Level.

     Debug  (type H for help)

     (! 10) ; We are now in first call (arg 10) before the multiply
     Source: (* N (! (1- N)))
     3] step*

     *Step*

     (! 10) ; We have finished evaluation of (1- n)
     Source: (1- N)
     3] step*

     *Breakpoint hit*

     Restarts:
       0: [CONTINUE] Return from BREAK.
       1: [ABORT   ] Return to Top-Level.

     Debug  (type H for help)

     (! 9) ; We hit the breakpoint in the recursive call
     Source: (* N (! (1- N)))
     3]

   Note: The ‘step*’ command differs from the single stepping commands
in that it also functions in compiled code which has not been compiled
with stepping instrumentation.  It simply steps to the next compiled
code location.  In the future, this form of stepping may be improved
enough to subsume the instrumentation based stepping commands, which
have much higher overhead.


File: sbcl.info,  Node: Function Tracing,  Next: Single Stepping,  Prev: Breakpoint Commands,  Up: Debugger

5.10 Function Tracing
=====================

The tracer causes selected functions to print their arguments and their
results whenever they are called.  Options allow conditional printing of
the trace information and conditional breakpoints on function entry or
exit.

   In SBCL, tracing can be done either by temporarily redefining the
function name (encapsulation), or using breakpoints.  When breakpoints
are used, the function object itself is destructively modified to cause
the tracing action.  The advantage of using breakpoints is that tracing
works even when the function is anonymously called via ‘funcall’, that
function object identity is preserved, and that anonymous and local
functions can also be traced.

 -- Macro: trace &rest specs
     ‘trace’ {Option Global-Value}* {Name {Option Value}*}*

     ‘trace’ is a debugging tool that provides information when
     specified functions are called.  In its simplest form:

                 (TRACE NAME-1 NAME-2 ...)

     The NAMEs are not evaluated.  Each may be one of the following:
        • ‘symbol’, denoting a function or macro.
        • ‘fname’, a valid function name, denoting a function.
        • (‘method’ ‘fname’ ‘qualifiers*’ (‘specializers*’)) denoting a
          method.
        • (‘compiler-macro’ ‘symbol’) denoting a compiler macro.
        • (‘labels’ ‘fname’ ‘:in’ ‘outer-name’) or (‘flet’ ‘fname’ ‘:in’
          ‘outer-name’) denoting a local function where ‘outer-name’ may
          be any of the previous names for functions, macros, methods or
          compiler macros.  Tracing local functions may require ‘debug’
          policy 3 to inhibit inlining.
        • ‘string’ denoting all functions fbound to symbols whose home
          package is the package with the given name.

     Options allow modification of the default behavior.  Each option is
     a pair of an option keyword and a value form.  Global options are
     specified before the first name, and affect all functions traced by
     a given use of ‘trace’.  Options may also be interspersed with
     function names, in which case they act as local options, only
     affecting tracing of the immediately preceding function name.
     Local options override global options.

     By default, ‘trace’ causes a printout on ‘*trace-output*’ each time
     that one of the named functions is entered or returns.  (This is
     the basic, ‘ansi’ Common Lisp behavior of ‘trace’.)

     The following options are defined:

     _‘:report’ Report-Type_
          If Report-Type is ‘trace’ (the default) then information is
          reported by printing immediately.  If Report-Type is ‘nil’,
          then the only effect of the trace is to execute other options
          (e.g.  ‘print’ or ‘break’).  Otherwise, Report-Type is treated
          as a function designator and, for each trace event, funcalled
          with 5 arguments: trace depth (a non-negative integer), a
          function name or a function object, a keyword (‘:enter’,
          ‘:exit’ or ‘:non-local-exit’), a stack frame, and a list of
          values (arguments or return values).

     _‘:condition’ Form_
     _‘:condition-after’ Form_
     _‘:condition-all’ Form_
          If ‘:condition’ is specified, then ‘trace’ does nothing unless
          Form evaluates to true at the time of the call.
          ‘:condition-after’ is similar, but suppresses the initial
          printout, and is tested when the function returns.
          ‘:condition-all’ tries both before and after.

     _‘:break’ Form_
     _‘:break-after’ Form_
     _‘:break-all’ Form_
          If specified, and Form evaluates to true, then the debugger is
          invoked at the start of the function, at the end of the
          function, or both, according to the respective option.

     _‘:print’ Form_
     _‘:print-after’ Form_
     _‘:print-all’ Form_
          In addition to the usual printout, the result of evaluating
          Form is printed at the start of the function, at the end of
          the function, or both, according to the respective option.
          Multiple print options cause multiple values to be printed.

     _‘:wherein’ Names_
          If specified, Names is a function name or list of names.
          ‘trace’ does nothing unless a call to one of those functions
          encloses the call to this function (i.e.  it would appear in a
          backtrace.)  Anonymous functions have string names like "DEFUN
          FOO".

     _‘:encapsulate’ {:DEFAULT | ‘t’ | NIL}_
          If ‘t’, the default, tracing is done via encapsulation
          (redefining the function name) rather than by modifying the
          function.  ‘:default’ is not the default, but means to use
          encapsulation for interpreted functions and funcallable
          instances, breakpoints otherwise.  When encapsulation is used,
          forms are *not* evaluated in the function's lexical
          environment, but ‘sb-debug:arg’ can still be used.

     _‘:methods’ {T | NIL}_
          If ‘t’, any function argument naming a generic function will
          have its methods traced in addition to the generic function
          itself.

     _‘:function’ Function-Form_
          This is a not really an option, but rather another way of
          specifying what function to trace.  The Function-Form is
          evaluated immediately, and the resulting function is traced.

     ‘:condition’, ‘:break’ and ‘:print’ forms are evaluated in a
     context which mocks up the lexical environment of the called
     function, so that ‘sb-debug:var’ and ‘sb-debug:arg’ can be used.
     The ‘-after’ and ‘-all’ forms can use also use ‘sb-debug:arg’.  In
     forms which are evaluated after the function call, (‘sb-debug:arg’
     ‘n’) returns the N-th value returned by the function.

   In the case of functions where the known return convention is used to
optimize, encapsulation may be necessary in order to make tracing work
at all.  The symptom of this occurring is an error stating
         Error in function FOO: :FUNCTION-END breakpoints are
         currently unsupported for the known return convention.
   in such cases we recommend using ‘(trace FOO :encapsulate t)’

 -- Macro: untrace &rest specs
     Remove tracing from the specified functions.  Untraces all
     functions when called with no arguments.

 -- Variable: *trace-indentation-step*
     the increase in trace indentation at each call level

 -- Variable: *max-trace-indentation*
     If the trace indentation exceeds this value, then indentation
     restarts at 0.

 -- Variable: *trace-encapsulate-default*
     the default value for the ‘:encapsulate’ option to ‘trace’

 -- Variable: *trace-report-default*
     the default value for the ‘:report’ option to ‘trace’


File: sbcl.info,  Node: Single Stepping,  Next: Enabling and Disabling the Debugger,  Prev: Function Tracing,  Up: Debugger

5.11 Single Stepping
====================

SBCL includes an instrumentation based single-stepper for compiled code,
that can be invoked via the ‘step’ macro, or from within the debugger.
*Note Debugger Policy Control::, for details on enabling stepping for
compiled code.

   The following debugger commands are used for controlling single
stepping.

 -- Debugger Command: start
     Selects the ‘continue’ restart if one exists and starts single
     stepping.  None of the other single stepping commands can be used
     before stepping has been started either by using ‘start’ or by
     using the standard ‘step’ macro.

 -- Debugger Command: step
     Steps into the current form.  Stepping will be resumed when the
     next form that has been compiled with stepper instrumentation is
     evaluated.

 -- Debugger Command: next
     Steps over the current form.  Stepping will be disabled until
     evaluation of the form is complete.

 -- Debugger Command: out
     Steps out of the current frame.  Stepping will be disabled until
     the topmost stack frame that had been stepped into returns.

 -- Debugger Command: stop
     Stops the single stepper and resumes normal execution.

 -- Macro: step form
     The form is evaluated with single stepping enabled.  Function calls
     outside the lexical scope of the form can be stepped into only if
     the functions in question have been compiled with sufficient
     ‘debug’ policy to be at least partially steppable.


File: sbcl.info,  Node: Enabling and Disabling the Debugger,  Prev: Single Stepping,  Up: Debugger

5.12 Enabling and Disabling the Debugger
========================================

In certain contexts (e.g., non-interactive applications), it may be
desirable to turn off the SBCL debugger (and possibly re-enable it).
The functions here control the debugger.

 -- Function: disable-debugger
     When invoked, this function will turn off both the ‘sbcl’ debugger
     and ‘ldb’ (the low-level debugger).  See also ‘enable-debugger’.

 -- Function: enable-debugger
     Restore the debugger if it has been turned off by
     ‘disable-debugger’.


File: sbcl.info,  Node: Efficiency,  Next: Beyond the ANSI Standard,  Prev: Debugger,  Up: Top

6 Efficiency
************

* Menu:

* Slot access::
* Stack allocation::
* Modular arithmetic::
* Recognized idioms::
* Global and Always-Bound variables::
* Miscellaneous Efficiency Issues::


File: sbcl.info,  Node: Slot access,  Next: Stack allocation,  Up: Efficiency

6.1 Slot access
===============

6.1.1 Structure object slot access
----------------------------------

Structure slot accessors are efficient only if the compiler is able to
open code them: compiling a call to a structure slot accessor before the
structure is defined, declaring one ‘notinline’, or passing it as a
functional argument to another function causes severe performance
degradation.

6.1.2 Standard object slot access
---------------------------------

The most efficient way to access a slot of a ‘standard-object’ is by
using ‘slot-value’ with a constant slot name argument inside a
‘defmethod’ body, where the variable holding the instance is a
specializer parameter of the method and is never assigned to.  The cost
is roughly 1.6 times that of an open coded structure slot accessor.

   Second most efficient way is to use a CLOS slot accessor, or
‘slot-value’ with a constant slot name argument, but in circumstances
other than specified above.  This may be up to 3 times as slow as the
method described above.

   Example:

     (defclass foo () ((bar)))

     ;; Fast: specializer and never assigned to
     (defmethod quux ((foo foo) new)
       (let ((old (slot-value foo 'bar)))
         (setf (slot-value foo 'bar) new)
         old))

     ;; Slow: not a specializer
     (defmethod quux ((foo foo) new)
       (let* ((temp foo)
              (old (slot-value temp 'bar)))
         (setf (slot-value temp 'bar) new)
         old))

     ;; Slow: assignment to FOO
     (defmethod quux ((foo foo) new)
       (let ((old (slot-value foo 'bar)))
         (setf (slot-value foo 'bar) new)
         (setf foo new)
         old))

   Note that when profiling code such as this, the first few calls to
the generic function are not representative, as the dispatch mechanism
is lazily set up during those calls.


File: sbcl.info,  Node: Stack allocation,  Next: Modular arithmetic,  Prev: Slot access,  Up: Efficiency

6.2 Stack allocation
====================

SBCL has fairly extensive support for performing allocations on the
stack when a variable or function is declared ‘dynamic-extent’.  The
‘dynamic-extent’ declarations are not verified, but are simply trusted
as long as ‘sb-ext:*stack-allocate-dynamic-extent*’ is true.

 -- Variable: *stack-allocate-dynamic-extent*
     If true (the default), the compiler believes ‘dynamic-extent’
     declarations and stack allocates otherwise inaccessible parts of
     the object whenever possible.

   SBCL recognizes any value which a variable declared ‘dynamic-extent’
can take on as having dynamic extent.  This means that, in addition to
the value a variable is bound to initially, a value assigned to a
variable by ‘setq’ is also recognized as having dynamic extent when the
variable is declared ‘dynamic-extent’.  Users can thus build complex
structures on the stack using iteration and ‘setq’.

   At present, SBCL implements stack allocation for the following kinds
of values when they are recognized as having dynamic extent:

   • ‘&rest’ lists

   • the results of ‘cons’, ‘list’, ‘list*’, and ‘vector’

   • the result of simple forms of ‘make-array’: stack allocation is
     possible only if the resulting array is known to be both simple and
     one-dimensional, and has a constant ‘:element-type’.

     *Note*: stack space is limited, so allocation of a large vector may
     cause stack overflow.  Stack overflow checks are done except in
     zero ‘safety’ policies.

   • closures defined with ‘flet’ or ‘labels’ with a bound
     ‘dynamic-extent’ declaration.

   • anonymous closures defined with ‘lambda’

   • user-defined structures when the structure constructor defined
     using ‘defstruct’ has been declared ‘inline’

     *Note*: structures with "raw" slots can currently be
     stack-allocated only on x86 and x86-64.  A "raw" slot is one whose
     declared type is a subtype of exactly one of: ‘double-float’,
     ‘single-float’, ‘(complex double-float)’, ‘(complex single-float)’,
     or ‘sb-ext:word’; but as an exception to the preceding, any subtype
     of ‘fixnum’ is not stored as raw despite also being a subtype of
     ‘sb-ext:word’.

   • otherwise-inaccessible parts of objects recognized to be dynamic
     extent.  The support for detecting when this applies is very
     sophisticated.  The compiler can do this detection when any value
     form for a variable contains conditional allocations, function
     calls, inlined functions, anonymous closures, or even other
     variables.  This allows stack allocation of complex structures.

   Examples:

     ;;; Declaiming a structure constructor inline before definition makes
     ;;; stack allocation possible.
     (declaim (inline make-thing))
     (defstruct thing obj next)

     ;;; Stack allocation of various objects bound to DYNAMIC-EXTENT
     ;;; variables.
     (let* ((list (list 1 2 3))
            (nested (cons (list 1 2) (list* 3 4 (list 5))))
            (vector (make-array 3 :element-type 'single-float))
            (thing (make-thing :obj list
                               :next (make-thing :obj (make-array 3))))
            (closure (let ((y ...)) (lambda () y))))
       (declare (dynamic-extent list nested vector thing closure))
       ...)

     ;;; Stack allocation of objects assigned to DYNAMIC-EXTENT variables.
     (let ((x nil))
       (declare (dynamic-extent x))
       (setq x (list 1 2 3))
       (dotimes (i 10)
         (setq x (cons i x)))
       ...)

     ;;; Stack allocation of arguments to a local function is equivalent
     ;;; to stack allocation of local variable values.
     (flet ((f (x)
              (declare (dynamic-extent x))
              ...))
       ...
       (f (list 1 2 3))
       (f (cons (cons 1 2) (cons 3 4)))
       ...)

     ;;; Stack allocation of &REST lists
     (defun foo (&rest args)
       (declare (dynamic-extent args))
       ...)

   As a notable exception to recognizing otherwise inaccessible parts of
other recognized dynamic extent values, SBCL does not as of 1.0.48.21
propagate dynamic-extentness through ‘&rest’ arguments - but another
conforming implementation might, so portable code should not rely on
this.

     (declaim (inline foo))
     (defun foo (fun &rest arguments)
       (declare (dynamic-extent arguments))
       (apply fun arguments))

     (defun bar (a)
       ;; SBCL will heap allocate the result of (LIST A), and stack allocate
       ;; only the spine of the &rest list -- so this is safe, but unportable.
       ;;
       ;; Another implementation, including earlier versions of SBCL might consider
       ;; (LIST A) to be otherwise inaccessible and stack-allocate it as well!
       (foo #'car (list a)))

   If dynamic extent constraints specified in the Common Lisp standard
are violated, the best that can happen is for the program to have
garbage in variables and return values; more commonly, the system will
crash.

   In particular, it is important to realize that this can interact in
suprising ways with the otherwise inaccessible parts criterion:

     (let* ((a (list 1 2 3))
            (b (cons a a)))
        (declare (dynamic-extent b))
        ;; Unless A is accessed elsewhere as well, SBCL will consider
        ;; it to be otherwise inaccessible -- it can only be accessed
        ;; through B, after all -- and stack allocate it as well.
        ;;
        ;; Hence returning (CAR B) here is unsafe.
        ...)

   SBCL also performs sophisticated escape analysis to enable automatic
stack allocation of local functions without any bound dynamic extent
declarations in many situations where the compiler can prove that no
uses escape (traditional Lisp terminology names this situation "all uses
are downward funargs").  For example, in the following function, the
local function ‘#'predicatep’ is stack allocated, because the compiler
understands that the built-in function ‘#'position-if’ only uses its
first argument as a downward funarg:

     (let ((acc 0))
       (flet ((predicatep (num) (plusp (+ num off))))
         (dotimes (i 10)
           (incf acc (position-if #'predicatep array)))
         (if (plusp off)
             (incf acc (if (positivep acc) 10 3))
             (incf acc (position-if #'predicatep array))))
       acc)

   Users can also declare that their own functions take downward funargs
by adding bound dynamic extent declarations on the function arguments.

     (defun trivial-hof (fun arg)
       (declare (dynamic-extent fun))
       (funcall fun 3 arg))

   Currently, such dynamic extent declarations only cause stack
allocation of downward funargs at call sites on sufficiently unsafe
policy.  This is partly because the compiler is currently not able to
detect incorrect usage of dynamic extent declarations.

     (defun autodxclosure1 (&optional (x 4))
       ;; Calling a higher-order function will only implicitly
       ;; stack-allocate a funarg if the callee is trusted (a CL: function)
       ;; or the caller is unsafe.
       (declare (optimize speed (safety 0) (debug 0)))
       (trivial-hof (lambda (a b) (+ a b x)) 92))


File: sbcl.info,  Node: Modular arithmetic,  Next: Recognized idioms,  Prev: Stack allocation,  Up: Efficiency

6.3 Modular arithmetic
======================

Some numeric functions have a property: N lower bits of the result
depend only on N lower bits of (all or some) arguments.  If the compiler
sees an expression of form ‘(logand EXP MASK)’, where EXP is a tree of
such "good" functions and MASK is known to be of type ‘(unsigned-byte
W)’, where W is a "good" width, all intermediate results will be cut to
W bits (but it is not done for variables and constants!).  This often
results in an ability to use simple machine instructions for the
functions.

   Consider an example.

     (defun i (x y)
       (declare (type (unsigned-byte 32) x y))
       (ldb (byte 32 0) (logxor x (lognot y))))

   The result of ‘(lognot y)’ will be negative and of type ‘(signed-byte
33)’, so a naive implementation on a 32-bit platform is unable to use
32-bit arithmetic here.  But modular arithmetic optimizer is able to do
it: because the result is cut down to 32 bits, the compiler will replace
‘logxor’ and ‘lognot’ with versions cutting results to 32 bits, and
because terminals (here--expressions ‘x’ and ‘y’) are also of type
‘(unsigned-byte 32)’, 32-bit machine arithmetic can be used.

   As of SBCL 0.8.5 "good" functions are ‘+’, ‘-’; ‘logand’, ‘logior’,
‘logxor’, ‘lognot’ and their combinations; and ‘ash’ with the positive
second argument.  "Good" widths are 32 on 32-bit CPUs and 64 on 64-bit
CPUs.  While it is possible to support smaller widths as well, currently
this is not implemented.

6.3.1 Signed modular arithmetic
-------------------------------

Sign-extending the result in the following way will be translated into
signed modular arithmetic:

     (defun add (a b)
       (declare (type (signed-byte 64) a b))
       (let ((u (ldb (byte 64 0) (+ a b))))
         (logior u (- (mask-field (byte 1 63) u)))))


File: sbcl.info,  Node: Recognized idioms,  Next: Global and Always-Bound variables,  Prev: Modular arithmetic,  Up: Efficiency

6.4 Recognized idioms
=====================

Common Lisp doesn't directly expose all features present in modern
hardware.  Some code patterns are recognized and turned into more
efficient hardware instructions without requiring the use of internal
features.

6.4.1 Count trailing zeros
--------------------------

     (defun ctz (n)
       (declare (type (unsigned-byte 64) n))
       (integer-length (ldb (byte 64 0) (lognor n (- n)))))
   is turned into hardware instructions on arm64 and x86-64.  It returns
64 when ‘n’ is 0.  ‘n’ can also be ‘(signed-byte 64)’ or ‘fixnum’.


File: sbcl.info,  Node: Global and Always-Bound variables,  Next: Miscellaneous Efficiency Issues,  Prev: Recognized idioms,  Up: Efficiency

6.5 Global and Always-Bound variables
=====================================

 -- Macro: defglobal name value &optional doc
     Defines ‘name’ as a global variable that is always bound.  ‘value’
     is evaluated and assigned to ‘name’ both at compile- and load-time,
     but only if ‘name’ is not already bound.

     Global variables share their values between all threads, and cannot
     be locally bound, declared special, defined as constants, and
     neither bound nor defined as symbol macros.

     See also the declarations ‘sb-ext:global’ and
     ‘sb-ext:always-bound’.

 -- Declaration: global

     Syntax: ‘(sb-ext:global symbol*)’

     Only valid as a global proclamation.

     Specifies that the named symbols cannot be proclaimed or locally
     declared ‘special’.  Proclaiming an already special or constant
     variable name as ‘global’ signal an error.  Allows more efficient
     value lookup in threaded environments in addition to expressing
     programmer intention.

 -- Declaration: always-bound

     Syntax: ‘(sb-ext:always-bound symbol*)’

     Only valid as a global proclamation.

     Specifies that the named symbols are always bound.  Inhibits
     ‘makunbound’ of the named symbols.  Proclaiming an unbound symbol
     as ‘always-bound’ signals an error.  Allows the compiler to elide
     boundness checks from value lookups.


File: sbcl.info,  Node: Miscellaneous Efficiency Issues,  Prev: Global and Always-Bound variables,  Up: Efficiency

6.6 Miscellaneous Efficiency Issues
===================================

FIXME: The material in the CMUCL manual about getting good performance
from the compiler should be reviewed, reformatted in Texinfo, lightly
edited for SBCL, and substituted into this manual.  In the meantime, the
original CMUCL manual is still 95+% correct for the SBCL version of the
Python compiler.  See the sections

   • Advanced Compiler Use and Efficiency Hints
   • Advanced Compiler Introduction
   • More About Types in Python
   • Type Inference
   • Source Optimization
   • Tail Recursion
   • Local Call
   • Block Compilation
   • Inline Expansion
   • Object Representation
   • Numbers
   • General Efficiency Hints
   • Efficiency Notes

   Besides this information from the CMUCL manual, there are a few other
points to keep in mind.

   • The CMUCL manual doesn't seem to state it explicitly, but Python
     has a mental block about type inference when assignment is
     involved.  Python is very aggressive and clever about inferring the
     types of values bound with ‘let’, ‘let*’, inline function call, and
     so forth.  However, it's much more passive and dumb about inferring
     the types of values assigned with ‘setq’, ‘setf’, and friends.  It
     would be nice to fix this, but in the meantime don't expect that
     just because it's very smart about types in most respects it will
     be smart about types involved in assignments.  (This doesn't affect
     its ability to benefit from explicit type declarations involving
     the assigned variables, only its ability to get by without explicit
     type declarations.)

   • Since the time the CMUCL manual was written, CMUCL (and thus SBCL)
     has gotten a generational garbage collector.  This means that there
     are some efficiency implications of various patterns of memory
     usage which aren't discussed in the CMUCL manual.  (Some new
     material should be written about this.)

   • SBCL has some important known efficiency problems.  Perhaps the
     most important are

        − The garbage collector is not particularly efficient, at least
          on platforms without the generational collector (as of SBCL
          0.8.9, all except x86).

        − Various aspects of the PCL implementation of CLOS are more
          inefficient than necessary.

   Finally, note that Common Lisp defines many constructs which, in the
infamous phrase, "could be compiled efficiently by a sufficiently smart
compiler".  The phrase is infamous because making a compiler which
actually is sufficiently smart to find all these optimizations
systematically is well beyond the state of the art of current compiler
technology.  Instead, they're optimized on a case-by-case basis by
hand-written code, or not optimized at all if the appropriate case
hasn't been hand-coded.  Some cases where no such hand-coding has been
done as of SBCL version 0.6.3 include

   • ‘(reduce #'f x)’ where the type of ‘x’ is known at compile time

   • various bit vector operations, e.g.  ‘(position 0 some-bit-vector)’

   • specialized sequence idioms, e.g.  ‘(remove item list :count 1)’

   • cases where local compilation policy does not require excessive
     type checking, e.g.  ‘(locally (declare (safety 1)) (assoc item
     list))’ (which currently performs safe ‘endp’ checking internal to
     assoc).

   If your system's performance is suffering because of some construct
which could in principle be compiled efficiently, but which the SBCL
compiler can't in practice compile efficiently, consider writing a patch
to the compiler and submitting it for inclusion in the main sources.
Such code is often reasonably straightforward to write; search the
sources for the string "‘deftransform’" to find many examples (some
straightforward, some less so).


File: sbcl.info,  Node: Beyond the ANSI Standard,  Next: External Formats,  Prev: Efficiency,  Up: Top

7 Beyond the ANSI Standard
**************************

SBCL is derived from CMUCL, which implements many extensions to the ANSI
standard.  SBCL doesn't support as many extensions as CMUCL, but it
still has quite a few.  *Note Contributed Modules::.

* Menu:

* Reader Extensions::
* Package-Local Nicknames::
* Package Variance::
* Garbage Collection::
* Generic Function Dispatch::
* Slot Access::
* Metaobject Protocol::
* Extensible Sequences::
* Support For Unix::
* Unicode Support::
* Customization Hooks for Users::
* Tools To Help Developers::
* Resolution of Name Conflicts::
* Hash Table Extensions::
* Random Number Generation::
* Timeouts and Deadlines::
* Miscellaneous Extensions::
* Stale Extensions::
* Efficiency Hacks::


File: sbcl.info,  Node: Reader Extensions,  Next: Package-Local Nicknames,  Up: Beyond the ANSI Standard

7.1 Reader Extensions
=====================

7.1.1 Extended Package Prefix Syntax
------------------------------------

SBCL supports extended package prefix syntax, which allows specifying an
alternate package instead of ‘*package*’ for the reader to use as the
default package for interning symbols:

     PACKAGE-NAME::FORM-WITH-INTERNING-INTO-PACKAGE

   Example:

       'foo::(bar quux zot) == '(foo::bar foo::quux foo::zot)

   ‘*package*’ is not rebound during the course of reading a form with
extended package prefix syntax; if ‘foo::bar’ would cause a read-time
package lock violation, so does ‘foo::(bar)’.

7.1.2 Symbol Name Normalization
-------------------------------

SBCL also extends the reader to normalize all symbols to Normalization
Form KC in builds with Unicode enabled.  Whether symbols are normalized
is controlled by

 -- Function: readtable-normalization readtable
     Returns ‘t’ if ‘readtable’ normalizes symbols to ‘nfkc’, and ‘nil’
     otherwise.  The ‘readtable-normalization’ of the standard readtable
     is ‘t’.

   Symbols created by ‘intern’ and similar functions are not affected by
this setting.  If ‘sb-ext:readtable-normalization’ is ‘t’, symbols that
are not normalized are escaped during printing.

7.1.3 Decimal Syntax for Rationals
----------------------------------

SBCL supports a decimal syntax for rationals, modelled after the
standard syntax for floating-point numbers.  If a number with
floating-point syntax has an exponent marker of ‘r’ or ‘R’ (rather than
one of the standard exponent markers), it is read as the rational with
the exact value of the decimal number expressed as a float.

   In addition, setting or binding the value of
‘*read-default-float-format*’ to ‘rational’ around a call to ‘read’ or
‘read-from-string’ has the effect that floating-point numbers without
exponent markers are read as rational numbers, as if there had been an
explicit ‘r’ or ‘R’ marker.

   Floating point numbers of all types are printed with an exponent
marker while the value of ‘*read-default-float-format*’ is ‘rational’;
however, rational numbers are printed in their standard syntax,
irrespective of the value of ‘*read-default-float-format*’.


File: sbcl.info,  Node: Package-Local Nicknames,  Next: Package Variance,  Prev: Reader Extensions,  Up: Beyond the ANSI Standard

7.2 Package-Local Nicknames
===========================

SBCL allows giving packages local nicknames: they allow short and
easy-to-use names to be used without fear of name conflict associated
with normal nicknames.

   A local nickname is valid only when inside the package for which it
has been specified.  Different packages can use same local nickname for
different global names, or different local nickname for same global
name.

   Symbol ‘:package-local-nicknames’ in ‘*features*’ denotes the support
for this feature.

 -- Macro: defpackage name [[option]]* ⇒ package

     Options are extended to include

        • ‘:local-nicknames (LOCAL-NICKNAME ACTUAL-PACKAGE-NAME)*’

          The package has the specified local nicknames for the
          corresponding actual packages.

     Example:

          (defpackage :bar (:intern "X"))
          (defpackage :foo (:intern "X"))
          (defpackage :quux (:use :cl) (:local-nicknames (:bar :foo) (:foo :bar)))
          (find-symbol "X" :foo) ; => FOO::X
          (find-symbol "X" :bar) ; => BAR::X
          (let ((*package* (find-package :quux)))
            (find-symbol "X" :foo))               ; => BAR::X
          (let ((*package* (find-package :quux)))
            (find-symbol "X" :bar))               ; => FOO::X

 -- Function: package-local-nicknames package-designator
     Returns an alist of (local-nickname .  actual-package) describing
     the nicknames local to the designated package.

     When in the designated package, calls to ‘find-package’ with the
     any of the local-nicknames will return the corresponding
     actual-package instead.  This also affects all implied calls to
     ‘find-package’, including those performed by the reader.

     When printing a package prefix for a symbol with a package local
     nickname, the local nickname is used instead of the real name in
     order to preserve print-read consistency.

     See also: ‘add-package-local-nickname’,
     ‘package-locally-nicknamed-by-list’,
     ‘remove-package-local-nickname’, and the ‘defpackage’ option
     ‘:local-nicknames’.

     Experimental: interface subject to change.
 -- Function: package-locally-nicknamed-by-list package-designator
     Returns a list of packages which have a local nickname for the
     designated package.

     See also: ‘add-package-local-nickname’, ‘package-local-nicknames’,
     ‘remove-package-local-nickname’, and the ‘defpackage’ option
     ‘:local-nicknames’.

     Experimental: interface subject to change.
 -- Function: add-package-local-nickname local-nickname actual-package
          &optional package-designator
     Adds ‘local-nickname’ for ‘actual-package’ in the designated
     package, defaulting to current package.  ‘local-nickname’ must be a
     string designator, and ‘actual-package’ must be a package
     designator.

     Returns the designated package.

     Signals a continuable error if ‘local-nickname’ is already a
     package local nickname for a different package, or if
     ‘local-nickname’ is one of "CL", "COMMON-LISP", or, "KEYWORD", or
     if ‘local-nickname’ is a global name or nickname for the package to
     which the nickname would be added.

     When in the designated package, calls to ‘find-package’ with the
     ‘local-nickname’ will return the package the designated
     ‘actual-package’ instead.  This also affects all implied calls to
     ‘find-package’, including those performed by the reader.

     When printing a package prefix for a symbol with a package local
     nickname, local nickname is used instead of the real name in order
     to preserve print-read consistency.

     See also: ‘package-local-nicknames’,
     ‘package-locally-nicknamed-by-list’,
     ‘remove-package-local-nickname’, and the ‘defpackage’ option
     ‘:local-nicknames’.

     Experimental: interface subject to change.
 -- Function: remove-package-local-nickname old-nickname &optional
          package-designator
     If the designated package had ‘old-nickname’ as a local nickname
     for another package, it is removed.  Returns true if the nickname
     existed and was removed, and ‘nil’ otherwise.

     See also: ‘add-package-local-nickname’, ‘package-local-nicknames’,
     ‘package-locally-nicknamed-by-list’, and the ‘defpackage’ option
     ‘:local-nicknames’.

     Experimental: interface subject to change.


File: sbcl.info,  Node: Package Variance,  Next: Garbage Collection,  Prev: Package-Local Nicknames,  Up: Beyond the ANSI Standard

7.3 Package Variance
====================

Common Lisp standard specifies that "If the new definition is at
variance with the current state of that package, the consequences are
undefined;" SBCL by default signals a full warning and retains as much
of the package state as possible.

   This can be adjusted using ‘sb-ext:*on-package-variance*’:

 -- Variable: *on-package-variance*
     Specifies behavior when redefining a package using ‘defpackage’ and
     the definition is in variance with the current state of the
     package.

     The value should be of the form:

            (:WARN [T | packages-names] :ERROR [T | package-names])

     specifying which packages get which behaviour ‘--’ with ‘t’
     signifying the default unless otherwise specified.  If default is
     not specified, ‘:warn’ is used.

     ‘:warn’ keeps as much state as possible and causes ‘sbcl’ to signal
     a full warning.

     ‘:error’ causes ‘sbcl’ to signal an error when the variant
     ‘defpackage’ form is executed, with restarts provided for user to
     specify what action should be taken.

     Example:

            (setf *on-package-variance* '(:warn (:swank :swank-backend) :error t))

     specifies to signal a warning if ‘swank’ package is in variance,
     and an error otherwise.


File: sbcl.info,  Node: Garbage Collection,  Next: Generic Function Dispatch,  Prev: Package Variance,  Up: Beyond the ANSI Standard

7.4 Garbage Collection
======================

SBCL provides additional garbage collection functionality not specified
by ANSI.

 -- Variable: *after-gc-hooks*
     Called after each garbage collection, except for garbage
     collections triggered during thread exits.  In a multithreaded
     environment these hooks may run in any thread.
 -- Function: gc &key full gen &allow-other-keys
     Initiate a garbage collection.

     The default is to initiate a nursery collection, which may in turn
     trigger a collection of one or more older generations as well.  If
     ‘full’ is true, all generations are collected.  If ‘gen’ is
     provided, it can be used to specify the oldest generation
     guaranteed to be collected.

7.4.1 Finalization
------------------

Finalization allows code to be executed after an object has been garbage
collected.  This is useful for example for releasing foreign memory
associated with a Lisp object.

 -- Function: finalize object function &key dont-save
     Arrange for the designated ‘function’ to be called when there are
     no more references to ‘object’, including references in ‘function’
     itself.

     If ‘dont-save’ is true, the finalizer will be cancelled when
     ‘save-lisp-and-die’ is called: this is useful for finalizers
     deallocating system memory, which might otherwise be called with
     addresses from the old image.

     In a multithreaded environment ‘function’ may be called in any
     thread.  In both single and multithreaded environments ‘function’
     may be called in any dynamic scope: consequences are unspecified if
     ‘function’ is not fully re-entrant.

     Errors from ‘function’ are handled and cause a ‘warning’ to be
     signalled in whichever thread the ‘function’ was called in.

     Examples:

            ;;; GOOD, assuming RELEASE-HANDLE is re-entrant.
            (let* ((handle (get-handle))
                   (object (make-object handle)))
             (finalize object (lambda () (release-handle handle)))
             object)

            ;;; BAD, finalizer refers to object being finalized, causing
            ;;; it to be retained indefinitely!
            (let* ((handle (get-handle))
                   (object (make-object handle)))
              (finalize object
                        (lambda ()
                          (release-handle (object-handle object)))))

            ;;; BAD, not re-entrant!
            (defvar *rec* nil)

            (defun oops ()
             (when *rec*
               (error "recursive OOPS"))
             (let ((*rec* t))
               (gc))) ; or just cons enough to cause one

            (progn
              (finalize "oops" #'oops)
              (oops)) ; GC causes re-entry to #'oops due to the finalizer
                      ; -> ERROR, caught, WARNING signalled
 -- Function: cancel-finalization object
     Cancel all finalizations for ‘object’, returning ‘t’ if it had a
     finalizer.

7.4.2 Weak Pointers
-------------------

Weak pointers allow references to objects to be maintained without
keeping them from being garbage collected: useful for building caches
among other things.

   Hash tables can also have weak keys and values: *note Hash Table
Extensions::.

 -- Function: make-weak-pointer object
     Allocate and return a weak pointer which points to ‘object’.
 -- Function: weak-pointer-value weak-pointer
     If ‘weak-pointer’ is valid, return the value of ‘weak-pointer’ and
     ‘t’.  If the referent of ‘weak-pointer’ has been garbage collected,
     returns the values ‘nil’ and ‘nil’.

7.4.3 Introspection and Tuning
------------------------------

 -- Variable: *gc-run-time*
     Total ‘cpu’ time spent doing garbage collection (as reported by
     ‘get-internal-run-time’.)  Initialized to zero on startup.  It is
     safe to bind this to zero in order to measure ‘gc’ time inside a
     certain section of code, but doing so may interfere with results
     reported by eg.  ‘time’.
 -- Function: bytes-consed-between-gcs
     The amount of memory that will be allocated before the next garbage
     collection is initiated.  This can be set with ‘setf’.

     On ‘gencgc’ platforms this is the nursery size, and defaults to 5%
     of dynamic space size.

     Note: currently changes to this value are lost when saving core.
 -- Function: dynamic-space-size
     Size of the dynamic space in bytes.
 -- Function: get-bytes-consed
     Return the number of bytes consed since the program began.
     Typically this result will be a consed bignum, so if you have an
     application (e.g.  profiling) which can't tolerate the overhead of
     consing bignums, you'll probably want either to hack in at a lower
     level (as the code in the ‘sb-profile’ package does), or to design
     a more microefficient interface and submit it as a patch.
 -- Function: gc-logfile
     Return the pathname used to log garbage collections.  Can be
     ‘setf’.  Default is ‘nil’, meaning collections are not logged.  If
     non-null, the designated file is opened before and after each
     collection, and generation statistics are appended to it.
 -- Function: generation-average-age generation
     Average age of memory allocated to ‘generation:’ average number of
     times objects allocated to the generation have seen younger objects
     promoted to it.  Available on ‘gencgc’ platforms only.

     Experimental: interface subject to change.
 -- Function: generation-bytes-allocated generation
     Number of bytes allocated to ‘generation’ currently.  Available on
     ‘gencgc’ platforms only.

     Experimental: interface subject to change.
 -- Function: generation-bytes-consed-between-gcs generation
     Number of bytes that can be allocated to ‘generation’ before that
     generation is considered for garbage collection.  This value is
     meaningless for generation 0 (the nursery)‘:’ see
     ‘bytes-consed-between-gcs’ instead.  Default is 5% of the dynamic
     space size divided by the number of non-nursery generations.  Can
     be assigned to using ‘setf’.  Available on ‘gencgc’ platforms only.

     Experimental: interface subject to change.
 -- Function: generation-minimum-age-before-gc generation
     Minimum average age of objects allocated to ‘generation’ before
     that generation is may be garbage collected.  Default is 0.75.  See
     also ‘generation-average-age’.  Can be assigned to using ‘setf’.
     Available on ‘gencgc’ platforms only.

     Experimental: interface subject to change.
 -- Function: generation-number-of-gcs-before-promotion generation
     Number of times garbage collection is done on ‘generation’ before
     automatic promotion to the next generation is triggered.  Default
     is 1.  Can be assigned to using ‘setf’.  Available on ‘gencgc’
     platforms only.

     Experimental: interface subject to change.
 -- Function: generation-number-of-gcs generation
     Number of times garbage collection has been done on ‘generation’
     without promotion.  Available on ‘gencgc’ platforms only.

     Experimental: interface subject to change.

7.4.4 Tracing Live Objects Back to Roots
----------------------------------------

     note: This feature is intended to help expert users diagnose rare
     low-level issues and should not be needed during normal usage.  On
     top of that, the interface and implementation are experimental and
     may change at any time without further notice.

   It is sometimes important to understand why a given object is
retained in the Lisp image instead of being garbage collected.  To help
with this problem, SBCL provides a mechanism that searches through the
different memory spaces, builds a path of references from a root to the
object in question and finally reports this paths:

 -- Function: search-roots weak-pointers &key criterion ignore print
     Find roots keeping the targets of ‘weak-pointers’ alive.

     ‘weak-pointers’ must be a single ‘sb-ext:weak-pointer’ or a list of
     those, pointing to objects for which roots should be searched.

     ‘criterion’ determines just how rooty (how deep) a root must be in
     order to be considered.  Possible values are:

     _‘:oldest’_
          This says we can stop upon seeing an object in the oldest gen
          to ‘gc’, or older.  This is the easiest test to satisfy.

     _‘:pseudo-static’_
          This is usually the same as ‘:oldest’, unless the oldest gen
          to ‘gc’ has been decreased.

     _‘:static’_
          To find a root of an image-backed object, you want to stop
          only at a truly ‘:static’ object.

     ‘ignore’ is a list of objects to treat as if nonexistent in the
     heap.  It can often be useful for finding a path to an interned
     symbol other than through its package by specifying the package as
     an ignored object.

     ‘print’ controls whether discovered paths should be returned or
     printed.  Possible values are

     _‘:verbose’_
          Return no values.  Print discovered paths using a verbose
          format with each node of each path on a separate line.

     _true (other than ‘:verbose’)_
          Return no values.  Print discovered paths using a compact
          format with all nodes of each path on a single line.

     _‘nil’_
          Do not print any output.  Instead return the discovered paths
          as a list of lists.  Each list has the form

          (‘target’ .  (‘root’ ‘node*’))

          where ‘target’ is one of the target of one of the
          ‘weak-pointers’.

          ‘root’ is a description of the root at which the path starts
          and has one of the following forms:

          ‘:static’ If the root of the path is a non-collectible heap
          object.

          ‘:pinned’ If an unknown thread stack pins the root of the
          path.

          ((‘thread-name’ | ‘thread-object’) ‘symbol’ ‘currentp’) If the
          path begins at a special binding of ‘symbol’ in a thread.
          ‘currentp’ is a ‘boolean’ indicating whether the value is
          current or shadowed by another binding.

          ((‘thread-name’ | ‘thread-object’) ‘guessed-pc’) If the path
          begins at a lexical variable in the function whose code
          contains ‘guessed-pc’.

          Each ‘node’ in the remainder of the path is a cons (‘object’ .
          ‘slot’) indicating that the slot at index ‘slot’ in ‘object’
          references the next path node.

     Experimental: subject to change without prior notice.

   An example of using this could look like this:

     * (defvar *my-string* (list 1 2 "my string"))
     *MY-STRING*

     * (sb-ext:search-roots (sb-ext:make-weak-pointer (third *my-string*)))
      -> ((SIMPLE-VECTOR 3)) #x10004E9EAF[2] -> (SYMBOL) #x5044100F[1] -> (CONS) #x100181FAE7[1] -> (CONS) #x100181FAF7[1] -> (CONS) #x100181FB07[0] -> #x100181F9AF

   The single line of output on ‘cl:*standard-output*’ shows the path
from a root to "my string": the path starts with SBCL's internal package
system data structures followed by the symbol (cl-user:*my-string*)
followed the three cons cells of the list.

   The ‘:print :verbose’ argument produces similar behavior but describe
the path elements in more detail:

     * (sb-ext:search-roots (sb-ext:make-weak-pointer (third *my-string*)) :print :verbose)
     Path to "my string":
      6       10004E9EAF [   2] a (simple-vector 3)
      0         5044100F [   1] COMMON-LISP-USER::*MY-STRING*
      0       100181FAE7 [   1] a cons
      0       100181FAF7 [   1] a cons
      0       100181FB07 [   0] a cons

   The ‘:print nil’ argument is a bit different:

     * (sb-ext:search-roots (sb-ext:make-weak-pointer (third *my-string*)) :print nil)
     (("my string" :STATIC (#(*MY-STRING* 0 0) . 2) (*MY-STRING* . 1)
       ((1 2 "my string") . 1) ((2 "my string") . 1) (("my string") . 0)))

   There is no output on ‘cl:*standard-output*’ and the return value is
a single path for the target object "my string".  As before, the path
shows the symbol and the three cons cells.


File: sbcl.info,  Node: Generic Function Dispatch,  Next: Slot Access,  Prev: Garbage Collection,  Up: Beyond the ANSI Standard

7.5 Generic Function Dispatch
=============================

If a generic function with standard or short method combination is
called, and the set of applicable methods does not include any primary
methods, then the generic function ‘no-primary-method’ will be invoked,
with the arguments to ‘no-primary-method’ being the invoked generic
function and its arguments, similar to the standard function
‘no-applicable-method’.  As with ‘no-applicable-method’, the default
method on ‘no-primary-method’ signals an error; programmers may define
methods on it.


File: sbcl.info,  Node: Slot Access,  Next: Metaobject Protocol,  Prev: Generic Function Dispatch,  Up: Beyond the ANSI Standard

7.6 Slot Access
===============

The slot access functions ‘slot-value’, ‘(setf slot-value)’,
‘slot-boundp’ and ‘slot-makunbound’ are defined to function as expected
on conditions (of metaclass ‘condition-class’) and, with some
limitations, on structures (of metaclass ‘structure-class’).

   For structures:

   • The name of a slot for the purposes of the slot access functions is
     the symbol used as the slot-name in the slot-description in the
     ‘defstruct’ form;

   • ‘slot-value’ and ‘slot-boundp’ function as expected, including (for
     ‘slot-value’) calling and respecting the return value of
     ‘slot-unbound’ if the slot is unbound;

   • ‘(setf slot-value)’ functions as expected, including performing
     type checks to verify that the new value is of an appropriate type
     for the slot;

   • ‘slot-makunbound’ makes the slot be unbound only when the slot
     corresponds to an ‘&aux’ argument with no default in a
     by-order-of-arguments (BOA) constructor.  In all other cases
     calling ‘slot-makunbound’ on a structure signals an error.

   • If any of the slot access functions is called with a structure
     instance which does not have a slot of the given name,
     ‘slot-missing’ is called and the return value of the effective
     method, if any, is respected.


File: sbcl.info,  Node: Metaobject Protocol,  Next: Extensible Sequences,  Prev: Slot Access,  Up: Beyond the ANSI Standard

7.7 Metaobject Protocol
=======================

7.7.1 AMOP Compatibility of Metaobject Protocol
-----------------------------------------------

SBCL supports a metaobject protocol which is intended to be compatible
with AMOP; present exceptions to this (as distinct from current bugs)
are:

   • ‘compute-effective-method’ only returns one value, not two.

     There is no record of what the second return value was meant to
     indicate, and apparently no clients for it.

   • The direct superclasses of ‘funcallable-standard-object’ are
     ‘(function standard-object)’, not ‘(standard-object function)’.

     This is to ensure that the ‘standard-object’ class is the last of
     the standardized classes before ‘t’ appearing in the class
     precedence list of ‘generic-function’ and
     ‘standard-generic-function’, as required by section 1.4.4.5 of the
     ANSI specification.

   • the arguments ‘:declare’ and ‘:declarations’ to
     ‘ensure-generic-function’ are both accepted, with the leftmost
     argument defining the declarations to be stored and returned by
     ‘generic-function-declarations’.

     Where AMOP specifies ‘:declarations’ as the keyword argument to
     ‘ensure-generic-function’, the Common Lisp standard specifies
     ‘:declare’.  Portable code should use ‘:declare’.

   • although SBCL obeys the requirement in AMOP that
     ‘validate-superclass’ should treat ‘standard-class’ and
     ‘funcallable-standard-class’ as compatible metaclasses, we impose
     an additional requirement at class finalization time: a class of
     metaclass ‘funcallable-standard-class’ must have ‘function’ in its
     superclasses, and a class of metaclass ‘standard-class’ must not.

     After a class has been finalized, it is associated with a class
     prototype which is accessible by a standard mop function
     ‘class-prototype’.  The user can then ask whether this object is a
     ‘function’ or not in several different ways: whether it is a
     function according to ‘typep’; whether its ‘class-of’ is ‘subtypep’
     ‘function’, or whether ‘function’ appears in the superclasses of
     the class.  The additional consistency requirement comes from the
     desire to make all of these answers the same.

     The following class definitions are bad, and will lead to errors
     either immediately or if an instance is created:
          (defclass bad-object (funcallable-standard-object)
            ()
            (:metaclass standard-class))
          (defclass bad-funcallable-object (standard-object)
            ()
            (:metaclass funcallable-standard-class))
     The following definition is acceptable:
          (defclass mixin ()
            ((slot :initarg slot)))
          (defclass funcallable-object (funcallable-standard-object mixin)
            ()
            (:metaclass funcallable-standard-class))
     and leads to a class whose instances are funcallable and have one
     slot.

     Note that this requirement also applies to the class
     ‘funcallable-standard-object’, which has metaclass
     ‘funcallable-standard-class’ rather than ‘standard-class’ as AMOP
     specifies.

   • the requirement that "No portable class C_p may inherit, by virtue
     of being a direct or indirect subclass of a specified class, any
     slot for which the name is a symbol accessible in the
     ‘common-lisp-user’ package or exported by any package defined in
     the ANSI Common Lisp standard."  is interpreted to mean that the
     standardized classes themselves should not have slots named by
     external symbols of public packages.

     The rationale behind the restriction is likely to be similar to the
     ANSI Common Lisp restriction on defining functions, variables and
     types named by symbols in the Common Lisp package: preventing two
     independent pieces of software from colliding with each other.

   • specializations of the ‘new-value’ argument to ‘(setf
     slot-value-using-class)’ are not allowed: all user-defined methods
     must have a specializer of the class ‘t’.

     This prohibition is motivated by a separation of layers: the
     ‘slot-value-using-class’ family of functions is intended for use in
     implementing different and new slot allocation strategies, rather
     than in performing application-level dispatching.  Additionally,
     with this requirement, there is a one-to-one mapping between
     metaclass, class and slot-definition-class tuples and effective
     methods of ‘(setf slot-value-using-class)’, which permits
     optimization of ‘(setf slot-value-using-class)’'s discriminating
     function in the same manner as for ‘slot-value-using-class’ and
     ‘slot-boundp-using-class’.

     Note that application code may specialize on the ‘new-value’
     argument of slot accessors.

   • the class named by the ‘name’ argument to ‘ensure-class’, if any,
     is only redefined if it is the proper name of that class;
     otherwise, a new class is created.

     This is consistent with the description of ‘ensure-class’ in AMOP
     as the functional version of ‘defclass’, which has this behaviour;
     however, it is not consistent with the weaker requirement in AMOP,
     which states that any class found by ‘find-class’, no matter what
     its ‘class-name’, is redefined.

   • an error is not signaled in the case of the ‘:name’ initialization
     argument for ‘slot-definition’ objects being a constant, when the
     slot definition is of type ‘structure-slot-definition’ (i.e.  it is
     associated with a class of type ‘structure-class’).

     This allows code which uses constant names for structure slots to
     continue working as specified in ANSI, while enforcing the
     constraint for all other types of slot.

   • the class named ‘t’ is not an instance of the ‘built-in-class’
     metaclass.

     AMOP specifies, in the "Inheritance Structure of Metaobject
     Classes" section, that the class named ‘t’ should be an instance of
     ‘built-in-class’.  However, it also specifies that
     ‘validate-superclass’ should return true (indicating that a direct
     superclass relationship is permissible) if the second argument is
     the class named ‘t’.  Also, ANSI specifies that classes with
     metaclass ‘built-in-class’ may not be subclassed using ‘defclass’,
     and also that the class named ‘t’ is the universal superclass,
     inconsistent with it being a ‘built-in-class’.

   • uses of ‘change-class’ and redefinitions of classes with ‘defclass’
     (or the functional interfaces ‘ensure-class’ or
     ‘ensure-class-using-class’) must ensure that for each slot with
     allocation ‘:instance’ or ‘:class’, the set of applicable methods
     on the ‘slot-value-using-class’ family of generic functions is the
     same before and after the change.

     This is required for correct operation of the protocol to update
     instances for the new or redefined class, and can be seen as part
     of the contract of the ‘:instance’ or ‘:class’ allocations.

   • Metaobject protocol users may wish to override
     ‘compute-discriminating-function’ for their own generic function
     classes.  Overriding implementations of
     ‘compute-discriminating-function’ must, in order to participate in
     the ‘no-applicable-method’ and ‘no-primary-method’ protocols,
     perform appropriate checks on the return value of
     ‘compute-applicable-methods’ before processing the effective
     method; the standard effective method contains error-invoking
     forms, but those forms have no access to the generic function
     invocation's arguments.

7.7.2 Metaobject Protocol Extensions
------------------------------------

In addition, SBCL supports extensions to the Metaobject protocol from
AMOP; at present, they are:

   • compile-time support for generating specializer metaobjects from
     specializer names in ‘defmethod’ forms is provided by the
     ‘make-method-specializers-form’ function, which returns a form
     which, when evaluated in the lexical environment of the
     ‘defmethod’, returns a list of specializer metaobjects.  This
     operator suffers from similar restrictions to those affecting
     ‘make-method-lambda’, namely that the generic function must be
     defined when the ‘defmethod’ form is expanded, so that the correct
     method of ‘make-method-specializers-form’ is invoked.  The
     system-provided method on ‘make-method-specializers-form’ generates
     a call to ‘find-class’ for each symbol specializer name, and a call
     to ‘intern-eql-specializer’ for each ‘(eql X)’ specializer name.

   • run-time support for converting between specializer names and
     specializer metaobjects, mostly for the purposes of ‘find-method’,
     is provided by ‘parse-specializer-using-class’ and
     ‘unparse-specializer-using-class’, which dispatch on their first
     argument, the generic function associated with a method with the
     given specializer.  The system-provided methods on those methods
     convert between classes and proper names and between lists of the
     form ‘(eql X)’ and interned eql specializer objects.

   • distinguishing unbound instance allocated slots from bound ones
     when using ‘standard-instance-access’ and
     ‘funcallable-standard-instance-access’ is possible by comparison to
     the symbol-macro ‘+slot-unbound+’.


File: sbcl.info,  Node: Extensible Sequences,  Next: Support For Unix,  Prev: Metaobject Protocol,  Up: Beyond the ANSI Standard

7.8 Extensible Sequences
========================

* Menu:

* Iterator Protocol::
* Simple Iterator Protocol::

ANSI Common Lisp has a class ‘sequence’ with subclasses ‘list’ and
‘vector’ on which the "sequence functions" like ‘find’, ‘subseq’, etc.
operate.  As an extension to the ANSI specification, SBCL allows
additional subclasses of ‘sequence’ to be defined (1).

   Users of this extension just make instances of ‘sequence’ subclasses
and transparently operate on them using sequence functions:
     (coerce (subseq (make-instance 'my-sequence) 5 10) 'list)
   From this perspective, no distinction between builtin and
user-defined ‘sequence’ subclasses should be necessary.

   Providers of the extension, that is of user-defined ‘sequence’
subclasses, have to adhere to a "sequence protocol" which consists of a
set of generic functions in the ‘sequence’ package.

   A minimal ‘sequence’ subclass has to specify ‘standard-object’ and
‘sequence’ as its superclasses and has to be the specializer of the
‘sequence’ parameter of methods on at least the following generic
functions:

 -- Generic Function: length sequence
     Returns the length of ‘sequence’ or signals a
     ‘protocol-unimplemented’ error if the sequence protocol is not
     implemented for the class of ‘sequence’.
 -- Generic Function: elt sequence index
     Returns the element at position ‘index’ of ‘sequence’ or signals a
     ‘protocol-unimplemented’ error if the sequence protocol is not
     implemented for the class of ‘sequence’.
 -- Generic Function: (setf elt)
     Replaces the element at position ‘index’ of ‘sequence’ with
     ‘new-value’ and returns ‘new-value’ or signals a
     ‘protocol-unimplemented’ error if the sequence protocol is not
     implemented for the class of ‘sequence’.
 -- Generic Function: adjust-sequence sequence length &key
          initial-element initial-contents
     Returns destructively modified ‘sequence’ or a freshly allocated
     sequence of the same class as ‘sequence’ of length ‘length’.
     Elements of the returned sequence are initialized to
     ‘initial-element’, if supplied, initialized to ‘initial-contents’
     if supplied, or identical to the elements of ‘sequence’ if neither
     is supplied.  Signals a ‘protocol-unimplemented’ error if the
     sequence protocol is not implemented for the class of ‘sequence’.
 -- Generic Function: make-sequence-like sequence length &key
          initial-element initial-contents
     Returns a freshly allocated sequence of length ‘length’ and of the
     same class as ‘sequence’.  Elements of the new sequence are
     initialized to ‘initial-element’, if supplied, initialized to
     ‘initial-contents’ if supplied, or undefined if neither is
     supplied.  Signals a ‘protocol-unimplemented’ error if the sequence
     protocol is not implemented for the class of ‘sequence’.

   ‘make-sequence-like’ is needed for functions returning
freshly-allocated sequences such as ‘subseq’ or ‘copy-seq’.
‘adjust-sequence’ is needed for functions which destructively modify
their arguments such as ‘delete’.  In fact, all other sequence functions
can be implemented in terms of the above functions and actually are, if
no additional methods are defined.  However, relying on these generic
implementations, in particular not implementing the iterator protocol
can incur a high performance penalty *Note Iterator Protocol::.

   When the sequence protocol is only partially implemented for a given
‘sequence’ subclass, an attempt to apply one of the missing operations
to instances of that class signals the following condition:

 -- Condition: protocol-unimplemented
     Class precedence list: ‘protocol-unimplemented’, ‘type-error’,
     ‘error’, ‘serious-condition’, ‘condition’, ‘t’

     This error is signaled if a sequence operation is applied to an
     instance of a sequence class that does not support the operation.

   In addition to the mandatory functions above, methods on the sequence
functions listed below can be defined.

   There are two noteworthy irregularities:
   • The function ‘sb-sequence:emptyp’ does not have a counterpart in
     the ‘cl’ package.  It is intended to be used instead of ‘length’
     when working with lazy or infinite sequences.

   • The functions ‘map’, ‘concatenate’ and ‘merge’ receive a type
     designator specifying the type of the constructed sequence as their
     first argument.  However, the corresponding generic functions
     ‘sb-sequence:map’, ‘sb-sequence:concatenate’ and
     ‘sb-sequence:merge’ receive a prototype instance of the requested
     ‘sequence’ subclass instead.

 -- Generic Function: emptyp sequence
     Returns ‘t’ if ‘sequence’ is an empty sequence and ‘nil’ otherwise.
     Signals an error if ‘sequence’ is not a sequence.

   • ‘sb-sequence:count’, ‘sb-sequence:count-if’,
     ‘sb-sequence:count-if-not’

   • ‘sb-sequence:find’, ‘sb-sequence:find-if’,
     ‘sb-sequence:find-if-not’

   • ‘sb-sequence:position’, ‘sb-sequence:position-if’,
     ‘sb-sequence:position-if-not’

   • ‘sb-sequence:subseq’

   • ‘sb-sequence:copy-seq’

   • ‘sb-sequence:fill’

   •  -- Generic Function: map result-prototype function sequence &rest
               sequences
          Implements ‘cl:map’ for extended sequences.

          ‘result-prototype’ corresponds to the ‘result-type’ of
          ‘cl:map’ but receives a prototype instance of an extended
          sequence class instead of a type specifier.  By dispatching on
          ‘result-prototype’, methods on this generic function specify
          how extended sequence classes act when they are specified as
          the result type in a ‘cl:map’ call.  ‘result-prototype’ may
          not be fully initialized and thus should only be used for
          dispatch and to determine its class.

          Another difference to ‘cl:map’ is that ‘function’ is a
          function, not a function designator.

   • ‘sb-sequence:nsubstitute’, ‘sb-sequence:nsubstitute-if’,
     ‘sb-sequence:nsubstitute-if-not’, ‘sb-sequence:substitute’,
     ‘sb-sequence:substitute-if’, ‘sb-sequence:substitute-if-not’

   • ‘sb-sequence:replace’

   • ‘sb-sequence:nreverse’, ‘sb-sequence:reverse’

   •  -- Generic Function: concatenate result-prototype &rest sequences
          Implements ‘cl:concatenate’ for extended sequences.

          ‘result-prototype’ corresponds to the ‘result-type’ of
          ‘cl:concatenate’ but receives a prototype instance of an
          extended sequence class instead of a type specifier.  By
          dispatching on ‘result-prototype’, methods on this generic
          function specify how extended sequence classes act when they
          are specified as the result type in a ‘cl:concatenate’ call.
          ‘result-prototype’ may not be fully initialized and thus
          should only be used for dispatch and to determine its class.

   • ‘sb-sequence:reduce’

   • ‘sb-sequence:mismatch’

   • ‘sb-sequence:search’

   • ‘sb-sequence:delete’, ‘sb-sequence:delete-if’,
     ‘sb-sequence:delete-if-not’, ‘sb-sequence:remove’,
     ‘sb-sequence:remove-if’, ‘sb-sequence:remove-if-not’,

   • ‘sb-sequence:delete-duplicates’, ‘sb-sequence:remove-duplicates’

   • ‘sb-sequence:sort’, ‘sb-sequence:stable-sort’

   •  -- Generic Function: merge result-prototype sequence1 sequence2
               predicate &key key
          Implements ‘cl:merge’ for extended sequences.

          ‘result-prototype’ corresponds to the ‘result-type’ of
          ‘cl:merge’ but receives a prototype instance of an extended
          sequence class instead of a type specifier.  By dispatching on
          ‘result-prototype’, methods on this generic function specify
          how extended sequence classes act when they are specified as
          the result type in a ‘cl:merge’ call.  ‘result-prototype’ may
          not be fully initialized and thus should only be used for
          dispatch and to determine its class.

          Another difference to ‘cl:merge’ is that ‘predicate’ is a
          function, not a function designator.

   In the spirit of ‘dolist’, generic sequences can be traversed using
the macro

 -- Macro: dosequence (element sequence &optional return) &body body
     Executes ‘body’ with ‘element’ subsequently bound to each element
     of ‘sequence’, then returns ‘return’.

   ---------- Footnotes ----------

   (1) A motivation, rationale and additional examples for the design of
this extension can be found in the paper ‘Rhodes, Christophe (2007):
User-extensible sequences in Common Lisp’ available for download at
<http://www.doc.gold.ac.uk/~mas01cr/papers/ilc2007/sequences-20070301.pdf>.


File: sbcl.info,  Node: Iterator Protocol,  Next: Simple Iterator Protocol,  Up: Extensible Sequences

7.8.1 Iterator Protocol
-----------------------

The iterator protocol allows subsequently accessing some or all elements
of a sequence in forward or reverse direction.  Users first call
‘make-sequence-iterator’ to create an iteration state and receive
functions to query and mutate it.  These functions allow, among other
things, moving to, retrieving or modifying elements of the sequence.  An
iteration state consists of a state object, a limit object, a from-end
indicator and the following six functions to query or mutate this state:
 -- Function: step function sequence iterator from-end
     Moves the iterator one position forward or backward in the
     associated sequence depending on the iteration direction.
 -- Function: endp function sequence iterator limit from-end
     Returns non-‘nil’ when the iterator has reached the end of the
     associated sequence with respect to the iteration direction.
 -- Function: element function sequence iterator
     Returns the sequence element associated to the current position of
     the iteration.
 -- Function: setf element function new-value sequence iterator
     Destructively modifies the associates sequence by replacing the
     sequence element associated to the current iteration position with
     a new value.
 -- Function: index function sequence iterator
     Returns the position of the iteration in the associated sequence.
 -- Function: copy function sequence iterator
     Returns a copy of the iteration state which can be mutated
     independently of the copied iteration state.

   An iterator is created by calling:

 -- Generic Function: make-sequence-iterator sequence &key from-end
          start end
     Returns a sequence iterator for ‘sequence’ or, if ‘start’ and/or
     ‘end’ are supplied, the subsequence bounded by ‘start’ and ‘end’ as
     nine values:

     1.  iterator state 2.  limit 3.  from-end 4.  step function 5.
     endp function 6.  element function 7.  setf element function 8.
     index function 9.  copy state function

     If ‘from-end’ is ‘nil’, the constructed iterator visits the
     specified elements in the order in which they appear in ‘sequence’.
     Otherwise, the elements are visited in the opposite order.

   Note that ‘make-sequence-iterator’ calls
‘make-simple-sequence-iterator’ when there is no specialized method for
a particular ‘sequence’ subclass.  *Note Simple Iterator Protocol::.

   The following convenience macros simplify traversing sequences using
iterators:

 -- Macro: with-sequence-iterator (&optional iterator limit from-end-p
          step endp element set-element index copy) (sequence &key
          from-end start end) &body body
     Executes ‘body’ with the elements of ‘vars’ bound to the iteration
     state returned by ‘make-sequence-iterator’ for ‘sequence’ and
     ‘args’.  Elements of ‘vars’ may be ‘nil’ in which case the
     corresponding value returned by ‘make-sequence-iterator’ is
     ignored.
 -- Macro: with-sequence-iterator-functions (&optional step endp elt
          setf index copy) (sequence &rest args &key from-end start end)
          &body body
     Executes ‘body’ with the names ‘step’, ‘endp’, ‘elt’, ‘setf’,
     ‘index’ and ‘copy’ bound to local functions which execute the
     iteration state query and mutation functions returned by
     ‘make-sequence-iterator’ for ‘sequence’ and ‘args’.  When some
     names are not supplied or ‘nil’ is supplied for a given name, no
     local functions are established for those names.  The functions
     established for ‘step’, ‘endp’, ‘elt’, ‘setf’, ‘index’ and ‘copy’
     have dynamic extent.


File: sbcl.info,  Node: Simple Iterator Protocol,  Prev: Iterator Protocol,  Up: Extensible Sequences

7.8.2 Simple Iterator Protocol
------------------------------

For cases in which the full flexibility and performance of the general
sequence iterator protocol is not required, there is a simplified
sequence iterator protocol consisting of a few generic functions which
can be specialized for iterator classes:

 -- Generic Function: iterator-step sequence iterator from-end
     Moves ‘iterator’ one position forward or backward in ‘sequence’
     depending on the iteration direction encoded in ‘from-end’.
 -- Generic Function: iterator-endp sequence iterator limit from-end
     Returns non-NIL when ‘iterator’ has reached ‘limit’ (which may
     correspond to the end of ‘sequence’) with respect to the iteration
     direction encoded in ‘from-end’.
 -- Generic Function: iterator-element sequence iterator
     Returns the element of ‘sequence’ associated to the position of
     ‘iterator’.
 -- Generic Function: (setf iterator-element)
     Destructively modifies ‘sequence’ by replacing the sequence element
     associated to position of ‘iterator’ with ‘new-value’.
 -- Generic Function: iterator-index sequence iterator
     Returns the position of ‘iterator’ in ‘sequence’.
 -- Generic Function: iterator-copy sequence iterator
     Returns a copy of ‘iterator’ which also traverses ‘sequence’ but
     can be mutated independently of ‘iterator’.

   Iterator objects implementing the above simple iteration protocol are
created by calling the following generic function:

 -- Generic Function: make-simple-sequence-iterator sequence &key
          from-end start end
     Returns a sequence iterator for ‘sequence’, ‘start’, ‘end’ and
     ‘from-end’ as three values:

     1.  iterator state 2.  limit 3.  from-end

     The returned iterator can be used with the generic iterator
     functions ‘iterator-step’, ‘iterator-endp’, ‘iterator-element’,
     (‘setf’ ‘iterator-element’), ‘iterator-index’ and ‘iterator-copy’.


File: sbcl.info,  Node: Support For Unix,  Next: Unicode Support,  Prev: Extensible Sequences,  Up: Beyond the ANSI Standard

7.9 Support For Unix
====================

* Menu:

* Command-line arguments::
* Querying the process environment::
* Running external programs::


File: sbcl.info,  Node: Command-line arguments,  Next: Querying the process environment,  Up: Support For Unix

7.9.1 Command-line arguments
----------------------------

The UNIX command line can be read from the variable
‘sb-ext:*posix-argv*’.


File: sbcl.info,  Node: Querying the process environment,  Next: Running external programs,  Prev: Command-line arguments,  Up: Support For Unix

7.9.2 Querying the process environment
--------------------------------------

The UNIX environment can be queried with the ‘sb-ext:posix-getenv’
function.

 -- Function: posix-getenv name
     Return the "value" part of the environment string "name=value"
     which corresponds to ‘name’, or ‘nil’ if there is none.


File: sbcl.info,  Node: Running external programs,  Prev: Querying the process environment,  Up: Support For Unix

7.9.3 Running external programs
-------------------------------

External programs can be run with ‘sb-ext:run-program’.  (1)

 -- Function: run-program program args &key env environment wait search
          pty input if-input-does-not-exist output if-output-exists
          error if-error-exists status-hook external-format directory
          preserve-fds
     ‘run-program’ creates a new process specified by ‘program’.  ‘args’
     is a list of strings to be passed literally to the new program.  In
     ‘posix’ environments, this list becomes the array supplied as the
     second parameter to the execv() or execvp() system call, each list
     element becoming one array element.  The strings should not contain
     shell escaping, as there is no shell involvement.  Further note
     that while conventionally the process receives its own pathname in
     argv[0], that is automatic, and the 0th string should not be
     present in ‘args’.

     The program arguments and the environment are encoded using the
     default external format for streams.

     ‘run-program’ will return a ‘process’ structure.  See the ‘cmu’
     Common Lisp Users Manual for details about the ‘process’ structure.

     Notes about Unix environments (as in the ‘:environment’ and ‘:env’
     args)‘:’

        • The ‘sbcl’ implementation of ‘run-program’, like Perl and many
          other programs, but unlike the original ‘cmu’ ‘cl’
          implementation, copies the Unix environment by default.
        • Running Unix programs from a setuid process, or in any other
          situation where the Unix environment is under the control of
          someone else, is a mother lode of security problems.  If you
          are contemplating doing this, read about it first.  (The Perl
          community has a lot of good documentation about this and other
          security issues in script-like programs.)

     _The ‘&key’ arguments have the following meanings:_
     _‘:environment’_
          a list of STRINGs describing the new Unix environment (as in
          "man environ").  The default is to copy the environment of the
          current process.

     _‘:env’_
          an alternative lossy representation of the new Unix
          environment, for compatibility with ‘cmu’ ‘cl’

     _‘:search’_
          Look for ‘program’ in each of the directories in the child's
          $PATH environment variable.  Otherwise an absolute pathname is
          required.

     _‘:wait’_
          If non-NIL (default), wait until the created process finishes.
          If ‘nil’, continue running Lisp until the program finishes.

     _‘:pty’ (not supported on win32)_
          Either ‘t’, ‘nil’, or a stream.  Unless ‘nil’, the subprocess
          is established under a ‘pty’.  If :pty is a stream, all output
          to this pty is sent to this stream, otherwise the
          ‘process-pty’ slot is filled in with a stream connected to pty
          that can read output and write input.

     _‘:input’_
          Either ‘t’, ‘nil’, a pathname, a stream, or ‘:stream’.  ‘t:’
          the standard input for the current process is inherited.
          ‘nil:’ /dev/null (nul on win32) is used.  pathname: the
          specified file is used.  stream: all the input is read from
          that stream and sent to the subprocess.  ‘:stream:’ the
          ‘process-input’ slot is filled in with a stream that sends its
          output to the process.  Defaults to ‘nil’.

     _‘:if-input-does-not-exist’ (when ‘:input’ is the name of a file)_
          can be one of: ‘:error’ to generate an error ‘:create’ to
          create an empty file ‘nil’ (the default) to return ‘nil’ from
          ‘run-program’

     _‘:output’_
          Either ‘t’, ‘nil’, a pathname, a stream, or ‘:stream’.  ‘t:’
          the standard output for the current process is inherited.
          ‘nil:’ /dev/null (nul on win32) is used.  pathname: the
          specified file is used.  stream: all the output from the
          process is written to this stream.  ‘:stream:’ the
          ‘process-output’ slot is filled in with a stream that can be
          read to get the output.  Defaults to ‘nil’.

     _‘:error’_
          Same as ‘:output’, additionally accepts ‘:output’, making all
          error output routed to the same place as normal output.
          Defaults to ‘:output’.

     _‘:if-output-exists’ (when ‘:output’ is the name of a file)_
          can be one of: ‘:error’ (the default) to generate an error
          ‘:supersede’ to supersede the file with output from the
          program ‘:append’ to append output from the program to the
          file ‘nil’ to return ‘nil’ from ‘run-program’, without doing
          anything

     _‘:if-error-exists’_
          Same as ‘:if-output-exists’, controlling ‘:error’ output to
          files.  Ignored when ‘:error’ ‘:output’.  Defaults to
          ‘:error’.

     _‘:status-hook’_
          This is a function the system calls whenever the status of the
          process changes.  The function takes the process as an
          argument.

     _‘:external-format’_
          The external-format to use for ‘:input’, ‘:output’, and
          ‘:error’ :STREAMs.

     _‘:directory’_
          Specifies the directory in which the program should be run.
          ‘nil’ (the default) means the directory is unchanged.

     _‘:preserve-fds’_
          A sequence of file descriptors which should remain open in the
          child process.

     _Windows specific options:_
     _‘:escape-arguments’ (default ‘t’)_
          Controls escaping of the arguments passed to CreateProcess.

     _‘:window’ (default ‘nil’)_
          When ‘nil’, the subprocess decides how it will display its
          window.  The following options control how the subprocess
          window should be displayed: ‘:hide’, ‘:show-normal’,
          ‘:show-maximized’, ‘:show-minimized’, ‘:show-no-activate’,
          ‘:show-min-no-active’, ‘:show-na’.  Note: console application
          subprocesses may or may not display a console window depending
          on whether the ‘sbcl’ runtime is itself a console or ‘gui’
          application.  Invoke ‘cmd’ /C ‘start’ to consistently display
          a console window or use the ‘:window’ ‘:hide’ option to
          consistently hide the console window.

   When ‘sb-ext:run-program’ is called with ‘wait’ equal to NIL, an
instance of class SB-EXT:PROCESS is returned.  The following functions
are available for use with processes:

 -- Function: process-p object
     ‘t’ if ‘object’ is a ‘process’, ‘nil’ otherwise.

 -- Function: process-input instance
     The input stream of the process or ‘nil’.

 -- Function: process-output instance
     The output stream of the process or ‘nil’.

 -- Function: process-error instance
     The error stream of the process or ‘nil’.

 -- Function: process-alive-p process
     Return ‘t’ if ‘process’ is still alive, ‘nil’ otherwise.  Can
     return a false positive on a closed process.

 -- Function: process-status process
     Return the current status of ‘process’.  The result is one of
     ‘:running’, ‘:stopped’, ‘:exited’, ‘:signaled’.

 -- Function: process-wait process &optional check-for-stopped
     Wait for ‘process’ to quit running for some reason.  When
     ‘check-for-stopped’ is ‘t’, also returns when ‘process’ is stopped.
     Returns ‘process’.

 -- Function: process-exit-code process
     The exit code or the signal of a stopped process.

 -- Function: process-core-dumped instance
     ‘t’ if a core image was dumped by the process.

 -- Function: process-close process
     Close all streams connected to ‘process’, stop maintaining the
     status slot.  After ‘process-close’, ‘process-alive-p’ and
     ‘process-exit-code’ can return stale information about a process,
     so should not be used.

 -- Function: process-kill process signal &optional whom
     Hand ‘signal’ to ‘process’.  If ‘whom’ is ‘:pid’, use the kill Unix
     system call.  If ‘whom’ is ‘:process-group’, use the killpg Unix
     system call.  Returns ‘t’ if successful, otherwise returns ‘nil’
     and error number (two values).

   ---------- Footnotes ----------

   (1) In SBCL versions prior to 1.0.13, ‘sb-ext:run-program’ searched
for executables in a manner somewhat incompatible with other languages.
As of this version, SBCL uses the system library routine ‘execvp(3)’,
and no longer contains the function, ‘find-executable-in-search-path’,
which implemented the old search.  Users who need this function may find
it in ‘run-program.lisp’ versions 1.67 and earlier in SBCL's CVS
repository here
<http://sbcl.cvs.sourceforge.net/sbcl/sbcl/src/code/run-program.lisp?view=log>.
However, we caution such users that this search routine finds
executables that system library routines do not.


File: sbcl.info,  Node: Unicode Support,  Next: Customization Hooks for Users,  Prev: Support For Unix,  Up: Beyond the ANSI Standard

7.10 Unicode Support
====================

SBCL provides support for working with Unicode text and querying the
standard Unicode database for information about individual codepoints.
Unicode-related functions are located in the ‘sb-unicode’ package.

   SBCL also extends ANSI character literal syntax to support Unicode
codepoints.  You can either specify a character by its Unicode name,
with spaces replaced by underscores, if a unique name exists (1) or by
giving its hexadecimal codepoint preceded by a "U", an optional "+", and
an arbitrary number of leading zeros.  You may also input the character
directly into your source code if it can be encoded in your file.  If a
character had an assigned name in Unicode 1.0 that was distinct from its
current name, you may also use that name (with spaces replaced by
underscores) to specify the character, unless the name is already
associated with a codepoint in the latest Unicode standard (such as
"BELL").

   For example, you can specify the codepoint U+00E1 ("Latin Small
Letter A With Acute") as
   • ‘#\LATIN_SMALL_LETTER_A_WITH_ACUTE’
   • ‘#\LATIN_SMALL_LETTER_A_ACUTE’
   • ‘#\á’ assuming a Unicode source file
   • ‘#\U00E1’
   • ‘#\UE1’
   • ‘#\U+00E1’

7.10.1 Unicode property access
------------------------------

The following functions can be used to find information about a Unicode
codepoint.

 -- Function: general-category character
     Returns the general category of ‘character’ as it appears in
     UnicodeData.txt

 -- Function: bidi-class character
     Returns the bidirectional class of ‘character’

 -- Function: combining-class character
     Returns the canonical combining class (‘ccc’) of ‘character’

 -- Function: decimal-value character
     Returns the decimal digit value associated with ‘character’ or
     ‘nil’ if there is no such value.

     The only characters in Unicode with a decimal digit value are those
     that are part of a range of characters that encode the digits 0-9.
     Because of this, '(decimal-digit c) <=> (digit-char-p c 10)' in
     #+sb-unicode builds

 -- Function: digit-value character
     Returns the Unicode digit value of ‘character’ or ‘nil’ if it
     doesn't exist.

     Digit values are guaranteed to be integers between 0 and 9
     inclusive.  All characters with decimal digit values have the same
     digit value, but there are characters (such as digits of number
     systems without a 0 value) that have a digit value but no decimal
     digit value

 -- Function: numeric-value character
     Returns the numeric value of ‘character’ or ‘nil’ if there is no
     such value.  Numeric value is the most general of the Unicode
     numeric properties.  The only constraint on the numeric value is
     that it be a rational number.

 -- Function: mirrored-p character
     Returns ‘t’ if ‘character’ needs to be mirrored in bidirectional
     text.  Otherwise, returns ‘nil’.

 -- Function: bidi-mirroring-glyph character
     Returns the mirror image of ‘character’ if it exists.  Otherwise,
     returns ‘nil’.

 -- Function: age character
     Returns the version of Unicode in which ‘character’ was assigned as
     a pair of values, both integers, representing the major and minor
     version respectively.  If ‘character’ is not assigned in Unicode,
     returns ‘nil’ for both values.

 -- Function: hangul-syllable-type character
     Returns the Hangul syllable type of ‘character’.  The syllable type
     can be one of ‘:l’, ‘:v’, ‘:t’, ‘:lv’, or ‘:lvt’.  If the character
     is not a Hangul syllable or Jamo, returns ‘nil’

 -- Function: east-asian-width character
     Returns the East Asian Width property of ‘character’ as one of the
     keywords ‘:n’ (Narrow), ‘:a’ (Ambiguous), ‘:h’ (Halfwidth), ‘:w’
     (Wide), ‘:f’ (Fullwidth), or ‘:na’ (Not applicable)

 -- Function: script character
     Returns the Script property of ‘character’ as a keyword.  If
     ‘character’ does not have a known script, returns ‘:unknown’

 -- Function: char-block character
     Returns the Unicode block in which ‘character’ resides as a
     keyword.  If ‘character’ does not have a known block, returns
     ‘:no-block’

 -- Function: unicode-1-name character
     Returns the name assigned to ‘character’ in Unicode 1.0 if it is
     distinct from the name currently assigned to ‘character’.
     Otherwise, returns ‘nil’.  This property has been officially
     obsoleted by the Unicode standard, and is only included for
     backwards compatibility.

 -- Function: proplist-p character property
     Returns ‘t’ if ‘character’ has the specified ‘property’.
     ‘property’ is a keyword representing one of the properties from
     PropList.txt, with underscores replaced by dashes.

 -- Function: uppercase-p character
     Returns ‘t’ if ‘character’ has the Unicode property Uppercase and
     ‘nil’ otherwise

 -- Function: lowercase-p character
     Returns ‘t’ if ‘character’ has the Unicode property Lowercase and
     ‘nil’ otherwise

 -- Function: cased-p character
     Returns ‘t’ if ‘character’ has a (Unicode) case, and ‘nil’
     otherwise

 -- Function: case-ignorable-p character
     Returns ‘t’ if ‘character’ is Case Ignorable as defined in Unicode
     6.3, Chapter 3

 -- Function: alphabetic-p character
     Returns ‘t’ if ‘character’ is Alphabetic according to the Unicode
     standard and ‘nil’ otherwise

 -- Function: ideographic-p character
     Returns ‘t’ if ‘character’ has the Unicode property Ideographic,
     which loosely corresponds to the set of "Chinese characters"

 -- Function: math-p character
     Returns ‘t’ if ‘character’ is a mathematical symbol according to
     Unicode and ‘nil’ otherwise

 -- Function: whitespace-p character
     Returns ‘t’ if ‘character’ is whitespace according to Unicode and
     ‘nil’ otherwise

 -- Function: soft-dotted-p character
     Returns ‘t’ if ‘character’ has a soft dot (such as the dots on i
     and j) which disappears when accents are placed on top of it.  and
     ‘nil’ otherwise

 -- Function: hex-digit-p character &key ascii
     Returns ‘t’ if ‘character’ is a hexadecimal digit and ‘nil’
     otherwise.  If ‘:ascii’ is non-NIL, fullwidth equivalents of the
     Latin letters A through ‘f’ are excluded.

 -- Function: default-ignorable-p character
     Returns ‘t’ if ‘character’ is a Default_Ignorable_Code_Point

 -- Function: grapheme-break-class char
     Returns the grapheme breaking class of ‘character’, as specified in
     ‘uax’ #29.

 -- Function: word-break-class char
     Returns the word breaking class of ‘character’, as specified in
     ‘uax’ #29.

 -- Function: sentence-break-class char
     Returns the sentence breaking class of ‘character’, as specified in
     ‘uax’ #29.

 -- Function: line-break-class character &key resolve
     Returns the line breaking class of ‘character’, as specified in
     ‘uax’ #14.  If ‘:resolve’ is ‘nil’, returns the character class
     found in the property file.  If ‘:resolve’ is non-NIL, certain
     line-breaking classes will be mapped to other classes as specified
     in the applicable standards.  Additionally, if ‘:resolve’ is
     ‘:east-asian’, Ambigious (class ‘:ai’) characters will be mapped to
     the Ideographic (‘:id’) class instead of Alphabetic (‘:al’).

7.10.2 String operations
------------------------

SBCL can normalize strings using:

 -- Function: normalize-string string &optional form filter
     Normalize ‘string’ to the Unicode normalization form ‘form’.
     Acceptable values for form are ‘:nfd’, ‘:nfc’, ‘:nfkd’, and
     ‘:nfkc’.  If ‘filter’ is a function it is called on each decomposed
     character and only characters for which it returns ‘t’ are
     collected.

 -- Function: normalized-p string &optional form
     Tests if ‘string’ is normalized to ‘form’

   SBCL implements the full range of Unicode case operations with the
functions

 -- Function: uppercase string &key locale
     Returns the full uppercase of ‘string’ according to the Unicode
     standard.  The result is not guaranteed to have the same length as
     the input.  If ‘:locale’ is ‘nil’, no language-specific case
     transformations are applied.  If ‘:locale’ is a keyword
     representing a two-letter ‘iso’ country code, the case transforms
     of that locale are used.  If ‘:locale’ is ‘t’, the user's current
     locale is used (Unix and Win32 only).

 -- Function: lowercase string &key locale
     Returns the full lowercase of ‘string’ according to the Unicode
     standard.  The result is not guaranteed to have the same length as
     the input.  ‘:locale’ has the same semantics as the ‘:locale’
     argument to ‘uppercase’.

 -- Function: titlecase string &key locale
     Returns the titlecase of ‘string’.  The resulting string can be
     longer than the input.  ‘:locale’ has the same semantics as the
     ‘:locale’ argument to ‘uppercase’.

 -- Function: casefold string
     Returns the full casefolding of ‘string’ according to the Unicode
     standard.  Casefolding removes case information in a way that
     allows the results to be used for case-insensitive comparisons.
     The result is not guaranteed to have the same length as the input.

   It also extends standard Common Lisp case functions such as
‘string-upcase’ and ‘string-downcase’ to support a subset of Unicode's
casing behavior.  Specifically, a character is ‘both-case-p’ if its case
mapping in Unicode is one-to-one and invertable.

   The ‘sb-unicode’ package also provides functions for
collating/sorting strings according to the Unicode Collation Algorithm.

 -- Function: unicode< string1 string2 &key start1 end1 start2 end2
     Determines whether STRING1 sorts before STRING2 using the Unicode
     Collation Algorithm.  The function uses an untailored Default
     Unicode Collation Element Table to produce the sort keys.  The
     function uses the Shifted method for dealing with variable-weight
     characters, as described in ‘uts’ #10

 -- Function: unicode= string1 string2 &key start1 end1 start2 end2
          strict
     Determines whether STRING1 and STRING2 are canonically equivalent
     according to Unicode.  The ‘start’ and ‘end’ arguments behave like
     the arguments to STRING=.  If ‘:strict’ is ‘nil’, UNICODE= tests
     compatibility equavalence instead.

 -- Function: unicode-equal string1 string2 &key start1 end1 start2 end2
          strict
     Determines whether STRING1 and STRING2 are canonically equivalent
     after casefolding (that is, ignoring case differences) according to
     Unicode.  The ‘start’ and ‘end’ arguments behave like the arguments
     to STRING=.  If ‘:strict’ is ‘nil’, UNICODE= tests compatibility
     equavalence instead.

 -- Function: unicode<= string1 string2 &key start1 end1 start2 end2
     Tests if STRING1 and STRING2 are either UNICODE< or UNICODE=

 -- Function: unicode> string1 string2 &key start1 end1 start2 end2
     Tests if STRING2 is UNICODE< STRING1.

 -- Function: unicode>= string1 string2 &key start1 end1 start2 end2
     Tests if STRING1 and STRING2 are either UNICODE= or UNICODE>

   The following functions are provided for detecting visually
confusable strings:

 -- Function: confusable-p string1 string2 &key start1 end1 start2 end2
     Determines whether STRING1 and STRING2 could be visually confusable
     according to the ‘idna’ confusableSummary.txt table

7.10.3 Breaking strings
-----------------------

The ‘sb-unicode’ package includes several functions for breaking a
Unicode string into useful parts.

 -- Function: graphemes string
     Breaks ‘string’ into graphemes according to the default grapheme
     breaking rules specified in ‘uax’ #29, returning a list of strings.

 -- Function: words string
     Breaks ‘string’ into words according to the default word breaking
     rules specified in ‘uax’ #29.  Returns a list of strings

 -- Function: sentences string
     Breaks ‘string’ into sentences according to the default sentence
     breaking rules specified in ‘uax’ #29

 -- Function: lines string &key margin
     Breaks ‘string’ into lines that are no wider than ‘:margin’
     according to the line breaking rules outlined in ‘uax’ #14.
     Combining marks will always be kept together with their base
     characters, and spaces (but not other types of whitespace) will be
     removed from the end of lines.  If ‘:margin’ is unspecified, it
     defaults to 80 characters

   ---------- Footnotes ----------

   (1) Please note that the codepoint U+1F5CF (PAGE) introduced in
Unicode 7.0 is named ‘UNICODE_PAGE’, since the name "Page" is required
to be assigned to form-feed (U+0C) by the ANSI standard.


File: sbcl.info,  Node: Customization Hooks for Users,  Next: Tools To Help Developers,  Prev: Unicode Support,  Up: Beyond the ANSI Standard

7.11 Customization Hooks for Users
==================================

The toplevel repl prompt may be customized, and the function that reads
user input may be replaced completely.

   The behaviour of ‘require’ when called with only one argument is
implementation-defined.  In SBCL, ‘require’ behaves in the following
way:

 -- Function: require module-name &optional pathnames
     Loads a module, unless it already has been loaded.  ‘pathnames’, if
     supplied, is a designator for a list of pathnames to be loaded if
     the module needs to be.  If ‘pathnames’ is not supplied, functions
     from the list ‘*module-provider-functions*’ are called in order
     with ‘module-name’ as an argument, until one of them returns
     non-NIL. User code is responsible for calling ‘provide’ to indicate
     a successful load of the module.
 -- Variable: *module-provider-functions*
     See function documentation for ‘require’.

   Although SBCL does not provide a resident editor, the ‘ed’ function
can be customized to hook into user-provided editing mechanisms as
follows:

 -- Function: ed &optional x
     Starts the editor (on a file or a function if named).  Functions
     from the list ‘*ed-functions*’ are called in order with ‘x’ as an
     argument until one of them returns non-NIL; these functions are
     responsible for signalling a ‘file-error’ to indicate failure to
     perform an operation on the file system.
 -- Variable: *ed-functions*
     See function documentation for ‘ed’.

   Conditions of type ‘warning’ and ‘style-warning’ are sometimes
signaled at runtime, especially during execution of Common Lisp defining
forms such as ‘defun’, ‘defmethod’, etc.  To muffle these warnings at
runtime, SBCL provides a variable ‘sb-ext:*muffled-warnings*’:

 -- Variable: *muffled-warnings*
     A type that ought to specify a subtype of ‘warning’.  Whenever a
     warning is signaled, if the warning is of this type and is not
     handled by any other handler, it will be muffled.


File: sbcl.info,  Node: Tools To Help Developers,  Next: Resolution of Name Conflicts,  Prev: Customization Hooks for Users,  Up: Beyond the ANSI Standard

7.12 Tools To Help Developers
=============================

SBCL provides a profiler and other extensions to the ANSI ‘trace’
facility.  For more information, see *note Macro common-lisp trace::.

   The debugger supports a number of options.  Its documentation is
accessed by typing ‘help’ at the debugger prompt.  *Note Debugger::.

   Documentation for ‘inspect’ is accessed by typing ‘help’ at the
‘inspect’ prompt.


File: sbcl.info,  Node: Resolution of Name Conflicts,  Next: Hash Table Extensions,  Prev: Tools To Help Developers,  Up: Beyond the ANSI Standard

7.13 Resolution of Name Conflicts
=================================

The ANSI standard (section 11.1.1.2.5) requires that name conflicts in
packages be resolvable in favour of any of the conflicting symbols.  In
the interactive debugger, this is achieved by prompting for the symbol
in whose favour the conflict should be resolved; for programmatic use,
the ‘sb-ext:resolve-conflict’ restart should be invoked with one
argument, which should be a member of the list returned by the condition
accessor ‘sb-ext:name-conflict-symbols’.


File: sbcl.info,  Node: Hash Table Extensions,  Next: Random Number Generation,  Prev: Resolution of Name Conflicts,  Up: Beyond the ANSI Standard

7.14 Hash Table Extensions
==========================

Hash table extensions supported by SBCL are all controlled by keyword
arguments to ‘make-hash-table’.

 -- Function: make-hash-table &key test size rehash-size
          rehash-threshold hash-function weakness synchronized
     Create and return a new hash table.  The keywords are as follows:

     _‘:test’_
          Determines how keys are compared.  Must a designator for one
          of the standard hash table tests, or a hash table test defined
          using ‘sb-ext:define-hash-table-test’.  Additionally, when an
          explicit ‘hash-function’ is provided (see below), any two
          argument equivalence predicate can be used as the ‘test’.

     _‘:size’_
          A hint as to how many elements will be put in this hash table.

     _‘:rehash-size’_
          Indicates how to expand the table when it fills up.  If an
          integer, add space for that many elements.  If a floating
          point number (which must be greater than 1.0), multiply the
          size by that amount.

     _‘:rehash-threshold’_
          Indicates how dense the table can become before forcing a
          rehash.  Can be any positive number <=1, with density
          approaching zero as the threshold approaches 0.  Density 1
          means an average of one entry per bucket.

     _‘:hash-function’_
          If unsupplied, a hash function based on the ‘test’ argument is
          used, which then must be one of the standardized hash table
          test functions, or one for which a default hash function has
          been defined using ‘sb-ext:define-hash-table-test’.  If
          ‘hash-function’ is specified, the ‘test’ argument can be any
          two argument predicate consistent with it.  The
          ‘hash-function’ is expected to return a non-negative fixnum
          hash code.  If ‘test’ is neither standard nor defined by
          ‘define-hash-table-test’, then the ‘hash-function’ must be
          specified.

     _‘:weakness’_
          When ‘:weakness’ is not ‘nil’, garbage collection may remove
          entries from the hash table.  The value of ‘:weakness’
          specifies how the presence of a key or value in the hash table
          preserves their entries from garbage collection.

          Valid values are:

          ‘:key’ means that the key of an entry must be live to
          guarantee that the entry is preserved.

          ‘:value’ means that the value of an entry must be live to
          guarantee that the entry is preserved.

          ‘:key-and-value’ means that both the key and the value must be
          live to guarantee that the entry is preserved.

          ‘:key-or-value’ means that either the key or the value must be
          live to guarantee that the entry is preserved.

          ‘nil’ (the default) means that entries are always preserved.

     _‘:synchronized’_
          If ‘nil’ (the default), the hash-table may have multiple
          concurrent readers, but results are undefined if a thread
          writes to the hash-table concurrently with another reader or
          writer.  If ‘t’, all concurrent accesses are safe, but note
          that ‘clhs’ 3.6 (Traversal Rules and Side Effects) remains in
          force.  See also: ‘sb-ext:with-locked-hash-table’.

 -- Macro: define-hash-table-test name hash-function
     Defines ‘name’ as a new kind of hash table test for use with the
     ‘:test’ argument to ‘make-hash-table’, and associates a default
     ‘hash-function’ with it.

     ‘name’ must be a symbol naming a global two argument equivalence
     predicate.  Afterwards both ‘'name’ and ‘#'name’ can be used with
     ‘:test’ argument.  In both cases ‘hash-table-test’ will return the
     symbol ‘name’.

     ‘hash-function’ must be a symbol naming a global hash function
     consistent with the predicate, or be a ‘lambda’ form implementing
     one in the current lexical environment.  The hash function must
     compute the same hash code for any two objects for which ‘name’
     returns true, and subsequent calls with already hashed objects must
     always return the same hash code.

     Note: The ‘:hash-function’ keyword argument to ‘make-hash-table’
     can be used to override the specified default hash-function.

     Attempting to define ‘name’ in a locked package as hash-table test
     causes a package lock violation.

     Examples:

            ;;; 1.

            ;; We want to use objects of type FOO as keys (by their
            ;; names.) EQUALP would work, but would make the names
            ;; case-insensitive -- which we don't want.
            (defstruct foo (name nil :type (or null string)))

            ;; Define an equivalence test function and a hash function.
            (defun foo-name= (f1 f2) (equal (foo-name f1) (foo-name f2)))
            (defun sxhash-foo-name (f) (sxhash (foo-name f)))

            (define-hash-table-test foo-name= sxhash-foo-name)

            ;; #'foo-name would work too.
            (defun make-foo-table () (make-hash-table :test 'foo-name=))

            ;;; 2.

            (defun == (x y) (= x y))

            (define-hash-table-test ==
              (lambda (x)
                ;; Hash codes must be consistent with test, so
                ;; not (SXHASH X), since
                ;;   (= 1 1.0)                   => T
                ;;   (= (SXHASH 1) (SXHASH 1.0)) => NIL
                ;; Note: this doesn't deal with complex numbers or
                ;; bignums too large to represent as double floats.
                (sxhash (coerce x 'double-float))))

            ;; #'== would work too
            (defun make-number-table () (make-hash-table :test '==))

 -- Macro: with-locked-hash-table (hash-table) &body body
     Limits concurrent accesses to ‘hash-table’ for the duration of
     ‘body’.  If ‘hash-table’ is synchronized, ‘body’ will execute with
     exclusive ownership of the table.  If ‘hash-table’ is not
     synchronized, ‘body’ will execute with other
     ‘with-locked-hash-table’ bodies excluded ‘--’ exclusion of
     hash-table accesses not surrounded by ‘with-locked-hash-table’ is
     unspecified.

 -- Function: hash-table-synchronized-p ht
     Returns ‘t’ if ‘hash-table’ is synchronized.

 -- Function: hash-table-weakness ht
     Return the ‘weakness’ of ‘hash-table’ which is one of ‘nil’,
     ‘:key’, ‘:value’, ‘:key-and-value’, ‘:key-or-value’.


File: sbcl.info,  Node: Random Number Generation,  Next: Timeouts and Deadlines,  Prev: Hash Table Extensions,  Up: Beyond the ANSI Standard

7.15 Random Number Generation
=============================

The initial value of ‘*random-state*’ is the same each time SBCL is
started.  This makes it possible for user code to obtain repeatable
pseudo random numbers using only standard-provided functionality.  See
‘seed-random-state’ below for an SBCL extension that allows to seed the
random number generator from given data for an additional possibility to
achieve this.  Non-repeatable random numbers can always be obtained
using ‘(make-random-state t)’.

   The sequence of numbers produced by repeated calls to ‘random’
starting with the same random state and using the same sequence of
‘limit’ arguments is guaranteed to be reproducible only in the same
version of SBCL on the same platform, using the same code under the same
evaluator mode and compiler optimization qualities.  Just two examples
of differences that may occur otherwise: calls to ‘random’ can be
compiled differently depending on how much is known about the ‘limit’
argument at compile time, yielding different results even if called with
the same argument at run time, and the results can differ depending on
the machine's word size, for example for limits that are fixnums under
64-bit word size but bignums under 32-bit word size.

 -- Function: seed-random-state &optional state
     Make a random state object.  The optional ‘state’ argument
     specifies a seed for deterministic pseudo-random number generation.

     As per the Common Lisp standard for ‘make-random-state’,
        • If ‘state’ is ‘nil’ or not supplied, return a copy of the
          default ‘*random-state*’.
        • If ‘state’ is a random state, return a copy of it.
        • If ‘state’ is ‘t’, return a randomly initialized state (using
          operating-system provided randomness where available,
          otherwise a poor substitute based on internal time and pid).

     As a supported ‘sbcl’ extension, we also support receiving as a
     seed an object of the following types:
        • (‘simple-array’ (‘unsigned-byte’ 8) (‘*’))
        • ‘unsigned-byte’
     While we support arguments of any size and will mix the provided
     bits into the random state, it is probably overkill to provide more
     than 256 bits worth of actual information.

     This particular ‘sbcl’ version also accepts an argument of the
     following type: (‘simple-array’ (‘unsigned-byte’ 32) (‘*’))

     This particular ‘sbcl’ version uses the popular MT19937 ‘prng’
     algorithm, and its internal state only effectively contains about
     19937 bits of information.
     http://www.math.sci.hiroshima-u.ac.jp/~m-mat/MT/emt.html

   Some notes on random floats: The standard doesn't prescribe a
specific method of generating random floats.  The following paragraph
describes SBCL's current implementation and should be taken as purely
informational, that is, user code should not depend on any of its
specific properties.  The method used has been chosen because it is
common, conceptually simple and fast.

   To generate random floats, SBCL evaluates code that has an equivalent
effect as
     (* limit
        (float (/ (random (expt 2 23)) (expt 2 23)) 1.0f0))
   (for single-floats) and correspondingly (with ‘52’ and ‘1.0d0’
instead of ‘23’ and ‘1.0f0’) for double-floats.  Note especially that
this means that zero is a possible return value occurring with
probability ‘(expt 2 -23)’ respectively ‘(expt 2 -52)’.  Also note that
there exist twice as many equidistant floats between 0 and 1 as are
generated.  For example, the largest number that ‘(random 1.0f0)’ ever
returns is ‘(float (/ (1- (expt 2 23)) (expt 2 23)) 1.0f0)’ while
‘(float (/ (1- (expt 2 24)) (expt 2 24)) 1.0f0)’ is the largest
single-float less than 1.  This is a side effect of the fact that the
implementation uses the fastest possible conversion from bits to floats.

   SBCL currently uses the Mersenne Twister as its random number
generator, specifically the 32-bit version under both 32- and 64-bit
word size.  The seeding algorithm has been improved several times by the
authors of the Mersenne Twister; SBCL uses the third version (from 2002)
which is still the most recent as of June 2012.  The implementation has
been tested to provide output identical to the recommended C
implementation.

   While the Mersenne Twister generates random numbers of much better
statistical quality than other widely used generators, it uses only
linear operations modulo 2 and thus fails some statistical tests(1).
For example, the distribution of ranks of (sufficiently large) random
binary matrices is much distorted compared to the theoretically expected
one when the matrices are generated by the Mersenne Twister.  Thus,
applications that are sensitive to this aspect should use a different
type of generator.

   ---------- Footnotes ----------

   (1) See chapter 7 "Testing widely used RNGs" in ‘TestU01: A C Library
for Empirical Testing of Random Number Generators’ by Pierre L'Ecuyer
and Richard Simard, ACM Transactions on Mathematical Software, Vol.  33,
article 22, 2007.


File: sbcl.info,  Node: Timeouts and Deadlines,  Next: Miscellaneous Extensions,  Prev: Random Number Generation,  Up: Beyond the ANSI Standard

7.16 Timeouts and Deadlines
===========================

SBCL supports three different ways of restricting the execution time
available to individual operations or parts of computations:

*Timeout Parameters*
     Some operations such as thread synchronization primitives accept a
     ‘:timeout’ parameter.  *Note Timeout Parameters::.

*Synchronous Timeouts (Deadlines)*
     Certain operations that may suspend execution for extended periods
     of time such as ‘cl:sleep’, thread synchronization primitives, IO
     and waiting for external processes respect deadlines established
     for a part of a computation.  *Note Synchronous Timeouts
     (Deadlines)::.

*Asynchronous Timeouts*
     Asynchronous timeouts can interrupt most computations at (almost)
     any point.  Thus, this kind of timeouts is the most versatile but
     it is also somewhat unsafe.  *Note Asynchronous Timeouts::.

* Menu:

* Timeout Parameters::
* Synchronous Timeouts (Deadlines)::
* Asynchronous Timeouts::
* Operations Supporting Timeouts and Deadlines::


File: sbcl.info,  Node: Timeout Parameters,  Next: Synchronous Timeouts (Deadlines),  Up: Timeouts and Deadlines

7.16.1 Timeout Parameters
-------------------------

Certain operations accept ‘:timeout’ keyword arguments.  These only
affect the specific operation and must be specified at each call site by
passing a ‘:timeout’ keyword argument and a corresponding timeout value
to the respective operation.  Expiration of the timeout before the
operation completes results in either a normal return with a return
value indicating the timeout or in the signaling of a specialized
condition such as ‘sb-thread:join-thread-error’.

   Example:

     (defun join-thread-within-5-seconds (thread)
       (multiple-value-bind (value result)
           (sb-thread:join-thread thread :default nil :timeout 5)
         (when (eq result :timeout)
           (error "Could not join ~A within 5 seconds" thread))
         value))

   The above code attempts to join the specified thread for up to five
seconds, returning its value in case of success.  If the thread is still
running after the five seconds have elapsed, ‘sb-thread:join-thread’
indicates the timeout in its second return value.  If a ‘:default’ value
was not provided, ‘sb-thread:join-thread’ would signal a
‘sb-thread:join-thread-error’ instead.

   To wait for an arbitrary condition, optionally with a timeout, the
‘sb-ext:wait-for’ macro can be used:

 -- Macro: wait-for test-form &key timeout
     Wait until ‘test-form’ evaluates to true, then return its primary
     value.  If ‘timeout’ is provided, waits at most approximately
     ‘timeout’ seconds before returning ‘nil’.

     If ‘with-deadline’ has been used to provide a global deadline,
     signals a ‘deadline-timeout’ if ‘test-form’ doesn't evaluate to
     true before the deadline.

     Experimental: subject to change without prior notice.


File: sbcl.info,  Node: Synchronous Timeouts (Deadlines),  Next: Asynchronous Timeouts,  Prev: Timeout Parameters,  Up: Timeouts and Deadlines

7.16.2 Synchronous Timeouts (Deadlines)
---------------------------------------

Deadlines, in contrast to timeout parameters, are established for a
dynamic scope using the ‘sb-sys:with-deadline’ macro and indirectly
affect operations within that scope.  In case of nested uses, the
effective deadline is the one that expires first unless an inner use
explicitly overrides outer deadlines.

 -- Macro: with-deadline (&key seconds override) &body body
     Arranges for a ‘timeout’ condition to be signalled if an operation
     respecting deadlines occurs either after the deadline has passed,
     or would take longer than the time left to complete.

     Currently only ‘sleep’, blocking ‘io’ operations, ‘get-mutex’, and
     ‘condition-wait’ respect deadlines, but this includes their
     implicit uses inside ‘sbcl’ itself.

     Unless ‘override’ is true, existing deadlines can only be
     restricted, not extended.  Deadlines are per thread: children are
     unaffected by their parent's deadlines.

     Experimental.

   Expiration of deadlines set up this way only has an effect when it
happens before or during the execution of a deadline-aware operation
(*note Operations Supporting Timeouts and Deadlines::).  In this case, a
‘sb-sys:deadline-timeout’ is signaled.  A handler for this condition
type may use the ‘sb-sys:defer-deadline’ or ‘sb-sys:cancel-deadline’
restarts to defer or cancel the deadline respectively and resume
execution of the interrupted operation.

 -- Condition: deadline-timeout
     Class precedence list: ‘deadline-timeout’, ‘timeout’,
     ‘serious-condition’, ‘condition’, ‘t’

     Signaled when an operation in the context of a deadline takes
     longer than permitted by the deadline.

   When a thread is executing the debugger, signaling of
‘sb-sys:deadline-timeout’ conditions for that thread is deferred until
it exits the debugger.

   Example:

     (defun read-input ()
       (list (read-line) (read-line)))

     (defun do-it ()
       (sb-sys:with-deadline (:seconds 5))
         (read-input)
         (sleep 2)
         (sb-ext:run-program "my-program"))

   The above code establishes a deadline of five seconds within which
the body of the ‘do-it’ function should execute.  All calls of
deadline-aware functions in the dynamic scope, in this case two
‘read-line’ calls, a ‘sleep’ call and a ‘sb-ext:run-program’ call, are
affected by the deadline.  If, for example, the first ‘read-line’ call
completes in one second and the second ‘read-line’ call completes in
three seconds, a ‘sb-sys:deadline-timeout’ condition will be signaled
after the ‘sleep’ call has been executing for one second.


File: sbcl.info,  Node: Asynchronous Timeouts,  Next: Operations Supporting Timeouts and Deadlines,  Prev: Synchronous Timeouts (Deadlines),  Up: Timeouts and Deadlines

7.16.3 Asynchronous Timeouts
----------------------------

Asynchronous timeouts are established for a dynamic scope using the
‘sb-sys:with-timeout’ macro:

 -- Macro: with-timeout expires &body body
     Execute the body, asynchronously interrupting it and signalling a
     ‘timeout’ condition after at least ‘expires’ seconds have passed.

     Note that it is never safe to unwind from an asynchronous
     condition.  Consider:

            (defun call-with-foo (function)
              (let (foo)
                (unwind-protect
                   (progn
                     (setf foo (get-foo))
                     (funcall function foo))
                 (when foo
                   (release-foo foo)))))

     If ‘timeout’ occurs after ‘get-foo’ has executed, but before the
     assignment, then ‘release-foo’ will be missed.  While individual
     sites like this can be made proof against asynchronous unwinds,
     this doesn't solve the fundamental issue, as all the frames
     potentially unwound through need to be proofed, which includes both
     system and application code ‘--’ and in essence proofing everything
     will make the system uninterruptible.

   Expiration of the timeout will cause the operation being executed at
that moment to be interrupted by an asynchronously signaled
‘sb-ext:timeout’ condition, (almost) irregardless of the operation and
its context.

 -- Condition: timeout
     Class precedence list: ‘timeout’, ‘serious-condition’, ‘condition’,
     ‘t’

     Signaled when an operation does not complete within an allotted
     time budget.


File: sbcl.info,  Node: Operations Supporting Timeouts and Deadlines,  Prev: Asynchronous Timeouts,  Up: Timeouts and Deadlines

7.16.4 Operations Supporting Timeouts and Deadlines
---------------------------------------------------

Operation                            Timeout            Affected by
                                     Parameter          Deadlines
---------------------------------------------------------------------------
‘cl:sleep’                           -                  since SBCL 1.4.3
‘cl:read-line’, etc.                 no                 yes
*note ‘wait-for’: Macro sb-ext wait-for.yes             yes
*note ‘process-wait’: Function sb-ext process-wait.no   yes
*note ‘grab-mutex’: Function sb-thread grab-mutex.yes   yes
*note ‘condition-wait’: Function sb-thread condition-wait.yesyes
*note ‘wait-on-semaphore’: Function sb-thread wait-on-semaphore.yesyes
*note ‘join-thread’: Function sb-thread join-thread.yes yes
*note ‘receive-message’: Function sb-concurrency receive-message.yesyes?
*note ‘wait-on-gate’: Function sb-concurrency wait-on-gate.yesyes?
*note ‘frlock-write’: Macro sb-concurrency frlock-write.yesyes?
*note ‘grab-frlock-write-lock’: Function sb-concurrency grab-frlock-write-lock.yesyes?


File: sbcl.info,  Node: Miscellaneous Extensions,  Next: Stale Extensions,  Prev: Timeouts and Deadlines,  Up: Beyond the ANSI Standard

7.17 Miscellaneous Extensions
=============================

 -- Function: array-storage-vector array
     Returns the underlying storage vector of ‘array’, which must be a
     non-displaced array.

     In ‘sbcl’, if ‘array’ is a of type (‘simple-array’ ‘*’ (‘*’)), it
     is its own storage vector.  Multidimensional arrays, arrays with
     fill pointers, and adjustable arrays have an underlying storage
     vector with the same ‘array-element-type’ as ‘array’, which this
     function returns.

     Important note: the underlying vector is an implementation detail.
     Even though this function exposes it, changes in the implementation
     may cause this function to be removed without further warning.
 -- Function: delete-directory pathspec &key recursive
     Deletes the directory designated by ‘pathspec’ (a pathname
     designator).  Returns the truename of the directory deleted.

     If ‘recursive’ is false (the default), signals an error unless the
     directory is empty.  If ‘recursive’ is true, first deletes all
     files and subdirectories.  If ‘recursive’ is true and the directory
     contains symbolic links, the links are deleted, not the files and
     directories they point to.

     Signals an error if ‘pathspec’ designates a file or a symbolic link
     instead of a directory, or if the directory could not be deleted
     for any reason.

     Both

             (DELETE-DIRECTORY "/tmp/foo")
             (DELETE-DIRECTORY "/tmp/foo/")

     delete the "foo" subdirectory of "/tmp", or signal an error if it
     does not exist or if is a file or a symbolic link.
 -- Function: get-time-of-day
     Return the number of seconds and microseconds since the beginning
     of the ‘unix’ epoch (January 1st 1970.)
 -- Function: assert-version->= &rest subversions
     Asserts that the current ‘sbcl’ is of version equal to or greater
     than the version specified in the arguments.  A continuable error
     is signaled otherwise.

     The arguments specify a sequence of subversion numbers in big
     endian order.  They are compared lexicographically with the runtime
     version, and versions are treated as though trailed by an unbounded
     number of 0s.

     For example, (assert-version->= 1 1 4) asserts that the current
     ‘sbcl’ is version 1.1.4[.0.0...]  or greater, and
     (assert-version->= 1) that it is version 1[.0.0...]  or greater.


File: sbcl.info,  Node: Stale Extensions,  Next: Efficiency Hacks,  Prev: Miscellaneous Extensions,  Up: Beyond the ANSI Standard

7.18 Stale Extensions
=====================

SBCL has inherited from CMUCL various hooks to allow the user to tweak
and monitor the garbage collection process.  These are somewhat stale
code, and their interface might need to be cleaned up.  If you have
urgent need of them, look at the code in ‘src/code/gc.lisp’ and bring it
up on the developers' mailing list.

   SBCL has various hooks inherited from CMUCL, like
‘sb-ext:float-denormalized-p’, to allow a program to take advantage of
IEEE floating point arithmetic properties which aren't conveniently or
efficiently expressible using the ANSI standard.  These look good, and
their interface looks good, but IEEE support is slightly broken due to a
stupid decision to remove some support for infinities (because it wasn't
in the ANSI spec and it didn't occur to me that it was in the IEEE
spec).  If you need this stuff, take a look at the code and bring it up
on the developers' mailing list.


File: sbcl.info,  Node: Efficiency Hacks,  Prev: Stale Extensions,  Up: Beyond the ANSI Standard

7.19 Efficiency Hacks
=====================

The ‘sb-ext:purify’ function causes SBCL first to collect all garbage,
then to mark all uncollected objects as permanent, never again
attempting to collect them as garbage.  This can cause a large increase
in efficiency when using a primitive garbage collector, or a more
moderate increase in efficiency when using a more sophisticated garbage
collector which is well suited to the program's memory usage pattern.
It also allows permanent code to be frozen at fixed addresses, a
precondition for using copy-on-write to share code between multiple Lisp
processes.  This is less important with modern generational garbage
collectors, but not all SBCL platforms use such a garbage collector.

   The ‘sb-ext:truly-the’ special form declares the type of the result
of the operations, producing its argument; the declaration is not
checked.  In short: don't use it.

 -- Special Operator: truly-the value-type form
     Specifies that the values returned by ‘form’ conform to the
     ‘value-type’, and causes the compiler to trust this information
     unconditionally.

     Consequences are undefined if any result is not of the declared
     type ‘--’ typical symptoms including memory corruptions.  Use with
     great care.

   The ‘sb-ext:freeze-type’ declaration declares that a type will never
change, which can make type testing (‘typep’, etc.)  more efficient for
structure types.


File: sbcl.info,  Node: External Formats,  Next: Foreign Function Interface,  Prev: Beyond the ANSI Standard,  Up: Top

8 External Formats
******************

External formats determine the coding of characters from/to sequences of
octets when exchanging data with the outside world.  Examples of such
exchanges are:

  1. Character streams associated with files, sockets and process
     input/output (See *note Stream External Formats:: and *note Running
     external programs::)

  2. Names of files

  3. Foreign strings (See *note Foreign Types and Lisp Types::)

  4. Posix interface (See *note sb-posix::)

  5. Hostname- and protocol-related functions of the BSD-socket
     interface (See *note Networking::)

   Technically, external formats in SBCL are named objects describing
coding of characters as well as policies in case de- or encoding is not
possible.  Each external format has a canonical name and zero or more
aliases.  User code mostly interacts with external formats by supplying
external format designators to functions that use external formats
internally.

* Menu:

* The Default External Format::
* External Format Designators::
* Character Coding Conditions::
* Converting between Strings and Octet Vectors::
* Supported External Formats::


File: sbcl.info,  Node: The Default External Format,  Next: External Format Designators,  Up: External Formats

8.1 The Default External Format
===============================

Most functions interacting with external formats use a default external
format if none is explicitly specified.  In some cases, the default
external format is used unconditionally.

   The default external format is UTF-8.  It can be changed via

   SB-EXT:*DEFAULT-EXTERNAL-FORMAT* and
SB-EXT:*DEFAULT-C-STRING-EXTERNAL-FORMAT*


File: sbcl.info,  Node: External Format Designators,  Next: Character Coding Conditions,  Prev: The Default External Format,  Up: External Formats

8.2 External Format Designators
===============================

In situations where an external format designator is required, such as
the ‘:external-format’ argument in calls to ‘open’ or ‘with-open-file’,
users may supply the name of an encoding to denote the external format
which is applying that encoding to Lisp characters.

   In addition to the basic encoding for an external format, options
controlling various special cases may be passed, by using a list (whose
first element must be an encoding name and whose rest is a plist) as an
external file format designator.

   More specifically, external format designators can take the following
forms:

‘:default’
     Designates the current default external format (See *note The
     Default External Format::).

‘KEYWORD’
     Designates the supported external format that has KEYWORD as one of
     its names.  (See *note Supported External Formats::).

‘(KEYWORD . OPTIONS-PLIST)’
     Designates an external format that is like the one designated by
     KEYWORD with options as specified in OPTIONS-PLIST.

   Valid options for OPTIONS-PLIST are:

‘:newline NEWLINE’

     An external format with an explicit ‘:newline’ option is like its
     KEYWORD parent, but recognizes certain characters or character
     sequences as newlines.  For ‘:lf’ (the default), the ‘#\Linefeed’
     character is treated as ‘#\Newline’ for both input and output.  For
     ‘:cr’, ‘#\Return’ is treated as ‘#\Newline’, while for ‘:crlf’ the
     two-character sequence ‘#\Return #\Linefeed’ is translated to and
     from ‘#\Newline’.

‘:replacement REPLACEMENT’

     An external format with an explicit ‘:replacement’ option is like
     its KEYWORD parent, but does not signal an error in case a
     character or octet sequence cannot be en- or decoded.  Instead, it
     inserts REPLACEMENT at the position in question.  REPLACEMENT has
     to be a string designator, that is a character or string.

     For example:
          (with-open-file (stream pathname :external-format '(:utf-8 :replacement #\?))
            (read-line stream))
     will read the first line of PATHNAME, replacing any octet sequence
     that is not valid in the UTF-8 external format with a question mark
     character.


File: sbcl.info,  Node: Character Coding Conditions,  Next: Converting between Strings and Octet Vectors,  Prev: External Format Designators,  Up: External Formats

8.3 Character Coding Conditions
===============================

De- or encoding characters using a given external format is not always
possible:

   • Decoding an octet vector using a given external format can fail if
     it contains an octet or sequence of octets that does not have an
     interpretation as a character according to the external format.

   • Conversely, a string may contain characters that a given external
     format cannot encode.  For example, the ASCII external format
     cannot encode the character ‘#\ö’.

   Unless the external format governing the coding uses the
‘:replacement’ keyword, SBCL will signal (continuable) errors under the
above circumstances.  The types of the condition signaled are not
currently exported or documented but will be in future SBCL versions.


File: sbcl.info,  Node: Converting between Strings and Octet Vectors,  Next: Supported External Formats,  Prev: Character Coding Conditions,  Up: External Formats

8.4 Converting between Strings and Octet Vectors
================================================

To encode Lisp strings as octet vectors and decode octet vectors as Lisp
strings, the following SBCL-specific functions can be used:

 -- Function: string-to-octets string &key external-format start end
          null-terminate
     Return an octet vector that is ‘string’ encoded according to
     ‘external-format’.

     If ‘external-format’ is given, it must designate an external
     format.

     If given, ‘start’ and ‘end’ must be bounding index designators and
     designate a subsequence of ‘string’ that should be encoded.

     If ‘null-terminate’ is true, the returned octet vector ends with an
     additional 0 element that does not correspond to any part of
     ‘string’.

     If some of the characters of ‘string’ (or the subsequence bounded
     by ‘start’ and ‘end’) cannot be encoded by ‘external-format’ an
     error of a subtype of ‘sb-int:character-encoding-error’ is
     signaled.

     Note that for some values of ‘external-format’ and ‘null-terminate’
     the length of the returned vector may be different from the length
     of ‘string’ (or the subsequence bounded by ‘start’ and ‘end’).
 -- Function: octets-to-string vector &key external-format start end
     Return a string obtained by decoding ‘vector’ according to
     ‘external-format’.

     If ‘external-format’ is given, it must designate an external
     format.

     If given, ‘start’ and ‘end’ must be bounding index designators and
     designate a subsequence of ‘vector’ that should be decoded.

     If some of the octets of ‘vector’ (or the subsequence bounded by
     ‘start’ and ‘end’) cannot be decoded by ‘external-format’ an error
     of a subtype of ‘sb-int:character-decoding-error’ is signaled.

     Note that for some values of ‘external-format’ the length of the
     returned string may be different from the length of ‘vector’ (or
     the subsequence bounded by ‘start’ and ‘end’).


File: sbcl.info,  Node: Supported External Formats,  Prev: Converting between Strings and Octet Vectors,  Up: External Formats

8.5 Supported External Formats
==============================

The following table lists the external formats supported by SBCL in the
form of the respective canonical name followed by the list of aliases:

‘:EUC-JP’
     ‘:EUCJP’, ‘:|eucJP|’

‘:GBK’
     ‘:CP936’

‘:SHIFT_JIS’
     ‘:SJIS’, ‘:|Shift_JIS|’, ‘:CP932’

‘:UCS-2BE’
     ‘:UCS2BE’

‘:UCS-2LE’
     ‘:UCS2LE’

‘:UCS-4BE’
     ‘:UCS4BE’

‘:UCS-4LE’
     ‘:UCS4LE’

‘:UTF-16BE’
     ‘:UTF16BE’

‘:UTF-16LE’
     ‘:UTF16LE’

‘:UTF-32BE’
     ‘:UTF32BE’

‘:UTF-32LE’
     ‘:UTF32LE’


File: sbcl.info,  Node: Foreign Function Interface,  Next: Pathnames,  Prev: External Formats,  Up: Top

9 Foreign Function Interface
****************************

This chapter describes SBCL's interface to C programs and libraries
(and, since C interfaces are a sort of _lingua franca_ of the Unix
world, to other programs and libraries in general.)

     Note: In the modern Lisp world, the usual term for this
     functionality is Foreign Function Interface, or FFI, where despite
     the mention of "function" in this term, FFI also refers to direct
     manipulation of C data structures as well as functions.  The
     traditional CMUCL terminology is Alien Interface, and while that
     older terminology is no longer used much in the system
     documentation, it still reflected in names in the implementation,
     notably in the name of the ‘SB-ALIEN’ package.

* Menu:

* Introduction to the Foreign Function Interface::
* Foreign Types::
* Operations On Foreign Values::
* Foreign Variables::
* Foreign Data Structure Examples::
* Loading Shared Object Files::
* Foreign Function Calls::
* Calling Lisp From C::
* Step-By-Step Example of the Foreign Function Interface::


File: sbcl.info,  Node: Introduction to the Foreign Function Interface,  Next: Foreign Types,  Up: Foreign Function Interface

9.1 Introduction to the Foreign Function Interface
==================================================

Because of Lisp's emphasis on dynamic memory allocation and garbage
collection, Lisp implementations use non-C-like memory representations
for objects.  This representation mismatch creates friction when a Lisp
program must share objects with programs which expect C data.  There are
three common approaches to establishing communication:

   • The burden can be placed on the foreign program (and programmer) by
     requiring the knowledge and use of the representations used
     internally by the Lisp implementation.  This can require a
     considerable amount of "glue" code on the C side, and that code
     tends to be sensitively dependent on the internal implementation
     details of the Lisp system.

   • The Lisp system can automatically convert objects back and forth
     between the Lisp and foreign representations.  This is convenient,
     but translation becomes prohibitively slow when large or complex
     data structures must be shared.  This approach is supported by the
     SBCL FFI, and used automatically when passing integers and strings.

   • The Lisp program can directly manipulate foreign objects through
     the use of extensions to the Lisp language.

   SBCL, like CMUCL before it, relies primarily on the automatic
conversion and direct manipulation approaches.  The ‘SB-ALIEN’ package
provides a facility wherein foreign values of simple scalar types are
automatically converted and complex types are directly manipulated in
their foreign representation.  Additionally the lower-level System Area
Pointers (or SAPs) can be used where necessary to provide untyped access
to foreign memory.

   Any foreign objects that can't automatically be converted into Lisp
values are represented by objects of type ‘alien-value’.  Since Lisp is
a dynamically typed language, even foreign objects must have a run-time
type; this type information is provided by encapsulating the raw pointer
to the foreign data within an ‘alien-value’ object.

   The type language and operations on foreign types are intentionally
similar to those of the C language.


File: sbcl.info,  Node: Foreign Types,  Next: Operations On Foreign Values,  Prev: Introduction to the Foreign Function Interface,  Up: Foreign Function Interface

9.2 Foreign Types
=================

Alien types have a description language based on nested list structure.
For example the C type

     struct foo {
         int a;
         struct foo *b[100];
     };

   has the corresponding SBCL FFI type

     (struct foo
       (a int)
       (b (array (* (struct foo)) 100)))

* Menu:

* Defining Foreign Types::
* Foreign Types and Lisp Types::
* Foreign Type Specifiers::


File: sbcl.info,  Node: Defining Foreign Types,  Next: Foreign Types and Lisp Types,  Up: Foreign Types

9.2.1 Defining Foreign Types
----------------------------

Types may be either named or anonymous.  With structure and union types,
the name is part of the type specifier, allowing recursively defined
types such as:

     (struct foo (a (* (struct foo))))

   An anonymous structure or union type is specified by using the name
‘nil’.  The ‘with-alien’ macro defines a local scope which "captures"
any named type definitions.  Other types are not inherently named, but
can be given named abbreviations using the ‘define-alien-type’ macro.


File: sbcl.info,  Node: Foreign Types and Lisp Types,  Next: Foreign Type Specifiers,  Prev: Defining Foreign Types,  Up: Foreign Types

9.2.2 Foreign Types and Lisp Types
----------------------------------

The foreign types form a subsystem of the SBCL type system.  An ‘alien’
type specifier provides a way to use any foreign type as a Lisp type
specifier.  For example,

     (typep FOO '(alien (* int)))

   can be used to determine whether FOO is a pointer to a foreign ‘int’.
‘alien’ type specifiers can be used in the same ways as ordinary Lisp
type specifiers (like ‘string’.)  Alien type declarations are subject to
the same precise type checking as any other declaration.  *Note Precise
Type Checking::.

   Note that the type identifiers used in the foreign type system
overlap with native Lisp type specifiers in some cases.  For example,
the type specifier ‘(alien single-float)’ is identical to
‘single-float’, since foreign floats are automatically converted to Lisp
floats.  When ‘type-of’ is called on an alien value that is not
automatically converted to a Lisp value, then it will return an ‘alien’
type specifier.


File: sbcl.info,  Node: Foreign Type Specifiers,  Prev: Foreign Types and Lisp Types,  Up: Foreign Types

9.2.3 Foreign Type Specifiers
-----------------------------

Note: All foreign type names are exported from the ‘sb-alien’ package.
Some foreign type names are also symbols in the ‘common-lisp’ package,
in which case they are reexported from the ‘sb-alien’ package, so that
e.g.  it is legal to refer to ‘sb-alien:single-float’.

   These are the basic foreign type specifiers:

   • The foreign type specifier ‘(* FOO)’ describes a pointer to an
     object of type FOO.  A pointed-to type FOO of ‘t’ indicates a
     pointer to anything, similar to ‘void *’ in ANSI C. A null alien
     pointer can be detected with the ‘sb-alien:null-alien’ function.

   • The foreign type specifier ‘(array FOO &rest dimensions)’ describes
     array of the specified ‘dimensions’, holding elements of type FOO.
     Note that (unlike in C) ‘(* FOO)’ and ‘(array FOO)’ are considered
     to be different types when type checking is done.  If equivalence
     of pointer and array types is desired, it may be explicitly coerced
     using ‘sb-alien:cast’.

     Arrays are accessed using ‘sb-alien:deref’, passing the indices as
     additional arguments.  Elements are stored in column-major order
     (as in C), so the first dimension determines only the size of the
     memory block, and not the layout of the higher dimensions.  An
     array whose first dimension is variable may be specified by using
     ‘nil’ as the first dimension.  Fixed-size arrays can be allocated
     as array elements, structure slots or ‘sb-alien:with-alien’
     variables.  Dynamic arrays can only be allocated using
     ‘sb-alien:make-alien’.

   • The foreign type specifier ‘(sb-alien:struct NAME &rest FIELDS)’
     describes a structure type with the specified NAME and FIELDS.
     Fields are allocated at the same offsets used by the
     implementation's C compiler, as guessed by the SBCL internals.  An
     optional ‘:alignment’ keyword argument can be specified for each
     field to explicitly control the alignment of a field.  If NAME is
     ‘nil’ then the structure is anonymous.

     If a named foreign ‘struct’ specifier is passed to
     ‘define-alien-type’ or ‘with-alien’, then this defines,
     respectively, a new global or local foreign structure type.  If no
     FIELDS are specified, then the fields are taken from the current
     (local or global) alien structure type definition of NAME.

   • The foreign type specifier ‘(sb-alien:union NAME &rest FIELDS)’ is
     similar to ‘sb-alien:struct’, but describes a union type.  All
     fields are allocated at the same offset, and the size of the union
     is the size of the largest field.  The programmer must determine
     which field is active from context.

   • The foreign type specifier ‘(sb-alien:enum NAME &rest SPECS)’
     describes an enumeration type that maps between integer values and
     symbols.  If NAME is ‘nil’, then the type is anonymous.  Each
     element of the SPECS list is either a Lisp symbol, or a list
     ‘(SYMBOL VALUE)’.  VALUE is an integer.  If VALUE is not supplied,
     then it defaults to one greater than the value for the preceding
     spec (or to zero if it is the first spec).

   • The foreign type specifier ‘(sb-alien:signed &optional BITS)’
     specifies a signed integer with the specified number of BITS
     precision.  The upper limit on integer precision is determined by
     the machine's word size.  If BITS is not specified, the maximum
     size will be used.

   • The foreign type specifier ‘(integer &optional BITS)’ is equivalent
     to the corresponding type specifier using ‘sb-alien:signed’ instead
     of ‘integer’.

   • The foreign type specifier ‘(sb-alien:unsigned &optional BITS)’ is
     like corresponding type specifier using ‘sb-alien:signed’ except
     that the variable is treated as an unsigned integer.

   • The foreign type specifier ‘(boolean &optional BITS)’ is similar to
     an enumeration type, but maps from Lisp ‘nil’ and ‘t’ to C ‘0’ and
     ‘1’ respectively.  BITS determines the amount of storage allocated
     to hold the truth value.

   • The foreign type specifier ‘single-float’ describes a
     floating-point number in IEEE single-precision format.

   • The foreign type specifier ‘double-float’ describes a
     floating-point number in IEEE double-precision format.

   • The foreign type specifier ‘(function RESULT-TYPE &rest ARG-TYPES)’
     describes a foreign function that takes arguments of the specified
     ARG-TYPES and returns a result of type RESULT-TYPE.  Note that the
     only context where a foreign ‘function’ type is directly specified
     is in the argument to ‘sb-alien:alien-funcall’.  In all other
     contexts, foreign functions are represented by foreign function
     pointer types: ‘(* (function ...))’.

   • The foreign type specifier ‘sb-alien:system-area-pointer’ describes
     a pointer which is represented in Lisp as a ‘system-area-pointer’
     object.  SBCL exports this type from ‘sb-alien’ because CMUCL did,
     but tentatively (as of the first draft of this section of the
     manual, SBCL 0.7.6) it is deprecated, since it doesn't seem to be
     required by user code.

   • The foreign type specifier ‘sb-alien:void’ is used in function
     types to declare that no useful value is returned.  Using
     ‘alien-funcall’ to call a ‘void’ foreign function will return zero
     values.

   • The foreign type specifier ‘(sb-alien:c-string &key external-format
     element-type not-null)’ is similar to ‘(* char)’, but is
     interpreted as a null-terminated string, and is automatically
     converted into a Lisp string when accessed; or if the pointer is C
     ‘NULL’ or ‘0’, then accessing it gives Lisp ‘nil’ unless ‘not-null’
     is true, in which case a type-error is signalled.

     External format conversion is automatically done when Lisp strings
     are passed to foreign code, or when foreign strings are passed to
     Lisp code.  If the type specifier has an explicit
     ‘external-format’, that external format will be used.  Otherwise a
     default external format that has been determined at SBCL startup
     time based on the current locale settings will be used.  For
     example, when the following alien routine is called, the Lisp
     string given as argument is converted to an ‘ebcdic’ octet
     representation.

          (define-alien-routine test int (str (c-string :external-format :ebcdic-us)))

     Lisp strings of type ‘base-string’ are stored with a trailing NUL
     termination, so no copying (either by the user or the
     implementation) is necessary when passing them to foreign code,
     assuming that the ‘external-format’ and ‘element-type’ of the
     ‘c-string’ type are compatible with the internal representation of
     the string.  For an SBCL built with Unicode support that means an
     ‘external-format’ of ‘:ascii’ and an ‘element-type’ of ‘base-char’.
     Without Unicode support the ‘external-format’ can also be
     ‘:iso-8859-1’, and the ‘element-type’ can also be ‘character’.  If
     the ‘external-format’ or ‘element-type’ is not compatible, or the
     string is a ‘(simple-array character (*))’, this data is copied by
     the implementation as required.

     Assigning a Lisp string to a ‘c-string’ structure field or variable
     stores the contents of the string to the memory already pointed to
     by that variable.  When a foreign object of type ‘(* char)’ is
     assigned to a ‘c-string’, then the ‘c-string’ pointer is assigned
     to.  This allows ‘c-string’ pointers to be initialized.  For
     example:

          (cl:in-package "CL-USER") ; which USEs package "SB-ALIEN"

          (define-alien-type nil (struct foo (str c-string)))

          (defun make-foo (str)
            (let ((my-foo (make-alien (struct foo))))
              (setf (slot my-foo 'str) (make-alien char (length str))
                    (slot my-foo 'str) str)
              my-foo))

     Storing Lisp ‘NIL’ in a ‘c-string’ writes C ‘NULL’ to the variable.

   • ‘sb-alien’ also exports translations of these C type specifiers as
     foreign type specifiers: ‘char’, ‘short’, ‘int’, ‘long’,
     ‘unsigned-char’, ‘unsigned-short’, ‘unsigned-int’, ‘unsigned-long’,
     ‘float’, ‘double’, ‘size-t’, and ‘off-t’.


File: sbcl.info,  Node: Operations On Foreign Values,  Next: Foreign Variables,  Prev: Foreign Types,  Up: Foreign Function Interface

9.3 Operations On Foreign Values
================================

This section describes how to read foreign values as Lisp values, how to
coerce foreign values to different kinds of foreign values, and how to
dynamically allocate and free foreign variables.

* Menu:

* Accessing Foreign Values::
* Coercing Foreign Values::
* Foreign Dynamic Allocation::


File: sbcl.info,  Node: Accessing Foreign Values,  Next: Coercing Foreign Values,  Up: Operations On Foreign Values

9.3.1 Accessing Foreign Values
------------------------------

 -- Function: deref POINTER-OR-ARRAY &rest INDICES

     The ‘sb-alien:deref’ function returns the value pointed to by a
     foreign pointer, or the value of a foreign array element.  When
     dereferencing a pointer, an optional single index can be specified
     to give the equivalent of C pointer arithmetic; this index is
     scaled by the size of the type pointed to.  When dereferencing an
     array, the number of indices must be the same as the number of
     dimensions in the array type.  ‘deref’ can be set with ‘setf’ to
     assign a new value.

 -- Function: slot STRUCT-OR-UNION SLOT-NAME

     The ‘sb-alien:slot’ function extracts the value of the slot named
     SLOT-NAME from a foreign ‘struct’ or ‘union’.  If STRUCT-OR-UNION
     is a pointer to a structure or union, then it is automatically
     dereferenced.  ‘sb-alien:slot’ can be set with ‘setf’ to assign a
     new value.  Note that SLOT-NAME is evaluated, and need not be a
     compile-time constant (but only constant slot accesses are
     efficiently compiled).

9.3.1.1 Untyped memory
......................

As noted at the beginning of the chapter, the System Area Pointer
facilities allow untyped access to foreign memory.  SAPs can be
converted to and from the usual typed foreign values using ‘sap-alien’
and ‘alien-sap’ (described elsewhere), and also to and from integers -
raw machine addresses.  They should thus be used with caution;
corrupting the Lisp heap or other memory with SAPs is trivial.

 -- Function: int-sap MACHINE-ADDRESS

     Creates a SAP pointing at the virtual address MACHINE-ADDRESS.

 -- Function: sap-ref-32 SAP OFFSET

     Access the value of the memory location at OFFSET bytes from SAP.
     This form may also be used with ‘setf’ to alter the memory at that
     location.

 -- Function: sap= SAP1 SAP2

     Compare SAP1 and SAP2 for equality.

   Similarly named functions exist for accessing other sizes of word,
other comparisons, and other conversions.  The reader is invited to use
‘apropos’ and ‘describe’ for more details

     (apropos "sap" :sb-sys)


File: sbcl.info,  Node: Coercing Foreign Values,  Next: Foreign Dynamic Allocation,  Prev: Accessing Foreign Values,  Up: Operations On Foreign Values

9.3.2 Coercing Foreign Values
-----------------------------

 -- Macro: addr ALIEN-EXPR

     The ‘sb-alien:addr’ macro returns a pointer to the location
     specified by ALIEN-EXPR, which must be either a foreign variable, a
     use of ‘sb-alien:deref’, a use of ‘sb-alien:slot’, or a use of
     ‘sb-alien:extern-alien’.

 -- Macro: cast FOREIGN-VALUE NEW-TYPE

     The ‘sb-alien:cast’ macro converts FOREIGN-VALUE to a new foreign
     value with the specified NEW-TYPE.  Both types, old and new, must
     be foreign pointer, array or function types.  Note that the
     resulting Lisp foreign variable object is not ‘eq’ to the argument,
     but it does refer to the same foreign data bits.

 -- Macro: sap-alien SAP TYPE

     The ‘sb-alien:sap-alien’ macro converts SAP (a system area pointer)
     to a foreign value with the specified TYPE.  TYPE is not evaluated.

     The TYPE must be some foreign pointer, array, or record type.

 -- Function: alien-sap FOREIGN-VALUE

     The ‘sb-alien:alien-sap’ function returns the SAP which points to
     ALIEN-VALUE's data.

     The FOREIGN-VALUE must be of some foreign pointer, array, or record
     type.


File: sbcl.info,  Node: Foreign Dynamic Allocation,  Prev: Coercing Foreign Values,  Up: Operations On Foreign Values

9.3.3 Foreign Dynamic Allocation
--------------------------------

Lisp code can call the C standard library functions ‘malloc’ and ‘free’
to dynamically allocate and deallocate foreign variables.  The Lisp code
shares the same allocator with foreign C code, so it's OK for foreign
code to call ‘free’ on the result of Lisp ‘sb-alien:make-alien’, or for
Lisp code to call ‘sb-alien:free-alien’ on foreign objects allocated by
C code.

 -- Macro: make-alien type &optional size
     Allocate an alien of type ‘type’ in foreign heap, and return an
     alien pointer to it.  The allocated memory is not initialized, and
     may contain garbage.  The memory is allocated using malloc(3), so
     it can be passed to foreign functions which use free(3), or
     released using ‘free-alien’.

     For alien stack allocation, see macro ‘with-alien’.

     The ‘type’ argument is not evaluated.  If ‘size’ is supplied, how
     it is interpreted depends on ‘type:’

        • When ‘type’ is a foreign array type, an array of that type is
          allocated, and a pointer to it is returned.  Note that you
          must use ‘deref’ to first access the array through the
          pointer.

          If supplied, ‘size’ is used as the first dimension for the
          array.

        • When ‘type’ is any other foreign type, then an object for that
          type is allocated, and a pointer to it is returned.  So
          (make-alien int) returns a (‘*’ int).

          If ‘size’ is specified, then a block of that many objects is
          allocated, with the result pointing to the first one.

     Examples:

            (defvar *foo* (make-alien (array char 10)))
            (type-of *foo*)                   ; => (alien (* (array (signed 8) 10)))
            (setf (deref (deref *foo*) 0) 10) ; => 10

            (make-alien char 12)              ; => (alien (* (signed 8)))
 -- Function: make-alien-string string &rest rest &key start end
          external-format null-terminate
     Copy part of ‘string’ delimited by ‘start’ and ‘end’ into freshly
     allocated foreign memory, freeable using free(3) or ‘free-alien’.
     Returns the allocated string as a (‘*’ ‘char’) alien, and the
     number of bytes allocated as secondary value.

     The string is encoded using ‘external-format’.  If ‘null-terminate’
     is true (the default), the alien string is terminated by an
     additional null byte.
 -- Function: free-alien alien
     Dispose of the storage pointed to by ‘alien’.  The ‘alien’ must
     have been allocated by ‘make-alien’, ‘make-alien-string’ or
     malloc(3).


File: sbcl.info,  Node: Foreign Variables,  Next: Foreign Data Structure Examples,  Prev: Operations On Foreign Values,  Up: Foreign Function Interface

9.4 Foreign Variables
=====================

Both local (stack allocated) and external (C global) foreign variables
are supported.

* Menu:

* Local Foreign Variables::
* External Foreign Variables::


File: sbcl.info,  Node: Local Foreign Variables,  Next: External Foreign Variables,  Up: Foreign Variables

9.4.1 Local Foreign Variables
-----------------------------

 -- Macro: with-alien VAR-DEFINITIONS &body BODY

     The ‘with-alien’ macro establishes local foreign variables with the
     specified alien types and names.  This form is analogous to
     defining a local variable in C: additional storage is allocated,
     and the initial value is copied.  This form is less analogous to
     ‘LET’-allocated Lisp variables, since the variables can't be
     captured in closures: they live only for the dynamic extent of the
     body, and referring to them outside is a gruesome error.

     The VAR-DEFINITIONS argument is a list of variable definitions,
     each of the form
          (NAME TYPE &optional INITIAL-VALUE)

     The names of the variables are established as symbol-macros; the
     bindings have lexical scope, and may be assigned with ‘setq’ or
     ‘setf’.

     The ‘with-alien’ macro also establishes a new scope for named
     structures and unions.  Any TYPE specified for a variable may
     contain named structure or union types with the slots specified.
     Within the lexical scope of the binding specifiers and body, a
     locally defined foreign structure type FOO can be referenced by its
     name using ‘(struct FOO)’.

     When a foreign function returns a structure by value, using
     ‘alien-funcall’ as the INITIAL-VALUE allows the returned struct to
     be stack-allocated directly into the local variable's storage,
     avoiding heap allocation:

          (with-alien ((result (struct point)
                               (alien-funcall
                                (extern-alien "make_point"
                                              (function (struct point) double double))
                                1.0d0 2.0d0)))
            (values (slot result 'x) (slot result 'y)))


File: sbcl.info,  Node: External Foreign Variables,  Prev: Local Foreign Variables,  Up: Foreign Variables

9.4.2 External Foreign Variables
--------------------------------

External foreign names are strings, and Lisp names are symbols.  When an
external foreign value is represented using a Lisp variable, there must
be a way to convert from one name syntax into the other.  The macros
‘extern-alien’, ‘define-alien-variable’ and ‘define-alien-routine’ use
this conversion heuristic:

   • Alien names are converted to Lisp names by uppercasing and
     replacing underscores with hyphens.

   • Conversely, Lisp names are converted to alien names by lowercasing
     and replacing hyphens with underscores.

   • Both the Lisp symbol and alien string names may be separately
     specified by using a list of the form

          (alien-string lisp-symbol)

 -- Macro: define-alien-variable NAME TYPE

     The ‘define-alien-variable’ macro defines NAME as an external
     foreign variable of the specified foreign ‘type’.  NAME and ‘type’
     are not evaluated.  The Lisp name of the variable (see above)
     becomes a global alien variable.  Global alien variables are
     effectively "global symbol macros"; a reference to the variable
     fetches the contents of the external variable.  Similarly, setting
     the variable stores new contents - the new contents must be of the
     declared ‘type’.  Someday, they may well be implemented using the
     ANSI ‘define-symbol-macro’ mechanism, but as of SBCL 0.7.5, they
     are still implemented using an older more-or-less parallel
     mechanism inherited from CMUCL.

     For example, to access a C-level counter FOO, one could write

          (define-alien-variable "foo" int)
          ;; Now it is possible to get the value of the C variable foo simply by
          ;; referencing that Lisp variable:
          (print foo)
          (setf foo 14)
          (incf foo)

 -- Function: get-errno

     Since in modern C libraries, the ‘errno’ "variable" is typically no
     longer a variable, but some bizarre artificial construct which
     behaves superficially like a variable within a given thread, it can
     no longer reliably be accessed through the ordinary
     ‘define-alien-variable’ mechanism.  Instead, SBCL provides the
     operator ‘sb-alien:get-errno’ to allow Lisp code to read it.

 -- Macro: extern-alien NAME TYPE

     The ‘extern-alien’ macro returns an alien with the specified TYPE
     which points to an externally defined value.  NAME is not
     evaluated, and may be either a string or a symbol.  TYPE is an
     unevaluated alien type specifier.


File: sbcl.info,  Node: Foreign Data Structure Examples,  Next: Loading Shared Object Files,  Prev: Foreign Variables,  Up: Foreign Function Interface

9.5 Foreign Data Structure Examples
===================================

Now that we have alien types, operations and variables, we can
manipulate foreign data structures.  This C declaration

     struct foo {
         int a;
         struct foo *b[100];
     };

   can be translated into the following alien type:

     (define-alien-type nil
       (struct foo
         (a int)
         (b (array (* (struct foo)) 100))))

   Once the ‘foo’ alien type has been defined as above, the C expression

     struct foo f;
     f.b[7].a;

   can be translated in this way:

     (with-alien ((f (struct foo)))
       (slot (deref (slot f 'b) 7) 'a)
       ;;
       ;; Do something with f...
       )

   Or consider this example of an external C variable and some accesses:

     struct c_struct {
             short x, y;
             char a, b;
             int z;
             c_struct *n;
     };
     extern struct c_struct *my_struct;
     my_struct->x++;
     my_struct->a = 5;
     my_struct = my_struct->n;

   which can be manipulated in Lisp like this:

     (define-alien-type nil
       (struct c-struct
               (x short)
               (y short)
               (a char)
               (b char)
               (z int)
               (n (* c-struct))))
     (define-alien-variable "my_struct" (* c-struct))
     (incf (slot my-struct 'x))
     (setf (slot my-struct 'a) 5)
     (setq my-struct (slot my-struct 'n))


File: sbcl.info,  Node: Loading Shared Object Files,  Next: Foreign Function Calls,  Prev: Foreign Data Structure Examples,  Up: Foreign Function Interface

9.6 Loading Shared Object Files
===============================

Foreign object files can be loaded into the running Lisp process by
calling ‘load-shared-object’.

 -- Function: load-shared-object pathname &key dont-save
     Load a shared library / dynamic shared object file / similar
     foreign container specified by designated ‘pathname’, such as a .so
     on an ‘elf’ platform.

     Locating the shared object follows standard rules of the platform,
     consult the manual page for dlopen(3) for details.  Typically paths
     specified by environment variables such as LD_LIBRARY_PATH are
     searched if the ‘pathname’ has no directory, but on some systems
     (eg.  Mac ‘os’ ‘x’) search may happen even if ‘pathname’ is
     absolute.  (On Windows LoadLibrary is used instead of dlopen(3).)

     On non-Windows platforms calling ‘load-shared-object’ again with a
     ‘pathname’ ‘equal’ to the designated pathname of a previous call
     will replace the old definitions; if a symbol was previously
     referenced through the object and is not present in the reloaded
     version an error will be signalled.  Reloading may not work as
     expected if user or library-code has called dlopen(3) on the same
     shared object or running on a system where dlclose(3) is a noop.

     ‘load-shared-object’ interacts with ‘sb-ext:save-lisp-and-die:’

     1.  If ‘dont-save’ is true (default is ‘nil’), the shared object
     will be dropped when ‘save-lisp-and-die’ is called ‘--’ otherwise
     shared objects are reloaded automatically when a saved core starts
     up.  Specifying ‘dont-save’ can be useful when the location of the
     shared object on startup is uncertain.

     2.  On most platforms references in compiled code to foreign
     symbols in shared objects (such as those generated by
     ‘define-alien-routine’) remain valid across ‘save-lisp-and-die’.
     On those platforms where this is not supported, a ‘warning’ will be
     signalled when the core is saved ‘--’ this is orthogonal from
     ‘dont-save’.

 -- Function: unload-shared-object pathname
     Unloads the shared object loaded earlier using the designated
     ‘pathname’ with ‘load-shared-object’, to the degree supported on
     the platform.

     Experimental.

