


CRASH(8)                                                 CRASH(8)


NAME
       crash - UNIX system failures

DESCRIPTION
       This section explains what happens when the system crashes
       and (very briefly) how to analyze crash dumps.

       When the system crashes voluntarily it prints a message of
       the form

              panic: why i gave up the ghost

       on the console, takes a dump on a mass storage peripheral,
       and  then  invokes  an  automatic  reboot   procedure   as
       described  in  reboot(8).   (If auto-reboot is disabled on
       the front panel of the machine the system will simply halt
       at  this  point.)  Unless some unexpected inconsistency is
       encountered in the state of the file systems due to  hard-
       ware  or  software  failure,  the  system will then resume
       multi-user operations.

       The system has a  large  number  of  internal  consistency
       checks;  if  one of these fails, then it will panic with a
       very short message indicating which one failed.   In  many
       instances,  this  will  be  the  name of the routine which
       detected the error,  or  a  two-word  description  of  the
       inconsistency.   A  full  understanding of most panic mes-
       sages requires perusal of the source code for the  system.

       The most common cause of system failures is hardware fail-
       ure, which can reflect itself in different ways.  Here are
       the  messages which are most likely, with some hints as to
       causes.  Left unstated in all  cases  is  the  possibility
       that  hardware  or  software error produced the message in
       some unexpected way.

       iinit  This cryptic panic message results from  a  failure
              to  mount  the root filesystem during the bootstrap
              process.  Either the root filesystem has been  cor-
              rupted,  or  the  system  is  attempting to use the
              wrong  device  as  root  filesystem.   Usually,  an
              alternate copy of the system binary or an alternate
              root filesystem can be used to bring up the  system
              to investigate.

       Can't exec /sbin/init
              This  is not a panic message, as reboots are likely
              to be futile.  Late in the bootstrap procedure, the
              system  was  unable  to locate and execute the ini-
              tialization process, init(8).  The root  filesystem
              is  incorrect or has been corrupted, or the mode or
              type of /sbin/init forbids execution.





                          April 20, 1991                        1





CRASH(8)                                                 CRASH(8)


       IO err in push
       hard IO err in swap
              The system encountered an error trying to write  to
              the  paging  device or an error in reading critical
              information from a disk drive.  The offending  disk
              should be fixed if it is broken or unreliable.

       realloccg: bad optim
       ialloc: dup alloc
       alloccgblk: cyl groups corrupted
       ialloccg: map corrupted
       free: freeing free block
       free: freeing free frag
       ifree: freeing free inode
       alloccg: map corrupted
              These  panic  messages  are among those that may be
              produced  when   filesystem   inconsistencies   are
              detected.   The  problem  generally  results from a
              failure  to  repair  damaged  filesystems  after  a
              crash,  hardware  failures, or other condition that
              should not normally occur.  A filesystem check will
              normally correct the problem.

       timeout table overflow
              This  really  shouldn't  be  a panic, but until the
              data structure involved is made to  be  extensible,
              running  out  of  entries  causes a crash.  If this
              happens, make the timeout table bigger.

       KSP not valid
       SBI fault
       CHM? in kernel
              These indicate either a serious bug in  the  system
              or,  more  often, a glitch or failing hardware.  If
              SBI faults recur, check out the  hardware  or  call
              field service.  If the other faults recur, there is
              likely a bug  somewhere  in  the  system,  although
              these  can  be  caused  by a flakey processor.  Run
              processor microdiagnostics.

       machine check %x:
              description

          machine dependent machine-check information
              Machine checks are different on each type  of  CPU.
              Most  of the internal processor registers are saved
              at the time of the fault and  are  printed  on  the
              console.   For  most  processors, there is one line
              that summarizes the type of machine check.   Often,
              the  nature  of  the  problem is apparent from this
              message and/or the contents of key registers.   The
              VAX  Hardware Handbook should be consulted, and, if
              necessary,  your  friendly  field  service   people
              should be informed of the problem.



                          April 20, 1991                        2





CRASH(8)                                                 CRASH(8)


       trap type %d, code=%x, pc=%x
              A  unexpected  trap has occurred within the system;
              the trap types are:

              0    reserved addressing fault
              1    privileged instruction fault
              2    reserved operand fault
              3    bpt instruction fault
              4    xfc instruction fault
              5    system call trap
              6    arithmetic trap
              7    ast delivery trap
              8    segmentation fault
              9    protection fault
              10   trace trap
              11   compatibility mode fault
              12   page fault
              13   page table fault

              The favorite trap types in system crashes are  trap
              types  8  and  9, indicating a wild reference.  The
              code is the referenced address, and the pc  at  the
              time  of the fault is printed.  These problems tend
              to be easy to track down if they  are  kernel  bugs
              since  the  processor stops cold, but random flaki-
              ness seems to cause this sometimes.   The  debugger
              can  be  used to locate the instruction and subrou-
              tine corresponding to the PC  value.   If  that  is
              insufficient  to suggest the nature of the problem,
              more detailed examination of the system  status  at
              the  time of the trap usually can produce an expla-
              nation.

       init died
              The system initialization process has exited.  This
              is  bad  news, as no new users will then be able to
              log in.  Rebooting is the only fix, so  the  system
              just does it right away.

       out of mbufs: map full
              The  network has exhausted its private page map for
              network  buffers.   This  usually  indicates   that
              buffers  are  being lost, and rather than allow the
              system to slowly degrade, it  reboots  immediately.
              The map may be made larger if necessary.

       That  completes  the list of panic types you are likely to
       see.

       When the system crashes it writes (or at least attempts to
       write)  an  image  of memory into the back end of the dump
       device, usually the same as the primary swap area.   After
       the  system  is rebooted, the program savecore(8) runs and
       preserves a copy of this core image and the current system



                          April 20, 1991                        3





CRASH(8)                                                 CRASH(8)


       in   a   specified   directory  for  later  perusal.   See
       savecore(8) for details.

       To analyze a dump you should begin by running adb(1)  with
       the  -k  flag  on the system load image and core dump.  If
       the core image is the result of a panic, the panic message
       is  printed.   Normally  the command ``$c'' will provide a
       stack trace from the point of the crash and this will pro-
       vide  a  clue  as to what went wrong.  For more detail see
       ``Using ADB to Debug the UNIX Kernel''.

SEE ALSO
       adb(1), reboot(8)
       VAX 11/780 System Maintenance Guide and VAX Hardware Hand-
       book for more information about machine checks.
       Using ADB to Debug the UNIX Kernel









































                          April 20, 1991                        4


