
                            APT Method Interface


-----------------------------------------------------------------------------
                                      
                               Jason Gunthorpe

                              <jgg@debian.org>

                                Version 3.3.0

-----------------------------------------------------------------------------

Copyright © 1998 Jason Gunthorpe

License Notice

    "APT" and this document are free software; you can redistribute them and/
    or modify them under the terms of the GNU General Public License as
    published by the Free Software Foundation; either version 2 of the
    License, or (at your option) any later version.

    For more details, on Debian systems, see the file /usr/share/
    common-licenses/GPL for the full license.

Abstract

This document describes the interface that APT uses to the archive access
methods.

-----------------------------------------------------------------------------

Table of Contents

1. Introduction
    1.1. General
    1.2. Terms
2. Specification
    2.1. Overview
    2.2. Message Overview
    2.3. Header Fields
    2.4. Notes

Chapter 1. Introduction

Table of Contents

1.1. General
1.2. Terms

1.1. General

    The APT method interface allows APT to acquire archive files (.deb),
    index files (Packages, Release, Mirrors) and source files (.tar.gz,
    .diff). It is a general, extensible system designed to satisfy all of
    these requirements:

     1. Remote methods that download files from a distant site

     2. Resume of aborted downloads

     3. Progress reporting

     4. If-Modified-Since (IMS) checking for index files

     5. In-Line MD5 generation
   
     6. No-copy in-filesystem methods

     7. Multi-media methods (like CD's)

     8. Dynamic source selection for failure recovery

     9. User interaction for user/password requests and media swaps

    10. Global configuration

    Initial releases of APT (0.1.x) used a completely different method
    interface that only supported the first 6 items. This new interface deals
    with the remainder.

1.2. Terms

    Several terms are used through out the document, they have specific
    meanings which may not be immediately evident. To clarify they are
    summarized here.

    source

        Refers to an item in source list. More specifically it is the broken
        down item, that is each source maps to exactly one index file.
        Archive sources map to Package files and Source Code sources map to
        Source files.

    archive file

        Refers to a binary package archive (.deb, .rpm, etc).

    source file

        Refers to one of the files making up the source code of a package. In
        debian it is one of .diff.gz, .dsc. or .tar.gz.

    URI

        Universal Resource Identifier (URI) is a super-set of the familiar
        URL syntax used by web browsers. It consists of an access
        specification followed by a specific location in that access space.
        The form is <access>:<location>. Network addresses are given with the
        form <access>://[<user>[:<pas>]@]hostname[:port]/<location>. Some
        examples:

        file:/var/mirrors/debian/
        ftp://ftp.example.org/debian
        https://deb.debian.org/debian
        ftp://jgg:MooCow@localhost:21/debian
        nfs://bigred/var/mirrors/debian
        rsync://debian.midco.net/debian
        cdrom:Debian 2.0r1 Disk 1/

    method

        There is a one to one mapping of URI access specifiers to methods. A
        method is a program that knows how to handle a URI access type and
        operates according to the specifications in this file.

    method instance

        A specific running method. There can be more than one instance of
        each method as APT is capable of concurrent method handling.

    message

        A series of lines terminated by a blank line sent down one of the
        communication lines. The first line should have the form xxx TAG
        where xxx are digits forming the status code and TAG is an
        informational string

    acquire

        The act of bring a URI into the local pathname space. This may simply
        be verifying the existence of the URI or actually downloading it from
        a remote site.

Chapter 2. Specification

Table of Contents

2.1. Overview
2.2. Message Overview
2.3. Header Fields
2.4. Notes

2.1. Overview

    All methods operate as a sub process of a main controlling parent. 3 FD's
    are opened for use by the method allowing two way communication and
    emergency error reporting. The FD's correspond to the well known unix
    FD's, stdin, stdout and stderr.

    Through operation of the method communication is done via http style
    plain text. Specifically RFC-822 (like the Package file) fields are used
    to describe items and a numeric-like header is used to indicate what is
    happening. Each of these distinct communication messages should be sent
    quickly and without pause.

    In some instances APT may pre-invoke a method to allow things like file
    URI's to determine how many files are available locally.

2.2. Message Overview

    The first line of each message is called the message header. The first 3
    digits (called the Status Code) have the usual meaning found in the http
    protocol. 1xx is informational, 2xx is successful and 4xx is failure. The
    6xx series is used to specify things sent to the method. After the status
    code is an informational string provided for visual debugging.

      * 100 Capabilities - Method capabilities

      * 101 Log - General Logging

      * 102 Status - Inter-URI status reporting (login progress)

      * 200 URI Start - URI is starting acquire

      * 201 URI Done - URI is finished acquire

      * 351 Aux Request - Method requests an auxiliary file

      * 400 URI Failure - URI has failed to acquire
   
      * 401 General Failure - Method did not like something sent to it

      * 402 Authorization Required - Method requires authorization to access
        the URI. Authorization is User/Pass

      * 403 Media Failure - Method requires a media change

      * 600 URI Acquire - Request a URI be acquired

      * 601 Configuration - Sends the configuration space

      * 602 Authorization Credentials - Response to the 402 message

      * 603 Media Changed - Response to the 403 message

    Only the 6xx series of status codes is sent TO the method. Furthermore
    the method may not emit status codes in the 6xx range. The Codes 351, 402
    and 403 require that the method continue reading all other 6xx codes
    until the proper 600/602/603 code is received. This means the method must
    be capable of handling an unlimited number of 600 messages.

    The flow of messages starts with the method sending out a 100
    Capabilities and APT sending out a 601 Configuration. After that APT
    begins sending 600 URI Acquire and the method sends out 200 URI Start, 
    201 URI Done or 400 URI Failure. No synchronization is performed, it is
    expected that APT will send 600 URI Acquire messages at -any- time and
    that the method should queue the messages. This allows methods like http
    to pipeline requests to the remote server. It should be noted however
    that APT will buffer messages so it is not necessary for the method to be
    constantly ready to receive them.

2.3. Header Fields

    The following is a short index of the header fields that are supported

    URI

        URI being described by the message

    Filename

        Location in the filesystem

    Last-Modified

        A time stamp in RFC1123 notation for use by IMS checks

    IMS-Hit

        The already existing item is valid

    Size

        Size of the file in bytes

    Resume-Point

        Location that transfer was started

    MD5-Hash

        Computed MD5 hash for the file

    Message

        String indicating some displayable message

    Media

        String indicating the media name required

    Site

        String indicating the site authorization is required for

    User

        Username for authorization

    Password

        Password for authorization

    Fail

        Operation failed

    Drive

        Drive the media should be placed in

    Config-Item

        A string of the form item=value derived from the APT configuration
        space. These may include method specific values and general values
        not related to the method. It is up to the method to filter out the
        ones it wants.

    Single-Instance

        Requires that only one instance of the method be run This is a yes/no
        value.

    Pipeline

        The method is capable of pipelining.

    Local

        The method only returns Filename: fields.

    Send-Config

        Send configuration to the method.

    Needs-Cleanup

        The process is kept around while the files it returned are being
        used. This is primarily intended for CD-ROM and File URIs that need
        to unmount filesystems.

    Version

        Version string for the method

    This is a list of which headers each status code can use

    100 Capabilities

        Displays the capabilities of the method. Methods should set the
        pipeline bit if their underlying protocol supports pipelining. The
        only known method that does support pipelining is http. Fields:
        Version, Single-Instance, Local-Only, Pipeline, Send-Config,
        Needs-Cleanup, Removable, AuxRequests, Send-URI-Encoded

    101 Log

        A log message may be printed to the screen if debugging is enabled.
        This is only for debugging the method. Fields: Message

    102 Status

        Message gives a progress indication for the method. It can be used to
        show pre-transfer status for Internet type methods. Fields: Message

    200 URI Start

        Indicates the URI is starting to be transferred. The URI is specified
        along with stats about the file itself. Fields: URI, Size,
        Last-Modified, Resume-Point

    201 URI Done

        Indicates that a URI has completed being transferred. It is possible
        to specify a 201 URI Done without a URI Start which would mean no
        data was transferred but the file is now available. A Filename field
        is specified when the URI is directly available in the local pathname
        space. APT will either directly use that file or copy it into another
        location. It is possible to return Alt-* fields to indicate that
        another possibility for the URI has been found in the local pathname
        space. This is done if a decompressed version of a .gz file is found.
        Fields: URI, Size, Last-Modified, Filename, MD5-Hash

    351 Aux Request

        Indicates a request for an auxiliary file to be downloaded by the
        acquire system (via another method) and made available for the
        requesting method. The requestor will get a 600 URI Acquire with the
        URI it requested and the filename will either be an existing file if
        the request was success or if the acquire failed for some reason the
        file will not exist. Fields: URI (of the file causing the need for
        the auxiliary file), MaximumSize, Aux-ShortDesc, Aux-Description,
        Aux-URI

    400 URI Failure
   
        Indicates a fatal URI failure. The URI is not retrievable from this
        source. As with 201 URI Done 200 URI Start is not required to precede
        this message Fields: URI, Message

    401 General Failure

        Indicates that some unspecific failure has occurred and the method is
        unable to continue. The method should terminate after sending this
        message. It is intended to check for invalid configuration options or
        other severe conditions. Fields: Message

    402 Authorization Required

        The method requires a Username and Password pair to continue. After
        sending this message the method will expect APT to send a 602
        Authorization Credentials message with the required information. It
        is possible for a method to send this multiple times. Fields: Site

    403 Media Failure

        A method that deals with multiple media requires that a new media be
        inserted. The Media field contains the name of the media to be
        inserted. Fields: Media, Drive

    600 URI Acquire

        APT is requesting that a new URI be added to the acquire list.
        Last-Modified has the time stamp of the currently cache file if
        applicable. Filename is the name of the file that the acquired URI
        should be written to. Fields: URI, Filename Last-Modified

    601 Configuration

        APT is sending the configuration space to the method. A series of
        Config-Item fields will be part of this message, each containing an
        entry from the configuration space. Fields: Config-Item.

    602 Authorization Credentials

        This is sent in response to a 402 Authorization Required message. It
        contains the entered username and password. Fields: Site, User,
        Password

    603 Media Changed

        This is sent in response to a 403 Media Failure message. It indicates
        that the user has changed media and it is safe to proceed. Fields:
        Media, Fail

2.4. Notes

    The methods supplied by the stock apt are:

     1. cdrom - For Multi-Disc CD-ROMs

     2. copy - (internal) For copying files around the filesystem

     3. file - For local files

     4. gzip - (internal) For decompression

     5. http - For HTTP servers

    The two internal methods, copy and gzip, are used by the acquire code to
    parallelize and simplify the automatic decompression of package files as
    well as copying package files around the file system. Both methods can be
    seen to act the same except that one decompresses on the fly. APT uses
    them by generating a copy URI that is formed identically to a file URI.
    The destination file is send as normal. The method then takes the file
    specified by the URI and writes it to the destination file. A typical set
    of operations may be:

    http://foo.com/Packages.gz -> /bar/Packages.gz
    gzip:/bar/Packages.gz -> /bar/Packages.decomp
    rename Packages.decomp to /final/Packages

    The http method implements a fully featured HTTP/1.1 client that supports
    deep pipelining and reget. It works best when coupled with an apache 1.3
    server. The file method simply generates failures or success responses
    with the filename field set to the proper location. The cdrom method acts
    the same except that it checks that the mount point has a valid cdrom in
    it. It does this by (effectively) computing a md5 hash of 'ls -l' on the
    mountpoint.

