# The generic API to SP

SP provides a generic API in addition to its native API. The generic interface is much simpler than the native interface. It is generic in the sense that it could be easily implemented using parsers other than SP. It provides all ESIS information as well as some other information about the instance that is commonly needed by applications. However, it doesn't provide access to all information available from SP; in particular, it doesn't provide information about the DTD. It is also slightly less efficient than the native interface.

The interface uses two related abstract classes. An `SGMLApplication` is an object that can handle a number of different kinds of event which correspond to information in an SGML document. An `EventGenerator` is an object that can generate a sequence of events of the kinds handled by an `SGMLApplication`. The `ParserEventGeneratorKit` class makes an `EventGenerator` that generates events using SP.

## Types

`SGMLApplication` has a number of local types that are used in several contexts:

`Char`
  This typedef is an unsigned integral type that represents a single bit combination (character). It is `unsigned short` if `SP_MULTI_BYTE` is defined and `unsigned char` otherwise.
`CharString`
  This struct represents a string of `Char`. It has the following members:
  `const Char *ptr`
    A pointer to the `Char`s of the string.
  `size_t len`
    The number of `Char`s in the string.
`Location`
  This struct holds information about a location in the entity structure of a document. It is constucted using an `OpenEntityPtr` and a `Position`. The `CharString`s in it will remain valid as long as the `OpenEntity` that is pointed to by the `OpenEntityPtr` that was used to construct it remains valid.

  It has the following members:

  `unsigned long lineNumber`
    The line number. `(unsigned long)-1` if invalid.
  `unsigned long columnNumber`
    The column number. Note that tabs are not treated specially. `(unsigned long)-1` if invalid.
  `unsigned long byteOffset`
    The number of bytes in the storage object preceding the location. `(unsigned long)-1` if invalid.
  `unsigned long entityOffset`
    The number of bit combinations in the entity preceding the location. `(unsigned long)-1` if invalid.
  `CharString entityName`
    The name of the external entity containing the location. An empty string if invalid.
  `CharString filename`
    The name of the file containing the location. An empty string if invalid.
  `const void *other`
    Other implementation-dependent information about the location. In the SP implementation it will be a pointer to a StorageObjectSpec. 0 if invalid.

  When a location is in an internal entity, the location of the reference to the entity will be used instead.

`OpenEntity`
  This class represents a currently open entity. The only use for an `OpenEntity` is, in conjunction with a `Position`, to create a `Location`. An `OpenEntity` is accessed using an `OpenEntityPtr`.
`OpenEntityPtr`
  This class is a reference-counted pointer to an `OpenEntity`.
`Position`
  This is an integral type that represents a position in an open entity. The meaning of a `Position` is completely determined by the `OpenEntity` object with which it is associated. The only use for an `Position` is, in conjunction with an `OpenEntity`, to create a `Location`.
`ExternalId`
  This struct represents an external identifier. It has the following members:
  `bool haveSystemId`
    True iff the external identifier included an explicit system identifier.
  `CharString systemId`
    The system identifier included in the external identifier. Valid only if `havePublicId` is true.
  `bool havePublicId`
    True iff the external identifier included an explicit public identifier.
  `CharString publicId`
    The public identifier included in the external identifier. Valid only if `havePublicId` is true.
  `bool haveGeneratedSystemId`
    True iff a system identifier was generated for the external identifier.
  `CharString generatedSystemId`
    The system identifier generated for the external identifier. Valid only if `haveGeneratedSystemId` is true.
`Notation`
  This struct represents a notation. It has the following members:
  `CharString name`
    The name of the notation.
  `ExternalId externalId`
    The external identifier of the notation.
`Entity`
  This struct represents an entity. It has the following members.
  `CharString name`
    The name of the entity.
  `Entity::DataType dataType`
    The type of the data of the entity.

    `Entity::DataType` is a local enum with the following possible values:

    `Entity::sgml`
    `Entity::cdata`
    `Entity::sdata`
    `Entity::ndata`
    `Entity::subdoc`
    `Entity::pi`
  `Entity::DeclType declType`
    The type of the declaration of the entity.

    `Entity::DeclType` is a local enum with the following possible values:

    `Entity::general`
      The entity is a general entity.
    `Entity::parameter`
      The entity is a parameter entity.
    `Entity::doctype`
      The entity was declared in a doctype declaration.
    `Entity::linktype`
      The entity was declared in a linktype declaration.
  `bool isInternal`
    True iff the entity is internal rather than external.
  `CharString text`
    The replacement text of the entity. Valid only if `isInternal` is true.
  `ExternalId externalId`
    The external identifier of the entity. Valid only if `isInternal` is false.
  `const Attribute *attributes`
    Pointer to the data attributes of the entity. Valid only if `isInternal` is false.
  `size_t nAttributes`
    The number of data attributes of the entity. Valid only if `isInternal` is false.
  `Notation notation`
    The entity's notation. An empty string if the entity has no notation. Valid only if `isInternal` is false.
`Attribute`
  This struct represents an attribute. More precisely it represents the assignment of an attribute value to an attribute name. It has the following members:
  `CharString name`
    The attribute name.
  `Attribute::Type type`
    An enumeration describing the type of the attribute.

    `Attribute::Type` is a local type with the following possible values:

    `Attribute::invalid`
      The attribute is invalid.
    `Attribute::implied`
      The attribute is an impliable attribute for which no value was specified.
    `Attribute::cdata`
      The attribute is a CDATA attribute.
    `Attribute::tokenized`
      The attribute is a tokenized attribute.
  `Attribute::Defaulted defaulted`
    An enumeration specifying whether the entity was defaulted, and, if so, how. This is non-ESIS information.

    `Attribute::Defaulted` is a local enum with the following possible values:

    `Attribute::specified`
      The value was explicitly specified.
    `Attribute::definition`
      The value was defaulted from the attribute definition.
    `Attribute::current`
      The value was defaulted using the CURRENT value of the attribute.
  `size_t nCdataChunks`
    The number of `Attribute::CdataChunk`s comprising the value of the attribute. Valid only if `type` is `cdata`.
  `const Attribute::CdataChunk *cdataChunks`
    The `Attribute::CdataChunk`s comprising the value of this attribute. Valid only if `type` is `cdata`.

    `Attribute::CdataChunk` is a local struct with the following members:

    `bool isSdata`
      True iff this chunk is the replacement text of an internal SDATA entity.
    `CharString data`
      The data of this chunk.
    `CharString entityName`
      The name of the internal SDATA entity that this chunk is the replacement text of. Valid only if `isSdata` is true. This is non-ESIS information.
  `CharString tokens`
    Valid only if `type` is `Attribute::tokenized`.
  `bool isId`
    True iff the declared value is ID. This is non-ESIS information.
  `size_t nEntities`
    The number of entities associated with this attribute. This will be zero unless the declared value is ENTITY or ENTITIES.
  `const Entity *entities`
    The entities associated with this attribute.
  `Notation notation`
    The notation associated with this attribute. If the declared value of the attribute is not NOTATION, then the name member will be an empty string.

## Events

For each event `xyzEvent` handled by `SGMLApplication`, there is a virtual function of `SGMLApplication` named `xyz` to handle the event, and a local struct of `SGMLApplication` named `XyzEvent`.

Pointers within an event `xyzEvent` are valid only during the call to `xyz`. None of the structs in events have copy constructors or assignment operators defined. It is up to the event handling function to make a copy of any data that it needs to preserve after the function returns.

Except as otherwise stated, the information in events is ESIS information. All position information is non-ESIS information.

There are the following types of event:

`AppinfoEvent`
  Generated for the APPINFO section of the SGML declaration. The event has the following members:
  `Position pos`
    The position of APPINFO parameter of the SGML declaration.
  `bool none`
    True iff APPINFO NONE was specified.
  `CharString string`
    The interpreted value of the minimum literal specified in the appinfo parameter of the SGML declaration. Valid only if `none` is false.
`PiEvent`
  Generated for a processing instruction. The event has the following members:
  `Position pos`
    The position of the processing instruction.
  `CharString data`
    The system data of the processing instuction.
  `CharString entityName`
    If the processing instruction was the result of the reference to a PI entity, the name of the entity. If not, an empty string. This is non-ESIS information.
`StartElementEvent`
  Generated for the start of an element. The event has the following members:
  `Position pos`
    The position of the start of the element.
  `CharString gi`
    The generic identifier of the element.
  `Element::ContentType contentType`
    The type of the element's content. This is non-ESIS information.

    `Element::ContentType` is an enum with the following possible values:

    `Element::empty`
      The element has empty content, either because it was declared as EMPTY or because there was a #CONREF attribute.
    `Element::cdata`
      The element has CDATA content.
    `Element::rcdata`
      The element has RCDATA content.
    `Element::mixed`
      The element has mixed content.
    `Element::element`
      The element has element content.
  `bool included`
    True iff the element was an included subelement (rather than a proper subelement). This is non-ESIS information.
  `size_t nAttributes`
    The number of attributes of this element.
  `const Attribute *attributes`
    A pointer to the attributes for this element.
`EndElementEvent`
  Generated for the end of an elemenet. The event has the following members:
  `Position pos`
    The position of the end of the element.
  `CharString gi`
    The generic identifier of the element.
`DataEvent`
  Generated for character data. Separate data events may be generated for consecutive data characters. Applications should make no assumptions about how character data is split into `DataEvent`s. The event has the following members:
  `Position pos`
    The position of the first character of the data.
  `CharString data`
    The data.
`SdataEvent`
  Generated for a reference to an internal sdata entity in content. The event has the following members:
  `Position pos`
    The position of the entity reference.
  `CharString text`
    The replacement text of the entity.
  `CharString entityName`
    The entity name. This is non-ESIS information.
`ExternalDataEntityRefEvent`
  Generated for a reference to an external data entity. The event has the following members:
  `Position pos`
    The position of the entity reference.
  `Entity entity`
    The referenced entity.
`SubdocEntityRefEvent`
  Generated for a reference to a subdoc entity. The event has the following members:
  `Position pos`
    The position of the entity reference.
  `Entity entity`
    The referenced entity.
`StartDtdEvent`
  Generated at the start of a document type declaration. This is non-ESIS information. The event has the following members:
  `Position pos`
    The position of the start of the document type declaration.
  `CharString name`
    The document type name.
  `bool haveExternalId`
    The external identifier for the entity declared in the document type declaration.
  `ExternalId externalId`
    Valid iff haveExternalId is true.
`EndDtdEvent`
  Generated at the end of a document type declaration. This is non-ESIS information. The event has the following members:
  `Position pos`
    The position of the end of the DTD.
  `CharString name`
`EndPrologEvent`
  Generated at the end of the prolog. The event has the following members:
  `Position pos`
    The position of the start of the instance.
`GeneralEntityEvent`
  Generated for each general entity in the name space of the governing doctype, but only if the `ParserEventGeneratorKit::outputGeneralEntities` option is enabled. This is non-ESIS information. The event has the following members:
  `Entity entity`
    The entity.

  No event will be generated for the declaration of the `#default` entity; instead an event will be generated when an entity reference uses the `#default` entity if that is the first time on which an entity with that name is used. This means that `GeneralEntityEvent` can occur after the end of the prolog.

`CommentDeclEvent`
  Generated for each comment declaration in the instance, but only if `ParserEventGeneratorKit::outputCommentDecls` option is enabled. This is non-ESIS information. The event has the following members:
  `Position pos`
    The position of the start of the comment declaration.
  `size_t nComments`
    The number of comments in the comment declaration.
  `const CharString *comments`
    The content of each comment in the declaration. This excludes the com delimiters.
  `const CharString *seps`
    The separator following each comment in the declaration.
`MarkedSectionStartEvent`
  Generated for the start of a marked section in the instance, but only if the `ParserEventGeneratorKit::outputMarkedSections` option is enabled. This is non-ESIS information. The event has the following members:
  `Position pos`
    The position of the start of the marked section declaration.
  `MarkedSectionStartEvent::Status status`
    The effective status of the marked section.

    `MarkedSectionStartEvent::Status` is a local enum with the following possible values:

    - `MarkedSectionStartEvent::include`
    - `MarkedSectionStartEvent::rcdata`
    - `MarkedSectionStartEvent::cdata`
    - `MarkedSectionStartEvent::ignore`
  `size_t nParams`
    The number of top-level parameters in the status keyword specification.
  `const MarkedSectionStartEvent::Param *params`
    The top-level parameters in the status keyword specification.

    `Param` is a local struct with the following members:

    `MarkedSectionStartEvent::Param::Type type`
      The type of top-level parameter:

      `MarkedSectionStartEvent::Param::Type` is a local enum with the following possible values:

      `MarkedSectionStartEvent::Param::temp`
      `MarkedSectionStartEvent::Param::include`
      `MarkedSectionStartEvent::Param::rcdata`
      `MarkedSectionStartEvent::Param::cdata`
      `MarkedSectionStartEvent::Param::ignore`
        The parameter is the corresponding keyword.
      `MarkedSectionStartEvent::Param::entityRef`
        The parameter is an entity reference.
    `CharString entityName`
      Valid when `type` is `MarkedSectionStartEvent::Param::entityRef`.
`MarkedSectionEndEvent`
  Generated for the end of a marked section in the instance, but only if the `ParserEventGeneratorKit::outputMarkedSections` option is enabled. This is non-ESIS information. The event has the following members:
  `Position pos`
    The position of the end of the marked section declaration.
  `MarkedSectionEndEvent::Status status`
    The effective status of the marked section.

    `MarkedSectionEndEvent::Status` is a local enum with the following possible values:

    - `MarkedSectionEndEvent::include`
    - `MarkedSectionEndEvent::rcdata`
    - `MarkedSectionEndEvent::cdata`
    - `MarkedSectionEndEvent::ignore`
`IgnoredCharsEvent`
  Generated for a sequence of characters in an ignored marked section in the instance, but only if the `ParserEventGeneratorKit::outputMarkedSections` option is enabled. This is non-ESIS information. The event has the following members:
  `Position pos`
    The position of the first of the ignored characters.
  `CharString data`
    The ignored characters.
`ErrorEvent`
  Generated for each error detected by the parser, and also for any other cases where the parser produces a message. This is non-ESIS information. It has the following members:
  `Position pos`
    The position at which the error was detected.
  `ErrorEvent::Type type`
    The type of error.

    `ErrorEvent::Type` is a local enum with the following possible values:

    `ErrorEvent::quantity`
      Exceeding a quantity limit.
    `ErrorEvent::idref`
      An IDREF to a non-existent ID.
    `ErrorEvent::capacity`
      Exceeding a capacity limit.
    `ErrorEvent::otherError`
      Any other kind of error.
    `ErrorEvent::warning`
      A warning. Not actually an error.
    `ErrorEvent::info`
      An informational message. Not actually an error.
  `CharString message`
    The message produced by the parser. If messages are not disabled, this will be the same as the message printed to standard error.

`SGMLApplication` also has a virtual function

    void openEntityChange(const OpenEntityPtr &);

which is similar to an event. An application that wishes to makes use of position information must maintain a variable of type `OpenEntityPtr` representing the current open entity, and must provide an implementation of the `openEntityChange` function that updates this variable. It can then use the value of this variable in conjunction with a `Position` to obtain a `Location`; this can be relatively slow. Unlike events, an `OpenEntityPtr` has copy constructors and assignment operators defined.

## EventGenerator

The `EventGenerator` interface provides the following functions:

`unsigned run(SGMLApplication &app)`
  Generate the sequence of events, calling the corresponding member of `app` for each event. Returns the number of errors. This must not be called more than once for any `EventGenerator`object.
`EventGenerator *makeSubdocEventGenerator(const SGMLApplication::Char *s, size_t n)`
  Makes a new `EventGenerator` for a subdocument of the current document. s and n together specify the system identifier of the subdocument entity. These should usually be obtained from the `generatedSystemId` member of the `externalId` member of the `Entity` object for the subdocument entity. This function can only be called after `run` has been called; the call to `run` need not have returned, but the `SGMLApplication` must have been passed events from the prolog or instance (ie the SGML declaration must have been parsed).
`void inhibitMessages(bool b)`
  If b is true, disables error and warning messages, otherwise enables them. Initially errors and warnings are enabled. This function may be called at any time, including while `run()` is executing.
`void halt()`
  Halt the generation of events by `run()`. This can be at any point during the execution of `run()`. It is safe to call this function from a different thread from that which called `run()`.

## ParserEventGeneratorKit

The `ParserEventGeneratorKit` class is used to create an `EventGenerator` that generate events using SP. It provides the following members:

`EventGenerator *makeEventGenerator(int nFiles, char *const *files)`
  This returns a new `EventGenerator` that will generate events for the SGML document whose document entity is contained in the `files`. The returned `EventGenerator` should be deleted when it is no longer needed. `makeEventGenerator` may be called more than once.
`void setOption(ParserEventGeneratorKit::Option opt)`
  This can be called any time before `makeEventGenerator()` is called.

  `ParserEventGeneratorKit::Option` is a local enum with the following possible values:

  `ParserEventGeneratorKit::showOpenEntities`
    This corresponds to the `-e` option of nsgmls.
  `ParserEventGeneratorKit::showOpenElements`
    This corresponds to the `-g` option of nsgmls.
  `ParserEventGeneratorKit::outputCommentDecls`
    This will cause `CommentDeclEvent`s to be generated.
  `ParserEventGeneratorKit::outputMarkedSections`
    This will cause `MarkedSectionStartEvent`s, `MarkedSectionStartEvent`s and `IgnoredCharsEvent`s to be generated.
  `ParserEventGeneratorKit::outputGeneralEntities`
    This will cause `GeneralEntityEvent`s to be generated.
`void setOption(ParserEventGeneratorKit::OptionWithArg opt, const char *arg)`
  This can be called any time before `makeEventGenerator()` is called.

  `ParserEventGeneratorKit::OptionWithArg` is a local enum with the following possible values:

  `ParserEventGeneratorKit::addCatalog`
    This corresponds to the `-m` option of nsgmls.
  `ParserEventGeneratorKit::includeParam`
    This corresponds to the `-i` option of nsgmls.
  `ParserEventGeneratorKit::enableWarning`
    This corresponds to the `-w` option of nsgmls.
  `ParserEventGeneratorKit::addSearchDir`
    This corresponds to the `-D` option of nsgmls.

## Using the interface

Creating an application with this interface involves the following steps:

- Derive a class from `SGMLApplication`, called, say, `MyApplication`.
- For each kind of event `FooEvent` that the application needs information from, define a member of `MyApplication` `void MyApplication::foo(const FooEvent &)`.
- Declare an object of type `ParserEventGeneratorKit`.
- Optionally set options for the `ParserEventGeneratorKit` using `ParserEventGeneratorKit::setOption`.
- Create an `EventGenerator` using `ParserEventGeneratorKit::makeEventGenerator`.
- Create an instance of `MyApplication` (usually on the stack).
- Call `EventGenerator::run` passing it a reference to the instance of `MyApplication`.
- Delete the `EventGenerator`.

The application must include the `ParserEventGeneratorKit.h` file (which in turn includes `EventGenerator.h` and `SGMLApplication.h`), which is in the `generic` directory. If your compiler does not support the standard C++ `bool` type, you must ensure that `bool` is defined as a typedef for `int`, before including this. One way to do this is to include `config.h` and then `Boolean.h` from the `lib` subdirectory of the SP distribution.

On Unix, the application must be linked with the `lib/libsp.a` library.

## Example

Here's a simple example of an application that uses this interface. The application prints an outline of the element structure of a document, using indentation to represent nesting.

    // The next two lines are only to ensure bool gets defined appropriately.
    #include "config.h"
    #include "Boolean.h"

    #include "ParserEventGeneratorKit.h"
    #include <iostream.h>

    ostream &operator<<(ostream &os, SGMLApplication::CharString s)
    {
      for (size_t i = 0; i < s.len; i++)
        os << char(s.ptr[i]);
      return os;
    }

    class OutlineApplication : public SGMLApplication {
    public:
      OutlineApplication() : depth_(0) { }
      void startElement(const StartElementEvent &event) {
        for (unsigned i = 0; i < depth_; i++)
          cout << "    ";
        cout << event.gi << '\n';
        depth_++;
      }
      void endElement(const EndElementEvent &) { depth_--; }
    private:
      unsigned depth_;
    };

    int main(int argc, char **argv)
    {
      ParserEventGeneratorKit parserKit;
      // Use all the arguments after argv[0] as filenames.
      EventGenerator *egp = parserKit.makeEventGenerator(argc - 1, argv + 1);
      OutlineApplication app;
      unsigned nErrors = egp->run(app);
      delete egp;
      return nErrors > 0;
    }

There's a bigger example in the `sgmlnorm` directory in the SP distribution. This uses the `SGMLApplication` interface, but it doesn't use the `ParserEventGeneratorKit` interface.