# erl_scan

### MODULE

erl_scan

### MODULE SUMMARY

The Erlang Token Scanner

### DESCRIPTION

This module contains functions for tokenizing characters into Erlang tokens.

### EXPORTS

`string(CharList,StartLine]) -> {ok, Tokens, EndLine} | Error`
`string(CharList) -> {ok, Tokens, EndLine} | Error`

Types:

`CharList = string()`
`StartLine = EndLine = Line = integer()`
`Tokens = [{atom(),Line}|{atom(),Line,term()}]`
`Error = {error, ErrorInfo, EndLine}`

Takes the list of characters `CharList` and tries to scan (tokenize) them. Returns `{ok, Tokens, EndLine}`, where `Tokens` are the Erlang tokens from `CharList`. `EndLine` is the last line where a token was found.

`StartLine` indicates the initial line when scanning starts. `string/1` is equivalent to `string(CharList,1)`.

`{error, ErrorInfo, EndLine}` is returned if an error occurs. `EndLine` indicates where the error occurred.

`tokens(Continuation, CharList, StartLine) ->Return`

Types:

`Return = {done, Result, LeftOverChars} | {more, Continuation}`
`Continuation = [] | string()`
`CharList = string()`
`StartLine = EndLine = integer()`
`Result = {ok, Tokens, EndLine} | {eof, EndLine}`
`Tokens = [{atom(),Line}|{atom(),Line,term()}]`

This is the re-entrant scanner which scans characters until a dot ('.' whitespace) has been reached. It returns:

`{done, Result, LeftOverChars}`
  This return indicates that there is sufficient input data to get an input. `Result` is:
  `{ok, Tokens, EndLine}`
    The scanning was successful. `Tokens` is the list of tokens including dot.
  `{eof, EndLine}`
    End of file was encountered before any more tokens.
  `{error, ErrorInfo, EndLine}`
    An error occurred.
`{more, Continuation}`
  More data is required for building a term. `Continuation` must be passed in a new call to `tokens/3` when more data is available.

`reserved_word(Atom) -> bool()`

Returns `true` if `Atom` is an Erlang reserved word, otherwise `false`.

`format_error(ErrorDescriptor) -> string()`

Types:

`ErrorDescriptor = errordesc()`

Takes an `ErrorDescriptor` and returns a string which describes the error or warning. This function is usually called implicitly when processing an `ErrorInfo` structure (see below).

### Error Information

The `ErrorInfo` mentioned above is the standard `ErrorInfo` structure which is returned from all IO modules. It has the following format:

        {ErrorLine, Module, ErrorDescriptor}

A string which describes the error is obtained with the following call:

    apply(Module, format_error, ErrorDescriptor)

### Notes

The continuation of the first call to the re-entrant input functions must be `[]`. Refer to Armstrong, Virding and Williams, 'Concurrent Programming in Erlang', Chapter 13, for a complete description of how the re-entrant input scheme works.

### See Also

io(3), erl_parse(3)

### AUTHORS

Robert Virding - support@erlang.ericsson.se

stdlib 1.14.5
Copyright © 1991-2007 Ericsson AB