Package lxml :: Package html :: Module clean :: Class Cleaner

[frames] | no frames]

# Class Cleaner

source code

    object --+
             |
            Cleaner

Instances cleans the document of each of the possible offending elements. The cleaning is controlled by attributes; you can override attributes in a subclass, or set them in the constructor.

`scripts`:
  Removes any `<script>` tags.
`javascript`:
  Removes any Javascript, like an `onclick` attribute.
`comments`:
  Removes any comments.
`style`:
  Removes any style tags or attributes.
`links`:
  Removes any `<link>` tags
`meta`:
  Removes any `<meta>` tags
`page_structure`:
  Structural parts of a page: `<head>`, `<html>`, `<title>`.
`processing_instructions`:
  Removes any processing instructions.
`embedded`:
  Removes any embedded objects (flash, iframes)
`frames`:
  Removes any frame-related tags
`forms`:
  Removes any form tags
`annoying_tags`:
  Tags that aren't wrong, but are annoying. `<blink>` and `<marque>`
`remove_tags`:
  A list of tags to remove.
`allow_tags`:
  A list of tags to include (default include all).
`remove_unknown_tags`:
  Remove any tags that aren't standard parts of HTML.
`safe_attrs_only`:
  If true, only include 'safe' attributes (specifically the list from feedparser).
`add_nofollow`:
  If true, then any <a> tags will have `rel="nofollow"` added to them.
`host_whitelist`:

  A list or set of hosts that you can use for embedded content (for content like `<object>`, `<link rel="stylesheet">`, etc). You can also implement/override the method `allow_embedded_url(el, url)` or `allow_element(el)` to implement more complex rules for what can be embedded. Anything that passes this test will be shown, regardless of the value of (for instance) `embedded`.

  Note that this parameter might not work as intended if you do not make the links absolute before doing the cleaning.

`whitelist_tags`:
  A set of tags that can be included with `host_whitelist`. The default is `iframe` and `embed`; you may wish to include other tags like `script`, or you may want to implement `allow_embedded_url` for more control. Set to None to include all tags.

This modifies the document in place.

Instance Methods

__init__(self, **kw) x.__init__(...) initializes x; see x.__class__.__doc__ for signature | source code

__call__(self, doc) Cleans the document. | source code

allow_follow(self, anchor) Override to suppress rel="nofollow" on some anchors. | source code

allow_element(self, el) | source code

allow_embedded_url(self, el, url) | source code

kill_conditional_comments(self, doc) IE conditional comments basically embed HTML that the parser doesn't normally see. | source code

clean_html(self, html) | source code

Inherited from `object`: `__delattr__`, `__getattribute__`, `__hash__`, `__new__`, `__reduce__`, `__reduce_ex__`, `__repr__`, `__setattr__`, `__str__`

Class Variables
| scripts = `True`
| javascript = `True`
| comments = `True`
| style = `False`
| links = `True`
| meta = `True`
| page_structure = `True`
| processing_instructions = `True`
| embedded = `True`
| frames = `True`
| forms = `True`
| annoying_tags = `True`
| remove_tags = `None`
| allow_tags = `None`
| remove_unknown_tags = `True`
| safe_attrs_only = `True`
| add_nofollow = `False`
| host_whitelist = ``(``)``
| whitelist_tags = ``set([``'``embed``'``,` `'``iframe``'``])``

Properties
Inherited from `object`: `__class__`

Method Details

### __init__(self, **kw)
(Constructor)

source code

x.__init__(...) initializes x; see x.__class__.__doc__ for signature

Overrides: object.__init__
  (inherited documentation)

### kill_conditional_comments(self, doc)

source code

IE conditional comments basically embed HTML that the parser doesn't normally see. We can't allow anything like that, so we'll kill any comments that could be conditional.